X.org vintage bugs, Google FOSS fixings and a dropzone – Snippets


  • Vintage bugs: Back in 1993, a use after free bug when handling ImageText wriggled its way into the X.org server and settled into what is believed to be every X.org server release that came after. Just over 20 years later, a security advisory and patch have been published for the bug. So look out for updates to your Linux distribution’s (or other Unix’s) X.org server in the near future. To many eyes, all bugs are eventually shallow. But, who really wants to look inside an X server.
  • Google’s FOSS-fixins: If you are looking for more than just bugs to fix, you can also check out Google’s latest bounty program which is offering rewards for proactively fixing up the security in well known open source applications. First up for rewards are OpenSSH, BIND, ISC DHCP, libjpeg, libjpeg-turbo, libpng, giflib, Chromium, Blink, OpenSSL, zlib and “Security-critical, commonly used components of the Linux kernel”. Help harden them up and you could be in line for up to $3133.7. The second phase will see that set of code joined by Apache httpd, lighttpd, nginx, Sendmail, Postfix, Exim, the toolchain security for GCC, binutils and LLVM and OpenVPN. I applaud Google for this as it goes beyond Google Summer of Code manpower and mentoring and should let a whole new set of contributors help harden the open source ecosystem.
  • DropzoneJS: Do you love sites which make it easy to upload images with a drag and a drop into the browser? The open source (MIT license) DropzoneJS library helps you do it with style, letting you drag files into the drop zone and showing uploads with thumbnails – its reported to have some trouble with hundreds of images, but also is easy to implement – if thats what you want to work with its there to be fixed.


GNU Make 4.0, Firefox OS 1.1, SSL Pulse and Linux defined – Snippets


  • GNU Make 4.0: GNU Make 4.0 is the latest version of the GNU Project’s version of the Make utility. The release’s headline feature is the integration of GNU Guile, the Scheme-based extention language recommended for GNU projects, into the compilation orchestrator. Other additions include an option to sync output to avoid jumbling results from parallel makes, tracing of targets, a switch to disable all debugging settings, various enhancements to the Windows version, the implementation of “::=” for POSIX portable make files and of “!=” for compatibility with BSD make files and the ability to write to a file directly. The announcement has details of where to download it and further information on the changes.
  • Firefox OS 1.1 on its way: The first proper update for Firefox OS, version 1.1 is heading to handsets soon. With MMS support, an API for Push Notifications, better app searching, enhanced contacts, faster app loading and scrolling (and it needs it), downloading of images and audio or video in the browser, autocorrect for the keyboard, a draft mode and attachment support for the email app and a lightly improved calendar app, there’s a lot of decent catching up with the rest of the pack. Mozilla say this will come with the second wave of Firefox OS phones, and appear for existing Firefox OS phones at the same time. We’ll be reporting on the update when it arrives on our ZTE Open.
  • Watching SSL: Wondering how widespread Forward Secrecy and RC4 are with SSL/HTTPS web sites? Wonder no more as the SSL Pulse report now tracks them too. Ivan Ristic explains whats being tracked in this blog post and how the results currently stack up.
  • Predefining Linux: An informative and interesing Stack Overflow moment as one person finds “linux” is defined as “1” and opens the door of the world of pre-defined symbols and other mystery meats that can be found in the larder of C compilers.

Apache Lucene and Solr go 4.5

solrThe text-search library Lucene and Solr, the search platform built on top of it, have both been updated to version 4.5. Version 4.4 came out in July so what’s changed in this version bump?

Well, first of all, for Lucene, the DocValues mechanism which allows typed storage to be associated with documents has been updated to allow for missing values and there’s now an in-memory supporting DocIDSet which is more efficient for carrying around smaller lists of documents. Other changes can be found in the Lucene 4.5 release notes.

Solr 4.5, as usual, benefits and supports these changes as it is built on Lucene, but the search platform has also had its own set of improvements. For example, when running a sharded cluster, its possible to now set up custom routing to the various shards, including routing based on field values. Faceted searches are now multi-threaded, the solr.xml configuration file is now storable in ZooKeeper and the CloudSolrServer has the ability to send updates directly to shard leaders. Again, more details are available in the Solr 4.5 release notes and the PDF of the updated Solr reference guide is available through the Apache mirrors. Both Lucene and Solr also have various bugfixes and performance improvements.

Updates for RethinkDB and FreeBSD and a 64-bit .NET JIT boost – Snippets


  • RethinkDB gets multi-indexing: The developers of the open source, NoSQL database RethinkDB have announced version 1.10 which comes with the ability to index rows with fields of multiple values, like say an list of tags for a blog entry. Looking for all records with a particular tag previously required slow iteration, but now with the multi-index it is possible to index the set of values within the field and then to “get_all” for a particular tag value using that index. RethinkDB server is written in C++ and AGPL licensed with Apache licensed client drivers.
  • FreeBSD 9.2 released: In the latest FreeBSD release ZFS gets added TRIM support for solid state drives and lz4 compression and there’s updates for OpenSSL (to 0.9.8y), DTrace (to 1.9.0), Sendmail (to 8.14.7) and OpenSSH (to 6.2p2). There’s also virtio drivers and enabled Dtrace in the “GENERIC” kernel. Read more in the FreeBSD 9.2 release announcement.
  • RyuJIT for .NET: Over in the world of .NET, interesting things are afoot with a new 64-bit just-in-time compiler, RyuJIT, making its debut as a CTP (Community Technical Preview). .NET’s had a 64-bit JIT for some time, though the JIT has apparently been quite slow. RyuJIT runs twice as fast and overall gives a 30% speed up to start up. One benchmark with regular expressions went off the scale, going from a 1.4GB working set and 60 seconds run time to 199MB and 1.8 seconds run time – yes the older compiler is particularly bad at regular expressions.

OpenStack costs, Boot2Gecko on APC, Python debugging and a storage warning – Snippets


  • OpenStack Hardware Calculator: Mirantis have an interesting OpenStack calculator which lets you how many and how big you want your average virtual machine, pick hardware and networking vendor and whether you want high availability or not. It comes back to you with a couple of configurations based on those requirements and $ pricing of the cloud’s hardware.
  • Boot2Gecko on Rock and Paper: Via has announced a preview of Boot2Gecko for it’s APC single board ARM-based PCs “Rock” and “Paper”. Boot2Gecko is the name of Firefox OS when its on unblessed devices as Liliputing pointed out, although the GitHub repository is still labelled APC-Firefox-OS. There’s plenty of known issues, but Via are offering free APCs to anyone who fixes them and sends a pull request. Wondering what to fix? There’s a list of bugs and enhancements awaiting work.
  • Python Debugging: Over on Hacker News, people are recommending pudb, the Python Urwid Debugger, which works in the console as a full terminal application. Older hands will get the “Turbo Pascal” vibe from it as it appears to have take some inspiration from there. So, Unix based Python programmers may want to check it out.
  • A warning about storage: A useful reminder from Christopher Deutsch’s blog about making sure that when you release an open source project you aren’t including any URLs which will cost you money. In Deutsch’s case it was a test file on Amazon S3 which was used by HiSRC to check bandwidth… and has just cost him $20 on his monthly Amazon bill.

ARM64, GNU Hurd and APL and curious binary – Snippets


  • ARM64 and iPhone explained: A useful look at what is actually changing with Apple’s A7 and ARM64 architecture from Mike Ash’s blog. Worth a read especially for the repurposed isa pointer.
  • GNU Hurd Updates: On the 30th anniversary of the GNU project, the Hurd developers released an update to the project’s operating system along with an update to GNU Mach and RPC translator GNU MIG.
  • GNU APL 1.0 Lands: APL is one of the venerable languages dating back to 1964 which has classically been associated with number crunching. And now, after some time in development, Jürgen Sauermann has announced the availability of GNU APL 1.0.
  • Polyglot binary code: CorkaMIX is a set of binaries for Windows, Linux and Mac OS X, each one of which is also a PDF document, a Java JAR file and a HTML document with JavaScript. As the author says “they serve no purpose, except proving that file formats not starting at offset 0 are a bad idea”.

Ubuntu 13.10 Betas, Rust 0.8 and Android drive-bys? – Snippets


  • Ubuntu 13.10’s only beta: The “Final Beta” for 13.10’s awfully codenamed “Saucy Salamander” has been announced so those wanting to give it a try before the 17 October final release, this is your chance. There’s an Ubuntu for phones image in among the images for the first time too. The release notes have details on how to upgrade and install. With only a 9 month supported lifespan from its release, you may want to consider waiting for next April’s 14.04LTS release.
  • Rust 0.8: Mozilla’s Rust language moves another step forward with September’s 0.8 release. Lots of details in the release notes; they’ve switched to Iterator based for loops, there’s a new (faster, more i18ny)way to format strings, changes in the FFI so there can be first-class foreign function pointers and importantly, the rewritten runtime and new experimental IO system.
  • Android drive-bys?: Interesting vulnerability in Android described by MWR Infosecurity where embedded ads in apps use WebView and how it would be possible to intercept the code going to these ad windows as its doen in the clear. And because WebView offers JavaScript functionality and because there is is JavaScript bridge, it is possible to execute arbitrary Java code. MWR worries about ad networks and Ad SDKs exposing a vector for infection and shows how they got all the way down to running system commands on the phone. There are a lot of dependencies, and it’ll take a Wifi man-in-the-middle attack to easily inject attack code – the Kismet wireless blog has a look at that problem.

Beta Ceylon, VLC 2.1 released, Whois research and Retro-browsing – Snippets


  • Ceylon goes beta: Red Hat’s own JVM-hosted language, Ceylon, has been declared feature-complete and released as a 1.0 beta. There’s a formal language spec, command line tools, SDK and a beta of an Eclipse based IDE for Ceylon too. Lots of language features have been added coming up to beta, including annotations, static methods, try for resources, switch statements that know strings and characters and more.
  • VLC 2.1 debuts: Every coder needs a video player that can handle any format. Thats my excuse anyway and here’s the newly released VLC 2.1 arriving to fill in the latest gaps in my playback capability. A new audio rendering pipeline, OpenGL ES support, new ports (Android from 2.1 to 4.3 for ARM, x86 and MIPS and iOS 5 to 7), a partial WinRT port, Microsoft Smoorth Streaming, support for VNC/rfb and remote desktop view-only modes, lots of new hardware decoding support on OS X, Android, Linux with VDPAU and Windows QuickSyncVideo. Oh yes and there’s the foundations for UltraHD support. And developers will find the code is amenable to integration with more software due to libVLC (and most of the modules) being under the LGPL2.1+.
  • Whois Privacy: An interesting study of whois and the identity proxies used to cover the identity of owners. Interesting in that the idea that only those with something nefarious to hide may use the obfuscating services is blown out of the water – “for example banks use privacy and proxy services almost as often as the registrants of domains used in the hosting of child sexual abuse images; and the registrants of domains used to host (legal) adult pornography use privacy and proxy services more often than most (but not all) of the different types of malicious activity that we studied”. Fixing Whois is going to be harder than we thought.
  • Browse like its 1992: Cern have launched [Line Mode Browser 2013], an emulation of 1992’s line mode browser, using Node.js and modern browser technology to recreate the glow green matrix of terminals of that era. You can find the code on GitHub.

Mozilla’s font, Fedora’s alpha, Java’s fixes and Gstreamer’s flow – Snippets


  • Mozilla’s Fira font: Mozilla has released a new open source (SIL Open Font Licence) font called Fira Sans. This is the Firefox OS typeface and comes in light, regular, medium and bold weights. There’s also a monospaced variant in regular and bold. Source for the font is available on GitHub.
  • Fedora 20 Alpha: On schedule for the revised schedule, the alpha of Fedora 20 has been released. As previously mentioned, and in the announcement, there’s lots of updates and enhancements including ARM as a primary architecture, latest GNOME and KDE, the undefaulting of SendMail and Syslog and a better NetworkManager. It’s ready to test and you can download the Alpha images but don’t press them into production for the worst that could happen will probably happen. Or not. Next stop, the beta release at the end of October.
  • Java Fixes and Pitfalls: Will Dormann,the author of the CERT Blog posting Don’t Sign that Applet has returned to the subject pointing out, in Signed Java Applet Security Improvements, that although applets running on Java7 before update 25 could be repurposed, that problem was addressed in Java 7 Update 25. In Update 25 though an applet can declare it must be executed in a sandbox and can be made to restrict where it loads code from. But you do need to be running update 25 or later (the current update version is 40) and Dormann does point out some gotchas which a developer needs to dodge for this change to be useful.
  • GStreamer 1.2.0 flows: The developers of the LGPL-licensed multimedia framework GStreamer have released version 1.2.0 of the framework with new API features, plugins for Microsoft Smooth Streaming, DASH adaptive streaming, bluez interation, openjpeg for JPEG2000 support, experimental VP9 encoding and decoding and many others.

PyCharm goes open source


JetBrains has announced that PyCharm 3, its Python IDE, is following the route pioneered by their Java IDE, IntelliJ IDEA and getting an open source community edition and a feature laden professional edition. The JetBrains idea is that the core features of an IDE, the editing and debugging, are better built in the open while they look at developing features that users can get a reasonable return on investment. “The ROI on code completion is huge” has been said by no-one ever while “Having the IDE handle my database models and framework integration has saved me hours” is a thing.

The differences between the two editions are listed in a comparison matrix. The shorter version is the community edition includes the smart code completing editor with error highlighting, code refactoring, integrated debugger and unit testing, version control integration and UI customisation options including VIM emulation with all of that under the Apache 2 licence. PyQt and PyGTK are also supported.

The professional version adds support for various Python frameworks, supports cross-language working (such as mixed JavaScript/HTML/Python development), remote host debug and testing and database support. Those frameworks include Django and Google’s App Engine. The professional edition starts at £76+VAT for the Personal version and £153+VAT for the commercial version.

Going open source doesn’t mean there aren’t any new features in PyCharm 3 though. The What’s New lists a fully featured terminal (so you don’t need to leave the IDE), new refactorings like invert boolean project-wide and replace duplicates, non-Python code injection, better analysis and improved type inference in both the community and professional versions. The professional version has acquired SQLAlchemy, Pyramid and Web2Py framework support, better Django code completion and duplicate code detection.

Download your preferred edition at the JetBrains site; there’s a 30 day trial for the professional version and you’ll need Java 6 or later installed along with Python 2.4 or later.