Apache Lucene and Solr go 4.5

solrThe text-search library Lucene and Solr, the search platform built on top of it, have both been updated to version 4.5. Version 4.4 came out in July so what’s changed in this version bump?

Well, first of all, for Lucene, the DocValues mechanism which allows typed storage to be associated with documents has been updated to allow for missing values and there’s now an in-memory supporting DocIDSet which is more efficient for carrying around smaller lists of documents. Other changes can be found in the Lucene 4.5 release notes.

Solr 4.5, as usual, benefits and supports these changes as it is built on Lucene, but the search platform has also had its own set of improvements. For example, when running a sharded cluster, its possible to now set up custom routing to the various shards, including routing based on field values. Faceted searches are now multi-threaded, the solr.xml configuration file is now storable in ZooKeeper and the CloudSolrServer has the ability to send updates directly to shard leaders. Again, more details are available in the Solr 4.5 release notes and the PDF of the updated Solr reference guide is available through the Apache mirrors. Both Lucene and Solr also have various bugfixes and performance improvements.

NetBSD 6.1.2, Lua JVM, Meego/Symbian’s long walk and MariaDB/Debian – Snippets


  • NetBSD 6.1.2 released: The second security/bug-fix release for NetBSD 6.1 is now available with one security fix and fixes for KVM shutdown, USB device enumeration, networking with npf, udf file systems and pthreads. There’s also updated timezone data, a corrected regression for some X apps and a fix for some Emacs 24 crashes.
  • A Lua JVM?: An intriguing experiment has appeared in the form of luje, a “toy” Java virtual machine written in Lua. It on-the-fly compiles the Java byte code into Lua scripts and then runs them with LuaJIT. “Right now it excels at anything which involves tight loops and float or doubles in local variables” say the developer, David Given, noting it can beat the Hotspot/JIT in those cases, but it does badly with longs and many other things. The code is a 0.1 release, is fragile and incompletly implemented, but if interesting JIT tricks are your thing, this is one to look at.
  • Crunch time for Meego/Symbian: It looks like the end for Nokia’s support of Meego and Symbian support of devices through the company’s store is coming with the blocking of new app publication from 1 January 2014.
  • MariaDB’s heading to Debian: Colin Charles passes on the news that Debian’s MySQL package team have a plan for MariaDB 5.5, that it’s been uploaded to Debian unstable and should appear in unstable in due course.

LibreOffice updated, iPython sponsored, Warden contained – Snippets


  • LibreOffice gets a maintenance bump: There’s an update for LibreOffice 4.1, the just announced 4.1.2 but the Document Foundation are still not up to recommending it for enterprise adoption and say a 4.0.5 (and soon 4.0.6) version of the office suite is still recommended for that. As usual they’ve scattered the changelogs over 3 different documents (at some point they might think about consolidating minor point updates changelogs into oooh a single release note), but in summary, things have been fixed most of which are listed in the RC1 changelog.
  • iPython gets Microsoft mad money: The iPython project, which creates a Python based architecture for interactive notebooks, visualisation, interpreters and parallel computing work, has just announced $100,000 sponsorship from Microsoft. Apparently they did one heck of a demo for the Microsoft Research folks. The sponsorship went through NumFocus who are sponsored by J.P.Morgan and Microsoft among others to promote open source scientific software.
  • More containers: Everyone’s got a plan for managing containers these days. This time it’s Cloud Foundry’s warden, an Apache licensed “simple API for managing isolated environments”. The server readme provides more details; apparently initially developed with LXC, Warden no longer depends on LXC. It uses aufs or overlayfs depending on edition of Ubuntu Linux and talks JSON over sockets between its server and clients.

Arduino’s x86 and TI/ARM treats

Arduino Tre – Bristling with connections

Arduino has been working with both Intel and TI to create two new boards, both of which are quite interesting departures from their previous designs. Both run Linux, in different ways but while one tries to replace the AVR microcontroller of the classic Arduino, the other hugs the classic Arduino deep into its design.

The first board announced was the Arduino Galileo which is powered by Intel’s Quark SoC X1000 running at 400Mhz and in due to be available in November and, according to some reports, will be “less than $80”. The processor is a 32-bit “Pentium-class” chip and the datasheet(pdf) details how the board has a set of 3.3V (or jumper settable 5V) connectors which are Arduino Uno R3 pin compatible. There’s also 10/100 Ethernet, a PCI Express mini card slot, micro-SD slot and USB client and USB host connectors on the board. This is very much an Intel rendering of what an Arduino would be with Intel’s Quark at its core; note, for example, that for a board of its spec, there’s no video out of any form, despite being closer to the Raspberry Pi and BeagleBone Black in pricing and that from the higher side. It’s an interesting iteration which gets Intel into the emerging market of small embeddable devices, but will it play with the makers – it doesn’t look easy to go from a Galileo to a production device.

The other board may be more exciting for makers, even if it is not available till next Spring and being shown now only as a sneak peek. Designed as part of a collaboration between Arduino and the BeagleBoard.org foundation, the Arduino Tre is a double processor sandwich with a 1GHz TI Sitara AM335x processor running Linux wrapped around a Atmega32u4 based Arduino – yes, the BeagleBone Black and Arduino have had a love-child and the Tre is the result; an ARM based Linux running processor to do the heavy compute lifting and networking and a classic Arduino to do the interfacing. The Tre is covered in connectors to wire up to, with the Arduino shield pins in the centre and the BeagleBone style cape connectors still on the board but separated by a whole Arduino now.

How this’ll work in practice, who knows, but it opens up a range of opportunities, especially as the Tre, unlike the Galileo, has HDMI video out too. Again no official pricing and this is down for a Spring landing so a bit of a wait. Till then, if you haven’t got one, get yourself a BeagleBone Black and interface it to your Arduino to simulate at least some of the experience.

Which reminds me… (he said getting his BeagleBone Black out)…

Google Containers, Freeseer 3.0 and free JavaScript books – Snippets


  • LMCTFY contains itself: A Google project, LMCTFY (Let Me Contain That For You) has emerged in the companies GitHub repository. It’s an open source version of Google’s container stack for Linux though it’s more application isolation and lacks Docker’s filesystem isolation. It’s apparently early days for the Apache licensed software but it will be one to keep an eye on as it could well turn into the basis for a Docker competitor.
  • Freeseer 3.0: The Freeseer project is an interesting platform for capturing in multiple video streams and presentations and managing them. Freeseer 3.0 came out about a month ago and has a completely rewritten backend, plugin system for GStreamer I/O and Mixing, RTMP streaming support and more. Built in Python and using GStreamer at its core, the software is GPLv3 licensed with source code on GitHub and it has been used to record many open source conferences. Find out more about the project in its documentation along with various guides.
  • JavaScript Allongé: The book JavaScript Allongé is now under a CC Attribution-ShareAlike 3.0 license and its author, Reginald Braithwaite, has written about why he has made it free. You can even get the source for the book in Markdown format and Braithwaite encourages posting issues, fixing and submitting pull requests to correct typos and other changes.

Apache CloudStack goes 4.2.0

cloudstackThe other other open source IaaS Cloud, CloudStack, has had an update with the release of CloudStack 4.2. What’s new? reveals a lot of work which the announcement summarises as 57 new features and 29 improved features such as the ability to plug in external or internal S3-compatible storage services and support for Cisco’s UCS compute chassis and SolidFire storage arrays.

A trawl through the release notes shows that there is far more than the headline items though. There’s a whole set of features to help support for regions, zone wide primary storage and a plug-in framework for writing UI extensions.

Networking has had a lot of work done to it too with initial support for IPv6 (as a technical preview), portable elastic IPs which can be transferred between zones, the ability to assign a VLAN to an isolated networks and persistent networks which can exist without VMs assigned to it. There’s also Cisco VNMC and VMware VDS support, enhanced support for Juniper gear and global server load balancing with health checks for load balanced instances.

Host support has not been left out. Windows 8 and Windows Server 2012 can now be VM Guest OS’s, ownership of VMs can now be changed by an administrator, resizable data disk volumes, storage migration (for XenServer and VMware), the ability to scale CPU and memory on running VMs (VMware and XenServer again), over-provisioning of memory and cpu (VMware, XenServer and KVM), bare-metal provisioning kickstarter, VM resetting on reboot and VMware VM snapshots.

Finally, there’s a who set of enhancements to the monitoring, maintenance and operations end of CloudStack, with support for auto purging alerts, API request throttling, forwarding of alerts to external SNMP and Syslog systems, a log collection tool, ability to change default password encryption and new VM snapshot and backup capabilities.

You can download the source or binaries (in deb and rpm packages) from cloudstack.apache.org where there is also documentation including installation and admin guides.

Ubuntu 13.10’s Mir-miss

Mir_diagram.svgThe news that XMir and Mir will not be in desktop Ubuntu 13.10 is hardly a surprise. Canonical set an aggressive development schedule and its one they are going to miss on the desktop. Ubuntu Touch is already running Mir as it has no legacy X apps due to it being yet to be released as a finished product and not supporting X anyway. But XMir is critical for the desktop if Canonical want to push Mir into the space they’ve assigned it as core to their graphics strategy.

Brief reminder: Canonical broke from the consensus development of a new display server technology to replace the X server called Wayland to develop their own display server called Mir. To make the transition and support current X applications, a way to run X applications on the new display technology is needed. For Mir, that’s XMir, for Wayland, there’s a rootless X server.

But the desktop schedule delay is going to have consequences. With 14.04 next April being an LTS release, if XMir is landed as a default in that release then it’s going to an interesting five years of support. If I was Canonical, I’d be looking at cancelling the 14.04 LTS status and moving it on to 14.10, using 14.04 as what 13.10 should have been – the release where all the glitches in XMir get shook out as the default desktop in the hands of real users. They can still release to their OEM partners with 14.04 after they’ve explained they’ll want to push an update in 2014Q4.

Meanwhile, the Wayland developers are working steadily away without the obvious pressure of OEMs and commercial schedules and will have a tech preview shipping with Fedora 20 if all goes to plan. There is no race to ship a new display server technology for Linux, but one player has been running while another walking and they don’t seem, at least from here, to be that far apart. For developers, there’s still no pressing need to choose a path either – keep developing with Qt or GTK+ or X raw and you’ll still be good for some years.