Docker 1.0: The Docker container management platform has hit version 1.0 though the major work had been done by version 0.11 – this is the project’s graduation, acknowledging its ready for production. The actual packaging and management software is going to be referred to as Docker Engine now as the announcement is also the signal for Docker (the company) to roll out 1.0 of Docker Cloud, a platform for sharing Docker packaged apps. Actual changes in 1.0 are things like a new COPY command and an improved ADD command for developers and the ability to pause and resume running containers, added XFS support and performance improvements in container removal. Make a note too. Posts 2375 and 2376 are now officially the HTTP and HTTPS ports for Docker. Docker has changes how people think about package and run applications on Linux and all it would need is for major players to adopt Docker and … oh Google’s added App Engine support for Docker to go with its Compute Engine support and then there’s…
RHEL 7.0: Red Hat has rolled out version 7.0 of its enterprise Linux and 7.0 is looking like a cracking release. Top of their highlights, containers and Docker support, XFS as default file system and new caching file systems (Btrfs is still experimental), systemd and new management components and more capabilities to work with Windows domains. The release notes as with all Red Hat releases are comprehensive and cover things like the switch to GNOME 3 on the desktop (while retaining a classic shell). RHEL 7 is the commercially supported upstream for other distributions, most notably the CentOS distribution which is working on its CentOS 7 release – no dates on that yet but it is the first test of the new relationship between Red Hat and CentOS.
Firefox 30.0: Thirty… As Firefox versioning heads out of the twenties, the [release of Firefox 30.0] has brought sidebar button for toolbar, support for GStreamer 1.0, command-E find selected, various developer changes and five critical and two high security fixes. Now it is thirty, Firefox is well on its way to settling down to a boring life where change is mostly about moving the furniture about and keeping an eye on the neighbours. The place to look for excitement is Mozilla’s Servo browser, being developed in Mozilla’s Rust language and is developing steadily.
Docker 0.9 unloads: Docker bumps its version number to Docker 0.9 and as it approaches version 1.0 makes a big change. Docker’s been pretty tightly tied to Linux Containers (LXC) technology to run applications packaged with it but in 0.9 there’s now execution drivers so the option to plug in any one of a range of isolation systems is now available. “OpenVZ, systemd-nspawn, libvirt-lxc, libvirt-sandbox, qemu/kvm, BSD Jails, Solaris Zones, and even good old chroot” are on Docker’s planned list with more to come from various projects. There’s also a new libcontainer which lets Docker plug straight into the Linux kernel to control things – this Go library is likely to see a lot of use outside of Docker too as it wraps up container configuration into a neat JSON specified bundle. Next stop for Docker is a production quality 0.10 which will serve as a release candidate for 1.0. Its lively down at the docks.
Vagrant 1.5 roams out: The developer environment manager Vagrant has been updated too. The new Vagrant 1.5 has added a sharing system to make collaboration easier, versioning for boxes, rsync and smb sync’d folders and Hyper-V support. Simpler SSH authentircation setup, a reworked plugin manager and support for Funtoo, NetBSD and TinyCore Linux as guests wrap out the wedge of features in this release. Alongside the release is the announcement of Vagrant Cloud, a hosted box sharing service built to use Vagrant 1.5’s sharing functions.
Xen 4.4 meditates: Meanwhile, the other Linux virtualisation platform, Xen, has made the first release on its aspirational six month cycle (taking 8 months in this case). The announcement for Xen 4.4 highlights an improved libvirt/libxl interface for better integration with VM managers or cloud platforms, a more flexible event channel interface allowing for over tens of thousands of guests and a rapidly maturing ARM port now with a stable ABI going forwards. There’s also a ‘tech preview’ of nested virtualisation on Intel.
- Firefox 26 digs in: Today we’ll see the release of Firefox 26, latest in the overly regular Firefox release cycle. From the (currently beta) release notes, we can see the big changes. All but the Flash plug-in are now click-to-play by default, Windows users can update their Firefox without having to write into the Firefox folders, the password manager can handle password fields generated by scripts and on Linux, if the installed gstreamer can handle h264, so can Firefox. A couple of fixes, some developer enhancements and thats about it. There’s also a Firefox for Android update due today. The release notes note some performance improvements, the same password manager enhancement and some fixes. The developer page for Firefox 26 covers changes of interest to developers in more detail. Firefox 26 will be turning up in updates and for download later today.
Netfix’s Suro goes open: From the people who brought you a cloud full of monkeys… Netflix’s latest open source release is Suro, an application monitoring system used by the video stream vendor to track the behaviour of their Amazon AWS deployed applications. Originally based on Apache Chukwa and adapted to fit Netflix’s demands, Suro pulls the company’s monitoring data from the various app clusters and pushes it to S3 (for Hadoop based analytics), to Apache Kafka (and on to Storm, Amazon ElasticSearch and Druid and to other event processors. There’s a lot more detail in the announcement including in production stats and how the pipeline is used to analyse errors.
Vagrant meets Docker: The latest update to Vagrant, version 1.4 has been announced and the big improvement in system that has traditionally been used to create automatically reproducible development environment is the addition of Docker support. The Docker provisioner can install Docker and then lets Vagrant cirtual machine pull and configure Docker containers within it. There’s also some enhancements to the scriptability of Vagrant itself, the ability to require a particular version of Vagrant and support for standalone file sync plugins.
websocketd: And finally, have you wanted to make a shell script or other app into a WebSocket server but lacked a library or access to the code to do it? Websocketd might be the answer as it turns anything with console I/O into a WebSocket server in a style rather reminiscent of CGI. Remember, most command line applications are not suitable for being exposed to the raw web, but the app could get you out of a hole when prototyping.
And, for reference, everything mentioned today is open source software.
Facebook, in their now traditional goal of taking on big data problems, solving them and then open sourcing the result, have open-sourced Presto, a distributed SQL query engine “optimized for ad-hoc analysis at interactive speed”. This type of app is designed for the folks who need to work out what people who like chips and cheese and rock but dont like bagels or opera also have, statistically, in common. Its a simple enough question, but when you get up to Facebook scale, its a hard question to answer. This is the land of Hadoop and Hadoop has its own SQL-like query engine, Hive.
But unlike Hive which converts queries into MapReduce tasks saving intermediate results to disk, Presto has a query and execution engine which runs in memory and is pipelined through the network. Presto is implemented in Java for easy integration with other parts of Facebook that are also built in Java and compiles parts of queries down to bytecode, letting the JVM JIT compile to machine code to get the best out of the Java environment. Although it doesn’t need Hive, Presto does need a datasource for its queries and it includes a plugin for Hive, though it only uses the Hive metastore service, presumably to obtain structural information, and then accesses the data over HDFS.
The Facebook announcement says “Presto is 10x better than Hive/MapReduce in terms of CPU efficiency and latency for most queries at Facebook” and has been in use internally since Spring of this year with multiple deployments and one cluster scaled to a thousand nodes. A thousand users actively use it with 30000 queries and processing a petabyte a day. Thats a good work out for any big data offering.
There’s plenty missing from Presto; various joins and aggregations are restricted and there’s no way to write results back into tables – they go straight to the client. Those issues, plus improved performance, query accelerators, hot cached data subsets and a high performance HBase connector are all on the roadmap for Presto.
Presto is licensed under the Apache License 2.0 but does not appear to be heading to the foundation with active development taking place around Facebook’s GitHub repository.
If you were using MongoHQ‘s SSD backed MongoDB hosting, be prepared for them to be in touch as they’ve been at the sharp end of a security breach. But it’s not just direct users of MongoHQ’s services that should be concerned – users of services which make use of MongoHQ need to put on their worrying hat too. For example, MongoHQ hosted Buffer‘s databases and that has been cited as the cause of the social media connector’s security breach. Another company, cloud based continuous integration specialists CircleCI, has also been compromised and issued its own security advice (through a statuspage.io supplied status page which as I write, has fallen over). They probably won’t be the only ones either.
With an interconnected set of reliant services, the services at the bottom of the stack are often the ones which have the biggest target on them. To draw a parallel, if you want to make the Jenga stack fall over going for the bricks at the bottom is a good strategy. Hitting popular data-service providers in the cloud pays big for an attacker; an original target may come with many bonus victims and the ripple out of awareness of the compromise can provide a bigger window for the attack to fill its swag bag and make out through the window. Which is why, when you are looking at a service provider in the cloud, you need to make sure they have good defences, an effective monitoring system and a notification system which lets clients react quickly… and that’s not a “service status page which updates regularly”. It’s the same list you should have for your in-house and condensate* systems too.
* systems that use cloud technology but aren’t actually up in the cloud.