Node.js and Docker realigned

sporkIt’s not really a surprise, but after just over six months since the “forking” of both Node.js and Docker, the two different projects have ended up back in some sort of alignment. For Node.js, it was the reunification with io.js under the Node.js Foundation, which was officially launched under the Linux Foundation’s umbrella. The Node.js and io.js technical development is now driven by a technical committee and hopefully this will all work out well for all.

The Docker situation is a little more complex. There’s no big group hug like with Node.js. Instead, there’s an official middle ground, the Open Container Project. The announcement of a vendor-neutral (how can it be vendor neutral when it’s founded by vendors?) project to come up with containerisation technology basically sees Docker throw its specs and CoreOS throw its specs for containers into the same ring and see what comes out.

OCP says it’ll try and come up with a future spec independent of what’s layered on top of it, not associated with any project or vendor and be portable. So no, there won’t be a standard command set or management layer, there shouldn’t be any lock-ins and there probably will emerge as standard with a scope so small that it’ll end up as a tiny checkbox on a requirements list.

On the plus side, with that out of the way, there’s room for people to get innovating with the rest of the containerisation stack, which is where all the vendors are heading right now. That list is long too – Amazon, Cisco, EMC, Fujitsu, Google, IBM, Red Hat, VMWare and more. With the essential core in neutral hands, the game always moves on. As for the spec itself? Keep your eyes on the OCP’s Github Repository where they say they’ll have something by end of July.

Let’s hope that OCP keeps to its goals better than that other OCP, you know, the one that was building Delta City in the soon-to-be ruins of Old Detroit. That just didn’t work out well at all.

Docker 1.5 and Node.js Foundations

sporkBack in December we saw two community splits, one in the Docker community and one in the Node.js community. It’s time to look back at both those splits.

Docker 1.5 just landed with IPV6 support, read only containers, a stats API and CLI commands for streaming results and the ability to specify what Dockerfile to use when building. Good updates.

Now, the other bits – There’s also an “Open Image Spec” which isn’t so much a spec as a formal declaration of whats currently implemented. Thats a documentation +1 but open it isn’t . It seems to be a response to Rocket, which is a specification by design. It’s a start but there’s still the question of how transparent and inclusive the development of that spec will be.

The openness question is, I guess, supposed to be addressed by the announcement of a new organisational structure for developing Docker, which is a good thing if it comes together and works, but Docker is still Docker’s ball.

Node.js owner Joyent has announced a the establishment of a Node.js Foundation. Billed as bringing “Neutral and Open Governance” to Node.js. The “Foundation” will be established by Joyent, IBM, PayPal, Microsoft, Fidelity and the Linux Foundation. One assumes the Linux Foundation will be providing the umbrella organisation for the Node.js Foundation to work under as they have for other organisations.

But, as Bradley Kuhn points out this isn’t really a foundation as an charity that works for the public good; in the US that’s a 501(c)(3). The Linux Foundation and the proposed Node.js Foundation are not that kind of foundation – they are trade associations run for benefit of their members, 501(c)(6) organisations. That means you don’t “magically get a neutral home and open governance”.

We don’t have enough information to see how the new organisation will be run but the announcement is just an announcement and the details may contain more than trace amounts of devil. For the Node.js community, this is just a milestone in a long road and a parallel road marked io.js is running alongside it for now.

Forking brilliant – Node/IO.js and Docker/Rocket

spork

What’s up with Node: So there’s been a fork in Node.js land with the appearance of IO.js. A group of core contributing developers have lost patience with Joyent, the developmental home of Node.js, and have set out to accelerate the development of the Async-JavaScript server side platform. This is the world of open source where people can vote with their time and effort.

It’s easy to see both sides of the fork. Joyent want steady, stable development as they move towards a foundationed, open-sourced release. That progress has been guided from within Joyent, as is their right, but it has ended up with a situation where old code, like an unsupported version of the V8 JavaScript engine, is still actively used.

The forkers wanted to move things foward faster. Some had been involved in a light fork, Node-forward, which was designed to make the enhancements and then offer pull requests to the Node project. But that wasn’t working for them. According to one of the better know users of Node, the fork has been a relatively polite affair in itself and most of the noise surrounding it has come from outside the Node developer community.

Which makes it all the more likely that this fork is going to be a good thing for the generality of the Node community. It’ll push both sides to compete on quality and progress and with commitments to compatibility from the forkers, the door is still open for changes to be backported. Of course, it could all go off the rails. Right now, we get to look forward to January 13, when IO.js will release its first alpha.

What’s up with Docker: Over with Docker, another case of long time contributers starting their own project has popped up. This time it’s all about containers. Containers in Linux let you run multiple systems off the same kernel. The problem was that LXC (Linux Containers) were hard work to set up and manage. Enter Docker in 2013 with an easy to configure and deploy solution to that problem. This was great stuff, bringing containers to more than just the pioneers who’d been harnessing them quietly.

It quickly started catching on and CoreOS contributed to the development by dotCloud, the original Docker company which eventually became Docker Inc, because they saw a use for a de facto standard container within CoreOS making app depolyment easy.

Time passed and as Docker Inc needed to grow it started a process of adding more and management elements to their Docker offering. Some of this was undermining CoreOS as they just needed a well matured container format to integrate with their server Linux. They weren’t happy with where development in Docker was heading and that it was bringing a big technical and architectural debt with it.

So CoreOS started building Rocket. Rocket isn’t a fork though; CoreOS started from scratch releasing a prototype to Github and specs for review. They started from scratch because one of their problems is what the see as the monolithic approach in Docker which they feel is counter a good security model. So rather than Docker tools talking to a single process and letting that do all the work, Rocket tools do the work themselves.

The company already is committed to Docker integrated into CoreOS and isn’t dropping it but it seems it wants to get building the foundations of a more secure container platform now, not wait till there’s an incident which blows out confidence. Rocket will notionally be done when it provides enough to create, package and run containers, containers defined by a specification which the Rocket developers created first. They hope that the spec will evolve and be implemented by others, including Docker.

Thoughts: These are two interestingly different splits. Both are powered by the force that powers most open source – enlightened self-interest. Both have the capacity to enhance the ecosystem that they are splitting from. And both are being created by developers who are already vested and have contributed, and probably will still contribute, in the the platforms they are splitting from. These have the potential to be sporks, splendid forks, if all parties are able to take as much as they give. Six months from now, both splits should have full releases and positions should be soldifying. How these things look a year from now is going to be very illustrative for open source in general. Just let me pop it in my diary now…

Developer Catchup: POODLE, Tails, Docker, Redis and more

developercatchupPOODLE yips: In what was a glorious nail in the coffin of SSLv3, the POODLE vulnerability(PDF) made sure no one would trust SSLv3 again. The simple fix is to turn off SSLv3 where its used. The bug itself is bad in terms of cryptography, in that it gives an attacker a route to completely decode a stream that has been encrypted, but in practice its not as bad because the attacker has to be a man in the middle to get started. So, using SSLv3 from the open Wi-Fi at the fast food cafe, a bad thing. More worthwhile reading includes Imperial Violet’s explanation and Zmap.io’s guide to disabling SSLv3 in servers.

Chasing Tails: The Tails Live Linux distro, which tries its level best to be an bootable anonymous secure distro, has had an update to Tails 1.2. In the wake of the POODLE hole, it’s switched over to Tor Browser, dropping the IceWeasel, and that change also happens to close its POODLE vulnerability. There’s also Tor and kernel updates and various other minor changes. If you use it, just upgrade.

Docker tightens security: Docker 1.3 has landed, or more accurately Docker Engine 1.3. Highlight is digital signature verification of repositories of images, albeit as a tech preview of the feature. A production option also lets you set SELinux and AppArmor profiles from the command line. Other goodies include the ability to inject a process into a running Docker app so you can wake up a shell when you need to debug something, create and start commands for containers (on top of existing the all in one run command) and most usefully to me at least, shared directories on Mac OS X. The more interesting (as in get the popcorn) move from Docker is its partnering with Microsoft with a long term goal of making Docker run on Windows containers, not just on an a VM with Linux inside. Big challenge there as Microsoft have to basically get cgroups and more onto Windows Server.

Redis Clustered: The Redis key/value cache and store has pushed a release candidate for Redis 3.0.0 out. This is a rather important release as @antirez explains in his blog, it’s the first version with Cluster support, a long in-development feature, which has reached “minimum viable product” level and is stable enough for testing.

Quickies: 6to5 – turns JavaScript ES6 code into plain ES5 code which could be well useful. Asciicinema – lets you record and playback terminal sessions (and could be even better with audio – hint). On the to read list – Building Web Apps with Go – MIT licensed book based around Heroku use but lots of interesting content. And Whiteout Mail has gone open source – it’s all about accessible secure mail and has been in the works since 2013.

Developer Catchup – Easier docker on Mac, versioning made hard, old school Unix on the Pi and new school packaging for Meteor

developercatchupLet’s go fly a Kitematic: There’s plenty of command line tools for Docker and command line driven ways to run it on Mac OS X. The latter’s harder because you need to run a VM and load it with an image and… well there’s boot2docker to help but… Enter Kitematic which takes the previous tools and rolls them with a neat UI and some extra neat tricks to make it a lot easier to start playing with the idea. Among those tricks are things like automatically creating an [appname].dev DNS entry so you can quickly connect to your new apps when they are up and running. If you like to run GUI tools alongside terminal sessions on your Mac, you might want to give Kitematic a go.

Versioning wars: Yes, people are arguing on the Internet and this time its over versioning. Some years back, Semantic Versioning appeared and set out some rules for when to bump the major, minor and patch numbers in a version number to embue it with some meaning. While this works for libraries where the consumer is often another program, it works less well with code to be consumed by people. The argument starts on Underscore’s Github where breaking changes as fixes were causing friction over what the version should actually be. This spilled out onto Hacker News which lead to the suggestion that “Semantic Versioning Isn’t” and back to HN where people continued to disagree. But it did get Fear-Driver Versioning (ferver) and the idea of romantic versioning a moment in the sun. From what I see, SemVer works but it does require discipline and transparency from the developers and the consumers of that code. Still… Developers eh?… because those bike sheds won’t pick what colour they are going to be by themselves.

Old School on a Pi: Want to run old school stylee? We’re talking Unix V5 here. Matt Hoskins has updated his 2005 presentation on how to do this (think PDP emulation and similar) so you can now do it all on a Raspberry Pi. Read on to learn the true old ways of Unix.

Meteor updates packaging: Meteor’s a supercool Node.js framework for building apps and its steadily approaching version 1.0. Version 0.9.0 has just been released and along side various additions, there’s now a new Isobuild packaging system because in the Twenty-Tens, no framework or language is complete without it’s own packaging system. IsoBuild does not make ISO images, by the way. The Iso is short for “isomorphic”. The devs explain that Isomorphic JavaScript is, in their model, JavaScript that runs on client or server and that there wasn’t a build system that worked with both. Isobuild lets you do that with Isopacks that deliver the right code for client or server automatically. They’ve already got 1800 Isopacks and have their eye on mobile apps too in their planning with something going on for Cordova. Watch out for it. 1.0 is not far away.

Catchup: New Pi?, MeArm & JavaScript, Docker goes big, harder Dart, Breach

CodescalingCatchup
Hot Pi: First up, a hot rumour via Hackaday is that there’s a Raspberry B+ with 4 USB ports, no composite, a 40 pin GPIO port and other changes… is this real? We shall see, but the Pi has been aching for an update and even if this isn’t it, there’s a gap to be filled.

MeARM by JavaScript: Two of my favourite projects, the low cost MeARM robot ARM and the JavaScript running Espruino board have been brought together for a fun little remote control project for the ARM.

Big Docks: Google’s Kubernates project has drawn in a whole load of contributors. Kubernates is a cluster manager for containers. Now, [according to the announcement], IBM, Red Hat, Microsoft, Docker (and CoreOS, Mesosphere and SaltStack) have all signed up to develop various aspects of the platform. Could be interesting… it’s already interesting that Docker has at once become the centre of and just a component of bigger effort all at the same time.

Dart hardens: Dart, Google’s replacement for JavaScript and poor relative of Go in terms of adoption, is now ECMA-408. Whether getting the ECMA badge will push Dart forward or will the cross-browser-centic JavaScript community continue not bothering to look… we shall see, though Dart seems to lack the features that engender excitement and adoption so…

HTML Breached: Ever wanted an all JavaScript browser? The Breach developers did and are creating just such a thing.Node.js eventloop and wrapping the Chromium App API so that the top side of the browser is all HTML/JS. There might be some interesting code to come from it but its still early days.

Developer Catchup: Docker 1.1.0, Rust 0.11.0, Python 2.7.8 and Dropbox Go Libraries

developercatchupDocker 1.1.0: The first post 1.0 update for Docker is in and Docker 1.1.0 now has a .dockerignore mechanism for ignoring file changes, containers that now pause when a commit it happening (rather than messing them up), container log tailing, the ability to feed tar archives to docker build and other changes which should make life a bit easier and more predictable.

Rust 0.11.0: The latest Rust announcement for version 0.11.0 is about smoothing out the type system to allow for dynamically sized types and refactoring the standard libraries to allow for that. It means that language embedded elements like ~ and @ have become library types called Box and Gc that should make the language easier to understand. It all brings Rust 1.0 closer – by the end of the year is the current hope.

Python Updates: The start of July saw a Python update for various security issues – Python 2.7.8 updated the OpenSSL library, fixed mimetypes and UNC paths regressions and blocked an arbitrary code execution hole in CGIHTTPServer. There were also a number of core and library fixes detailed in the release notes. There was no corresponding update for Python 3.x though the CGIHTTPServer issue is scheduled to be fixed in Python 3.4.2 according to the in progress changelog.

Dropbox Go: Dropbox, a big Python user, has also been working with Go and has been moving its infrastructure to Go based code. In the process, they’ve written a lot of libraries to support that work and now they are open sourcing those Go libraries with a 3 clause BSD license. There’s code for caching, an improved error interface, a programmatic SQL statement builder, a memcahce client library, connection management and a space efficient hash library. And they will be doing it the right way – they’ve committed to using the public versions of the libraries inhouse (rather than maintain their own branch). You’ll find the documentation for all the libraries over at Godoc.org.