Node.js and Docker realigned

sporkIt’s not really a surprise, but after just over six months since the “forking” of both Node.js and Docker, the two different projects have ended up back in some sort of alignment. For Node.js, it was the reunification with io.js under the Node.js Foundation, which was officially launched under the Linux Foundation’s umbrella. The Node.js and io.js technical development is now driven by a technical committee and hopefully this will all work out well for all.

The Docker situation is a little more complex. There’s no big group hug like with Node.js. Instead, there’s an official middle ground, the Open Container Project. The announcement of a vendor-neutral (how can it be vendor neutral when it’s founded by vendors?) project to come up with containerisation technology basically sees Docker throw its specs and CoreOS throw its specs for containers into the same ring and see what comes out.

OCP says it’ll try and come up with a future spec independent of what’s layered on top of it, not associated with any project or vendor and be portable. So no, there won’t be a standard command set or management layer, there shouldn’t be any lock-ins and there probably will emerge as standard with a scope so small that it’ll end up as a tiny checkbox on a requirements list.

On the plus side, with that out of the way, there’s room for people to get innovating with the rest of the containerisation stack, which is where all the vendors are heading right now. That list is long too – Amazon, Cisco, EMC, Fujitsu, Google, IBM, Red Hat, VMWare and more. With the essential core in neutral hands, the game always moves on. As for the spec itself? Keep your eyes on the OCP’s Github Repository where they say they’ll have something by end of July.

Let’s hope that OCP keeps to its goals better than that other OCP, you know, the one that was building Delta City in the soon-to-be ruins of Old Detroit. That just didn’t work out well at all.

Developer Catchup: Rust 1.0 and Node reunification

Rust wakes updevelopercatchup

First up, Rust has reached version 1.0, though this is an announcement that was hardly unexpected. It has a lot to live up to given the Rust web site goes for such unloaded language as “blazingly fast, prevents nearly all segfaults, and guarantees thread safety”. The real test for Rust, at least for me, is how well Servo, Mozilla’s browser written in Rust and the application Rust was created with in mind. It seems this is the best possible test case, so…

There’s already a minimised ARM port which looks to bring Rust’s safety features to RTOS/embedded environments and I’ve come across some systems programmers who are interested in Rust, but not noted much momentum. Rust is living in the higher 50 (51-100) of the Tiobe index which is a very approximate guesstimate at momentum, but better than nothing. What this says to me is that despite arriving at 1.0 complete with packaging system and more, Rust is going to have a long journey ahead of it.

Node reunifies

Back in December, when Node.js and io.js forked, I expressed the hope that it would be a positive fork. Well, now that fork is coming to an end with the reunification, under the umbrella of the Node Foundation. Except there won’t be any merging of code and the io.js repository is being turned into the node.js repository.

Io.js folks will join the Node Foundations technical committee and, going forward, the next Node.js will be based on Io.js code. It’s well done to Io.js for taking action and making good practical and solid engineering steps that made it practically a no-brainer to take Node.js forward. We don’t seem to be completely done yet. The structure for future releases and development still needs nailing down.

So here’s looking forward to the next two releases of Node.js… the one which brings us all Io.js’s improvements like an up-to-date V8 JavaScript engines, and the one after which will probably come with more detail on how the desire for a faster development cycle and the need for stable long lived versions will be sorted out.

If you are looking to get your head around ES6, the next and arriving generation of JavaScript, try out Understanding ECMAScript 6, which is a CC-NC book via LeanPub – currently 30% complete and already full of useful details.

Snippets – JavaScript, Node, Git, HTTP2 and Regexps

snippets07In this Snippets, 6to5 becomes Babel, Node.js 0.12 on Pi, Git 2.3, HTTP2 explained and regular expressions from chained methods.

6to5 becomes Babel – As ES6, the next generation JavaScript, starts arriving in browsers the 6to5 transpiler, which converts ES6 code into current ES5 code so you can run your JavaScript apps on old and new browsers, has been looking to its future and changed its name to Babel to reflect its future plans. In a blog post the project explains that the transpiler’s codebase is not just useful for ES6 to ES5 conversion but to a whole range of IDEs and tools to come, so they’ve changed name and will begin opening up the API to let other projects plug into it. Smooth move.

Node.js 0.12 on a Pi – If you’re trying to build Node.js on your older Raspberry Pi, you may have problems. Not now – Thanks to Conor O’Neill who has built Node.js getting around a problem with identifying the version of ARM processor by… applying some patches from io.js. You can download the built version from his blog… which will save you many hours of build time. Comments suggest not rushing as it seems slower and you can already get a nightly release for ARM v6 for io.js.

Git 2.3 is out – The latest version of Git adds a push-to-deploy option so rather than log in to your server and git pull the latest version down, you can automatically have the server download new versions. Handy, but potential for huge blowback, use after considering the probable issues. There’s also a new trick where cloning can borrow assets from another local clone.

HTTP2 Explained – In HTTP2 Explained Daniel Stenberg is pulling together everything you need to know about HTTP2 in one living document. HTTP2 is going to be a big part of everyone’s web future, so it’s a good time to get reading.

Regexps from chains – The interesting idea from RegExpBuilder is why not use chained JavaScript functions to create regular expressions. Pro, wordier syntax explains more. Con, wordier syntax vs Regexp’s confusing compactness. Interesting idea though.

Docker 1.5 and Node.js Foundations

sporkBack in December we saw two community splits, one in the Docker community and one in the Node.js community. It’s time to look back at both those splits.

Docker 1.5 just landed with IPV6 support, read only containers, a stats API and CLI commands for streaming results and the ability to specify what Dockerfile to use when building. Good updates.

Now, the other bits – There’s also an “Open Image Spec” which isn’t so much a spec as a formal declaration of whats currently implemented. Thats a documentation +1 but open it isn’t . It seems to be a response to Rocket, which is a specification by design. It’s a start but there’s still the question of how transparent and inclusive the development of that spec will be.

The openness question is, I guess, supposed to be addressed by the announcement of a new organisational structure for developing Docker, which is a good thing if it comes together and works, but Docker is still Docker’s ball.

Node.js owner Joyent has announced a the establishment of a Node.js Foundation. Billed as bringing “Neutral and Open Governance” to Node.js. The “Foundation” will be established by Joyent, IBM, PayPal, Microsoft, Fidelity and the Linux Foundation. One assumes the Linux Foundation will be providing the umbrella organisation for the Node.js Foundation to work under as they have for other organisations.

But, as Bradley Kuhn points out this isn’t really a foundation as an charity that works for the public good; in the US that’s a 501(c)(3). The Linux Foundation and the proposed Node.js Foundation are not that kind of foundation – they are trade associations run for benefit of their members, 501(c)(6) organisations. That means you don’t “magically get a neutral home and open governance”.

We don’t have enough information to see how the new organisation will be run but the announcement is just an announcement and the details may contain more than trace amounts of devil. For the Node.js community, this is just a milestone in a long road and a parallel road marked io.js is running alongside it for now.

Snippets: Io.js, FreeBSD in the Cloud and 6502 Basic

Io.js 1.0 beta lands

Io.js is the spork of Node.js which is trying to put features the developers think have languished too long in development hell into a production codebase. We talked about it at the end of last year. Well, now there’s something tangible – a 1.0.0 in development release. What thats means is, top of the list, ES6 support with generators, templates and new string methods and more. Boom, huge improvement in JavaScript for developers living the Node thing. Io.js is all unstable at the moment but its already moving at a pace. There’s a further list of changes between Node.js 0.10.35 and io.js 1.0.0. One can only hope this helps leapfrog forward the entire Node development process.

FreeBSD sets sail

FreeBSD hasn’t been out in the clouds that much but that may be changing. DigitalOcean has announced FreeBSD on their cloud and thats a company who has till now only done Linux as their OS. Someone quickly posted the Dmesg output to show it was a real thing too. This could be a very special year for FreeBSD.

6502 Basic

This takes me back – the source code for Microsoft’s Basic for 6502 is now available. Written in MACRO-10 assembler so the PDP-10 could compile it, Paul Allen made macros to turn make the MACRO-10 engine handle 6502 assembly. That is some fine work. Now… where is that wait 6502,255 code… the associated article tells all and explains how it works.

PS: Regular visitors might notice the look of the site has changed. We’re experimenting with something a little fresher and we’d love to know what you think.

Forking brilliant – Node/IO.js and Docker/Rocket

spork

What’s up with Node: So there’s been a fork in Node.js land with the appearance of IO.js. A group of core contributing developers have lost patience with Joyent, the developmental home of Node.js, and have set out to accelerate the development of the Async-JavaScript server side platform. This is the world of open source where people can vote with their time and effort.

It’s easy to see both sides of the fork. Joyent want steady, stable development as they move towards a foundationed, open-sourced release. That progress has been guided from within Joyent, as is their right, but it has ended up with a situation where old code, like an unsupported version of the V8 JavaScript engine, is still actively used.

The forkers wanted to move things foward faster. Some had been involved in a light fork, Node-forward, which was designed to make the enhancements and then offer pull requests to the Node project. But that wasn’t working for them. According to one of the better know users of Node, the fork has been a relatively polite affair in itself and most of the noise surrounding it has come from outside the Node developer community.

Which makes it all the more likely that this fork is going to be a good thing for the generality of the Node community. It’ll push both sides to compete on quality and progress and with commitments to compatibility from the forkers, the door is still open for changes to be backported. Of course, it could all go off the rails. Right now, we get to look forward to January 13, when IO.js will release its first alpha.

What’s up with Docker: Over with Docker, another case of long time contributers starting their own project has popped up. This time it’s all about containers. Containers in Linux let you run multiple systems off the same kernel. The problem was that LXC (Linux Containers) were hard work to set up and manage. Enter Docker in 2013 with an easy to configure and deploy solution to that problem. This was great stuff, bringing containers to more than just the pioneers who’d been harnessing them quietly.

It quickly started catching on and CoreOS contributed to the development by dotCloud, the original Docker company which eventually became Docker Inc, because they saw a use for a de facto standard container within CoreOS making app depolyment easy.

Time passed and as Docker Inc needed to grow it started a process of adding more and management elements to their Docker offering. Some of this was undermining CoreOS as they just needed a well matured container format to integrate with their server Linux. They weren’t happy with where development in Docker was heading and that it was bringing a big technical and architectural debt with it.

So CoreOS started building Rocket. Rocket isn’t a fork though; CoreOS started from scratch releasing a prototype to Github and specs for review. They started from scratch because one of their problems is what the see as the monolithic approach in Docker which they feel is counter a good security model. So rather than Docker tools talking to a single process and letting that do all the work, Rocket tools do the work themselves.

The company already is committed to Docker integrated into CoreOS and isn’t dropping it but it seems it wants to get building the foundations of a more secure container platform now, not wait till there’s an incident which blows out confidence. Rocket will notionally be done when it provides enough to create, package and run containers, containers defined by a specification which the Rocket developers created first. They hope that the spec will evolve and be implemented by others, including Docker.

Thoughts: These are two interestingly different splits. Both are powered by the force that powers most open source – enlightened self-interest. Both have the capacity to enhance the ecosystem that they are splitting from. And both are being created by developers who are already vested and have contributed, and probably will still contribute, in the the platforms they are splitting from. These have the potential to be sporks, splendid forks, if all parties are able to take as much as they give. Six months from now, both splits should have full releases and positions should be soldifying. How these things look a year from now is going to be very illustrative for open source in general. Just let me pop it in my diary now…

Developer Catchup: Synchronous Node, Serviced Polyfills, Sparks Sparked, Tangrams Mapped and SHAaaaaaa!

developercatchupNode.js synchronously: Node.js is sweet if you can adapt to the asynchronous model of start thing, say what you want to do when its done, do everything else anyway. Good for web request handling but bleh for trying to emulate a shellscript. Turns out that in Node.js 0.12 (coming soon? anyone? Bueller?) we get synchronous child processes to now you can run that curl or find or whatever and just wait till its returned with its results. The folks at Strongloop have written about these synchronous child process methods and how they make writing command line utilities in Node easier. Check it out Noders.

Serviced Polyfills: Polyfills fill gaps in browser functionality and standards compliance. The older the browser, the more Polyfill you need to fill the gaps and the newer, the less. But it gets hard working out how much Polyfill you are going to need. Fear not, as Samuel Giles at FT Labs has an answer, “Polyfills as a Service“. Add a simple script tag pointing at a source from the polyfill.io content delivery network to your pages and whatever browser views your page, it gets the polyfill it needs. This is because the system sniffs the browser agent and works out the best set of polyfill based on that. Neat idea, potentially very handy – and you can run your own private version if you need to.

Spark sparks: Apache Spark just got a 1.1 release. Spark is Hadoop data processing engine which can run on YARN-based Hadoop clusters or in standalone mode. Spark 1.1 improves the performance (and they already say they are up to 100 times faster than Hadoop MapReduce) and has SQL layer enhancements. 1.1 also adds more statistical functions, can take steaming data fromAmazon Kinesis and pull data from Apache Flume and more. If your into clusters and data crunching and haven’t looked at Spark, you might want to look into it.

Tangram Mapping: Do you want to render cool 2D and 3D maps? Check out Tangram, a Mapping Library then as it is building out from a WebGL implementation to other OpenGL platforms to make oodly cool dynamic map renders. Very slick.

SHAaaaaaa!: We mention the Google sunsetting of SHA-1 the other week. If you were unsure why this was important, can we send you off to Why Google is Hurrying the Web to Kill SHA-1 which explains why it all and includes a brief history of collision attacks in the wild.

RHEL 6.5 and Docker, Ruby Fixes and Epic Node.js Bugfixing – Snippets

Snippets.png

  • RHEL 6.5 docks?: Red Hat Enterprise Linux 6.5 has been released and as is usual for the point releases of RHEL, has a number of enhancements like Precision Time Protocol support (for microsecond synchronisation accuracy), better network data for admins, GlusterFS integration for KVM and NVMe (PCI SSD) support. Mentioned in the announcement is Docker, the container deployment platform, but oddly there appears to be no mention of it in the technical notes or release notes. LXC (Linux Containers) is also still on the Technology Preview list. Still, it’s a Red Hat update and the timer is now running for the equivalent CentOS, Scientific Linux and Oracle Linux updates to appear.

  • Ruby security update: There’s updates for Ruby 1.9.3 and Ruby 2.0.0 which both address a security problem, namely a heap overflow when parsing floating point numbers. The same issue is also fixed in the new preview2 of Ruby 2.1.0, just released.

  • Epic Node.js Bugfixing: There’s nothing like a good detailed walkthrough of the debugging hell someone’s been through to offer a chance to say (a) good work, (b) glad I didn’t have to do that and (c) you found what where? In the spirit of this, Joyent have documented the Walmart Node.js Memory Leak from identification to final elimination, in text and three video talks. The leak itself (in file closing) was subtle and the hunt hard – The fix is in the latest Node.js releases.