Forking brilliant – Node/IO.js and Docker/Rocket

spork

What’s up with Node: So there’s been a fork in Node.js land with the appearance of IO.js. A group of core contributing developers have lost patience with Joyent, the developmental home of Node.js, and have set out to accelerate the development of the Async-JavaScript server side platform. This is the world of open source where people can vote with their time and effort.

It’s easy to see both sides of the fork. Joyent want steady, stable development as they move towards a foundationed, open-sourced release. That progress has been guided from within Joyent, as is their right, but it has ended up with a situation where old code, like an unsupported version of the V8 JavaScript engine, is still actively used.

The forkers wanted to move things foward faster. Some had been involved in a light fork, Node-forward, which was designed to make the enhancements and then offer pull requests to the Node project. But that wasn’t working for them. According to one of the better know users of Node, the fork has been a relatively polite affair in itself and most of the noise surrounding it has come from outside the Node developer community.

Which makes it all the more likely that this fork is going to be a good thing for the generality of the Node community. It’ll push both sides to compete on quality and progress and with commitments to compatibility from the forkers, the door is still open for changes to be backported. Of course, it could all go off the rails. Right now, we get to look forward to January 13, when IO.js will release its first alpha.

What’s up with Docker: Over with Docker, another case of long time contributers starting their own project has popped up. This time it’s all about containers. Containers in Linux let you run multiple systems off the same kernel. The problem was that LXC (Linux Containers) were hard work to set up and manage. Enter Docker in 2013 with an easy to configure and deploy solution to that problem. This was great stuff, bringing containers to more than just the pioneers who’d been harnessing them quietly.

It quickly started catching on and CoreOS contributed to the development by dotCloud, the original Docker company which eventually became Docker Inc, because they saw a use for a de facto standard container within CoreOS making app depolyment easy.

Time passed and as Docker Inc needed to grow it started a process of adding more and management elements to their Docker offering. Some of this was undermining CoreOS as they just needed a well matured container format to integrate with their server Linux. They weren’t happy with where development in Docker was heading and that it was bringing a big technical and architectural debt with it.

So CoreOS started building Rocket. Rocket isn’t a fork though; CoreOS started from scratch releasing a prototype to Github and specs for review. They started from scratch because one of their problems is what the see as the monolithic approach in Docker which they feel is counter a good security model. So rather than Docker tools talking to a single process and letting that do all the work, Rocket tools do the work themselves.

The company already is committed to Docker integrated into CoreOS and isn’t dropping it but it seems it wants to get building the foundations of a more secure container platform now, not wait till there’s an incident which blows out confidence. Rocket will notionally be done when it provides enough to create, package and run containers, containers defined by a specification which the Rocket developers created first. They hope that the spec will evolve and be implemented by others, including Docker.

Thoughts: These are two interestingly different splits. Both are powered by the force that powers most open source – enlightened self-interest. Both have the capacity to enhance the ecosystem that they are splitting from. And both are being created by developers who are already vested and have contributed, and probably will still contribute, in the the platforms they are splitting from. These have the potential to be sporks, splendid forks, if all parties are able to take as much as they give. Six months from now, both splits should have full releases and positions should be soldifying. How these things look a year from now is going to be very illustrative for open source in general. Just let me pop it in my diary now…

Developer Catchup: Go libraries, easy Charts, Tumblr frameworks, Zsh secrets and secret Android compilers

developercatchupFacebook Go: When you develop a lot in Go, you make a lot of libraries and tools in Go. Facebook must be doing plenty because their new Facebook Go repository is full of code, much of it useful utilities for managing HTTP connections, mocking for tests, apps to test libraries like MySQL and MongoDB drivers and so on. Add to your resource list.

HTTP2 Go: While we’re talking Go, there’s a HTTP2 library in development by Google’s Brad Fitzpatrick. While you probably will never touch this directly – it’s designed to be hidden behind net/http – it’s good to know it’s being worked on and it tracking the drafts of the next generation HTTP.

Simpler Charts: When you pick up a powerful charting library does your head spin with the number of virtual buttons, knobs, sliders and dials youo can adjust? And do you get disheartened when no matter how much you twiddle, things just don’t look good enough? Metricsgraphics.js might be what you need. Underneath it uses the D3.js library, but to the user it presents a simple, opinionated API which is designed to need the least twiddling for a good chart. An examples page shows what you can do and an interactive demo lets you play.

Tumblr Services: Seems the folks at Tumblr have been wrestling with microservices, performance and reusability. To take that on they have built Colossus and blogged about it – it’s a Scala/NIO/Akka based framework designed to rapidly and concurrently to process many small client requests. It’s still a work in progress and the release is “pre-1.0″ but the code is up on Github. The most interesting part is probably that its coming out of Tumblr.

Zsh features: Zsh is a neat shell, but at first look not overly compelling. This article on Zsh features shows why Zsh is neat. It talks about smart directory completion on ‘cd’ commands, shorthand pathing, partial command searching, tab completion for the kill command, expanding environment variables, git and general help and more. I’ve switched over to zsh but there’s still so much to find and so much familiar from bash.

Covert Compilers for Android: Interesting article about Jack and Jill, two compilers for Android and a new intermediate byte code called Jayce. It appears Google are pushing out a new build chain which does away with dex and has Jack generating Dalvik bytecode directly. For libraries, they are compiled into Jayce bytecode by Jill and are consumed by Jack. These aren’t announced yet but it will represent a major change to the Android build system, sufficient to allow Google to start moving away from standard Java.

Making Catchup: Pi A+, Beagle X15, 68K prototyped and cheap Wifi hacking

makingcatchupRaspberry Pi Model A+ breaks cover: It seems that there’s been a leak on the Pi A+, the compacted version of the Pi less Ethernet, as its being reported. The cut-down Pi now has microSD and a 40 pin GPIO to match the B+. It still lacks the features that made the ODROID/W so interesting – LiPo battery support and real time clock on board. It does retain one thing from the Model A, the question of who’s it actually for.

BeagleBoard X15 leaks: What next for the BeagleBoard, the original board for the BeagleBoard project and predecessor of the most neat BeagleBone Black. Well, the answer appears to be the BeagleBoard X15. On the board, a dual core A15 CPU clocking at 1.5GHz, 2GB RAM, hardware video decoding, bristling with ports including SATA, two GB Ethernet ports and USB3.0. It looks splendid and we look forward to it landing in February 2015.

Going 68K in a week: We previously mentioned a project to build a 68K single board computer and Hackaday has an update in that it seems the developer put a prototype together in a week. It’s epic retro-computing work and highlghts the challenges that early system builders had in bringing early processor power to play.

Cheap Wifi: There’s a board, ESP8266, which offers a Wifi board for $5 or so. Madly cheap, terribly documented and a real challenge for hackers. Hackaday points us at a project on Instructables which shows how to use the board and an Arduino to pic up email and display details on an LCD screen. Useful.

Developer Catchup: FreeBSD at 21, Meteor at 1.0, tunnels, disklessness, neurons and 68008s

developercatchup

  • FreeBSD hits 21:FreeBSD is 21 today and you can see the original announcement preserved on the FreeBSD site and the most recent status report shows where current development was at the end of the third quarter. Looking forward to tier 1 support for more ARM platforms in FreeBSD 11.

  • Meteor hits 1.0: After a good long maturation with plenty of reworking and changes for the better – rather than those long betas which see no changes and never end – the rather splendid Meteor framework has hit version 1.0. It lets you build apps which are really smart about keeping all the users in sync with each other and builds on Node, JavaScript (on the server and browser) and other great open source foundations. And it’s open source itself. Having written apps in the past using it, I recommend it for the modern single screen web app. There’s a step by step tutorial on building an app too. If I had to pick a flaw its that it uses the curl/wget to shell anti-pattern – `curl https://install.meteor.com/ | sh – that has become rather cool but still boils down to running an unviewed, unfiltered script on your system. We need a fix for this, and we don’t need another package manager. A simple “download/scan/report&alert and offer to run” utility would do – want to be a popular person out there? Go write it!

  • Tunnelling out: I have to admit I only just found out about this one but ngrok is a useful service which lets you create a tunnel from the net to a single port on a machine without fiddling with firewalls and other stuff. Download an executable, run it with a port number and it’ll do the rest. And you can inspect the traffic easily for simple debugging.

  • Redis goes diskless: Replication usually involves disks and disks change performance and when you are all about the performance, thats critical. That’s why @antirez has been working on diskless replication for Redis. Read his introductory article to the motivation and implementation.

  • Neural networks in JavaScript: To be honest, I’ve never though about doing neural networks in the browser but it seems Juan Cazala has and his Synaptic library lets you experiment with them too.

And a little making

  • Different single board processors: Remember the 68000 series? The folks at Big Mess O Wires do and are working on building a single board computer around a 68008 (the un-power-house at the heart of the classic Sinclair QL). The aim is to get it running Linux.

Developer Catchup: POODLE, Tails, Docker, Redis and more

developercatchupPOODLE yips: In what was a glorious nail in the coffin of SSLv3, the POODLE vulnerability(PDF) made sure no one would trust SSLv3 again. The simple fix is to turn off SSLv3 where its used. The bug itself is bad in terms of cryptography, in that it gives an attacker a route to completely decode a stream that has been encrypted, but in practice its not as bad because the attacker has to be a man in the middle to get started. So, using SSLv3 from the open Wi-Fi at the fast food cafe, a bad thing. More worthwhile reading includes Imperial Violet’s explanation and Zmap.io’s guide to disabling SSLv3 in servers.

Chasing Tails: The Tails Live Linux distro, which tries its level best to be an bootable anonymous secure distro, has had an update to Tails 1.2. In the wake of the POODLE hole, it’s switched over to Tor Browser, dropping the IceWeasel, and that change also happens to close its POODLE vulnerability. There’s also Tor and kernel updates and various other minor changes. If you use it, just upgrade.

Docker tightens security: Docker 1.3 has landed, or more accurately Docker Engine 1.3. Highlight is digital signature verification of repositories of images, albeit as a tech preview of the feature. A production option also lets you set SELinux and AppArmor profiles from the command line. Other goodies include the ability to inject a process into a running Docker app so you can wake up a shell when you need to debug something, create and start commands for containers (on top of existing the all in one run command) and most usefully to me at least, shared directories on Mac OS X. The more interesting (as in get the popcorn) move from Docker is its partnering with Microsoft with a long term goal of making Docker run on Windows containers, not just on an a VM with Linux inside. Big challenge there as Microsoft have to basically get cgroups and more onto Windows Server.

Redis Clustered: The Redis key/value cache and store has pushed a release candidate for Redis 3.0.0 out. This is a rather important release as @antirez explains in his blog, it’s the first version with Cluster support, a long in-development feature, which has reached “minimum viable product” level and is stable enough for testing.

Quickies: 6to5 – turns JavaScript ES6 code into plain ES5 code which could be well useful. Asciicinema – lets you record and playback terminal sessions (and could be even better with audio – hint). On the to read list – Building Web Apps with Go – MIT licensed book based around Heroku use but lots of interesting content. And Whiteout Mail has gone open source – it’s all about accessible secure mail and has been in the works since 2013.

Making Catchup: 1Sheeld, Codebender, Odroid/W, Beans, Metawear and more

2014-10-10 18.15.26First of all a catchup on some of my making. I presented a short talk at Oggcamp 2014 on using the 1Sheeld with an Android phone to make experimenting with Arduino much simpler. The 1Sheeld sits on Arduino’s serial ports and using Bluetooth, talks to an Android phone app. The app is able to emulate a whole range of devices, like keypads and LEDs, and sensors, such as gyroscopes and barometers, and act as a proxy to web services like Twitter and Facebook. You just click on the things you need active and write code for the 1Sheeld library that talks to the board and onwards to the phone.

The demo involved using a Nexus 5’s gyroscope to roll a pixel around an Adafruit Neopixel shield and you can check out that code for that on my Rollapixel page on Codebender.cc. Want to see that working? Here’s a bit of video:

Big shout out to the Codebender.cc folk as they have the 1Sheeld libraries and examples all online as part of their splendid online IDE – it’s great to be able to cut code without spending time wrestling Java and the Arduino IDE into shape and even better to be able to quickly share it.

Other devices I’ve been playing with recently….

The ODroid/W Raspberry Pi-clone: Lovely bit of work by the HardKernel folk. It’s built to go into those smaller devices that the Pi doesn’t address, has LiPo battery support, real time clock and it’s well compact. That Broadcom cut off the supplies is more a worry for Pi owners as it looks like your locked into a Pi Foundation organised ecosystem. The HardKernel folk still have their tiny quad core ARMs like the 4core Odroid/U3 and octocore Exynos-based Odroid/XU3, one of which is mounted behind a monitor here (the smller one).

The Light Blue Bean: A small BLE/Arduino compatible… the software’s a bit hairy and Mac OS X/iOS centric at the moment but its a little board with a lot of potential. The ones I have will probably all end up being turned into iBeacons at some point.

The Metawear wearable: Andother BLE/ARM-core controller combo, this is really tiny, so much so I’m not brandishing a soldering iron near it till I get some really tiny tips. Waiting to see where the creators go with it as the world of wearables is, well, odd.

Other catchups:

Developer Catchup: Bashed, Qubes R2, Linux from Scratch, RethinkDB, Material Bootstrapped and… COBOL?

developercatchupBashed: So the Bash bug is out there and real. These quick notes are still valid. The point is that this hideous feature (really, exporting function definitions through environment variables) is horrid and leaky by design and it’s only this bug in how that feature is implemented thats bringing it to the fore. CGI scripting, Qmail, some SSH and DHCP services are all potentially vulnerable, so patch away but be prepared to patch again because the lid is off this can of worms. Safest end point is, most probably, that the functionality goes away, but thats unlikely and even if it did there’ll still be old bash installs out there. Least helpful response – the FSF statement which fails to apologise and then pats itself on the back that free software let the patches be shared and then rattles the donation tin. Funniest response – Brian J Fox, Bash creator, quoted in the NYT joking his first response was “Aha, my plan worked”.

Security in a Qube too: The Qubes OS developers have been working away steadily on their virtualisation-compartmented desktop operating system and now Joanna Rutkowska has announced Qubes OS Release 2. The OS is now described as “a powerful desktop OS” rather than a proof-of-concept, and to reinforce that, Casper Bowden, is joining the advisory board for Qubes to see if it can be brought to a wider world. If you’ve not met Qubes, imagine a desktop Linux where each app or group of apps are run in their own virtualised sandbox while the OS works to make it easy for the user to not be bothered by that. If you were looking for a “post-Snowden” OS, Qubes should be on your list – check the site for downloads, resources and white papers explaining whats in the OS.

Linux from Scratch: You may, “post-Snowden” want to go through every bit of code is in your running systems. One place to start there is Linux from Scratch which takes you through assembling your own Linux system (and automated or hardened versions) from component parts. It’s just been [updated to LFS version 7.6], along with updated to Beyond Linux From Scratch (BLFS) and systemd editions of LFS and BLFS.

RethinkDB 1.15: NoSQL… no come back… Cool NoSQL database RethinkDB just got updated to version 1.15 getting a huge set of geospatial functions to add to its already interesting suite of functions. There’s also server-side UUID generation and performance boosts through lazy deserialisation.

Material world: Some folks love Google’s Material look and feel. Well, now they can have some of that on thje web with Bootstrap Material Design, a Bootstrap theme what brings the stylings and gives a nice flat look to apps.

Finally: Via Adafruit, a picture of Grace Hopper teaching COBOL.