Node-RED updated, Hadoop 2.3.0 out, NetBeans 8.0 RCs and Skrollr scrolls – Snippets

Node-RED updated: The most excellent graphical UI for connecting the Internet of Things (or just things in general), Node-RED has been updated to version 0.6. The announcement notes the process of separating the admin and server authentication to make deployment more robust has begun. Node-RED has nodes that accept HTTP connections and has a HTTP admin front end and previously these were all under one HTTP authentication mechanism – now the UI and nodes are more separate with the option to set a user/password for each. There’s some UI changes like a search filter for the palette of available nodes and easier flow importing by just dragging and dropping JSON onto the UI. In the node-red-nodes library, they’ve added Postgres, Amazon DynamoDB and Emoncms for more connections. There’s also fixes for the MQTT keepalive handing, an added socket timeout settings for TCP sockets and support for all 17 pins of WiringPi. More generally, there’s a range generating node now and the inject node can send empty payloads if needed. Finally, the MongoDB node now can send a user name and password – something I found I needed when writing this for MongoHQ.

Hadoop 2.3.0 released: In case you missed it, version 2.3.0 of the Apache Hadoop project got a release. The release notes list all the details. The short version is this is mostly about HDFS, the distributed file system and the changes include the ability to class the storage under HDFS so you can make tradeoffs between say spinning media, SSDs and memory, an ability to explicitly cache files or directories under HDFS (and local zero-copy reading from the cache) and the use of HDFS and YARN to simplify deploying MapReduce code. Hortonworks has a good writeup which also looks forward to Hadoop 2.4.0 with HDFS ACLs and rolling upgrades.

NetBeans 8 gets an RC: The NetBeans IDE has hit release candidate for 8.0. This is the version that will include JDK 8 support in the editor, Java SE Embedded and Java ME Embedded support, PrimeFaces code generators, AngularJS navigation and code completion, PHP 5.5 support and much more. There’s a summary in the announcement, a lot more detail in the New and Noteworthy wiki page and a pencilled in release date of mid-April.

Skrollr scrolls in: Recently spotted – Skrollr, a compact parallax scrolling and scrolling animation library for all your Webtml5.0 styled sites including the ability to “scale, skew and rotate the sh** out of any element”.

Facebook’s Conceal, Callback hell and a listening Pi – Snippets


Facebook’s Conceal revealed: Facebook have open sourced Conceal, a library for encrypting files on Android devices. The company uses the library for encrypting data that its apps store on SD cards. It uses pre-selected OpenSSL algorithms, picked for efficient memory management and speed, and gets the library down to 85KB by not trying to be a general purpose crypto kit. An interesting bit of pragmatism which means Facebook’s apps can happily encrypt on low-end Android devices, Conceal is available under a BSD licence with its source on GitHub.

Callback hell: Callbacks in Node.js can get pretty gnarly if you do everything with inline anonymous functions. This blog posting from Strongloop is a handy summary of some of the ways, from nesting, modularisation, async, promises and (soon to come to Node) ES6 generators. So callback, much techniques.

A Pi that listens: Meanwhile, a nice little Instructable covers converting an old bakelite Televox intercom into a voice controlled personal assistant by popping a Raspberry Pi inside.. and a sound board… and some software of course… It’ll probably be quite hard to find another fine bakelite intercom, but the rest of the projects a good starting point for assembling your own style of smart box…

Ruby 2.1 rolls out a performance push

Ruby_logoRuby 2.1 has been released on Christmas day and is billed as offering “speedup without severe incompatibilities”. The performance boost is down to a new method cache in the VM and a new generational garbage collection system. The old method cache was cleared eacg time a new method was defined but now only that cache damage has been tracked down and reduced and a future of a more optimal larger cache has been opened up.

The generational GC was added as part of an optimisation of Ruby’s object management which also adds hooks for allocation and deallocation – there’s a deeper look into these changes in Koichi Sasada’s presentation(PDF) from RubyConf 2013. The GNU Multiple Precision Arithmetic Library is also now used to accelerate Bignum calculations.

Beyond the runtime changes, there are some language changes such as a new method for Array – to_h – to turn key/value pairs into a hash, a new syntax for rational numbers so that 1//2 is the same as Rational(1,2) and the addition of an explaining cause to Exceptions, but they seem to be few and far between. A full list is in the NEWS file which also notes any compatibility issues. Source for Ruby 2.1 is available now for download and will most probably be appearing in various binary forms over the coming weeks.

The performance improvements, although useful, will not help the perception that, outside of the CRUD web applications where Ruby and Rails made their mark, the native form of the language benchmarks as one of the slower languages out there. They will at least though help Ruby when it comes to a developer deciding which tradeoffs they want to make.

Enlightenment 0.18 lit, FreeNAS 9.2 released and Java 8 brews – Snippets


  • Enlightenment Updated: The Enlightenment/EFL window manager/libraries/desktop has been updated to version 0.18.0, just a year after the long silence that led up to the release of Englightenment 0.17.0. A full list of bug fixes and improvements is in the NEWS file for the release – compositing has been merged into the core, ten crashing bugs have been fixed and modules for music control, bluetooth, DBus application menus and compositing control have been added. Downloads are available from the project’s site.

  • FreeNAS 9.2 goes final: The network storage platform FreeNAS has been updated too with over 260 fixes and a rebasing on FreeBSD 9.2. The developers say it should sport improved performance, especially with encryption if appropriate hardware is available, and be more able to cope with higher loads. The release notes offer further details – Items I like on the list are full registration of all services through multicast DNS using Avahi, which should make a server much easier to just drop into a network, and the addition of a REST API for FreeNAS for remote control.

  • Java 8’s final draft: The final draft for the Java 8 specification is now available and this is going to be the reference document to the changes being made in Java 8, due in March 2014. Lambda expressions, new date and time APIs and type annotations are referenced with pointers out to the various JSRs to where Java will be next year.

Firefox 26, Netflix’s Suro, Vagrants and Dockers and Websockets for all – Snippets


  • Firefox 26 digs in: Today we’ll see the release of Firefox 26, latest in the overly regular Firefox release cycle. From the (currently beta) release notes, we can see the big changes. All but the Flash plug-in are now click-to-play by default, Windows users can update their Firefox without having to write into the Firefox folders, the password manager can handle password fields generated by scripts and on Linux, if the installed gstreamer can handle h264, so can Firefox. A couple of fixes, some developer enhancements and thats about it. There’s also a Firefox for Android update due today. The release notes note some performance improvements, the same password manager enhancement and some fixes. The developer page for Firefox 26 covers changes of interest to developers in more detail. Firefox 26 will be turning up in updates and for download later today.

  • Netfix’s Suro goes open: From the people who brought you a cloud full of monkeys… Netflix’s latest open source release is Suro, an application monitoring system used by the video stream vendor to track the behaviour of their Amazon AWS deployed applications. Originally based on Apache Chukwa and adapted to fit Netflix’s demands, Suro pulls the company’s monitoring data from the various app clusters and pushes it to S3 (for Hadoop based analytics), to Apache Kafka (and on to Storm, Amazon ElasticSearch and Druid and to other event processors. There’s a lot more detail in the announcement including in production stats and how the pipeline is used to analyse errors.

  • Vagrant meets Docker: The latest update to Vagrant, version 1.4 has been announced and the big improvement in system that has traditionally been used to create automatically reproducible development environment is the addition of Docker support. The Docker provisioner can install Docker and then lets Vagrant cirtual machine pull and configure Docker containers within it. There’s also some enhancements to the scriptability of Vagrant itself, the ability to require a particular version of Vagrant and support for standalone file sync plugins.

  • websocketd: And finally, have you wanted to make a shell script or other app into a WebSocket server but lacked a library or access to the code to do it? Websocketd might be the answer as it turns anything with console I/O into a WebSocket server in a style rather reminiscent of CGI. Remember, most command line applications are not suitable for being exposed to the raw web, but the app could get you out of a hole when prototyping.

And, for reference, everything mentioned today is open source software.

IDEA 13, Java crypto, FreeBSD 10 beta 4, Rails update, Go 1.2 – Snippets


  • IntelliJ IDEA 13: Jetbrains has rolled out the latest version of its IntelliJ IDEA Java IDE. Version 13 gets a big refresh on the user interface with new light look and feel on Windows and Linux and toolbars hidden by default, better visualisation of errors and warnings with “lens mode”, comment/string only searching, built in SSH terminal, Java 8 support and a presentation mode for talking about coding. All those features, along with enhancements to Android, Gradle, Groovy, Scala and version control support are in the community version. The commercial Ultimate edition includes JSF 2.2 support, batch job code assistance, JAX-RS 2.0 annotation handling, more app server support, Spring context configuration and MVC view, improved JavaScript debugger, CSS extract refactorings, DART support and many enhancements to the database viewing and support. Full details are in the What’s New page for the new release. The open source Community version and a 30 day trial of the commercial version are both available to download.

  • Bouncy Castle Crypto update: Adding support for client side TLS 1.2 and DTLS 1.2, along with ECDH and ECDSA for the OpenPGP library and many other cryptography options, the splendidly named Legion of the Bouncy Castle have updated their Java Crypto libraries to version 1.5.0 – further details in the release notes.

  • FreeBSD 10 beta 4: The announcement of FreeBSD 10 Beta 4 has also seen the gentle push of the scheduled release date to 2 January 2014 with a December full of release candidates. The in-development release notes give an idea of what to expect as will this article from September.

  • Rails updates for security: There’s updated Rails with the release of 3.2.16 and 4.0.2 which address four or five CVE-numbered vulnerabilities. The problems fixed include various XSS vulnerabilities, a denial of service hole and fixes for a previous incomplete security fix.

  • Go 1.2 is go: Go 1.2 is now official with the announcement that, after 7 months, the latest modifications to the language, library and toolchain are now available. Full details in the release notes. Updates are expected to come on something closer to the 7 month cycle in future.

Docker for all Linux distros, DPorts and more for DragonFlyBSD and advice for coders – Snippets


  • Docker 0.7 unloading: With Docker 0.7, the Docker developers have made a big leap in Linux coverage. (If you are new to Docker, read the introduction to it I did for the Linux Foundation). Under the covers, Docker has used storage drivers to maintain images on disk, but up till now they’d needed a patched Linux kernel for that to work. A patch from Red Hat has changed that though and adds “DEVICEMAPPER”, a storage driver which used copy-on-write LVM snapshots and doesn’t need a patched kernel, to the list of storage drivers. The selection of the driver needed is done automagically and the resultant images are interchangable between different drivers so there’s no driver lock-in. That all means that Docker now runs on Fedora, RHEL, Ubuntu, Debian, SUSE, Arch, Gentoo and others. More drivers are coming too, for BtrFS, ZFS, Gluster and Ceph. Other additions, merged in the 0.6 cycle include offline image transfer, better port redirection, linkable containers and descriptive names for containers.

  • DragonFlyBSD updated: Version 3.6 of DragonFlyBSD – the now ten year old BSD project that sets out to give BSD native optimised clustering capabilities – has been released. The update standardises on Dports and pkg for installation tools, making around 20,000 packages available, and the process of building those 20,000 packages in parallel has allowed for the testing and near elimination of kernel contention with more cores scaling up the improvements made. There’s also i915 and KMS support, albeit experimental, and updated localisation. DragonFlyBSD is still using its HAMMER filesystem with work on HAMMER2 carrying on into DragonFlyBSD 3.7.

  • Coding Advice: Whether your learning or experienced, this article offers sage advice on how to approach coding. While we’re on the subject of advice, here’s some false things that programmers believe are true about geography, addresses, names and time.

FreeBSD 10.0beta3, SQL Injections, Rust stacks, InfluxDB and Circus renewal – Snippets


Catching up on Codescaling with some of the less mentioned things worth noting…

  • FreeBSD 10.0’s latest beta: It’s into the home/RC straight for FreeBSD 10 with the release of the third and hopefully last beta of the development cycle. The original schedule would have seen RC2 available around now, but with a focus on a quality release, there’s been a bit of slippage. Check out this FreeBSD News item from September for a feel of what’s going in. I’m looking forward to the switch to LLVM/Clang and seeing how the tickless kernel works out.
  • SQL injection attacks by Google?: Sucuri have come across an odd thing, Google doing SQL Injection attacks. Basically, Google’s bots crawl a site with links which would carry out an SQLi attack if followed… and then follow them like the bots they are which carries out the attack. Google may want to add at least some filtering to their bots in future, but its something to remind any application that ingests URLs from the web to follow them that URLs are not necessarily passive.
  • Rust reworks stack plan: For those interested in the implementation of languages, the Rust developers have decided to drop segmented stacks. Segmented stacks were stacks that were allocated small and expanded as needed. This would have allowed threads to have a much smaller footprint, but it didn’t quite work out that way. Followups on the thread discuss the cost of memory, both having it and accessing it, and alternative strategies.
  • InfluxDB: Databases for time series data are in and the latest open source addition to the game is InfluxDB which prides itself in no external dependencies. The Go-based MIT-licensed code has a JSONic HTTP API, an SQLish query language and a playground server to get running with. Its early days for InfluxDB, but its off to a good start.
  • Mozilla’s Circus Renewed: Mozilla’s Services project has announced a new version of its process/socket manager called Circus. Built using Python and ZeroMQ and recently redeveloped to be Python 3 compatible and fully asynchronous, the software lets an administrator manage processes and sockets on servers through a command line, Python API or web console. You can find the code on mozilla-services github.

Talend go Apache, Mozilla and Xiph, Oracle and Java and Virtualbox updates – Snippets


  • Talend go Apache: Talend, makers of integration, ETL and other data management products, have long been proponents of the GPL license for their products. I’ve asked them about this in the past and they’ve been robust in their reasoning about why the GPL is right for them. It appears though that that era has come to an end with an announcement that the company will be stepping towards more permissive licensing. They first plan to move to LGPL with version 5.4 of their products then to Apache in 2014. They’ve been steadily exposed to permissive licensing as they have built Talend ESB on Apache projects and when they went to release “Talend Open Studio for Big Data” they decided to go with Apache for better compatibility with the Hadoop ecosystem. That product, they say, is “arguably the most adopted product from Talend, ever” and that inspired a licensing rethink. An interesting change (and if you’ve not looked at Talend’s software, check it out… there’s some powerful integration mojo in there).
  • Mozilla’s new video hire: founder Monty Montgomery is off to Mozilla amicably leaving his current employer, Red Hat, for a chance to work at Mozilla with the other Xiph developers. Current work in progress is the Dalaa video codec which is setting out to be a free to implement and use, and technically superior alternative to h.265 and Google’s VP9. Mozilla is primary sponsor on the project and talking to Gigaom, Montgomery says progress on Dalaa is solid and there could be commercial products using it by the end of 2015. It looks like Mozilla are making sure that they aren’t caught again between a rock (h.264) and a hard place (VP8) in the future.
  • Oracle and Java fix time: It’s time for Oracle to drop its metric shedload of fixes for October. Short version, there’s a Java 7 update 45 (release notes) now available with security fixes for 51 vulnerabilities nearly all of which are remotely exploitable and with eleven scoring the full 10.0 on CVSS scores and nine scoring 9.3. Typically, most Java holes are around the sandbox, WebStart and applets, but two of the 10.0 critical holes affect servers too. Update your Java 7; if you are still on Java 6, you now have two problems.
  • VirtualBox 4.3: A new version of Oracle’s open source VirtualBox has arrived. The changes in version 4.3 are sufficient for it to be called a major update. The VT-x code and AMD-V code, the guts of the virtualisation, has been rewritten to fix bugs and improve performance. There’s a new instruction interpreter that can step in when hardware virtualisation isn’t able to handle something. New notifications, better keyboard short cuts and support for video capture have been added to the GUI while support for emulating USB touch devices, webcam passthrough and SCSI CD-ROM emulation have also been added. There is also a new virtual router mode which lets multiple VMs share one NAT service. And obviously, there’s oodles of bug fixes.

The Rubinus/Ruby Ruckus

rubinius_logo_black_on_whiteIt seems to be all going on with the developers of Rubinus, the LLVM JIT-powered Ruby implementation which recently hit version 2.0.0. First came the news that Engine Yard had ended their sponsorship for the project saying that “we no longer feel like the project needs any help from us to accelerate” – the ending of sponsorship will, they say, let them invest more in other emerging projects.

With that announcement made, Rubinus lead Brian Shirai said “I have been working to simplify and focus the project”; funding changes do tend to allow projects to step back and look at their goals. In this case, the results of that work turned up in an announcement of Rubinus X.

Rubinus X is billed as an experiment in modernising Ruby, which the project’s home page declares as no longer relevant to modern computing. The plan appears to be a major reworking of Ruby – concurrency with Promises and non-blocking I/O, persistent data structures, object composition, code loading as an API, consistency through composing operations from system primitives, immutable strings and a more explicit OO model. There is also a clean up of the language planned with a single encoding used within running programs, the removal of position-significant arguments in API calls, a defined numeric tower and the purging of Perl-inspired features and global variables.

Shirai declared “Ruby is a dying language” which triggered much discussion over on sites like Hacker News about whether Ruby was dying or not and whether Node.js was the new Ruby/Rails but very little conversation on the merits of the Rubinus X plan. More details on how the Rubinus X project will develop will be disclosed in the coming days via the mailing list (which has already got 570 signups). Rubinus X does seem to have some ambitious goals and with the development taking place outside the mainstream of Ruby, it shouldn’t have an impact on Ruby development at least until it has proven itself. What affect it will have on Rubinus development is unclear though; with no one paid to develop it, resources will be tight. Definitely one to keep an eye on.