The State of Node and Relational Databases

15 Apr 2013 | By Alex Young | Comments | Tags databases node modules sql

Recently I started work on a Node project that was built using Sequelize with MySQL. It was chosen to ease the transition from an earlier version written with Ruby on Rails. The original’s ActiveRecord models mapped quite closely to their Sequelize equivalents, which got things started smoothly enough.

Although Sequelize had some API quirks that didn’t feel very idiomatic alongside other Node code, the developers have hungrily accepted pull requests and it’s emerging as a reasonable ORM solution. However, like many others in the Node community I feel uncomfortable with ORM.

Why? Well, some of us have learned how to use relational databases correctly. Joining an established project that uses ORM only to find there’s no referential integrity or sensible indexes is to be expected these days, as programmers have moved away from caring about databases to application-level schemas. I’ve had my head down in MongoDB/Mongoose and Redis code for the last few years, but relational databases aren’t going away any time soon so either programmers need to get the hang of them or we need better database modules.

This all prompted me to look at alternative solutions to relational databases in Node. First, I broke down the problem into separate areas:

  • Driver: The module that manages database connections, sends queries, and responds with data
  • Abstraction layer: Provide tools for escaping queries to avoid SQL injection attacks, and wrap multiple drivers so it’s easy to port applications between MySQL/PostgreSQL/SQLite
  • Validator: Validates data against a schema prior to sending it to the database. Aids with the generation of human-readable error messages
  • Query generator: Generates SQL queries based on a more JavaScript-programmer-friendly API
  • Schema management: Keep schema up-to-date when fields are added or removed

Some projects won’t need to support all of these areas – you can mix and match them as needed. I prefer to create simple veneer-like “model” classes that wrap more low-level database operations. This works well in a web application where it can be make sense to decouple the HTTP layer from the database.

Database Driver

The mysql and pg modules are actively maintained, and are usually required by “abstraction layer” modules and ORM solutions.

A note about configuration: when it comes to connecting to the database, I strongly prefer modules that support connection URIs. It makes it a lot easier to deploy web applications to services like Heroku, because a single environmental variable can be set that contains the connection details for your production database.

Abstraction Layer

This level sits above the driver layer, and should offer lightweight JavaScript sugar. There are many examples of this, but a good one is transactions. Transactions are particularly useful in JavaScript because they can help create APIs that are less dependent on heavily nested callbacks. For example, it makes sense to model transactions as an EventEmitter descendent that allows operations to be pushed to an internal stack.

The author of the pg module, Brian Carlson, who occasionally stops by the #dailyjs IRC channel on Freenode, recently mentioned his new relational project that aims to provide a saner approach to ORM in Node. This module feels more like an abstraction layer API, but it’s gunning to be a formidable new ORM solution.

There are some popular libraries that tackle the abstraction layer, including any-db and massive.


I usually find myself dealing with errors in web forms, so anything that makes error handling easier is an advantage. Validation and schemas are closely related, which is why ORM libraries usually combine them.

It’s possible to treat them separately, and in the JavaScript community we have solutions based on or inspired by JSON Schema. The jayschema module by Nate Silva is one such project. It’s really aimed at validating JSON, but it could be used to validate JavaScript objects spat out by a database driver.

Validator has some simple tools for validating data types, but it also has optional Express middleware that makes it easy to drop into a web application. Another similar project is conform by Oleg Korobenko.

Query Generator

The sql module by Brian Carlson is an SQL builder – it has a chainable API that turns JavaScript into SQL:


He’s using this to build the previously mentioned relational module as well.

Schema Management

Sequelize has an API for managing database migrations. It can migrate to a given version and back, and it can also “sync” a model’s schema to the database (creating the table if it doesn’t exist).

There are also dedicated migration modules, like db-migrate by Jeff Kunkle.


The Node community has created a rich set of modules for working with relational databases, and although there’s a strong anti-ORM sentiment interesting projects like relational are appearing.

Although these modules don’t address my concerns about the way in which ORM gets used with apathy towards best practices, it’s promising to see lower-level modules that can be used as building blocks for more application-specific solutions.

All of this has come at a time when relational database projects are adapting, changing, and even growing in popularity despite the recent attention the NoSQL movement has been given. PostgreSQL is going from strength to strength, and Heroku provides it as default. MariaDB is a drop-in replacement for MySQL that has a non-blocking Node module. SQLite is probably technically growing in usage as it backs Core Data in iCloud applications – Android developers also use SQLite.

Let other readers know how you deal with SQL-backed Node projects in the comments!

ZestJS, backbone-pageable, Marionette and Chaplin

12 Apr 2013 | By Alex Young | Comments | Tags frameworks libraries backbone.js mvc


Øyvind Smestad sent in ZestJS (GitHub: zestjs, License: Apache 2.0), which offers an interesting take on client and server-side modularity:

ZestJS provides client and server rendering for static and dynamic HTML widgets (Render Components) written as AMD modules providing low-level application modularity and portability.

It treats widgets as AMD components, with separate files for markup, JavaScript, and CSS. It can then render the results on either the server or client. The server-side renderer, zest-server, is a small Node project that is capable of rendering views and serving static files. It also handles routing, essentially mapping HTTP routes to what Zest calls “Render Components”.

Some aspects of ZestJS remind me of Backbone.js – the data loading can be performed on the initial page load, but remote APIs are easy to integrate as well. It also uses r.js for building and optimizing single page web apps, which is similar to the workflow of many Backbone.js developers.

ZestJS was created by Guy Bedford, and there are client and server quick start guides.


backbone-pageable (GitHub: wyuenho / backbone-pageable, License: MIT) by Jimmy Yuen Ho Wong is a drop-in replacement for Backbone.Collection that adds support for server and client-side pagination. It includes options for sorting, infinite paging, and caching:

It comes with reasonable defaults and works well with existing server APIs. Besides being really good at pagination and sorting, it is also really smart as syncing changes across pages while paginating on the client-side. It is also extremely lightweight - only 4k minified and gzipped

It supports query parameters, so you can easily set up your pagination links with variables to meet your server’s requirements (or perhaps to allow multiple pagination controls on a page).

var Book = Backbone.Model.extend({});

var Books = Backbone.PageableCollection.extend({
  model: Book,
  url: '',

  state: {
    firstPage: 0,
    currentPage: 2,
    totalRecords: 200

  queryParams: {
    currentPage: 'current_page',
    pageSize: 'page_size'

The project comes with some serious test coverage, including a test suite geared up for Zepto which is a nice touch.

The author is currently working on releasing a new version that adds Backbone 1.0 support. It should be 1.2.2, so keep a lookout for that if you’re already on Backbone 1.0.

Comparison of Marionette and Chaplin

Mathias Schäfer, who is one of the creators of Chaplin.js, sent in a comparison of Marionette and Chaplin. Both libraries attempt to address various limitations of Backbone.js – Chaplin.js adds better support for CoffeeScript class hierarchies, stricter memory management, lazy-loading modules, and publish/subscribe for cross-module communication.

The comparison is detailed and reveals some of the thinking that went into Chaplin.js in the first place:

Compared to Marionette, Chaplin acts more like a framework. It’s more opinionated and has stronger conventions in several areas. It took ideas from server-side MVC frameworks like Ruby on Rails which follow the convention over configuration principle. The goal of Chaplin is to provide well-proven guidelines and a convenient developing environment.

The post already has interesting comments – Mathias seems to be following up questions with some well thought out responses.

Google, Twitter, and AngularJS

11 Apr 2013 | By Alex Young | Comments | Tags angularjs mvc angularfeeds


My behemoth of a Backbone.js tutorial series has run its course, so I wanted to follow it up with some posts about AngularJS. One thing that intrigues me about AngularJS is the emerging relationship between Google and Twitter. Or between prominent Google and Twitter developers. I don’t think there’s a overarching plan at the management level to create an open source partnership, just a set of coincidences as projects have aligned along certain vectors that are pushing front-end web development forward.

Most of what I’m going to talk about here has been packaged up into Yeoman, which has some Google employees behind it (Paul Irish, Addy Osmani) and includes technology from Twitter (Bower). The rest of this post will break down the emerging next generation Google/Twitter open source stack.

Data Binding, Views, Routes

AngularJS from Google is the obvious choice for data binding. It’s definitely growing in popularity, and Yeoman includes a generator for it.


Bootstrap is also supported by Yeoman, and offers a fine set of extensible UI widgets. Although seeing vanilla Bootstrap sites has become massively clichéd, it doesn’t take much effort to customise it.

Build/Preview: Grunt

In my Backbone.js tutorial, we used Node to create a small web server with RequireJS for local development. I included some details on Grunt purely because I use GNU Make for my own projects, so I wanted to look at Grunt more seriously. The developers behind Yeoman have selected Grunt as their build/preview tool.

Test: Karma, Mocha

Yeoman bundles in Mocha, and Karma can be used to script browsers. It’s used to test AngularJS, so that’s where the connection comes from, and there’s karma-mocha – an adapter for Karma to use Mocha.

Package Management: Bower

Bower, from Twitter, is a lightweight package manager. I’ve talked a bit about it on DailyJS before, and I try to include links to Bower package names when featuring front-end modules. Yeoman comes with Bower.

I covered Backbone.sync a fair bit in the Backbone.js tutorial series because it’s so flexible I was able to do some cool things with it like sync data with Google’s JavaScript APIs. This is interesting when you consider that Backbone is configured to talk to a Rails-style REST server out of the box.

So, what about this brave new world of Google/Twitter open source projects? What data syncing solutions are there? To my knowledge there isn’t anything generic, yet, but there is the Yeoman Express Stack:

A proof of concept end-to-end stack for development using Yeoman 0.9.6, Express and AngularJS. Note: This experimental branch is providing for testing purposes only (at present) and does not provide any guarantees.

This is a small project that builds on Yeoman and AngularJS to sync data with a Node/Express server. This came from a weekend hack project that involved Addy Osmani, who has contributed to many of the projects mentioned here.

There is also Angular Seed, which persists data to a Node/Express server using Socket.IO.

Also significant is a proposal, Entity-Driven Tooling from the Yeoman developers about adding CRUD generator support:

tl;dr: what if one command could scaffold out the CRUD models/views for your client and server side code, with baked in offline support. Would this help you? Would this solve a pain point of yours? Are there better ways to do this than what’s described below?

Although I’ve often wished for something like this, the post seems to imply a localStorage-based syncing solution would be developed. This would allow the browser-based portion of the project to behave like a client, making data available in localStorage for offline use.

However, syncing data can be difficult, so providing a generator that can do this would be more involved than Backbone.sync. Perhaps basing it around CouchDB’s eventual consistency model would work? Locally available records would have a version parameter which would be used to safely sync concurrently with the server. This would leave conflict resolution up to the developer of the application – some servers might store the latest version of a record, and others might throw an error, perhaps causing the client to display a conflict resolution dialog.

There may be a localStorage sync project the Yeoman developers have in mind.

I can’t help feeling that there’s more to Google open source than Closure Library. If you’ve used Android since Jelly Bean, Chrome OS, or Google Plus, then you know Google’s designers have been pushing things far beyond where the company was just a few years ago. Although Closure Library is a formidable set of tools, the widgets don’t fit in well with the Yeoman generation’s open source projects, and I’m eager to see what a next generation version of Bootstrap would look like.

But for now I’m looking at AngularJS, Grunt, Bootstrap, Bower, Mocha for my next tutorial series. I’ll have to find something interesting to sync data with, because I enjoyed figuring out the Google Tasks API.

Node Roundup: 0.8.23, indev, compressjs

10 Apr 2013 | By Alex Young | Comments | Tags node modules build compression
You can send in your Node projects for review through our contact form.

Node 0.8.23

In case you haven’t switched to 0.10 yet, Node 0.8.23 was released yesterday. This version adds bug fixes for the http, tls, child_process, and crypto modules.


indev (GitHub: azer / indev, License: BSD, npm: indev) by Azer Koçulu is a lightweight alternative to Makfiles. It supports “Devfiles” which can be written in either CoffeeScript or JavaScript, and includes shortcuts for lots of shell commands through ShellJS.

The inclusion of ShellJS makes it feel closer to make than Grunt, so if Grunt isn’t quite what you want then indev might be what you’re looking for.


compressjs (GitHub: cscott / compressjs, License: GPLv2, npm: compressjs) by C. Scott Ananian features several compression algorithms, implemented in pure JavaScript. It can run in browsers, and includes bzip2, LZP3, a modified LZJB, PPM-D, and an implementation of Dynamic Markov Compression.

The readme includes benchmarks for each algorithm, and a script is included so you can use it to compress things on the command-line.

jQuery Roundup: 2.0 Beta 3, betterToggle, Cavendish.js

09 Apr 2013 | By Alex Young | Comments | Tags jquery plugins css slideshows
Note: You can send your plugins and articles in for review through our contact form.

jQuery 2.0 Beta 3

jQuery 2.0 Beta 3 is out:

… we really need your help in finding and fixing any bugs that may be hiding in the nooks and crannies of jQuery 2.0. We want to get all the problems ironed out before this version ships, and the only way to do that is to find out whether it runs with your code.

This release introduces Node compatibility, so you can now load it with require(). It also makes it work in Windows 8 Store apps.


betterToggle (GitHub: kanakiyajay / betterToggle, License: GPLv2) by Jay Kanakiya is a plugin for toggling elements with CSS3 transforms. As an added bonus it allows multiple elements to be toggled.

Usage is similar to .toggle: $(selector).betterToggle(), and the project’s homepage has plenty of demos.


Cavendish.js (GitHub: michaek / cavendish.js, License: MIT) by Michael Hellein is a slide manager plugin aimed at front-end developers well-versed in CSS. It has a plugin-based API that allows it to support different styles for displaying and navigating slides:

var cavendish = $('.cavendish').cavendish('cavendish');

The bundled plugins include a simple player that pauses on hover, a pager, previous and next arrows, and a parallax scrolling effect. The API also exposes the events used, so you can add listeners to see when Cavendish has been initialised and after a slide has been transitioned.

LungoJS, Math.js, Collage

08 Apr 2013 | By Alex Young | Comments | Tags mobile maths frameworks libraries ui



LungoJS (GitHub: TapQuo / Lungo.js, License: GPLv3) from TapQuo is a framework for HTML5 apps that aims to be cross-device. It supports mobile, desktop, and TV devices. The JavaScript API has support for working with the DOM, localStorage, caching, navigation routing, remote services, and views.

There’s a designer-focused tutorial that explains how to create an application with Lungo, and a Google group (which currently requires permission to join).


Math.js (GitHub: josdejong / mathjs, License: Apache 2.0, npm: mathjs, bower: mathjs) by Jos de Jong is a maths library for client-side JavaScript and Node. It supports complex numbers, units, strings, arrays, and matrices, built-in functions and constants, as well as mathematical expression parsing.

It has no dependencies and is compatible with the built-in Math library. One feature I particularly like is the expression parser:

var parser = math.parser();
parser.eval('1.2 / (2.3 + 0.7)'); // 0.4
parser.eval('a = 5.08 cm');
parser.eval('a in inch');         // 2 inch
parser.eval('sin(45 deg) ^ 2');   // 0.5

This opens up some interesting possibilities for storing mathematical expressions in databases then safely evaluating them later on.

The project includes unit tests, and detailed documentation can be found in the readme file.


Collage (GitHub: ozanturgut / collage, License: Apache 2.0) by Ozan Turgut is a framework for creating interactive collages. It can knit together remote APIs then present media in a two-dimensional space.

This example demonstrates some of the APIs that are supported as standard:

var collage = Collage.create(document.getElementById('PopcornCollage'));
collage.load('popcorn media', {
  flickr: [{ tags: 'popcorn'}],
  googleNews: ['popcorn'],
  twitter: [{ query: 'popcorn' }]
  collage.start('popcorn media');

AngularCollection, datamock.js, store.js

05 Apr 2013 | By Alex Young | Comments | Tags mvc angularjs testing databases localStorage


AngularCollection (GitHub: tomkuk / angular-collection, License: MIT, bower: angular-collection) by Tomasz Kuklis is a collection module for AngularJS. It allows objects to be added, removed, updated, and fetched at a specific index. It also has a sort method and last.

It comes with a Grunt build script and some unit tests.


datamock.js (GitHub: marksteve / datamock.js, License: MIT) by Mark Steve Samson is a small library for generating sample data for mockups. Data attributes can be used to bind mocked data, like this: <p data-mock="lorem">Lorem ipsum here...</p>.

It includes some other value types, like names and emails. The author has included a bookmarklet task in the build script so you can generate a bookmarklet that fills a page in with test sample data.


store.js (GitHub: nbubna / store, License: MIT) by Nathan Bubna is a friendlier API for localStorage and sessionStorage. Basic usage is store(key, data), but it has other functions like store.setAll, store.getAll, and support for namespaces.

The sessionStorage API is accessed through store.session, for example: store.session(key, 'value'). The project includes a Grunt build script and PhantomJS-powered tests.

Backbone.js Tutorial: jQuery Plugins and Moving Tasks

04 Apr 2013 | By Alex Young | Comments | Tags backbone.js mvc node backgoog bootstrap jquery


Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit 705bcb4
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone
cd dailyjs-backbone-tutorial
git reset --hard 705bcb4

Using Backbone with jQuery Plugins

Although Backbone doesn’t need to be used with jQuery specifically, a lot of people use it with jQuery (and RequireJS) to get access to the diverse plugins made by the jQuery community. In this tutorial I’ll explain how to use jQuery plugins with Backbone projects, and how to find ones that will work well.

The example I’ve used is integrating a drag-and-drop “sortable” plugin to allow tasks to be reordered.

HTML5 Sortable

The plugin I’ve used for drag-and-drop is the HTML5 Sortable Plugin by Ali Farhadi. The reason I used this particular plugin is it has a simple event-based API that allows the plugin to be unloaded and sort events to be captured and responded to. It just needs a container element and the child elements that need to be sorted. The unordered list of tasks in this project directly translates to the expected markup.

Sometimes it’s easier to just write out data attributes to elements rather than trying to create relationships between the DOM nodes used by plugins and models. HTML5 Sortable emits a 'sortupdate' event when a node has been dragged and dropped, and it’ll pass the relevant element to the listener callback. From this we need to figure out which model has changed, then translate that into something Google’s API can understand.

Loading Plugins with RequireJS

In an earlier tutorial I demonstrated how to load non-AMD libraries using RequireJS. If you want a recap, just check out app/js/main.js and look at the shim property in the RequireJS configuration:

  baseUrl: 'js',

  paths: {
    text: 'lib/text'

  shim: {
    'lib/underscore-min': {
      exports: '_'
    'lib/backbone': {
      deps: ['lib/underscore-min']
    , exports: 'Backbone'
    'app': {
      deps: ['lib/underscore-min', 'lib/backbone', 'lib/jquery.sortable']

The 'app' property expresses a dependency between the main Backbone application file and lib/jquery.sortable, which means /lib/jquery.sortable.js will get automatically loaded (or compiled in by r.js when creating a production build of the app).

Google Tasks Ordering API

It would be too easy if HTML5 Sortable’s API was a one-to-one match with the Google Task’s ordering API. Google’s API has a specific method for moving tasks, and it’s based around moving one task to occupy the position of another one:

gapi.client.tasks.tasks.move({ tasklist: listId, task: id, previous: previousId });

Moving a task to the top of the list is handled by passing null for previous.

Next I’ll explain how to create some simple interface elements for the draggable handle, and then we’ll look at how to persist moved tasks by translating Google’s API into Backbone model and collection code.

Implementation: Views and Templates

I added a little handle by using a Bootstrap icon and an anchor element in app/js/templates/tasks/task.html:

<a href="#" class="handle pull-right"><i class="icon-move"></i></a>

Next I added the code that maps between the Backbone view and the jQuery HTML5 Sortable plugin to app/js/views/tasks/index.js:

makeSortable: function() {
  var $el = this.$el.find('#task-list');
  if (this.collection.length) {
    $el.sortable({ handle: '.handle' }).bind('sortupdate', _.bind(this.saveTaskOrder, this));

saveTaskOrder: function(e, o) {
  var id = $(o.item).find('.check-task').data('taskId')
    , previous = $(o.item).prev()
    , previousId = previous.length ? $(previous).find('.check-task').data('taskId') : null
    , request

  this.collection.move(id, previousId, this.model);

The makeSortable method makes an element that appears within TasksIndexView “sortable” – that is, HTML Sortable has been wrapped around it. The plugin’s 'sortupdate' method is then bound to saveTaskOrder.

The saveTaskOrder method gets the current task’s ID by looking at the checkbox, because I’d already added a data attribute to that element in the template. This ID is then passed to the collection with the previous task’s ID. In this case, the previous task is the one adjacent to it, which Google’s API needs to figure out how to move the task.

The collection property in this view is a Tasks property, so let’s take a look at how to implement the move method that causes the changes to be persisted.

Implementation: Models and Collections

Open app/js/collections/tasks.js and add a new method called move:

move: function(id, previousId, list) {
  var model = this.get(id)
    , toModel = this.get(previousId)
    , index = this.indexOf(toModel) + 1

  this.remove(model, { silent: true });
  this.add(model, { at: index, silent: true });

  // Persist the change
  list.moveTask({ task: id, previous: previousId });

This method just exists to trigger remove and add calls on the collection so cause the objects to be reshuffled internally. It then calls moveTask on the TaskList model (in app/js/models/tasklist.js):

moveTask: function(options) {
  options['tasklist'] = this.get('id');
  var request = gapi.client.tasks.tasks.move(options);

  Backbone.gapiRequest(request, 'update', this, options);

The gapiRequest method forms the basis for the custom Backbone.sync method used in this project, which I’ve talked about in previous tutorials. I wasn’t able to figure out how to make Backbone.sync cope with moving items in a way that made sense given how gapi.client.tasks.tasks.move works, but I was able to at least reuse some of the syncing functionality by creating a request and calling the “standard” request handler.


When you can’t find a suitable Backbone plugin for something and want to use a jQuery plugin, my advice is to look for plugins that have event-based APIs and can be cleanly unloaded. That will make them easy to hook into your Backbone views.

The full source for this tutorial can be found in alexyoung / dailyjs-backbone-tutorial, commit e9edfa3.

Node Roundup: 0.11.0, Dependo, Mashape OAuth, node-windows

03 Apr 2013 | By Alex Young | Comments | Tags node modules dependencies oauth authentication windows
You can send in your Node projects for review through our contact form.

0.11.0, 0.10.2

Node 0.11.0 has been released, which is the latest unstable branch of Node. Node 0.10.12 was also released, which includes some fixes for the stream module, an update for the internal uv library, and various other fixes for cryptographic modules and child_process.



Dependo (GitHub: auchenberg / dependo, License: MIT, npm: dependo) by Kenneth Auchenberg is a visualisation tool for generating force directed graphs of JavaScript dependencies. It can interpret CommonJS or AMD dependencies, and uses MaDGe to generate the raw dependency graph. D3.js is used for drawing the results.

Mashape OAuth

Mashape OAuth (GitHub: Mashape / mashape-oauth, License: MIT, npm: mashape-oauth) by Nijiko Yonskai is a set of modules for OAuth and OAuth2. It has been designed to work with lots of variations of OAuth implementations, and includes some lengthy Mocha unit tests.

The authors have also written a document called The OAuth Bible that explains the main concepts behind each supported OAuth variation, which is useful because the OAuth terminology isn’t exactly easy to get to grips with.


node-windows (GitHub: coreybutler / node-windows, License: MIT/BSD, npm: node-windows) by Corey Butler is a module designed to help write long-running Windows services with Node. It supports event logging and process management without requiring Visual Studio or the .NET Framework.

Using native node modules on Windows can suck. Most native modules are not distributed in a binary format. Instead, these modules rely on npm to build the project, utilizing node-gyp. This means developers need to have Visual Studio (and potentially other software) installed on the system, just to install a native module. This is portable, but painful… mostly because Visual Studio itself is over 2GB.

node-windows does not use native modules. There are some binary/exe utilities, but everything needed to run more complex tasks is packaged and distributed in a readily usable format. So, no need for Visual Studio… at least not for this module.

jQuery Roundup: Sidr, Huey, Backbone.Advice

02 Apr 2013 | By Alex Young | Comments | Tags jquery plugins backbone.js graphics components
Note: You can send your plugins and articles in for review through our contact form.



Sidr (GitHub: artberri / sidr, License: MIT) by Alberto Varela creates menus that look like the sidebars found in recent iOS apps. It can cope with multiple menus on a page, and can load content remotely. It’s also responsive, so it should work well in mobile projects.

The author has written up documentation complete with demos, and has included a Grunt build script. It seems like the exact sort of UI component that the next great web-based RSS reader might use…


Huey (GitHub: michaelrhodes / huey, License: MIT) by Michael Rhodes will find the dominant colour of an image and return it as an RGB array. This is all performed client-side, and doesn’t even depend on jQuery. It could be used to create the kind of effect seen in iTunes, where the background colour changes to suit the selected album art.


Backbone.Advice (GitHub: rhysbrettbowen / Backbone.Advice, License: MIT) by Rhys Brett-Bowen is a Backbone plugin based on Angus Croll’s advice pattern. It basically adds functional mixins to Backbone objects, and can be wrapped like this:


Rhys sent in a whole bunch of other Backbone-related projects, including Backbone.ComponentView and Backbone.ModelRegistry. Backbone.ComponentView is based on goog.ui.component from Closure Library, and also works with Backbone.Advice.

Five Minute Guide to Streams2

01 Apr 2013 | By Alex Young | Comments | Tags streams streams2 node 5min

Node 0.10 is the latest stable branch of Node. It’s the branch you should be using for Real Work™. The most significant API changes can be found in the stream module. This is a quick guide to streams2 to get you up to speed.

The Base Classes

There are now five base classes for creating your own streams: Readable, Writable, Duplex, Transform, and PassThrough. These base classes inherit from EventEmitter so you can attach listeners and emit events as you normally would. It’s perfectly acceptable to emit custom events – this might make sense, for example, if you’re writing a streaming parser. The parser could emit events like 'headers' to indicate the headers have been parsed, perhaps for a CSV file.

To make your own Readable stream class, inherit from stream.Readable and implement the _read(size) method. The size argument is “advisory” – a lot of Readable implementations can safely ignore it. Once your _read method has collected data from an underlying I/O source, it can send it by calling this.push(chunk) – internally data will be placed into a queue so “clients” of your class can deal with it when they’re ready.

The Writable class should also be inherited from, but this time a _write(chunk, encoding, callback) method should be implemented. Once you’ve written data to the underlying I/O source, callback can be called, passing an error if required.

The Duplex class is like a Readable and Writable stream in one – it allows data sources that transmit and receive data to be modelled. This makes sense when you think about it – TCP network sockets transmit and receive data. To implement a Duplex stream, inherit from stream.Duplex and implement both the _read and _write methods.

The Transform class is useful for implementing parsers, like the CSV example I mentioned earlier. In general, streams that change data in some way should be implemented using stream.Transform. Although Transform sounds a bit like a Duplex stream, this time you’ll need to implement a _transform(chunk, encoding, callback) method. I’ve noticed several projects in the wild that use Duplex streams with a stubbed _read method, and I wondered if these would be better served by using a Transform class instead.

Finally, the PassThrough stream inherits from Transform to do… nothing. It relays the input to the output. That makes it ideal for sitting inside a pipe chain to spy on streams, and people have been using this to write tests or instrument streams in some way.


Pipes must follow this pattern: readable.pipe(writable). As Duplex and Transform streams can both read and write, they can be placed in either position in the chain. For example, I’ve been using process.stdin.pipe(csvParser).pipe(process.stdout) where csvParser is a Transform stream.


The general pattern for inheriting from the base classes is as follows:

  1. Create a constructor function that calls the base class using, options)
  2. Correctly inherit from the base class using Object.create or util.inherits
  3. Implement the required underscored method, whether it’s _read, _write, or _transform

Here’s a quick stream.Writable example:

var stream = require('stream');

GreenStream.prototype = Object.create(stream.Writable.prototype, {
  constructor: { value: GreenStream }

function GreenStream(options) {, options);

GreenStream.prototype._write = function(chunk, encoding, callback) {
  process.stdout.write('\u001b[32m' + chunk + '\u001b[39m');

process.stdin.pipe(new GreenStream());

Forwards Compatibility

If you want to use streams2 with Node 0.8 projects, then readable-stream provides access to the newer APIs in an npm-installable module. Since the stream core module is implemented in JavaScript, then it makes sense that the newer API can be used in Node 0.8.

Some open source module authors are including readable-stream as a dependency and then conditionally loading it:

var PassThrough = require('stream').PassThrough;

if (!PassThrough) {
  PassThrough = require('readable-stream/passthrough');

This example is taken from until-stream.

Streams2 in the Wild

There are some interesting open source projects that use the new streaming API that I’ve been collecting on GitHub. multiparser by Jesse Tane is a stream.Writable HTML form parser. until-stream by Evan Oxfeld will pause a stream when a certain signature is reached.

Hiccup by naomik uses the new streams API to simulate sporadic throughput, and the same author has also released bun which can help combine pipes into composable units, and Burro which can package objects into length-prefixed JSON byte streams. Conrad Pankoff used Burro to write Pillion, which is an RPC system for object streams.

There are also less esoteric modules, like csv-streamify which is a CSV parser.

Edge.js, Bespoke.js, Barcode39

29 Mar 2013 | By Alex Young | Comments | Tags Canvas node Microsoft presentations libraries


Edge.js (GitHub: tjanczuk / edge, License: Apache 2, npm: edge) by Tomasz Janczuk is an in-process interoperability layer between .NET and Node. This allows things like CPU-bound operations to be processed by .NET, or Node to access the Win32 APIs through C#.

The .NET code can be executed asynchronously and may be passed as a multiline comment or a string. A basic example looks like this:

var edge = require('edge');

var helloWorld = edge.func('async (input) => { return ".NET Welcomes " + input.ToString(); }');

helloWorld('JavaScript', function(err, result) {
  if (err) throw err;

Running this example would display “.NET welcomes JavaScript”.

Other CLR languages can be supported, should you be interested in playing with F# for example.

This project requires Windows, and needs Visual Studio 2012, Python 2.7, and npm-gyp to build.



Bespoke.js (GitHub: markdalgleish / bespoke.js, License: MIT, bower: bespoke.js) by Mark Dalgleish is a small but slick presentation library. It works with keyboard and touch events, and is intended to be used with CSS transitions.

It’s built using ECMAScript 5, so you’ll want to run your presentations on a compatible browser.

Creating presentations involves wrapping HTML slide content in <section> containers. Horizontal and vertical deck styles are supported, and Mark has documented the CSS classes in the project’s readme so you can hook into the provided JavaScript and styles.


Barcode39 (GitHub: erik5388 / barcode-39.js, License: MIT) by Erik Zettersten is a Code 39 implementation – it basically generates barcodes that almost all barcode readers can cope with. It can output data URIs and supports Canvas for drawing.

The JavaScript API is new Barcode39(elementId, type, delimeter), but it will also look for an element with the default ID of barcode and read the element’s content for the barcode’s source text.

Backbone.js Tutorial: Updates for 1.0, Clear Complete

28 Mar 2013 | By Alex Young | Comments | Tags backbone.js mvc node backgoog bootstrap


Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit 711c9f6
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone
cd dailyjs-backbone-tutorial
git reset --hard 711c9f6

Updating to Backbone 1.0

I updated bTask to work with Backbone 1.0, which required two small changes. The first was a change to the behaviour of callbacks in Backbone.sync – the internal call to the success callback now only needs one argument, which is the response data. I think I’ve mentioned that on DailyJS before, but you shouldn’t need to worry about this in your own Backbone projects unless you’ve written a custom Backbone.sync implementation.

The second change was the collection add events were firing when the views called fetch. I fixed this by passing reset: true to the fetch options. Details on this have been included in Backbone’s documentation under “Upgrading to 1.0”:

If you want to smartly update the contents of a Collection, adding new models, removing missing ones, and merging those already present, you now call set (previously named “update”), a similar operation to calling set on a Model. This is now the default when you call fetch on a collection. To get the old behavior, pass {reset: true}.

Adding “Clear Complete”

When a task in Google Tasks is marked as done, it will appear with strike-through and hang around in the list until it is cleared or deleted. Most Google Tasks clients will have a button that says “Clear Complete”, so I added one to bTask.

I added a method called clear to the Tasks collection which calls the .clear method from the Google Tasks API (rather than going through Backbone.sync):

define(['models/task'], function(Task) {
  var Tasks = Backbone.Collection.extend({
    model: Task,
    url: 'tasks',

    clear: function(tasklist, options) {
      var success = options.success || function() {}
        , request
        , self = this

      options.success = function() {
        self.remove(self.filter(function(task) {
          return task.get('status') === 'completed';


      request = gapi.client.tasks.tasks.clear({ tasklist: tasklist });
      Backbone.gapiRequest(request, 'update', this, options);

  return Tasks;

I also added a button (using Bootstrap’s built-in icons) to app/js/templates/app.html, and added an event to AppView (in app/js/views/app.js):

var AppView = Backbone.View.extend({
  // ...
  events: {
    'click .clear-complete': 'clearComplete'

  // ...
  clearComplete: function() {
    var list = bTask.views.activeListMenuItem.model;
    bTask.collections.tasks.clear(list.get('id'), { success: function() {
      // Show some kind of user feedback
    return false;

I had to change app/js/views/lists/menuitem.js to set the current collection in the open method to make this work.


Because I’ve been reviewing Backbone’s evolution as it progressed to 1.0 for DailyJS, updating this project wasn’t too much effort. In general the 1.0 release is backwards compatible, so you should definitely consider upgrading your own projects. Also, now bTask has ‘Clear Complete’, I feel like it does enough of the standard Google Tasks features for me to actually use it regularly.

Remember that you can try it out for yourself at

The full source for this tutorial can be found in alexyoung / dailyjs-backbone-tutorial, commit 705bcb4.

Node Roundup: wish, Vow, shell-jobs

27 Mar 2013 | By Alex Young | Comments | Tags node modules testing promises async time daemons unix
You can send in your Node projects for review through our contact form.


wish (GitHub: EvanBurchard / wish, License: MIT, npm: wish) by Evan Burchard is an assertion module designed to raise meaningful, human-readable errors. When assertions fail, it parses the original source to generate a useful error message, which means the standard comparison operators can be used.

For example, if wish(a === 5) failed an error like this would be displayed:

  Expected "a" to be equal(===) to "5".

If assert(a === 5) had been used instead, AssertionError: false == true would have been raised. A fairer comparison would be assert.equal, which would produce AssertionError: 4 == 5, but it’s interesting that wish is able to introspect the variable name and add that to the error.


Vow (GitHub: dfilatov / jspromise, License: MIT/GPL, npm: vow) by Filatov Dmitry is a Promises/A+ implementation. Promises can be created, fulfilled, and rejected – you should be able to get the hang of it if you’ve used libraries with then methods elsewhere, but there are some differences to Promises/A which feels like it actually simplifies some of the potentially messier parts of the original CommonJS specification.

Here’s an example of the Vow API:

var promise1 = Vow.promise(),
    promise2 = Vow.promise();

Vow.all([promise1, promise2, 3])
  .then(function(value) {
    // value is [1, 2, 3]


The author has written some pretty solid looking tests, and benchmarks are included as well. The project performs favorably when compared to other popular promise libraries:

 mean timeops/sec


I like seeing daemons made in Node, and Azer Koçulu recently sent in a cron-inspired daemon called shell-jobs (GitHub: azer / shell-jobs, License: MIT, npm: shell-jobs). It uses .jobs files that are intended to be human readable. All you need to do is write a shell command followed by a # => and then a time:

cowsay "Hello" > /tmp/jobs.log # => 2 minutes

The shell-jobs script will then parse this file and output the following:

  jobs Starting "cowsay "Hello" > /tmp/jobs.log" [2 minutes] +2ms

After two minutes has passed the job will be executed:

  exec 1. Running cowsay "Hello" > /tmp/jobs.log. +0ms

jQuery Roundup: Individual Memberships, Bootstrap Tag Autocomplete, CDNJS

26 Mar 2013 | By Alex Young | Comments | Tags jquery plugins bootstrap cdn
Note: You can send your plugins and articles in for review through our contact form.

jQuery Foundation Individual Memberships

The jQuery Foundation has allowed corporations to become members for a year now, and they’ve just opened up the programme to individuals. If you’re interested in effectively sponsoring the jQuery Foundation, the page has details on pricing and rewards.

Each pricing tier includes a gift, starting with a t-shirt, and the top $400 tier also includes “access to individual members only benefits at jQuery Foundation events”. I’m not sure what these individual benefits are, but where I come from $400 gets you a lot of benefits for your buck, so consider me cautiously intrigued.

Bootstrap Tag Autocomplete

When you’re writing Bootstrap-based projects, including any old jQuery plugin sometimes requires a bit of extra work to tailor the required markup and CSS to fit in with Bootstrap’s defaults. That means Bootstrap plugins are in demand from developers and designers. Nada Aldahleh recently sent in Bootstrap Tag Autocomplete (GitHub: Sandglaz / bootstrap-tagautocomplete, License: Apache 2.0), which is a Bootstrap and jQuery UI component for creating Twitter-like autocomplete interfaces.

It’s built on Bootstrap’s Typeahead library, and includes its own caret position library for getting and setting the caret position.

QUnit tests have been included, and the project’s website includes documentation and code samples.


Ryan Kirkman sent in CDNJS, which is an open source CDN. They’re looking for feedback on which libraries should be included – there are currently 325 listed. The code that runs the project is available on GitHub at cdnjs / cdnjs, and it’s based on Node.

Scripts can be added to the CDN by forking the GitHub project and following the instructions in the readme file. The general rule of thumb is that projects must have over 100 watchers on GitHub, but as long as sufficient popularity can be demonstrated the authors will consider including a new project. That means the list of libraries on is useful for finding high quality scripts.

Dependent Types for JavaScript, radioactive.js, Minimail

25 Mar 2013 | By Alex Young | Comments | Tags computer-science education email apps

Dependent Types for JavaScript

Dependent Types for JavaScript published on Lambda the Ultimate is about Dependent JavaScript (DJS), a statically-typed dialect of JavaScript by Ravi Chugh, David Herman, and Ranjit Jhala (pdf):

DJS supports the particularly challenging features such as run-time type-tests, higher-order functions, extensible objects, prototype inheritance, and arrays through a combination of nested refinement types, strong updates to the heap, and heap unrolling to precisely track prototype hierarchies

The paper has a summary of related work that will be interesting to those of you who are experimenting with dialects of JavaScript with different type models.


radioactive.js (GitHub: reinpk / radioactive, License: MIT) by Peter Reinhardt is a library for modelling nuclear physics. Its intended use is for creating interactive demonstrations of radioactive decay:

One of the biggest problems I’ve encountered in writing about nuclear reactors is that people don’t understand radioactive decay. This is a huge problem because it means that 99% of the population is totally unqualified to decide anything about nuclear energy.

Suppose I have 1 kg of Cesium-134, with a half-life of 30 years. And 1 kg of Uranium-238, with a half-life of 4.5 billion years. I’m going to give you one of the blocks, and you have to sleep with it tonight like a teddy bear. Which one do you want?

If you guessed Cesium-134, you’re dead.

So, if you often find yourself presented with various radioactive isotopes and don’t want to die, Peter’s library may be of use to you. Or else you’re creating presentations or simulations using D3.js that you want to have some level of accuracy.


Minimail (GitHub: emailbox / minimail_mobileapp, License: BSD3) by Nicholas Reed is a mobile and server-side project to create a developer-friendly email client:

It is at the alpha stage, which means it kinda, sorta runs on Android and iOS, and is usable as a replacement mobile client (with changes synced to your Gmail web interface). I made it because there currently are no mobile email clients that are built with common frontend web languages. I’d like to see anyone able to run their own custom email client that fits their workflow.

It’s built using PhoneGap, and the server is Node with MongoDB.

KievII, Capturing, Mobify 2.0

22 Mar 2013 | By Alex Young | Comments | Tags audio mobile

KievII Host

The KievII audio plugin host

KievII Host (GitHub: janesconference / KievIIHost, License: MIT) by Cristiano Belloni is an audio plugin host based on the Web Audio API. The plugins are loaded dynamically using RequireJS from a GitHub repository, and already include lots of cool effects like a phaser, wah-wah, and pitch shift.

If you want to try out the KievII demo, which is a lot of fun, you’ll need Chrome and to enable Web Audio Input in chrome://flags. I tested it in Chrome 25 on a Mac and it ran pretty solidly. It allows audio to be routed from the mic through various effects, and there’s also a sample player with a keyboard for triggering audio samples at different pitches.

There’s more information about KievII at, which has a demo of a step sequencer.

Capturing for Responsive Design

Shawn Jansepar sent in his post at the Mozilla Hacks blog, called Capturing - Improving Performance of the Adaptive Web, about a client-side API he’s developed called Capturing:

Our approach to give you resource control is done by capturing the source markup before it has a chance to be parsed by the browser, and then reconstructing the document with resources disabled.

The ability to control resources client-side gives you an unprecedented amount of control over the performance of your website.

Capturing was a key feature of Mobify.js 1.1, our framework for creating mobile and tablet websites using client-side templating. We have since reworked Mobify.js in our 2.0 release to be a much more modular library that can be used in any existing website, with Capturing as the primary focus.

One example Shawn uses is using a polyfill for the picture element, which only includes an extra img tag for browsers without JavaScript turned on. This is in contrast to other solutions that require noscript tags.

Mobify.js 2.0

Meanwhile, Mobify.js 2.0 (GitHub: mobify / mobifyjs, License: MIT) has been released as a developer preview. This includes the Capturing API mentioned in Shawn’s post above.

The goal of Mobify.js is to help existing sites better support mobile devices. The examples included are the picture polyfill mentioned above, using media queries, and templating.

Node Roundup: 0.10.1, Express Group Handlers, Fox,

21 Mar 2013 | By Alex Young | Comments | Tags node modules express middleware testing p2p
You can send in your Node projects for review through our contact form.

Node 0.10.1

Node 0.10.1 has been released, hot on the heels of 0.10.0. This version improves the performance of the non-streaming crypto APIs, fixes some tls and net module issues, and makes missing callbacks in streams2 show a warning rather than raising an exception.

Express Group Handlers

Express Group Handlers (GitHub: tldrio / express-group-handlers, License: MIT, npm: express-group-handlers) by Louis Chatriot provides a little bit of sugar for managing Express middleware. It allows routes to be wrapped with beforeEach and afterEach so middleware can be confined to certain routes.

The beforeEach method can accept multiple middlewares to run, and it’s easy to wrap it around existing code:

var groupHandlers = require('express-group-handlers');


app.beforeEach(groupHandler, function(app) {
  app.get('/route3', finalHandler3); // GET /route3 will execute groupHandler, then finalHandler3


Fox (GitHub: azer / fox, License: BSD, npm: fox) by Azer Koçulu is a test framework that is largely compatible with Mocha, as long as you don’t have nested invocations of describe. It works with both Node and client-side projects, and injects chai so you automatically get assertions without having to load an extra library.

Passing the -b flag to the command-line program will cause Fox to compile the scripts necessary to run the tests in a browser. It also includes tests written with itself, of course! (GitHub: InstantWebP2P /, License: MIT, npm: by Tom Zhou is a set of projects designed to send HTTP over UDP, the goal being to take advantage of UDP’s inherent performance benefits. It supports TURN and STUN channeling with WebSockets, for realtime streaming.

It can run web services using peer-to-peer protocols, behind NAT and firewalls. The module includes a binary called node-httpp, which provides the HTTP over UDP handling. The project includes installation instructions, and a brief roadmap.

I’ve seen a few peer-to-peer Node projects, but I think this is the first one I’ve seen that uses UDP as the transport layer protocol. It’s also interesting that the author is directly addressing NAT issues.

Backbone 1.0

20 Mar 2013 | By Alex Young | Comments | Tags backbone.js mvc libraries frameworks

Today Jeremy Ashkenas announced that Backbone.js 1.0 has been released, after a whole year on the 0.9 branch. This release adds support for HTTP PATCH requests, where partial updates are sent to the server. It also provides some sugar for data filtering methods like where and omit (from Underscore.js). Jeremy also notes that the annotated source has been improved.

Although Backbone 1.0 includes internal refactoring, the API should be largely backwards compatible – at least from the tests I’ve been doing with my own Backbone projects against the releases on GitHub. There may be quirks that I’ve missed, however, so as always make sure you’ve carefully tested your code before releasing it.

Jeremy also intimates that he’s looking for new major features, but doesn’t specifically promise anything in the roadmap:

In an ecosystem where overarching, decides-everything-for-you frameworks are commonplace, and many libraries require your site to be restructured to suit their look, feel, and default behavior - Backbone should continue to be a tool that gives you the freedom to design the full experience of your web application.

I feel like Backbone has earned its 1.0 status, and I’ll be watching Backbone’s GitHub repository very carefully over the next few months.

jQuery Roundup: Panzoom, jQuery.Feedback, shurikenJS

19 Mar 2013 | By Alex Young | Comments | Tags jquery plugins animation frameworks
Note: You can send your plugins and articles in for review through our contact form.


Panzoom (GitHub: timmywil / jquery.panzoom, License: MIT) by Timmy Willison is a plugin for adding panning and zooming behaviour to an element. It works in jQuery 1.9+, and uses CSS transforms and matrix functions to take advantage of GPU acceleration where available.

Any element can be potentially panned or zoomed, including Canvas and videos. It supports touch gestures, including pinch, so it should work well in a mobile project.

There’s a demo in the form of Panzoom’s unit tests.


jQuery.Feedback (GitHub: siong1987 /, License: MIT) by Teng Siong Ong is a clone of the feedback widget found on Branch. It has some nice CSS transform animations, and just expects a callback for handling sending the resulting message.



shurikenJS (GitHub: shurikenjs, License: MIT) by Stephan Fischer is a JavaScript framework that uses recent ECMAScript methods and an API based around native DOM objects. The Node object is enhanced with new chainable methods, for example: Node.css('color', 'red').hide().show().

Stephan hasn’t included any tests yet, but I’ve included it here because it’s so different to jQuery it should get you thinking. The combination of modern ECMAScript with native DOM code is an interesting approach to client-side development.