As we’ve seen over the last few weeks, Node is a great platform for building portable Unix programs. This makes it a useful tool for developing daemons. I often find myself dealing with daemons, whether working as a web developer, sysadmin, or general Unix developer.

Rationale

Why bother making a Node process behave like a daemon? There are several reasons, but generally because it’s a service that should be running at all times, in the background, and will automatically start after a reboot.

There are different ways to do this depending on the operating system. Most Linux distributions use System V compatible init scripts with runlevels. An alternative is upstart which is installed by default on Ubuntu systems.

BSD’s init behaves differently – there are no run levels. Solaris has the Service Management Facility which describes services using XML service manifests – I often find myself writing a simple XML file that wraps around a start/stop init script.

Finally, Mac OS has launchd which is a system wide and user-oriented daemon manager. It uses XML plist files, and the documentation for these can be found in man launchd.plist.

Linux Standard Base

Guidance for writing init scripts can be found in the Linux Standard Base’s section on Init Script Actions. This states that applications conforming to the standard should accept a single argument which should include one of the following:

• start
• stop
• restart
• force-reload
• status

The status action works by returning specific exit status codes:

• 0: The service is running correctly
• 1: The program is dead and a /var/run pid file exists
• 2: The program is dead and a /var/lock lock file exists
• 3: The program is not running
• 150 to 199 are reserved for your own requirements

Other actions are also expected to return specific status codes:

• 1: Unspecified error
• 2: Invalid arguments
• 3: Unimplemented feature
• 4: Insufficient privileges
• 5: Program is not installed
• 6: Program is not configured
• 7: Program is not running
• 150 to 199 are reserved for your own requirements

Most daemons are distributed without implementing all of these actions and status codes, but it’s a good idea to be aware of them.

Node Examples

StatsD (License: MIT, npm: statsd) from Etsy is a network daemon for aggregating statistics. It comes with the files for a Debian package, and includes an upstart script:

description "statsd"
author      "rdio"

start on startup
stop on shutdown

script
    # We found $HOME is needed. Without it, we ran into problems
    export HOME="/root"

    exec sudo -u nobody /usr/share/statsd/scripts/start
end script

Part of the installation process sets up log files with the correct permissions, and installs a configuration file to /etc/statsd. It’s usually a good idea to support a system-wide configuration file that can be overridden for the purpose of distributing daemons.

The log file is created in /var/log/statsd/, which is standard as well.

This daemon loads the configuration file based on the command-line argument:

config.configFile(process.argv[2], function(config, oldConfig) {
  // Process the configuration file
});

Some daemons may opt to check for configuration files in default locations, loading them in order of preference:

• /etc/mydaemon/config.js
• $HOME/.mydaemon/config.js

Init Module

Developing a Linux-friendly init script is made easy by the Init Node module by Paul Driver. He’s actually implemented the correct status codes, and the daemon module has been used, so you essentially get a real background daemon with very little code:

var init = require('init');

init.simple({
  pidfile: '/var/run/myprog.pid',
  logfile: '/var/log/myprog.log',
  command: process.argv[3],
  run: function() {
    setInterval(function() {
      // Will be redirected to /var/log/myprog.log
      console.log('Running');
    }, 1000);
  }
});

Notice that console output will be redirected to a log file, so in this case “Running” will be printed to /var/log/myprog.log.

Wrapper Scripts

The Init module turns a Node script into a daemon, but what if your daemon needs to accept various command-line parameters rather than just start/stop/restart? In this case, most people either write a shell script to provide a System V init wrapper, or they might use an upstart script as we saw with StatsD.

Of course, other Unix systems have different init implementations, but a simple init script or init script-compatible Node module can usually be coerced into working with systems like Solaris’ SMF.

You’ll need basic shell scripting skills to write an init script, but there’s an example of one in Fedora’s packaging documentation: SysVInitScript.

Packaging Guidelines

If you want to reach a wider audience, packing guidelines often have useful tips on how to distribute daemons as well:

• Basics of the Debian package management system
• Ubuntu packaging guide
• Fedora RPM
• Arch packaging
• Solaris pkgadd

Node 0.6.14

Node 0.6.14 was released soon after 0.6.13. This release includes bug fixes and updates npm to 1.1.12.

Mocha 1.0

Mocha 1.0 has been released. There’s now a GitHub-flavoured Markdown reporter, and JavaScript “compilers” are now supported with mocha --compilers coffee:coffee-script.

An API has also been added, so it’s possible to programatically create tests:

var mocha = new Mocha;
mocha.reporter('spec').ui('bdd');

mocha.addFile('test/suite.js');
mocha.addFile('test/runner.js');
mocha.addFile('test/runnable.js');

ncluster

ncluster (npm: ncluster) by Ben Murphy is a zero-downtime clustering solution. It’ll reload the application when a SIGHUP is issued, and gracefully quit when SIGQUIT is sent.

Ben has designed it to be used with Capistrano. He’s also created a Node/Ubuntu Vagrant deployment solution with instructions for deploying his demo app to a virtual machine (nodejs_vagrant_helloworld).

ENVy

ENVy (npm: envy) by Elio Capella Sánchez is a small module for managing JSON settings files. It’ll use require to load a file called config.json in the project’s root directory.

It also uses a convention for automatically determining the environment and loading the appropriate properties. Given a JSON file like this:

{
  "environment": "development",

  "development": {
    "test": "Development property"
  },

  "production": {
    "test": "Production property"
  }
}

Then require('envy').config will load the development settings, so config.test will return “Development property”.

jQuery 1.7.2

jQuery 1.7.2 has been released. This release mainly includes fixes, but there are also tweaks that improves API consistency. There’s an important note for jQuery Mobile users on which jQuery version to use:

If you’re using jQuery Mobile, please use jQuery 1.7.2 only with jQuery Mobile 1.1. For previous versions of jQuery Mobile, stay with jQuery core 1.7.1 or earlier.

Filtrify

Filtrify (GitHub: luis-almeida / filtrify, License: MIT) by Luís Almeida is a Chosen-inspired tag filtering plugin. It allows multiple tags to be selected and searched, and multiple Filtrify instances can be used on a single page.

The markup is based on data attributes:

<div id="placeHolder"></div>

<ul id="container">
  <li data-genre="pop, rock, british, classic rock"> The Beatles </li>
  <li data-genre="rock, british, blues, classic rock"> The Rolling Stones </li>
  <li data-genre="alternative, electronic, female vocalists"> Björk </li>
  <li data-genre="rock, alternative, grunge"> Foo Fighters </li>
  <li data-genre="rock, classic rock"> Bruce Springsteen </li>
</ul>

Now calling $.filtrify('container', 'placeHolder') will add the selector to the page.

The plugin accepts lots of options, including a property for supplying a callback that fires when tags are added or removed.

JsBehaviour

JsBehaviour (License: MIT) by DracoBlue is similar to Declarative that we featured last week – both are essentially tools for associating complex behaviour with markup as unobtrusively as possible.

Handlers must be registered with jsb:

jsb.registerHandler('example', function(el, options) {
  element.textContent = 'I am loaded with name: ' + options.name;
});

Then parameters can be supplied to JsBehaviour like this:

<span><input class="jsb_ jsb_example" type="hidden" value="{&quot;name&quot;:&quot;Jan&quot;}" />Are you loaded?</span>

Running this example would produce I am loaded with name: Jan.

JsBehaviour expects “enhanced” elements to include a jsb_ class. The author has written more about this library with examples on the JsBehaviour blog.

Rest Test Test

Rest Test Test (GitHub: jeroenooms / resttesttest) by Jeroen Ooms is a jQuery and Bootstrap-based HTTP testing tool. It allows various HTTP methods to be issued, along with parameters. The results are displayed with the associated HTTP headers.

The author recommends using this with Firebug for testing REST services and Cross Origin Resource Sharing (CORS).

Gazel

Gazel (GitHub: matthewp / gazel, License: MPL 2.0) by Matthew Phillips is a key-value store for browsers, based on Redis. The API is similar to Redis, and has a friendly chainable syntax:

var client = gazel.createClient();

client.on('error', function(err) {
  console.error('Error:', err);
});

client.multi()
  .set('key', 1)
  .incrby('key', 10)
  .get('key')
  .exec(function(results) {
    var key = results.pop()[0];
  });

It uses IndexedDB for storage, and will attempt to use vendor-specific versions where available. The author has written a sizeable suite of Mocha tests that can be run in a browser.

Heroku Node Version Support

Heroku now permit Node and npm versions to be requested using your application’s package.json file:

{
  "name": "myapp",
  "version": "0.0.1",
  "engines": {
    "node": "0.6.x",
    "npm":  "1.0.x"
  }
}

The available versions are listed here:

• Node.js version manifest
• npm version manifest

Heroku defaults to their oldest version of Node 0.4, so it’s probably a good idea to specify the latest 0.6 version.

Read more here: Specifying a version of Node.js / npm

nios

nios by Sebastian Waisbrot is a Node port for iOS. The objective is to allow Node projects to run on an iOS device with minimal modifications. It uses WebViewJavascriptBridge to run Node’s libraries.

The example that comes with the project runs a Node HTTP server within a native iOS app.

tQuery Web Audio

Jerome Etienne has added support for the Web Audio API to the development branch of his tQuery project. tQuery.WebAudio shows off the API, and he’s also written a tutorial with a screencast: tQuery WebAudio for More Realistic 3D.

JavaScript Motion Tracking

Romuald Quantin sent in an article on JavaScript Motion Tracking which has a demo and some explanation behind the Canvas-based effect.

Reified

Reified (License: MIT, npm: reified) by Brandon Benvie is a binary library for Node and browsers. Each supported binary type is implemented using buffers:

var reified = require('reified')
  , int32 = new reified('Uint32', 10000000)
  , int16 = new reified('Uint16', int32)
  , int8 = new reified('Uint8', int16);

int8.write(100);
// Uint8 100

int8.write('test');
// Exception: TypeError: Invalid value for Uint8: undefined

There are also constructors for arrays, structs, and bitfields:

var DescriptorFlags = reified('DescriptorFlags', {
  ENUMERABLE   : 1,
  CONFIGURABLE : 2,
  READONLY     : 3,
  WRITABLE     : 4,
  FROZEN       : 5,
  HIDDEN       : 6,
  NOTPRIVATE   : 7,
}, 1);

var desc = new DescriptorFlags;
desc.HIDDEN = true;

// { ‹DescriptorFlags›
//   ENUMERABLE:   false,
//   CONFIGURABLE: true,
//   READONLY:     true,
//   WRITABLE:     true,
//   FROZEN:       true,
//   HIDDEN:       true,
//   NOTPRIVATE:   true }

Thing.js

Thing.js (GitHub: matthewp / thingjs, License: MPL 2.0, npm: thingjs) by Matthew Phillips is a small library that wraps around the ES5 Object.create method to provide mixins and object initialisation:

var Thing = require('thingjs')
  , A = { name_a: 'a' }
  , B = { name_b: 'b' }
  , C;

C = Thing.create([A, B], {
  init: function() {
    console.log(this.name_a);
    console.log(this.name_b);
  }
});

// Returns an instance of C
Thing.create(C, true);

Last week in Unix and Node: Signals I discussed how to interact with Node processes using POSIX signals. We often also need to run multiple processes, but how exactly can we launch processes from Node, and how can we manage multiple processes programatically?

This depends on the architecture of your application. If you’ve got a situation where consumers take work from a queue – perhaps from a messaging system like RabbitMQ or from a centralised database – then multiple processes can be launched from the shell. The standard Unix rules apply:

• Consider writing a PID file using process.pid
• If the processes are long-running, then listen for the appropriate signals
• Exit with a non-zero code using process.exit when an unrecoverable error occurs
• Use the right output streams – log errors with console.error and messages with console.log, or consider using a library like winston to handle logging

If your Node programs adhere to these rules then it’s fairly easy to manipulate them at the system level using shell scripts or makefiles.

Spawning Processes

Running a child process using child_process.spawn has a familiar event-based API:

var spawn = require('child_process').spawn
  , ls = spawn('ls', ['-latr']);

ls.stdout.on('data', function(data) {
  console.log(data.toString());
});

ls.on('exit', function(code) {
  console.log('Child process exited with code:', code);
});

The object returned by spawn has the standard streams, which also have event-based APIs. It also makes grabbing the exit code pretty easy.

An alternative is child_process.exec which has a lighter syntax:

var exec = require('child_process').exec;

exec('ls -latr', function(err, stdout, stderr) {
  console.log(stdout);
});

The interesting thing about exec is it takes a second argument that includes useful options like timeout and env. If you’re running something resource intensive but only want it to run for a few seconds, then exec('command', { timeout: 1000 }) is extremely convenient.

There’s another related method which is execFile – this one doesn’t run a subshell, so you’ll want to use the full path to the command.

Node processes can be spawned using child_process.fork which allows messages to be passed to the child process. There’s a full example in the Node documentation on fork. If you need to pass messages between processes then this is an easy way to do it without involving something more complex.

Cluster

Node’s Cluster API offers a programmatic solution to process management. This is particularly attractive when you need to take advantage of multiple cores or CPUs but don’t want to use a full-blown messaging system.

Workers can be spawned using cluster.fork, and when they die a death event is fired. This example is adapted from Node’s documentation:

var cluster = require('cluster')
  , http = require('http')
  , numCPUs = require('os').cpus().length
  , i
  , worker
  , numReqs = 0;

if (cluster.isMaster) {
  console.log('Master PID:', process.pid);

  // Start a set of workers based on the number of CPUs
  for (i = 0; i < numCPUs; i++) {
    var worker = cluster.fork();
    worker.on('message', function(msg) {
      if (msg.cmd && msg.cmd == 'notifyRequest') {
        numReqs++;
        console.log('Total requests:', numReqs);
      }
    });
  }

  // Log when the worker dies
  cluster.on('death', function(worker) {
    console.log('Worker died, with PID:', worker.pid);
  });
} else {
  console.log('Worker PID:', process.pid);

  http.Server(function(req, res) {
    res.writeHead(200);
    res.end('Hello from ' + process.env.NODE_WORKER_ID + '\n');

    process.send({ cmd: 'notifyRequest' });
  }).listen(8000);

  console.log('Listening on port 8000');
}

First, cluster.isMaster is used to determine if the current process is the master or a worker. Next, cluster.fork is used to spawn a new worker – this can only be called from the master process. A listener is set up on the worker for a message event – later on process.send is used to notify the master process of an event. This is a useful form of message passing that allows JavaScript objects to be sent as messages.

If the process isn’t the master then a HTTP server is started; each worker is now effectively sharing HTTP requests across CPU cores. The interesting thing about cluster.fork is it allows TCP servers to be shared across workers, and notice that the message passing API used here is available with child_process.fork as well.

Running this demonstrates that the PID files work as expected:

➜ node cluster.js
Master PID: 31839
Worker PID: 31840
Listening on port 8000
Worker PID: 31842
Listening on port 8000
Worker PID: 31844
Listening on port 8000
Worker PID: 31843
Listening on port 8000
Worker PID: 31841
Listening on port 8000
Worker PID: 31845
Listening on port 8000
Worker PID: 31846
Listening on port 8000
Worker PID: 31847
Listening on port 8000

Killing a worker behaves as expected, too:

➜ kill 31845
Worker died, with PID: 31845

The servers will still be running, and Node won’t try to access the dead worker no matter how hard you try.

Killing Clusters

This all works fairly well and may reduce the amount of work it takes to run groups of Node processes, but until recently Node didn’t automatically kill workers when the master is killed. In the previous example, kill 31839 (which was the master’s PID) will leave the children hanging around.

There are many ways to deal with this – Linux comes with setsid which can be used to run programs in sessions. The master process would then be the session leader, and the signals would be sent to each worker.

It can also be handled manually by keeping an array of workers around:

process.on('SIGTERM', function() {
  console.log('Master killed');

  workers.forEach(function(w) {
    w.kill();
  });

  process.exit(0);
});

Issuing SIGINT by pressing ctrl c seems to have the desired effect by default.

Node 0.6.13 and 0.7.6

Node 0.6.13 (stable) and 0.7.6 (unstable) have been released. These releases also update npm, and I noticed npm can now take OS and CPU properties in package.json. This is documented in the man page, so to read more type man npm-json or view the online version here: npmjs.org/doc/json.html.

Architectures and operating systems can even be blacklisted, so if you’re sure your module runs fine everywhere except Windows, then this can be represented with "os" : [ "!win32" ].

ShareJS

ShareJS (GitHub: josephg / ShareJS, License: MIT, npm: share) by Joseph Gentle is an application that can be used to add concurrent editing to any web application. The server is Node, and the client is expected to use Ace for editing.

The author has already put together a ShareJS wiki with extra documentation (although the readme is pretty good too). Also of note is the decision to use Operational Transformations:

Most wikis have a ‘save’ button and do locking. OT is a class of algorithms that do multi-site realtime concurrency. OT is like realtime git. It works with any amount of lag (from zero to an extended holiday). It lets users make live, concurrent edits with low bandwidth.

According to the project’s documentation, Joseph is an ex-Google Wave engineer (his LinkedIn profile mentions “Engineering Intern at Google”). Also, ShareJS has a 1:1 code to test LoC ratio.

Route66

Route66 (GitHub: vdemedes / route66, License: MIT, npm: route66) by Vadim Demedes is a middleware for routing in Connect 2.0. Vadim missed the connect.router middleware that was removed from Connect, so he created a replacement.

The decision behind removing the router was discussed at length in the Remove router ticket in Connect’s GitHub issues.

Declarative

Declarative (License: MIT, npm: declarative) by Alex Lawrence allows HTML to be mapped to behavioural JavaScript using mappings. The author’s first example is a search form with a character counter that uses the following HTML:

<form action="/" method="POST">
  <input id="search" name="search" type="text" maxlength="50" />
  <span data-widget-counter="target: 'search', text: '{0} characters left'"></span>
  <input type="submit">
</form>

Notice the use of data attributes to supply options to the JavaScript mapping:

declarative.mappings.add({
  id: 'counter',
  prefix: 'data-widget-',
  types: ['counter']
  callback: function(counter, type, options) {
    var input = document.getElementById(options.target);
    var maxlength = input.getAttribute('maxlength');
    countCharacters(input, counter, maxlength);
  }
});

Once a mapping has been declared, it can be mapped to the whole DOM using apply:

declarative.apply('counter').to(document);

None of this depends on jQuery, but the author has provided examples that demonstrate jQueryUI integration. Jasmine tests and examples are included in the project’s source.

jQR

jQR (GitHub: Gottox / jQR, License: GPL3) by Enno Boland is a QR Code generator for jQuery. It’s similar to jquery.qrcode.js by Jerome Etienne, featured previously on DailyJS – both use the same method name:

$('#qrcode').qrcode('Hello World');

Jerome’s plugin includes qrcode.js by Kazuhiko Arase, whereas Enno’s plugin is a rewrite that’s influenced by Kazuhiko’s original code.

Ender-Carousel

Ender-Carousel (GitHub: nemeseri / ender-carousel, ender: ender-carousel, npm: ender-carousel) by Andras Nemeseri is a carousel plugin for Ender that’s jQuery-compatible. The Ender-Carousel Basic Configuration Tutorial has sample HTML, CSS, and JavaScript, which is just $('.carousel').carousel().

Stapes.js

Stapes.js (GitHub: hay / stapes, License: MIT) by Hay Kranen is a small JavaScript MVC framework. Like other recent takes on MVC, it’s based around events and inheritance. It also works nicely with RequireJS, jQuery, and Zepto.

Stapes uses modules:

var Module = Stapes.create();

Module.extend({
  init: function() {
    this.emit('ready');
  }
});

Modules include data methods for getting and setting attributes:

var module = Stapes.create();

module.set({
  name: 'Alex'
, title: 'Mr'
});

module.get('name'); // Alex

Attributes can be removed, filtered, and updated. The author has written up full documentation and a rationale behind the project at the Stapes.js homepage.

Shardjs

Shardjs (License: MIT) by Francesco Sullo is a module for managing keys of up to 3844 Redis nodes. The author was inspired by Sharding and IDs at Instagram where the Instagram developers discuss a solution to generating unique IDs at scale.

To get around the lack of 64-bit integers in JavaScript, the author has opted to use 62-bit strings. He discusses his algorithm for generating unique keys and where the number 3844 comes from in the project’s readme:

The advantage of this approach is that since all the keys generated at the same millisecond will go on the same virtual shard and there is the incremental sequence, we can mantain the key well sorted by time.

soma.js

soma.js (GitHub: somajs / somajs, License: MPL) by Romuald Quantin is a new MVC framework that uses the observer pattern with native events, and the command pattern, with the ultimate goal of encouraging development of decoupled modules:

The command pattern aims to encapsulate method invocation, requests or operations into a single object and gives you the ability to both parameterize and pass method calls around that can be executed at your discretion. In addition, it enables you to decouple objects invoking the action from the objects which implement them, giving you a greater degree of overall flexibility in swapping out concrete ‘classes’.

– Addy Osmani on the command pattern

In soma.js applications, there are wires, models, views, and commands. Wires are used to represent the logic of an application and coordinate between other elements of the framework. Models don’t necessarily imply a particular storage solution, they’re more like a convention for integrating with other solutions like localStorage or Knockout – there’s a demo showing how to use soma.js with Knockout’s data bindings.

The project’s website includes demos and a tutorial that explain how everything works.

jsprops

jsprops (License: MIT) by Tobiasz Cudnik adds class-based properties to prototypes, the goal being to reduce redundancy when copying properties to instances. It also includes inheritance-based signalling.

Tobiasz has included CoffeeScript and JavaScript examples, but I’ve reproduced the JavaScript one here because it’s a little bit easier to understand:

var property = require('jsprops').property;

function Klass() {}

Klass.prototype.foo = property('foo');
Klass.prototype.bar = property('bar', null, 'def_value');
Klass.prototype.baz = property('baz', {
  set: function(set, val) {
    set(val.replace(/a/, 'b'));
  }
});

The project comes with tests that further illustrate the intended usage.

Augmented Reality in the Browser

In Augmented Reality in the Browser, Jerome Etienne discusses WebRTC and how to combine WebGL with a live video feed – all in a browser.

To simplify this process, Jerome has created tquery.jsartoolkit, which uses his tquery library to wrap around JSARToolKit. JSARToolKit, by Ilmari Heikkinen, is the core of the augmented reality functionality.

 Springbase

Springbase (GitHub: springbase / node-springbase, npm: springbase) have released a Node driver for their relational data store. This is a hosted database solution that includes a web data browser and management interface, and even SQL support.

The API is event based:

var springbase = require('springbase')
  , conn
  , query;

conn = new springbase.data.Connection({
  application: '1GmTdS2ayPts',
  username: '<email>',
  password: '<password>'
});

query = conn.getQuery('qryStoreInventoryByZipCode');
query.on('ready', function() {
  var reader = query.execute({ zipCode: 94611 });
  reader.on('row', function(row) {
    console.log('Found store:', row);
  });
});

They’re currently offering a free account with 100 MB storage and 1 GB per month bandwidth (I get no remuneration for mentioning this).

jQRange

jQRange (GitHub: Gottox / jQRange) by Enno Boland attempts to help deal with W3C and Internet Explorer ranges. The current selection can be obtained with $.range('^'), and ranges can be spanned with $(selector).range().

The plugin even supports regular expressions, so any range matching a given pattern can be spanned. The author has included QUnit tests, and a lot more functionality is documented in the readme file.

Signals represent a limited form of inter-process communication. When a signal is issued to a process, it will be interrupted and a signal handler will be executed. If there is no signal handler, the default handler will be called instead. This sounds a lot like asynchronous events in Node, and that’s exactly how signals are implemented; using process.on:

process.on('SIGINT', function() {
  console.log('Interrupted');
});

Typing man sigaction in a terminal will display some background on signals, including a useful list of signal names and their descriptions.

Using Signals

Create a file called signals.js that contains the following, and chmod +x it:

#!/usr/bin/env node

process.on('SIGINT', function() {
  console.log('Got a SIGINT');
  process.exit(1);
});

process.on('SIGHUP', function() {
  console.log('Got a SIGHUP');
});

setInterval(function() {
    console.log('Daemon running');
}, 10000);

console.log('PID:', process.pid);

Now chmod +x signals.js and run it with ./signals.js. It’ll print out its PID – this can be used with the kill command to issue signals:

➜ ./signals.js &
PID: 12626
➜ kill -s SIGHUP 12626
Got a SIGHUP

Running the command with & to put it in the background will actually display the PID in most shells, but I put process.pid in there just to demonstrate how to get the PID in Node. Once you’ve sent a few signals to it, issuing a SIGINT will end the process.

Usage in Node

A common use of signal events in Node is to perform some kind of cleanup when a process is terminated. Where signals really come in handy is for communicating with daemons. Many daemons accept SIGHUP to reload configuration files – I followed the same convention when I built an IRC daemon in Node. Another convention is to listen for SIGTERM to perform a graceful shutdown. This could be used to close down sockets, database connections, or remove any temporary files.

I believe Node is a great solution for writing Unix daemons. It’s a good idea to follow established conventions, so add listeners for SIGHUP and SIGTERM.

Command-Option-Argument

Command-Option-Argument (npm: coa) by Sergey Berezhnoy is a command-line option parser that uses a chainable API:

require('coa').Cmd() // main (top level) command declaration
  .name(process.argv[1]) // set top level command name from program name
  .title('My awesome command line util') // title for use in text messages
  .helpful() // make command "helpful", i.e. options -h --help with usage message
  .opt() // add an option
    .name('version') // name for use in API
    .title('Version') // title for use in text messages
    .short('v') // short key: -v
    .long('version') // long key: --version
    .flag() // for options without value
    .act(function(opts) { // add action for option
      // return message as result of action
      return JSON.parse(require('fs').readFileSync(__dirname + '/package.json'))
          .version;
    })
    .end() // end option chain and return to main command

The API’s chainable design uses stack-like hierarchies, so when an option is declared with opt() subsequent method calls will apply to that option. The author has included a makefile which runs tests with Vows.

Nodemailer

Nodemailer (License: MIT, npm: nodemailer) by Andris Reinman is a module for sending emails that supports Unicode, attachments, connection pools, and SSL/STARTTLS. It’s even designed to work with popular mail services like Gmail and Hotmail.

The API centres around SMTP transports:

var nodemailer = require('nodemailer')
  , smtpTransport = nodemailer.createTransport('SMTP', {})
  , options = { from: 'alex@example.com', subject: 'Subject', text: 'Hello' };

smtpTransport.sendMail(options, function(err, response) {
});

Notice the use of the standard leading error argument in the callback. This project includes a test suite written with Nodeunit.

Node.js-Ultra-REPL

Node.js-Ultra-REPL (License: MIT, npm: ultra-repl) by Brandon Benvie is a REPL for Node that’s intended to be used as a development environment. It provides a command-line UI where JavaScript can be executed and explored. Before using it, I strongly recommend pressing F1 to learn the basic commands. The command prompt stays at the bottom of the screen, while the result of any code is displayed above. Results can be read through by using page up and down. New V8 contexts can be created using control shift up, then switched between using control up and control down.

jQuery 1.7.2 RC1

jQuery 1.7.2 RC1 has been released. Several bugs have been fixed since the last beta release, which are fully documented in the announcement blog post. The announcement also contains an interesting call for IE6 help:

If you are particularly interested in IE6 support, please help us out. We are having sporadic trouble running the unit tests in IE6. It hasn’t been possible for us to determine the cause of these problems, but the problem doesn’t happen consistently and the sheer size of our test suite may just be overwhelming a browser that is more than a decade old.

jQuery OEmbed

jQuery OEmbed All (GitHub: starfishmod / jquery-oembed-all, License: MIT) can embed content like YouTube videos simply by providing a URL to the item’s page. It was originally created by Richard Chamorro, but this version has been forked by Andrew Mee to rely less on the OEmbed API.

The plugin will attempt to use various third party services to embed the content, including JSONP oEmbed and YQL. Services that require an API key are also supported:

$('.oembed').oembed(null,{
  embedMethod: 'auto',
  apikeys: {
    etsy: '<your etsy key>'
  }
});

Ignition

Ignition (GitHub: daytona / ignition, License: MIT/GPL) by Johan Sahlén of Daytona Communication AB is a jQuery MVC framework. It was sent in by Wade Stebbings who said he’d struggled to find information on it after it was recommended by a friend. It’s definitely an interesting framework, even though it seems to have stalled development in 2009.

It supports URL routing:

var $i = new Ignition({ modules: ['UrlManager', 'HistoryDispatcher'] });

$i.addRoute('articles/(\d+)', function(article_id) {
  $i.c.Articles.show(article_id);
});

And models:

$i.m('Article', {
  find_by_id: function(id, callback) {
    this.json($i.getUrl('article', { id: id }), { success: callback });
  }
});

Controllers and programmatic views are also included:

$i.c('Articles', {
  show: function(id) {
    $i.m.Article.find_by_id(id, $i.v.Articles.show);
  }
});

$i.v('Articles', {
  show: function(data) {
    var list = $('<ul id="articles"></ul>');
    for (var i = 0; i < data.articles.length; i++) {
      var article = data.articles[i];
      var listItem = $('<li><a href="'+article.url+'">'+article.title+'</a></li>');
      list.append(listItem);
    }

    $('body').append(list);
  }
});

API documentation is available if you want to dig a little deeper into the framework: Ignition API documentation.

nextVal

nextVal (GitHub: jukebox42 / nextVal, License: MIT) by John Norton is a form validation plugin. It uses a validate attribute with options that work a little bit like a mini validation DSL:

<input type="text" validate="email" placeholder="Please enter a proper email address" />

Then $(selector).nextVal() can be called on the form. Custom validation rules can also be added:

$(function() {
   $('form[name=FORMNAME]').nextVal({
      customRules:[
         ['matchpassword',function($o){return !($o.val() == $('#password').val());},''],
         ['xheader',function($o){return !$o.val().match(/^X-[a-zA-Z0-9_\-]+$/);},'Please enter a valid xheader. For example X-UserData1'],
      ]
   });
});

The only problem I have with this approach is the use of the validate attribute. To my knowledge, there is no validate attribute, so this should probably use a data attribute instead. Also, HTML5 already provides some validation attributes (required, pattern, min and max, step, and maxlength).

jquery-ui-rails

jquery-ui-rails makes adding jQuery UI 1.8.18 and its associated assets easy to install in Rails projects. It’s configured to work with the Rails asset pipeline, so it’s easy to require specific modules and get jQuery UI projects off the ground quickly.

Nools

Nools (License: MIT, npm: nools) by Doug Martin is a rules engine. Rules are contained in “flows”, and instances of flows are known as “sessions” – these are used to add or retract facts from the engine:

var nools = require('nools')
  , session
  , message;

function Message(message) {
  this.message = message;
}

session = flow.getSession();
message = new Message('hello');

// Add the fact to the engine
session.assert(message);

// Change it
message.message = 'Bye';
session.modify(message);

// Remove it
session.retract(message);

For examples of how to define flows, have a look at the Nools readme.

Procstreams

Procstreams (License: MIT, npm: procstreams) by Marco Rogers is an attempt to create a more idiomatic JavaScript shell scripting API:

var $p = require('procstreams');
$p('cat lines.txt').pipe('wc -l')
  .data(function(stdout, stderr) {
      console.log(stdout); // prints number of lines in the file lines.txt
  });

$p('mkdir foo')
  .and('cp file.txt foo/')
  .and('rm file.txt')
    .on('exit', function() {
      console.log('done');
    });

As we’ve seen many times on DailyJS, chainable APIs offer an elegant solution to reducing unnecessary callbacks in asynchronous APIs, so this module is definitely appealing on that level.

Kalendae

Kalendae (License: MIT) by Jarvis Badgley provides a date picker without any dependencies. There’s a demo of Kalendae showing calendars instantiated with various options.

If jQuery is available, then a plugin will also be available through $(selector).kalendae(options).

The author has also provided a makefile that can build the project, minimise it, and run it through Google Closure.

JavaScript Tile-Based RPG

Probed / RPG is a tile-based RPG written with server-side JavaScript. It’s built with Node, but the author has only deployed it to Windows so far. There’s a video of the game here:

The goal of the project is to create a random game world with quests, a battle system, loot, equipment, and levelling.

 Luca

Luca.JS (GitHub: http://datapimp.github.com/luca/) by Jonathan Soeder is a library for Backbone.js that provides augmented View and Collection classes:

Luca [includes] best practices and optimizations that I have personally come up with developing some rather large Backbone.js apps. It assumes a style of progamming very similar to ExtJS where you define almost all of your views and components as large nested JSON structures.

Luca is built on Twitter’s Bootstrap CSS, and includes CoffeeScript examples. It’s initially distributed as a Ruby library for Rails projects. The author is planning to add JavaScript examples and distribute an alternate version through npm in the future.

Regardless, it may provide some inspiration for those who want to further their Backbone.js knowledge.

KSS

KSS (GitHub: hughsk / kss-node, npm: kss) by Hugh Kennedy is a Node implementation of Knyle Style Sheets which is a methodology for writing maintainable CSS.

Specifically, KSS is a documentation specification and styleguide format. It is not a preprocessor, CSS framework, naming convention, or specificity guideline.

Installing the KSS module provides a kss-node command-line tool that will generate styleguide documentation based on your LESS or CSS stylesheets.

The project’s site includes a demo – take a look at the KSS Modules documentation for an example.

The module also includes an API, and it includes tests written with Mocha.

An anonymous pipe can be used to connect the standard streams of multiple processes.

tput setaf 48 ; whoami | figlet | tr _ … | tr \\ \` | tr \| ¡ | tr / √

This example prints out the result of whoami using FIGlet, and also uses tput to colourise the output.

Pipes are typically denoted with the vertical bar character. Both the | notation and concept of pipes were created by Doug McIlroy, who was quoted in part one of this series for his “Write programs that do one thing and do it well” quote.

Streams

Node provides an abstract representation of a stream through the Stream class. It’s actually widely used within Node, and represents the Unix text streams we’ve been discussing very neatly. In fact, it even includes a pipe method:

stream.pipe(destination, [options])

That means we can actually connect the standard input and output like this:

#!/usr/bin/env node

process.stdin.resume();
process.stdin.pipe(process.stdout);

The resume method is called here because standard input is paused by default.

➜ chmod 700 cat.js 
➜ echo 'Unix has a funny idea about what a cat is' | ./cat.js
Unix has a funny idea about what a cat is

The nice thing about streams in Node is they work like everything else: they’re asynchronous and event-based:

stream.on('data', function(data) {
  console.log('Received data:', data);
});

Whenever a stream is passed to a readable stream’s pipe method, the pipe event will be emitted.

Example

By combining what we’ve learned in the last few Unix and Node articles, we can start to build command-line tools that resemble real Unix tools. The following example uses Commander.js, which can be installed with npm install commander.

#!/usr/bin/env node

var program = require('commander')
  , c = 90;

program
  .version('1.0.0')
  .option('-r, --red', 'Red text')
  .option('-g, --green', 'Green text')
  .option('-b, --blue', 'Blue text')
  .parse(process.argv);

function colour(c, str) {
  return '\033[' + c + 'm' + str + '\033[0m';
}

if (program.red) {
  c = 91;
} else if (program.green) {
  c = 92;
} else if (program.blue) {
  c = 94;
}

process.stdin.resume();
process.stdin.setEncoding('utf8');
process.stdin.on('data', function(data) {
  process.stdout.write(colour(c, data));
});

Running echo "Some text" | ./colour.js --red will display red text. The expected encoding is set with process.stdin.setEncoding('utf8'), else the data argument passed to the data event on the stdin stream would be a plain buffer. The output is written to stdout using stream.write(string, [encoding], [fd]).

Conclusion

By having such a simple asynchronous API for streams, building Node programs that can work well with pipelines in shell scripts is straightforward. Other methods like stream.pipe are also useful. Read Node’s Stream documentation for more details and examples.

Plunker

Plunker (GitHub: ggoodman / stsh, License: MIT) by Geoff Goodman is a web application for editing web snippets. It’s inspired by other popular services like jsFiddle, but has Node source that you can fork, and has an interface more suited to editing multiple files. It has all the obvious features like syntax highlighting, using the venerable Ace editor. The interface has been built with Twitter’s Bootstrap, so it also serves as an example of a Bootstrap web application.

There’s a public API for Plunker which allows “plunks” to be created, updated, retrieved, and deleted.

Mixr

Mixr (npm: mixr) by Alex Barlow is an Express compiler and pre-processor for client-side JavaScript and CSS. It can also handle other formats like LESS and CoffeeScript. To use Mixr, write client-side code in separate files, then include other files using comments:

//= require lib/jquery.min.js
//= require main.js
//= require something.js.coffee

In this example, Mixr would automatically process the CoffeeScript file.

During development, it’s possible to compile assets on each request:

app.configure(function(){
  Mixr.addHelpers(app);
});

app.configure('development', function(){
  // Add Mixr routes for development mode only
  Mixr.addExpressRoutes(app);
});

Then during deployment, a compiled file can be generated with ./node_modules/mixr/bin/mixr compile.

FnQueue

FnQueue (License: MIT, npm: fnqueue) by Kilian Ciuffolo is a utility for chaining functions:

new FnQueue({
  // This will wait for 'searchSomething'
  funnyStuff: function(processSomething, callback) {
    callback(null, 'ciao!');
  },
  searchSomething: function(callback) {
    callback(err, results);
  },
}, function(err, data) {
    if (err) {
      throw(err);
    }

    console.log(data.searchSomething);
  }
});

If you want to read more about Node’s wealth of control flow libraries, check out New Control Flow Libraries.

timekeeper

I often find myself needing to travel time, so rather than hacking Date I’d prefer to use mocks through Veselin Todorov’s timekeeper (License: MIT, npm: timekeeper) library. Time can be frozen by using timekeeper.freeze(date). Once the normal passage of time is required, the fourth dimension can be corrected simply by calling timekeeper.reset().

The only caveat is setTimeout and setInteval won’t work as expected until timekeeper.reset() is called.

StyleDocco

_______ __         __        _____
|     __|  |_.--.--|  |-----.|     \-----.----.----.-----.
|__     |   _|  |  |  |  -__||  --  | _  |  __|  __|  _  |
|_______|____|___  |__|_____||_____/_____|____|____|_____|
             |_____|

StyleDocco (GitHub: jacobrask / styledocco, License: MIT, npm: styledocco) by Jacob Rask generates documentation and style guides from stylesheets.

It’s distributed as a command-line tool, and can be installed using npm:

npm install -g styledocco

Adding descriptive comments is enough for StyleDocco to generate its results. For an example, see StyleDocco 0.3.0 default styles.

noUiSlider

noUiSlider (GitHub: leongersen / noUiSlider, License: MPL 1.1/2.0) by Leon Gersen generates sliders. The basic usage is $('.sliderbar').noUiSlider(), but it also supports a wide range of options and events.

The sliders can be conventional value sliders, or represent a range with two draggable controls. The author has supplied CSS as well, so it can be used quite quickly out of the box.

jecookie

jecookie (License: MIT) by Alejandro El Informático is a friendly cookie library. JSON is supported, and cookies can be marked as secure.

var cookie = new jecookie('cookie_name', { secure: true, domain: '.example.org' });
cookie.data = { name : value, content : value };

It’s been tested in IE 6+, Firefox 3+, Chrome 10+, and Opera 10+.

Jed

Jed (GitHub: SlexAxton / Jed, License: WTFPL/Dojo CLA, npm: jed) by Alex Sexton is a JavaScript implementation of gettext. Plural forms are parsed using a Jison grammar (in plurals.jison)), and it has a chainable API:

var n = 5;

i18n
  .translate("%d key doesn't exist")
  .ifPlural(n, "%d keys don't exist")
  .fetch();

This can be used in both browsers and Node, and the author also demonstrates using it in the AMD style. As a bonus, Jed includes a useful sprintf implementation.

messageformat.js

messageformat.js (License: WTFPL/Dojo CLA, npm: messageformat) also by Alex Sexton handles both pluralization and gender in translations. Despite sounding similar to Jed, this is a different approach to gettext:

Gettext can generally go only one level deep without hitting some serious roadblocks. For example, two plural elements in a sentence, or the combination of gender and plurals.

Basic interpolation looks like this:

var mf = new MessageFormat('en')
  , message = mf.compile('His name is {NAME}.');

message({ 'NAME' : 'Jed' });
// "His name is Jed."

More complex replacements can use SelectFormat and PluralFormat:

var message = mf.compile('{GENDER, select, male{He} female{She} other{They}} liked this.');
message({ 'GENDER' : 'male' });
// "He liked this."

message({ 'GENDER' : 'female' });
// "She liked this."

Blossom

Blossom (GitHub: fohr / blossom, License: GPLv3 for non-commercial projects, npm: blossom) is an early beta of what Erich Ocean hopes represents the future of SproutCore:

Blossom is a serious attempt to advance SproutCore 1.x forward after the failure of SproutCore 2. I put forward the rational behind Blossom in early December, 2011 and also demo’d early work I had done on Blossom at the SproutCore User Group in Washington D.C. late January, 2012. […] Blossom is designed to become the future of SproutCore, but it also stands on its own.

Blossom has a Node-based build system, and it can be installed with npm install blossom. It includes the latest datastore and statechart code from SproutCore 1.8, and Node-based unit tests for foundation and datastore.

Currently, the SproutCore documentation should suffice for working with parts of Blossom. The Blossom developers are currently working towards their first release candidate on April 1st. The current code on GitHub is 1.0.0-beta.1.

Despite its SproutCore heritage, it’s important to understand where Blossom is different:

instead of simulating a web browser, Blossom’s native runtime has high-performance, platform-native implementations of the various Blossom and SproutCore classes, such as SC.Request (for networking), SC.Layer (for animation), as well as the various JavaScript runtime objects such as CanvasRenderingContext2D, Float32Array and ArrayBuffer that are exposed to Blossom developers.

reMarked.js

reMarked.js (GitHub: leeoniya / reMarked.js, License: MIT) by Leon Sorokin converts a client-side DOM fragment into Markdown. The reMarked.js homepage has several links to GitHub READMEs on the left-hand side, and pressing the ‘reMark it!’ button will convert their HTML into Markdown.

The author’s ultimate goal is to use this in a WYSIWYG editor. It seems like a difficult project, so it features tests as well.

Scroll.js

Scroll.js (License: MIT) by David Babaioff is a cross-browser library for managing the scroll event:

Scroll.bind('scroll_1', function() {
    // Your event handler code goes here.
}).bind({
    'scroll_2': function() {
        // Your event handler code goes here.
    },
    'scroll_3': function() {
        // Your event handler code goes here.
    }
});

It doesn’t have any dependencies, so it may be useful in cases where you don’t want to use a larger library like jQuery.

Andro.js

Andro.js (GitHub: maryrosecook / androjs, License: MIT) by Mary Rose Cook combines the idea of reusable behaviour with events:

Behaviour mixins are good for writing video games. They help with the concoction of game object logic: no more nightmarish inheritance hierarchies or weird bundles of functions that invent cryptic interfaces for the objects upon which they operate.

The author’s example is a cube that gets augmented with behaviours to provide logic based around events:

var Andro = require('andro').Andro
  , andro = new Andro()
  , soundBehaviour
  , cube;

function Cube() {
  this.touch = function(contact) {
    andro.eventer(this).emit('touch', contact);
  };
}

soundBehaviour = {
  setup: function(owner, eventer) {
    eventer.bind(this, 'touch', function() {
      console.log('SFX: Leo_the_Lion.wav');
    });
  }
};

cube = new Cube();
andro.setup(cube);
andro.augment(cube, firstTouchBehaviour);
cube.touch('added');

Here I’ve paraphrased the original example at androjs.maryrosecook.com/ just to give an idea of the syntax. I agree with the author that this library provides some useful tools for representing game logic, and the documentation also provides examples of situations where this might not work so well.

IndexedDB Examples

These IndexedDB examples by Parashuram Narasimhan show what works in Chrome, Firefox, and Internet Explorer 8+. The standards are still changing, but Parashuram has been working hard to track the current state of API changes in both his blog and the jquery-indexeddb plugin.

For more on IndexedDB, see the usual suspects: W3C: Indexed Database API, MDN: IndexedDB.