The JavaScript blog.


testing node modules time async promises unix daemons

Node Roundup: wish, Vow, shell-jobs

Posted on .

You can send in your Node projects for review through our contact form.


wish (GitHub: EvanBurchard / wish, License: MIT, npm: wish) by Evan Burchard is an assertion module designed to raise meaningful, human-readable errors. When assertions fail, it parses the original source to generate a useful error message, which means the standard comparison operators can be used.

For example, if wish(a === 5) failed an error like this would be displayed:

  Expected "a" to be equal(===) to "5".

If assert(a === 5) had been used instead, AssertionError: false == true would have been raised. A fairer comparison would be assert.equal, which would produce AssertionError: 4 == 5, but it's interesting that wish is able to introspect the variable name and add that to the error.


Vow (GitHub: dfilatov / jspromise, License: MIT/GPL, npm: vow) by Filatov Dmitry is a Promises/A+ implementation. Promises can be created, fulfilled, and rejected -- you should be able to get the hang of it if you've used libraries with then methods elsewhere, but there are some differences to Promises/A which feels like it actually simplifies some of the potentially messier parts of the original CommonJS specification.

Here's an example of the Vow API:

var promise1 = Vow.promise(),  
    promise2 = Vow.promise();

Vow.all([promise1, promise2, 3])  
  .then(function(value) {
    // value is [1, 2, 3]


The author has written some pretty solid looking tests, and benchmarks are included as well. The project performs favorably when compared to other popular promise libraries:

 mean timeops/sec


I like seeing daemons made in Node, and Azer Koçulu recently sent in a cron-inspired daemon called shell-jobs (GitHub: azer / shell-jobs, License: MIT, npm: shell-jobs). It uses .jobs files that are intended to be human readable. All you need to do is write a shell command followed by a # => and then a time:

cowsay "Hello" > /tmp/jobs.log # => 2 minutes  

The shell-jobs script will then parse this file and output the following:

  jobs Starting "cowsay "Hello" > /tmp/jobs.log" [2 minutes] +2ms

After two minutes has passed the job will be executed:

  exec 1. Running cowsay "Hello" > /tmp/jobs.log. +0ms


node unix daemons

Node Daemon Architecture

Posted on .

I've been researching the architecture of application layer server implementations in Node. I'm talking SMPT, IMAP, NNTP, Telnet, XMPP, and all that good stuff.

Node has always seemed like the perfect way to write network oriented daemons. If you're a competent JavaScript developer, it unlocks powerful asynchronous I/O features. In The Architecture of Open Source Applications: nginx by Andrew Alexeev, the author explains nginx's design in detail -- in case you don't know, nginx is a HTTP daemon that's famous for solid performance. Andrew's review states the following:

It was actually inspired by the ongoing development of advanced event-based mechanisms in a variety of operating systems. What resulted is a modular, event-driven, asynchronous, single-threaded, non-blocking architecture which became the foundation of nginx code.


Connections are processed in a highly efficient run-loop in a limited number of single-threaded processes called workers. Within each worker nginx can handle many thousands of concurrent connections and requests per second.

Highly efficient run-loop and event-based mechanisms? That sounds exactly like a Node program! In fact, Node comes with several built-in features that make dealing with such an architecture a snap.


If you read DailyJS you probably know all about EventEmitter. If not, then this is the heart of Node's event-based APIs. Learn EventEmitter and the Stream API and you'll be able to easily learn Node's other APIs very quickly.

EventEmitter is the nexus of Node's APIs. You'll see it underlying the network APIs, including the HTTP and HTTPS servers. You can happily stuff it into your own classes with util.inherits -- and you should! At this point, many popular third-party Node modules use EventEmitter or one of its descendants as a base class.

If you're designing a server of some kind, it would be wise to consider basing it around EventEmitter. And once you realise how common this is, you'll find all kinds of ways to improve the design of everything from daemons to web applications. For example, if I need to notify disparate entities within an Express application that something has happened, knowing that Express mixes EventEmitter into the app object means I can do things like app.on and app.emit rather than requiring access to a global app object.


Guess what else is an instance of EventEmitter? The process global object. It can be used to manage the current process -- including events for signals.


Domains can be used to group I/O operations -- that means working with errors in nested callbacks is a little bit less painful:

If any of the event emitters or callbacks registered to a domain emit an error event, or throw an error, then the domain object will be notified, rather than losing the context of the error in the process.on('uncaughtException') handler, or causing the program to exit with an error code.

Domains are currently experimental, but from my own experiences writing long-running daemons with Node, they definitely bring a level of sanity to my spaghetti code.


The Cluster module is also experimental, but makes it easier to spawn multiple Node processes that share server ports. These processes, or workers, can communicate using IPC (Inter-process communication) -- all using the EventEmitter-based API you know and love.

In the Wild

I've already mentioned that Express "mixes in" EventEmitter. This is in contrast to the inheritance-based approach detailed in Node's documentation. It's incorrect to say Express does this because it's actually done by Connect, in connect.js:

function createServer() {  
  function app(req, res){ app.handle(req, res); }
  utils.merge(app, proto);
  utils.merge(app, EventEmitter.prototype);
  app.route = '/';
  app.stack = [];
  for (var i = 0; i < arguments.length; ++i) {
  return app;

The utils.merge method copies properties from one object to another:

exports.merge = function(a, b){  
  if (a && b) {
    for (var key in b) {
      a[key] = b[key];
  return a;

There's also a unit test that confirms that the authors intended to mix in EventEmitter.

An extremely popular way to daemonize (never demonize, which means to "portray as wicked and threatening") a program is to use the forever module. It can be used as a command-line script or as a module, and is built on some modules that are useful for creating Node daemons, like forever-monitor and winston.

However, what I'm really interested in is the architecture of modules that provide services rather than utility modules for managing daemons. One such example is statsd from Etsy. It's a network daemon for collecting statistics. The core server code, stats.js, uses net.createServer and a switch statement to execute commands based on the server's protocol. Notable uses of EventEmitter include backendEvents for asynchronously communicating with the data storage layer, and automatic configuration file reloading. I particularly like the fact the configuration file is reloaded -- it's a good use of Node's built-in features.

James Halliday's smtp-protocol can be used to implement SMTP servers (it isn't itself an SMTP server). The server part of the module is based around a protocol parser, ServerParser -- a prototype class and a class for representing clients (Client). Servers are created using net.createServer, much like the other projects I've already mentioned.

This module is useful because it demonstrates how to separate low-level implementation details from the high-level concerns of implementing a real production-ready server. Completely different SMTP servers could be built using smtp-protocol as the foundation. Real SMTP servers need to deal with things like relaying messages, logging, and managing settings, so James has separated that out whilst retaining a useful level of functionality for his module.

I've also been reading through the telnet module, which like smtp-protocol can be used to implement a telnet server.

At the moment there seems to be a void between these reusable server modules and daemons that can be installed on production servers. Node makes asynchronous I/O more accessible, which will lead to novel server implementations like Etsy's stats server. If you've got an idea for a niche application layer server, then why not build it with Node?


tutorials node unix daemons

Unix and Node: Daemons

Posted on .

As we've seen over the last few weeks, Node is a great platform for building portable Unix programs. This makes it a useful tool for developing daemons. I often find myself dealing with daemons, whether working as a web developer, sysadmin, or general Unix developer.


Why bother making a Node process behave like a daemon? There are several reasons, but generally because it's a service that should be running at all times, in the background, and will automatically start after a reboot.

There are different ways to do this depending on the operating system. Most Linux distributions use System V compatible init scripts with runlevels. An alternative is upstart which is installed by default on Ubuntu systems.

BSD's init behaves differently -- there are no run levels. Solaris has the Service Management Facility which describes services using XML service manifests -- I often find myself writing a simple XML file that wraps around a start/stop init script.

Finally, Mac OS has launchd which is a system wide and user-oriented daemon manager. It uses XML plist files, and the documentation for these can be found in man launchd.plist.

Linux Standard Base

Guidance for writing init scripts can be found in the Linux Standard Base's section on Init Script Actions. This states that applications conforming to the standard should accept a single argument which should include one of the following:

  • start
  • stop
  • restart
  • force-reload
  • status

The status action works by returning specific exit status codes:

  • 0: The service is running correctly
  • 1: The program is dead and a /var/run pid file exists
  • 2: The program is dead and a /var/lock lock file exists
  • 3: The program is not running
  • 150 to 199 are reserved for your own requirements

Other actions are also expected to return specific status codes:

  • 1: Unspecified error
  • 2: Invalid arguments
  • 3: Unimplemented feature
  • 4: Insufficient privileges
  • 5: Program is not installed
  • 6: Program is not configured
  • 7: Program is not running
  • 150 to 199 are reserved for your own requirements

Most daemons are distributed without implementing all of these actions and status codes, but it's a good idea to be aware of them.

Node Examples

StatsD (License: MIT, npm: statsd) from Etsy is a network daemon for aggregating statistics. It comes with the files for a Debian package, and includes an upstart script:

description "statsd"  
author      "rdio"

start on startup  
stop on shutdown

    # We found $HOME is needed. Without it, we ran into problems
    export HOME="/root"

    exec sudo -u nobody /usr/share/statsd/scripts/start
end script  

Part of the installation process sets up log files with the correct permissions, and installs a configuration file to /etc/statsd. It's usually a good idea to support a system-wide configuration file that can be overridden for the purpose of distributing daemons.

The log file is created in /var/log/statsd/, which is standard as well.

This daemon loads the configuration file based on the command-line argument:

config.configFile(process.argv[2], function(config, oldConfig) {  
  // Process the configuration file

Some daemons may opt to check for configuration files in default locations, loading them in order of preference:

  • /etc/mydaemon/config.js
  • $HOME/.mydaemon/config.js

Init Module

Developing a Linux-friendly init script is made easy by the Init Node module by Paul Driver. He's actually implemented the correct status codes, and the daemon module has been used, so you essentially get a real background daemon with very little code:

var init = require('init');

  pidfile: '/var/run/myprog.pid',
  logfile: '/var/log/myprog.log',
  command: process.argv[3],
  run: function() {
    setInterval(function() {
      // Will be redirected to /var/log/myprog.log
    }, 1000);

Notice that console output will be redirected to a log file, so in this case "Running" will be printed to /var/log/myprog.log.

Wrapper Scripts

The Init module turns a Node script into a daemon, but what if your daemon needs to accept various command-line parameters rather than just start/stop/restart? In this case, most people either write a shell script to provide a System V init wrapper, or they might use an upstart script as we saw with StatsD.

Of course, other Unix systems have different init implementations, but a simple init script or init script-compatible Node module can usually be coerced into working with systems like Solaris' SMF.

You'll need basic shell scripting skills to write an init script, but there's an example of one in Fedora's packaging documentation: SysVInitScript.

Packaging Guidelines

If you want to reach a wider audience, packing guidelines often have useful tips on how to distribute daemons as well:


tutorials node unix daemons signals

Unix and Node: Signals

Posted on .

Signals represent a limited form of inter-process communication. When a signal is issued to a process, it will be interrupted and a signal handler will be executed. If there is no signal handler, the default handler will be called instead. This sounds a lot like asynchronous events in Node, and that's exactly how signals are implemented; using process.on:

process.on('SIGINT', function() {  

Typing man sigaction in a terminal will display some background on signals, including a useful list of signal names and their descriptions.

Using Signals

Create a file called signals.js that contains the following, and chmod +x it:

#!/usr/bin/env node

process.on('SIGINT', function() {  
  console.log('Got a SIGINT');

process.on('SIGHUP', function() {  
  console.log('Got a SIGHUP');

setInterval(function() {  
    console.log('Daemon running');
}, 10000);

console.log('PID:', process.pid);  

Now chmod +x signals.js and run it with ./signals.js. It'll print out its PID -- this can be used with the kill command to issue signals:

➜ ./signals.js &
PID: 12626  
➜ kill -s SIGHUP 12626
Got a SIGHUP  

Running the command with & to put it in the background will actually display the PID in most shells, but I put process.pid in there just to demonstrate how to get the PID in Node. Once you've sent a few signals to it, issuing a SIGINT will end the process.

Usage in Node

A common use of signal events in Node is to perform some kind of cleanup when a process is terminated. Where signals really come in handy is for communicating with daemons. Many daemons accept SIGHUP to reload configuration files -- I followed the same convention when I built an IRC daemon in Node. Another convention is to listen for SIGTERM to perform a graceful shutdown. This could be used to close down sockets, database connections, or remove any temporary files.

I believe Node is a great solution for writing Unix daemons. It's a good idea to follow established conventions, so add listeners for SIGHUP and SIGTERM.