GammaJS: A WebGL platformer game engine

Gamma (GitHub: Royce / GammaJS, MIT license) by Royce Townsend is a library for creating browser-based 2.5D platform games. It uses a few libraries like RequireJS, and there’s documentation that takes you through setting up Gamma.

Creating a new project with Gamma basically looks like this:

require(['gma/base', 'gma/manager'],
    function(gma) {
        // The game specification is contained within this function.
        var manager = gma.manager();
        manager.storeLevels({});
        manager.init();
    }
);

The manager object contains all the basic settings for a game, like the width, height, a canvas element, and other higher-level objects like a list of levels.

This project has a lot of documentation, so if you’re looking into building a platform game with a traditional structure this might take a lot of the drudgery out of getting a project started.

emscripten

Speaking of games, emscripten was released recently which is a LLVM-to-JavaScript compiler:

It takes LLVM bitcode (which can be generated from C/C++, using llvm-gcc or clang, or any other language that can be converted into LLVM) and compiles that into JavaScript, which can be run on the web (or anywhere else JavaScript can run).

I really enjoyed looking at the ray tracing demo. If you want to run the WebGL examples you’ll need a compatible browser.

Welcome to part 44 of Let’s Make a Framework, the ongoing series about building a JavaScript framework.

If you haven’t been following along, these articles are tagged with lmaf. The project we’re creating is called Turing.

Over the last few weeks we’ve built a test framework based on the CommonJS assert module, a suitable test runner, and started converting Turing’s tests to use it. The test framework is called Turing Test.

Patterns

This week I want to discuss the patterns we’ve used to create the framework. I realise a lot of new readers find the prospect of reading a 44 part tutorial series daunting, so I thought I’d go back and explore some of the important patterns that you can use in your own JavaScript code.

Anonymous Functions

Anonymous functions are often used as callbacks for API calls:

var a = 1;
MyAPI.method('parameter', function() {
  console.log(a);
});

Because this is a closure the a variable is visible to the anonymous function.

Another important use of anonymous functions is to control scope. This is used throughout our framework to create self-contained modules that only expose a small part of their functionality, thereby creating private scopes:

(function(global) {
  function privateFunction() {
    console.log('Hello from privateFunction');
  }

  var MyAPI = {
    method: function() {
      privateFunction();
    }
  };

  global.MyAPI = MyAPI;
})(window);

MyAPI.method();
// Hello from privateFunction

CommonJS Module Compatibility

The previous example expects window to be defined. It can be made CommonJS module compatible like this:

function getGlobal() {
  if (typeof window !== 'undefined') {
    return window;
  } else if (typeof exports !== 'undefined') {
    return exports;
  }
}

(function(global) {
  function privateFunction() {
    console.log('Hello from privateFunction');
  }

  var MyAPI = {
    method: function() {
      privateFunction();
    }
  };

  global.MyAPI = MyAPI;
})(getGlobal() || this);

Now in a CommonJS-compatible interpreter this should work:

> var MyAPI = require('./test').MyAPI;
> MyAPI.method();

Many predominantly browser-based libraries are now adding CommonJS module compatibility so people can reuse them in server-side projects (Underscore and Backbone are good examples of this).

Iterators

To get around poor iterator support in some browsers, we had to build our own enumerable module. This is all built from the each function, which jQuery defines in its core. We built each like this:

turing.enumerable = {
  Break: {},

  each: function(enumerable, callback, context) {
    try {
      if (Array.prototype.forEach && enumerable.forEach === Array.prototype.forEach) {
        enumerable.forEach(callback, context);
      } else if (turing.isNumber(enumerable.length)) {
        for (var i = 0, l = enumerable.length; i < l; i++) callback.call(enumerable, enumerable[i], i, enumerable);
      } else {
        for (var key in enumerable) {
          if (hasOwnProperty.call(enumerable, key)) callback.call(context, enumerable[key], key, enumerable);
        }
      }
    } catch(e) {
      if (e != turing.enumerable.Break) throw e;
    }

    return enumerable;
  },

  // etc.

This approach is used by many JavaScript frameworks, and works like this:

1. Array.prototype.forEach is checked to see if we can use the existing functionality based into most interpreters and browsers
2. Else if the passed-in collection has a length, use a for loop to repeatedly call each value
3. If the collection is not an Array, use a for ... in ... loop instead
4. If Break has been thrown by a callback, catch it and return the collection

Chained APIs

We’re used to chained APIs from jQuery (and now a growing number of JavaScript libraries):

$('selector')
  .attr('attr', 'value')
  .css({ 'attr', 'value' })
  .find('selector')
    .css({ 'attr', 'value' })

This is a useful technique because it cuts down the noise that a lot of callbacks would introduce. It’s expressive and easy to get the hang of.

We built this in turing.enumerable.js by defining a Chainer class that collects results, and a method that wraps each chainable call into a form that returns this after each method has been called.

Returning this and collecting results is how jQuery works. Because our module was designed to return results, the chained API style is triggered by calling chain first:

turing.enumerable.chain([1, 2, 3, 4])
      .filter(function(n) { return n % 2 == 0; })
      .map(function(n) { return n * 10; })
      .values();

This is similar to Underscore.

Prototypal Classes

I’ve used JavaScript’s native prototypal classes in Turing a lot, rather than relying on a library that adds classical object-oriented features. Prototypal classes are easy to get the hang of, and they’re a quick way to organise your code:

function MyClass() {
  this.instanceVar = 'a dull dude';
}

MyClass.classMethod = function() {
  console.log('Greetings from MyClass');
};

MyClass.prototype.method = function() {
  console.log('All work and no play makes Alex', this.instanceVar);
};

MyClass.classMethod();
var myClass = new MyClass();
myClass.method();

At first it seems strange using the function keyword to define what appears to be a class, but once you’ve got used to prototypes it becomes a powerful code organisation tool.

Prototypal Inheritance

In JavaScript 1.8.5, Mozilla introduced Object.create which can be used to inherit from objects:

function User() {
}

User.prototype.toString = function() {
  return 'User';
};

User.prototype.state = function() {
  return 'Logged in';
}

function Admin() {
}

// Some browsers won't have Object.create,
// this is a simple version
if (typeof Object.create !== 'function') {
  Object.create = function(o) {
    function F() {}
    F.prototype = o;
    return new F();
  };
}

Admin.prototype = Object.create(User.prototype);

Admin.prototype.toString = function() {
  return 'Admin';
}

var user = new User(),
    admin = new Admin();

console.log('User:', user);
console.log('Admin:', admin);
console.log(user.state());
console.log(admin.state());

This example uses Crockford’s Object.create in browsers and interpreters that don’t have Object.create. It creates two prototypal classes, User and Admin, then extends Admin’s prototype to include the methods in User.

References

• Underscore
• turing.enumerable.js
• Object.create at MDC
• Prototypal Inheritance in JavaScript by Douglas Crockford

Welcome to the Node Roundup. Send in your apps and libraries using our contact form or @dailyjs.

Tobi

Tobi (GitHub: LearnBoost / tobi, MIT license) is yet another project by TJ Holowaychuk and Guillermo Rauch from LearnBoost for server-side browser-based testing, in a similar vein to Webrat. Tobi uses jQuery and jsdom to make writing tests intuitive for us seasoned JavaScripters.

TJ’s example demonstrates using jQuery’s selectors and the kind of chained API we’re now familiar with:

var tobi = require('tobi'),
    app = require('./my/app'),
    browser = tobi.createBrowser(app);

browser.get('/login', function(res, $) {
  $('form')
    .fill({ username: 'tj', password: 'tobi' })
    .submit(function(res, $) {
      res.should.have.status(200);
      res.should.have.header('Content-Length');
      res.should.have.header('Content-Type', 'text/html; charset=utf-8');
      $('ul.messages').should.have.one('li', 'Successfully authenticated');
      browser.get('/login', function(res, $) {
        res.should.have.status(200);
        $('ul.messages').should.have.one('li', 'Already authenticated');
        // We are finished testing, close the server
        app.close();
      });
    });
});

Tobi is actually test framework agnostic, and they’ve added hooks for TJ’s should.js assertion library (used in the example). That means the syntax for writing assertions is expressive and succinct.

Because this project is test framework (or test runner) agnostic, Tobi could be used with something like Expresso to run the tests. This is an example of using should.js with Expresso:

var lib = require('mylib'),
    should = require('should');

module.exports = {
  'test .version': function() {
    lib.version.should.match(/^\d+\.\d+\.\d+$/);
  }
};

Zombie

Zombie (GitHub: assaf / zombie, MIT license) by Assaf Arkin is a similar project that simulates a browser to perform tests. Assaf’s examples use the CommonJS assertion module:

var zombie = require("zombie");

// Load the page from localhost
zombie.visit('http://localhost:3000/', function(err, browser) {

  // Fill email, password and submit form
  browser.
    fill('email', 'zombie@underworld.dead').
    fill('password', 'eat-the-living').
    pressButton('Sign Me Up!', function(err, browser) {
      // Form submitted, new page loaded.
      assert.equal(browser.text('title'), 'Welcome To Brains Depot');
    });
});

These tests could also be run with a suitable test runner.

The author claims this library is insanely fast, but I haven’t verified its performance. Performance is important for testing because people are less likely to bother running them if they take too long to run, so it’s probably worth trying both Zombie and Tobi out to see how they fare compared to your current functional/integration/acceptance testing solutions.

Node-validator

node-validator (License) by Chris O’Hara is a library for string validation, filtering and sanitization.

It can be used in a browser or Node, but the really interesting thing about this library is the author’s example of using it to validate requests in a Node app. There’s a gist that makes the following possible:

get('/', function(req, res) {
    req.onValidationError(function() {
        // Redirect the user...
    });

    // Validate user input
    req.check('zip', 'Please enter a valid ZIP code').len(4,5).isInt();
    req.check('email', 'Please enter a valid email').len(6,64).isEmail();
    req.checkHeader('referer').contains('localhost');

    // Sanitize user input
    req.sanitize('textarea').xss();
    req.sanitize('foo').toBoolean();

    // etc.
});

Now requests to a URL with parameters like http://localhost:8080/?zip=12345&foo=1&textarea=large_string will be validated. This could be very useful for sanity checking in an Express app, for example.

Welcome to the jQuery roundup 38. You can send your plugins and articles in for review through our contact form or @dailyjs.

jQuery Community Updates for December

Addy Osmani has posted the jQuery Community Updates for December to the jQuery blog. There are details on preparations for jQuery 1.5, which features a major overhaul of the jQuery.ajax code and API.

jQuery BBQ

jQuery BBQ (MIT and GPL licenses) by Ben Alman helps manage bookmarkable #hash history. There’s a “project page”, unit tests, and lots of examples.

This plugin can cope with multiple history states per page, as demonstrated in the advanced example.

jQuery History Plugin

jQuery History Plugin by Mikage Sawatari and Takayuki Miwa is another hashchange event-based history plugin. The API is slightly different to jQuery BBQ, using the $.history.init() and $.history.load() methods to work with history.

blockUI

blockUI (GitHub: malsup / blockui) by Mike Alsup allows you to block interactions with a page until asynchronous events have completed. It fades the page and shows a message, but it also sets the mouse cursor to a suitable busy icon.

This example makes a nice rounded-corner style display:

$(document).ready(function() { 
  $('#demo2').click(function() { 
    $.blockUI({ css: { 
      border: 'none', 
      padding: '15px', 
      backgroundColor: '#000', 
      '-webkit-border-radius': '10px', 
      '-moz-border-radius': '10px', 
      opacity: .5, 
      color: '#fff' 
    } }); 

    setTimeout($.unblockUI, 2000); 
  });
});

Welcome to part 7 of Let’s Make a Web App, a tutorial series about building a web app with Node. This series will walk you through the major areas you’ll need to face when building your own applications. These tutorials are tagged with lmawa.

Previous tutorials:

• Part 1: Introduction
• Part 2: Installation and Skeleton App, source code commit: 4ea936b
• Part 3: RESTful Methods and Testing, source code commit: 39e66cb
• Part 4: Templates, Partials, Creating and Editing Documents, source code commit: f66fdb
• Part 5: Authentication, Sessions, Access Control Middleware, source code commit: 03fe9b
• Part 6: Interface Basics, source code commit: f2261c

Package Versions

I’ve updated the Nodepad README to include the versions of Node and Mongo that I’m using. It also includes the versions of the packages I’ve used. That should help you actually get the code running if it doesn’t seem to work. I’ve tested it on Mac OS and Debian.

Also remember that you need to restart Node whenever you change code (but not Jade templates).

We’re using npm to install packages, and it sets up path names so specific package versions can be required. To install a package at a specific version, do this:

npm install express@1.0.0

Then to use it, do this:

var express = require('express@1.0.0');

You can verify this works by typing node and entering the previous line:

> express = require('express@1.0.0')
{ version: '1.0.0'
, Server: { [Function: Server] parseQueryString: [Function] }
, createServer: [Function]
}

Jade Tricks

When I first demonstrated Jade I hardcoded all the attributes. It’s possible to save a lot of effort by writing selectors as a shorthand for classes and IDs:

div#left.outline-view
  div#DocumentTitles
    ul#document-list
      - for (var d in documents)
        li
          a(id='document-title-' + documents[d].id, href='/documents/' + documents[d].id)
            =documents[d].title

Notice that an ID selector has been combined with a class name: div#left.outline-view.

The default tag is div, which means the previous example can be reduced to this:

#left.outline-view
  #DocumentTitles
    ul#document-list
      - for (var d in documents)
        li
          a(id='document-title-' + documents[d].id, href='/documents/' + documents[d].id)
            =documents[d].title

Error Pages

Express allows us to define a custom error handler with app.error:

// Error handling
function NotFound(msg) {
  this.name = 'NotFound';
  Error.call(this, msg);
  Error.captureStackTrace(this, arguments.callee);
}

sys.inherits(NotFound, Error);

// This method will result in 500.jade being rendered
app.get('/bad', function(req, res) {
  unknownMethod();
});

app.error(function(err, req, res, next) {
  if (err instanceof NotFound) {
    res.render('404.jade', { status: 404 });
  } else {
    next(err);
  }
});

app.error(function(err, req, res) {
  res.render('500.jade', {
    status: 500,
    locals: {
      error: err
    } 
  });
});

Error handlers get four parameters, the error, req, res, and next. The next method can be used to pass the error on to the next handler. In the previous example, the 404 handler passes on errors that aren’t NotFound, and we effectively catch all other errors and consider them 500s.

Visiting /bad in the browser will display the custom 500 page. Notice that I specify the HTTP status code in the options for render — it’s important to correctly specify status codes else a 200 will be returned instead of a 404 or 500.

Error Handling within Mongoose Code

The next method is available to all of our app HTTP verb methods, which means we can use it to render a custom 404 page:

app.get('/documents/:id.:format?/edit', loadUser, function(req, res, next) {
  Document.findById(req.params.id, function(d) {
    if (!d) return next(new NotFound('Document not found'));
    // Else render the template...

This is the pattern that I found easiest to read when using Mongoose. Simply using throw new NotFound within a Mongoose callback will cause the application to crash rather than Express’s error handlers being triggered.

Conclusion

When distributing or deploying Node applications it’s important to be specific about package versions. Many key packages are still under heavy development, so distribution in particular can be difficult.

Express makes it fairly easy to create custom error handlers with templates, but make sure you specify the HTTP status code and use next(exception) when in a callback.

This version of Nodepad was in commit 929f564.

Crafty

Crafty (dual MIT and GPL license) is a JavaScript game engine by Louis Stow with a jQuery-inspired API.

Crafty.init(50, 580, 225);

var player = Crafty.e('2D, DOM, gravity, controls, twoway, image')
	.attr({x: Crafty.viewport.width / 2, w: 16, h: 16})
	.gravity('floor')
	.twoway(2)
	.image('favicon.png', 'no-repeat');

var floor = Crafty.e('2D, floor')
	.attr({x:0, y: 225, w: 580, h: 5});

As you can see, the API uses chained calls like jQuery. At the moment Crafty is an early release (version 0.1), but the Crafty blog has a post about upcoming features.

The author is keen to point out that the project is a game framework, rather than an engine. Typically game engines are less general than the types of frameworks we’re used to working with as web developers, so this probably suits our way of working (and should suit many types of game).

Opera Widgets for Android

Opera Labs has announced an Android runtime that can run WAC widgets (Wholesale Applications Community): WAC 1.0 – Supporting runtime for Android: Installation instructions and release notes. This means Android devices can now run a type of widget that’s designed around open, standardised specifications.

More details on about WAC Widgets can be found in: Introduction to Widget Handset APIs in WAC 1.0.

It’s not just a superficial cross platform library, however:

Using these new APIs we can take pictures, check GPS location, browse our address books, access accelerometer information, and even send SMS, MMS and E-Mail messages

Exceptional and Node

Exceptional, a service that manages exception notifications from web applications, now supports Node. In Exceptional and node.js, the developers announced a new library called exceptional-node that makes exception reporting a breeze:

var Exceptional = require(./'exceptional');
Exceptional.API_KEY = **YOUR-API-KEY**

process.addListener('uncaughtException', function(err) {
  Exceptional.handle(err);
});

try {
  // Your Code
} catch(error) {
  // Your own error processing
  Exceptional.handle(error);
}

It has a package.json file so you should be able to install it with npm. It’s released under the MIT license.

Welcome to part 43 of Let’s Make a Framework, the ongoing series about building a JavaScript framework.

If you haven’t been following along, these articles are tagged with lmaf. The project we’re creating is called Turing.

Over the last few weeks we’ve built a test framework based on the CommonJS assert module, a suitable test runner, and started converting Turing’s tests to use it. The test framework is called Turing Test.

JavaScript Testing

I get a lot of questions about JavaScript tests, so before continuing converting Turing’s tests I’m going to just explain a little bit about the basics behind JavaScript testing.

If you’re primarily a front-end developer, a lot of the innovative new work in JavaScript testing might appear confusing. The needs of server-side testing and client-side diverge, so it’s perfectly acceptable to use a different testing framework for each part of your project.

According to our 2010 JavaScript survey, Qunit is the most popular test framework. So you could use Qunit for your browser tests, and the server-side developers could use something like nodeunit or jasmine.

There are different types of test libraries. The one we’ve been developing on this tutorial series is a unit testing framework. The assert module defined in Unit Testing/1.0 by CommonJS is an example of a large part of a unit testing framework.

Libraries like Jasmine are Behavior Driven Development (BDD) libraries. Some frameworks like Cucumber take this to another extreme, where tests are written as executable plain-text.

There are more flavours of test libraries, and some become fashionable and widely used. Because tests can be useful documentation for another developer, if you’re writing an open source project it might be a good idea to use what your community uses. If you’re writing a jQuery plugin, why not use Qunit?

If you’re working on your own project or in a small team, the choice of test libraries can seem bewildering. You could fall back on the CommonJS modules and treat that as the “standard” way to write tests, as it’s likely that other people will be able to understand your tests. The point is to actually write tests in a way that makes you feel productive — some people get bogged down with the abstraction of BDD libraries, other people find that style of testing can help communicate business logic to clients or managers.

Test Conversion

I’ve converted the core tests to Turing Test (from RiotJS).

The enumerable tests demonstrated where using assert.equal or assert.deepEqual are appropriate:

exports.testEnumerable = {
  'test array iteration with each': function() {
    var a = [1, 2, 3, 4, 5],
        count = 0;
    turing.enumerable.each(a, function(n) { count += 1; });
    assert.equal(5, count, 'count should have been iterated');
  },

  'test array iteration with map': function() {
    var a = [1, 2, 3, 4, 5],
        b = turing.enumerable.map(a, function(n) { return n + 1; });
    assert.deepEqual([2, 3, 4, 5, 6], b, 'map should iterate');
  },

The second test here need deepEqual to check the arrays are the same. It’s important to remember which type of equal to use when writing CommonJS assertions.

The OO tests had to be reworked slightly to suit CommonJS module constraints in the browser.

Running All Tests

I’ve made run.html run all of the tests in the browser. It was actually very easy to do, I just had to list all of the unit test files in script tags:

<script src="events_test.js" type="text/javascript"></script>
<script src="alias_test.js" type="text/javascript"></script>
<script src="anim_test.js" type="text/javascript"></script>
<script src="dom_test.js" type="text/javascript"></script>
<script src="net_test.js" type="text/javascript"></script>
<script src="oo_test.js" type="text/javascript"></script>

Future Updates

When I started writing the Turing Test tutorials I mentioned I’d like to be able to run all tests, in a similar way to Qunit. This kind of works now, but isn’t presented particularly well. The test runner we’ve built doesn’t summarise all of the tests, and the layout isn’t quite clear enough.

Turing Test also needs to display benchmarks, and according to our survey 12% of people benchmarking code use unit tests as an indicator of performance.

I’m pleased with the results now everything has been converted to CommonJS tests. As usual, more next week!

References

• 2010 JavaScript survey
• Qunit
• nodeunit
• jasmine
• Unit testing
• Behavior Driven Development
• Unit Testing/1.0

Welcome to the Node Roundup. Send in your apps and libraries using our contact form or @dailyjs.

Node 0.3.2 Released

Node 0.3.2 has been released. I’ve annotated the release notes with links to the relevant documentation:

• V8 3.0.3
• New TLS implementation
• New os and tty modules
• Callback parameters for stream.write and stream.connect
• DNS module now supports CNAME lookups
• cmake support
• make lint
• oprofile support, with ./configure --oprofile
• Bug fixes, including fixes for ChildProcess:Spawn memory leak, global variable leaks
• Deprecated process.compile, process.ENV (Process API)

Kod

Kod by Rasmus Andersson is an editor for Mac OS that uses Node as the scripting language. The interface looks and behaves a lot like Google’s Chrome, and the editor and syntax highlighting are styled with CSS. It’s designed to be concurrent so the app should feel fast, and it’s built using Objective-C.

If you’re a Mac-based JavaScript developer, Kod might suit you. If you want to try it, register for a beta invite and take a look at this discussion: Kod Beta thread.

Node, USB, and Kinect

Schakko, who writes node-usb, has added node-usb-kinect.js which talks to Microsoft’s Kinect. So if you get one for Christmas and quickly tire of the boring arm-waving games, maybe you could hack it with Node!

Welcome to the jQuery roundup 37. You can send your plugins and articles in for review through our contact form or @dailyjs.

FullCalendar

FullCalendar (GitHub: arshaw / fullcalendar, GPL/MIT Dual License) by Adam Shaw is an incredibly elaborate jQuery calendar plugin.

It reminds me of Google’s calendar, but he’s also got a jQuery UI themed demo which shows a calendar with a theme. It doesn’t support editing, but it would be possible to add support using the events, which are listed in FullCalendar’s documentation.

Even better, the project comes with tests and a Makefile! It’s not often I come across jQuery plugins that are packaged this well, believe me.

Capty

Capty (GitHub: wbotelhos / capty, MIT License) by Washington Botelho is a little plugin that displays a caption over an image based on the alt attribute:

$('#capty').capty();

<img id="capty" src="image.jpg" alt="Example" />

If the image has a name attribute with a selector, it’ll use the selector for the caption’s content (see the A custom caption with a external content example).

The author has more interesting plugins on his GitHub account, wbotelhos. In particular the Wizard plugin caught my eye, which I thought was something to do with incredible magical powers, but it’s actually just for making multi-stage form wizards.

Absolution Updates

Michaël Vanderheeren has been working on his Absolution jQuery UI theme. This theme uses CSS3, and has some interesting demos:

• Registration form demo
• Equalizer demo
• File, progress bars, and tabs

He’s been making a lot of small tweaks lately so it uses more CSS3 and looks cleaner than before.

This is one of the themes I previously covered in a general jQuery UI roundup.

Welcome to part 6 of Let’s Make a Web App, a tutorial series about building a web app with Node. This series will walk you through the major areas you’ll need to face when building your own applications. These tutorials are tagged with lmawa.

Previous tutorials:

• Part 1: Introduction
• Part 2: Installation and Skeleton App, source code commit: 4ea936b
• Part 3: RESTful Methods and Testing, source code commit: 39e66cb
• Part 4: Templates, Partials, Creating and Editing Documents, source code commit: f66fdb
• Part 5: Authentication, Sessions, Access Control Middleware, source code commit: 03fe9b

Before starting this tutorial remember to start up a mongo daemon if your computer doesn’t run one automatically.

In the last part we looked at authentication and sessions. We used the particularly cool concept of middleware to build some seamless access control. This week I’m going to demonstrate how to make the interface more interesting using jQuery.

Interface Design

When I design interfaces I usually plan out a rough idea before developing the app. This is usually known as top-down design. Once the interface seems workable, I move on to develop the API with a simple interface and tests, just like we’ve been doing so far in this tutorial series.

I like to sketch out interfaces with graph paper, a mechanical pencil, and a good eraser. I keep the sketches rough so my colleagues know they’re not finished designs, they’re ideas that will evolve as we discuss the project.

A simple sketch of Nodepad suggested the following:

• The interface could use two panels like a desktop app, one for the list of notes and the other for the note body
• A save button could appear at the bottom, but an auto-save feature would be nice too
• Single-clicking a note loads it, double-click allows the title to be edited
• We need account settings for email/password change
• Editing could work with XMLHttpRequest, because we already have JSON support

Cheating

One of the most important things when building interfaces is to cheat as much as possible. When I’m programming I don’t spend hours working on code that’s already available in a well-written library, and the same can be said for design. I don’t need to draw every icon I need, and I don’t need to reinvent the wheel for layouts either.

Today there are a lot of solutions available, from CSS frameworks to GUI-heavy projects like Cappuccino. For Nodepad we’re going to use jQuery UI which sits in the middle. It gives us a lot of bang for buck when combined with a solid theme.

For the theme, I’ve decided to use Aristo (demo). It’s not necessarily the best theme out there, but I’ve got a lot of experience using it and I think it looks great.

Including Aristo and jQuery UI

I’ve downloaded Aristo from GitHub and put it in public/stylesheets/aristo. Then we just need to load jQuery UI and the new stylesheet in the views/layout.jade template:

link(rel='stylesheet', href='/stylesheets/aristo/jquery-ui-1.8.5.custom.css')
script(type='text/javascript', src='https://ajax.googleapis.com/ajax/libs/jqueryui/1.8.7/jquery-ui.min.js')

Page Structure

Our interface needs two columns, a header, a document body for editing, and some buttons for document management. In Jade it might look like this:

div(class='outline-view', id='left')
  div(id='DocumentTitles')
    ul(id='document-list')
      - for (var d in documents)
        li
          a(id='document-title-' + documents[d].id, href='/documents/' + documents[d].id)
            =documents[d].title

  ul(class='toolbar')
    li
      a(href='/documents/new')
        +
    li
      a(href='#', id='delete-document')
        -

div(class='content-divider')

div(class='content')
  div(id='editor-container')
    textarea(name='d[data]', id='editor')
      =d.data || ''

ul(id='controls',class='toolbar')
  li
    a(href='#', id='save-button') Save

The first part, outline-view is a div that contains the document list. This is the code we’ve seen before. I’ve positioned these with absolute and I’m using some JavaScript on resize and focus to resize the document list and button toolbars.

The selected document is displayed using some CSS gradients:

.outline-view ul .selected {
  color: #fff;
  background-color: #8897ba;
  background: -webkit-gradient(linear, left top, left bottom, from(#b2bed7), to(#8897ba));
  background: -moz-linear-gradient(top,  #b2bed7,  #8897ba);
}

It doesn’t matter if browsers don’t support CSS3 gradients, it’ll look fine with just the darker colour.

Selecting Documents

Recall that our API requires .json to be appended to the URL to get JSON data back. We just need some simple jQuery event handling to load the document from the server:

$('#document-list li a').live('click', function(e) {
  var li = $(this);

  $.get(this.href + '.json', function(data) {
    $('#document-list .selected').removeClass('selected');
    li.addClass('selected');
    $('#editor').val(data.data);
    $('#editor').focus();
  });

  e.preventDefault();
});

This binds a click handler that will fire whenever a document title is clicked. It fills out the textarea with the data value from the JSON response. Rather than a normal event, live is used to set up a delegate in case the document list changes.

Saving Documents

Something I like to do when creating dynamic interfaces is to set id attributes based on database IDs, using a simple naming convention. In Nodepad I’ve used DOM ids like this: document-844ce17994ba1b87d359000001. To retrieve the database ID we just need to get the last part, which is easy if we split on the hyphens.

Once that convention is established, we can use a mini jQuery plugin to extract item IDs:

$.fn.itemID = function() {
  try {
    var items = $(this).attr('id').split('-');
    return items[items.length - 1];
  } catch (exception) {
    return null;
  }
};

This makes saving documents when the save button is pressed straightforward enough:

$('#save-button').click(function() {
  var id = $('#document-list .selected').itemID(),
      params = { d: { data: $('#editor').val(), id: id } };
  $.put('/documents/' + id + '.json', params, function(data) {
    // Saved, will return JSON
  });

There isn’t actually a HTTP verb for put in jQuery, so I defined one:

$.put = function(url, data, success) {
  data._method = 'PUT';
  $.post(url, data, success, 'json');
};

Progress

We haven’t actually used any jQuery UI so far, but we’ll get to that in the next part. So far we’ve got a simple little editor that’s starting to look like something more friendly and usable.

I haven’t had time to browser test yet, so stick to WebKit or Firefox.

The latest Nodepad commit was dd40b.

Update: I’ve changed the Nodepad README to include more details on library, binary and operating systems that I’ve tested it against. This may help people who’ve been having trouble running it.

Distributed Mandelbrot Generation

When I was at university my dissertation was on distributed computing with arbitrary-precision arithmetic. Which basically meant I used an entire lab to generate pretty pictures with very large numbers.

So when I found mandlebrotapp.appspot.com I got all misty-eyed and nostalgic. It uses browsers to generate sections of the Mandelbrot set. It has a basic job management system so you can submit new jobs, and it’ll use different colours during rendering so you can see which blocks other people generated.

Given the news of recent password leaks from major sites, if anyone has the guile to create a rainbow table hash cracker using JavaScript delivered over ad networks the Internet will probably fall apart. Don’t do that! Unless it’s purely for research purposes, of course.

What to Read to Get Up to Speed in JavaScript

What to Read to Get Up to Speed in JavaScript by Rey Bango is list of books and blogs for beginner, intermediate, and experienced JavaScript developers. It mentions the classics, like JavaScript: The Good Parts, and even includes DailyJS (thanks!)

1k Christmas Tree

More animated weather effects. This is becoming a DailyJS meme now. This animated Christmas tree by @romancortes uses Canvas to render a rotating 3D tree, complete with falling snow.

Mobile Web Application Best Practices

Mobile Web Application Best Practices is a W3C Recommendation published on 14th December, 2010. It consists of a list of 32 best practices, with explanations for each point:

Use Cookies Sparingly

Cookies are a common and effective means to store small amounts of state on the client. […] Information stored in cookies, however, is sent to the server for every request and so using them for excessive amounts of data can negatively impact performance, particularly on a mobile network.

I actually think this would be a great resources for developers new to mobile web development, and even useful as a sanity check for more experienced developers and designers.

Welcome to part 42 of Let’s Make a Framework, the ongoing series about building a JavaScript framework.

If you haven’t been following along, these articles are tagged with lmaf. The project we’re creating is called Turing.

In this part I’ll discuss using our new CommonJS-based testing framework, turing-test.js, to test our framework.

Animation Tests

The old animation tests were fairly basic. Once I’d rewritten them I noticed something interesting:

exports.testAnimations = {
  'test hex colour converts to RGB': function() {
    assert.equal('rgb(255, 0, 255)', turing.anim.parseColour('#ff00ff').toString());
  },

  'test RGB colours are left alone': function() {
    assert.equal('rgb(255, 255, 255)', turing.anim.parseColour('rgb(255, 255, 255)').toString());
  },

  'test chained animations': function() {
    var box = document.getElementById('box');
    turing.anim.chain(box)
      .highlight()
      .pause(250)
      .move(100, { x: '100px', y: '100px', easing: 'ease-in-out' })
      .animate(250, { width: '1000px' })
      .fadeOut(250)
      .pause(250)
      .fadeIn(250)
      .animate(250, { width: '20px' });

    // New:
    setTimeout(function() { assert.equal(box.style.top, '100px'); }, 350);
    setTimeout(function() { assert.equal(box.style.width, '20px'); }, 2000);
  }
};

The chained calls at the end didn’t have any tests before, so I tried adding some with setTimeout to test a few steps that would be executed as part of the chain. Another way to test time-based events like this would be to put the asserts inside any callbacks the API supports.

With the current version of the test library, exceptions will be logged for the failed tests rather than printing them to the test reports. There’d need to be a mechanism for waiting for asserts in setTimeout to handle this correctly (maybe an async test helper).

DOM Tests

The DOM tests can be pretty much translated directly from the original tests:

exports.testDOM = {
  'test tokenization': function() {
    assert.equal('class', turing.dom.tokenize('.link').finders(), 'return class for .link');
    assert.equal('name and class', turing.dom.tokenize('a.link').finders(), 'return class and name for a.link');
  },

  'test selector finders': function() {
    assert.equal('dom-test', turing.dom.get('#dom-test')[0].id, 'find with id');
    assert.equal('dom-test', turing.dom.get('div#dom-test')[0].id, 'find with id and name');
    assert.equal('Example Link', turing.dom.get('a')[0].innerHTML, 'find with tag name');

// etc.

It’s a good idea to have a message for each assertion to make it easier to track down failing tests.

Events Tests

The events tests look a little bit different because my other testing library will run functions when they’re passed:

given('a delegate handler', function() {
  var clicks = 0;
  turing.events.delegate(document, '#events-test a', 'click', function(e) {
    clicks++;
  });

  should('run the handler when the right selector is matched', function() {
    turing.events.fire(turing.dom.get('#events-test a')[0], 'click');
    return clicks;
  }).equals(1);

  should('only run when expected', function() {
    turing.events.fire(turing.dom.get('p')[0], 'click');
    return clicks;
  }).equals(1);
});

This is the old test which binds a counter, clicks, then triggers events with handlers that change clicks. Because assert.equal won’t run functions, it needs to be rewritten like this:

exports.testEvents = {
  'test delegate handlers': function() {
    var clicks = 0;
    turing.events.delegate(document, '#events-test a', 'click', function(e) {
      clicks++;
    });

    turing.events.fire(turing.dom.get('#events-test a')[0], 'click');
    assert.equal(1, clicks, 'run the handler when the right selector is matched');

    turing.events.fire(turing.dom.get('p')[0], 'click');
    assert.equal(1, clicks, 'run the handler when the right selector is matched');
  }
};

Being acutely aware of closure binding is the key to testing events. I’ve also ported the other event tests and they run fine with the assertion module.

Conclusion

These browser-based tests were straightforward to port from the BDD-style library, Riot.js, to CommonJS. So far it’s only really been a cosmetic, syntax shift.

I feel like these CommonJS-based tests read well, but I’ve worked with unit testing libraries more intensely than more modern BDD libraries. There’s no reason why BDD libraries can’t be built on CommonJS assertions though, which is what many open source developers have been working on.

This code is available in commit c3175f.

Welcome to the Node Roundup. Send in your apps and libraries using our contact form or @dailyjs.

Onvalid

Onvalid (License) by Mark Doffman is a validation library for Node (but should work elsewhere). The Onvalid documentation is quite detailed and contains a lot of examples. The suggested usage looks like this:

var __ = Onvalid = require('./onvalid');

var schema = {
  name: __.exists
};

var user = {
  'name': 'Bob'
};

if (Onvalid.validate(user, schema)) {
  console.log('OK');
}

Error messages are supplied to a callback as the third parameter of the validate method.

I like the idea of using decoupled validation libraries, a library like this would work well with our Mongoose-based Nodepad tutorial project.

node-rlog

node-rlog (Apache License 2.0) by J. Brisbin is a distributed logging solution for Riak and Node in the form of a library and command-line utility. The Node usage is very simple:

var rlog = require('node-rlog'),
    log = rlog.getLogger('bucketname');
log.debug('This is a debug line: ', myobj, myobj2);

Logs can then be viewed with the noderlog utility:

noderlog -b bucketname

Install it with npm from the GitHub repository:

git clone https://github.com/jbrisbin/node-rlog.git
npm install ./node-rlog

Wave-Node

Wave-Node (Apache License 2.0) by scottbw is a Node implementation of the Google Wave Gadget API. It uses WebSockets, which I thought was interesting because I’m fairly sure the last time I used Wave it polled aggressively with XMLHttpRequests.

Apparently it can be integrated with the Apache Wookie W3C Widget server.

If you want to try out the server and examples, you’ll need to check the code out from GitHub and install redis.

Welcome to the jQuery roundup 36. You can send your plugins and articles in for review through our contact form or @dailyjs.

jQuery.highlightRegex

jQuery.highlightRegex (MIT License, demo) by Johann Burkard highlights text based on a regular expression:

$('selector').highlightRegex(/text/ig);

It’s the kind of thing I could imagine being useful in a web application that deals with a lot of text (like a notepad, ereader, etc.)

jQuery Weather Widget

The jQuery weather widget 1.0 (Creative Commons Attribution) by Seth Wright displays a nice little weather widget based on data from an XML file. The author uses PHP to parse the XML, but I thought a possible extension to the project would be to use jQuery.find to read the XML from a JSONP weather API (if there is one, if not a server-side component will be required).

Booklet

Booklet (dual MIT and GPL License) is a plugin by Will Grauvogel that displays content in a flip-book layout. There’s a tutorial that uses this plugin to create an impressive Moleskine-themed page: Moleskine Notebook with jQuery Booklet (demo).

The Booklet API is very simple:

$(function() {
  $('#mybook').booklet({
    width:  600,
    height: 200
  });
});

The JavaScript Developer Survey results are in! The results are available here: JavaScript Developer Survey Results, and the raw data is here: JavaScript Survey Raw Data.

1218 people responded, which is up from 151 last year.

Analysis

Note that many of the questions allowed multiple responses, which is why the percentages might not add up to 100%.

Audience Background

98% of respondents say they work with JavaScript in the browser, and 35% are working on server-side code. 3% said they’re working with WebOS, which may see more interest from our community since WebOS 2.0 has Node support for services. 11% are working on “other” server-based code, which might indicate the growing interesting in Node for network daemons.

Testing and Debugging

66% of respondents do not test their JavaScript (down from 68% last year). There were 6 entries for nodeunit and 10 for Expresso. JsTestDriver got some love with 15 people saying they use it. Two people selected Selenium as well. There are a lot of other responses in this category, I recommend scrolling through the raw data if you’re interested in testing.

JQUnit/QUnit (I’ll change this to QUnit next year) was the most popular test library, with 34%. Jasmine is also pretty popular, with 18%.

Firebug is the most popular web-oriented debugger, at 78%. Second was WebKit, with 49%. As I’d expect people are using multiple tools to debug their browser-based code, but given the popularity of Chrome and Safari I thought Firebug’s lead might have been less far ahead.

Firebug is also the most popular benchmarking tool (71%). 12% benchmark based on unit tests.

Static Analysis, Packaging

A huge amount of people are using JSLint, with a 95% share of those who use some form of static analysis.

As for minifying code, 50% of people use YUI Compressor, and 41% use Closure Compiler. JSMIN seems to be losing popularity, with 25%.

Hosting and Collaboration

75% of respondents use GitHub to find code, which beats search engines at 68%. 38% use news sites like reddit and Hacker News. GitHub is also the preferred hosting service, with 86%. Launchpad, Codaset, and Codebase have a handful of votes each.

Preferred Interpreter

Those of us who work with server-side JavaScript said we prefer Node as our interpreter, with 78%. Second was Rhino, with 18%.

Other Languages

PHP is the most popular language among our readers, at 46%. Ruby is second at 33%, and Java is third at 25%. Python is ahead of the C-family of languages, including Objective-C, at 24%.

Conclusion

Thanks for taking part in the survey, we got a fascinating cross-section of the JavaScript community.

I don’t think it’s too surprising that only half of respondents test their JavaScript, because it does seem like extra hard work for frontend developers. If you’re a frontend developer who’s confused about how and what to test, I could write up more tutorials in that area. Let me know via the comments.

The popularity of PHP surprised me, but it’s possible that this survey may have been featured on prominent PHP blogs. However, both JavaScript and PHP are primarily web languages so it does make sense. I recommend thinking outside of your comfort zone a little bit — if you’re a web developer there’s no reason you can’t pick up a bit of Objective-C for iPhone, Java for Android, or write some TCP/IP daemons in Node.

Opera Dragonfly and Microsoft’s JavaScript debugger tools didn’t get much interest. Why not try those out too? You might learn something interesting!

The strong interest in Node leads me to believe that we’ll see a lot of startups working with Node or providing services for the Node community in the near future. If you’re working in this area why not meet up with us for beers to discuss your next project?

JavaScript Hits #1 On GitHub

JavaScript is now the most popular language on GitHub. It only seems natural considering the popularity of web applications — have you worked on a web application that doesn’t use JavaScript somewhere? But it’s not just projects like jQuery at the top, key server-side projects are in the most watched overall category, like Express.

BitmapData.js

BitmapData.js (GitHub: BitmapData.js) by Peter Nitsch is a Canvas implementation of the AS3 BitmapData class. I haven’t used much AS3 myself, but the examples look pretty good:

• Perlin Noise
• Animated Plasma
• Blendmodes

This is another useful library for ActionScript developers who are moving into Canvas-based development.

WebKit Clock

WebKit Clock by Ono Takehiko uses no images (apart from the social network sharing buttons) to generate an amazing world clock. There are delicious animations for switching between cities, and an Apple Dashboard-style clock. It uses a lot of box-shadow, border-radius, and a Canvas for the map.

I was particularly impressed with the map animation. If you’re looking for more inspiration, the author has dozens of other interesting projects on onotakehiko.com.

Welcome to part 41 of Let’s Make a Framework, the ongoing series about building a JavaScript framework.

If you haven’t been following along, these articles are tagged with lmaf. The project we’re creating is called Turing.

The last few parts have been concerned with building a test framework based on CommonJS:

• Part 36 – An introduction to how JavaScript frameworks handle testing
• Part 37 – A review of CommonJS test specifications, and examples of how to run assert module tests in a browser with a simple test runner
• Part 38 and commit 9a4c33e – Assertions part 1
• Part 39 and commit 8dd48b5 – Assertions part 2
• Part 40 and commit 5a6cbf61 – Test runner

In this part I’ll show you how to use external libraries through git submodules, and how to simulate CommonJS modules in a browser (in a limited fashion).

Submodules

Writing good reusable code isn’t just about well-written code, it also involves project management. I don’t mean a pointy-haired boss telling you what to do, just practical techniques for managing projects and sharing code between them.

I’ve already hit on the issues around packaging JavaScript frameworks, and we’ve looked at libraries like jQuery and Prototype to see how they package their code. Now that turing-test.js is in a usable state we need a way of distributing it with Turing, without having to manually update it every time we make changes.

One way Real Open Source Projects™ do this is using git submodules. From Pro Git:

It often happens that while working on one project, you need to use another project from within it. Perhaps it’s a library that a third party developed or that you’re developing separately and using in multiple parent projects.

Adding Submodules

Adding a submodule is easy:

git submodule add git@github.com:alexyoung/turing-test.js.git turing-test

Getting and Updating Submodules

Git can’t do everything, so it’s important to communicate the fact we’re using submodules in our README. I’ve put a note in Turing’s README with instructions on how to get the required submodules. Of course, users don’t really need to worry about this, it’s only for people who want to run tests.

If you check Turing out from Git, you’ll need to do this:

git submodule init
git submodule update

If the submodule has been updated, you’ll need to run git submodule update to get the latest version.

Integrating Turing Test

Now we have a process for sharing Turing-Test with Turing, we need to actually use the library to test something. For now I’ve put the test library submodule in test/turing-test/ because it makes the path handling easier between Node and the web-based file tests.

The goal of Turing Test was to make tests use CommonJS assertions and a CommonJS test runner, which could work in a browser and server-side JavaScript. We’ve got most of what we need so far, except for one thing:

var module = require('library').property;

Supporting modules in browsers isn’t easy because it’s incredibly awkward to block execution until something has loaded. Turing Test addressed this initially by mocking out require and forcing the user to load scripts with script tags in a HTML stub.

This is a simple solution, but can we go even further?

Giving Browsers CommonJS Module Support

I decided to see how close I could get a browser-based test to resemble a CommonJS test. This will run using Turing Test in both environments:

require.paths.unshift('./turing-test/lib');

turing = require('../turing.core.js').turing;
var test = require('test'),
    assert = require('assert'),
    $t = require('../turing.alias.js');

exports.testAlias = {
  'test turing is present': function() {
    assert.ok(turing, 'turing.core should have loaded');
  },

  'test alias exists': function() {
    assert.ok($t, 'the $t alias should be available');
  }
};

test.run(exports);

This test, alias_test.js will work in both a browser and Node. How? Well, the browser has a HTML test harness file that performs some setup before the test is run:

<html>
  <head>
    <title>Alias Test</title>

    <!-- Test Library -->
    <link rel="stylesheet" href="turing-test/stylesheets/screen.css"></link>
    <script src="turing-test/turing-test.js" type="text/javascript"></script>
  </head>
  <body>
    <ul id="results"></ul>
    <script src="alias_test.js" type="text/javascript"></script>
  </body>
</html>

The file turing-test.js referenced here is where some browser patching occurs to provide require. Now require in the browser loads scripts by inserting a script tag and watching for the script to finish loading. Certain parts of the library block with setTimeout until the scripts have finished loading.

function addEvent(obj, type, fn)  {
  if (obj.attachEvent) {
    obj['e' + type + fn] = fn;
    obj[type + fn] = function(){obj['e' + type + fn](window.event);}
    obj.attachEvent('on' + type, obj[type + fn]);
  } else
    obj.addEventListener(type, fn, false);
}

var scriptTag = document.createElement('script'),
    head = document.getElementsByTagName('head');
this.loading();
addEvent(scriptTag, 'load', tt.doneLoading);
scriptTag.setAttribute('type', 'text/javascript');
scriptTag.setAttribute('src', script);
head[0].insertBefore(scriptTag, head.firstChild);

The first call to require causes Turing Test to install some browser-patching code. The final call to test.run at the end of the tests will actually hit a fake test runner that uses setTimeout to block until the scripts have loaded:

fakeTest: {
  run: function(tests) {
    if (tt.isLoading) {
      setTimeout(function() { tt.fakeTest.run(tests); }, 10);
    } else {
      return tt.testRunner.run(tests);
    }
  }
}

As long as our tests haven’t been run yet they won’t hit the empty objects returned by our hacked version of require.

Conclusion

At this point we’ve got a test framework that looks 100% CommonJS in the browser and console, with a very simple format for the HTML loading stubs. That doesn’t mean tests that depend on the DOM will work, and module loading doesn’t really work very well outside of this limited example.

In fact, the insanity I had to pull to get this to work was quite ridiculous, and took hours of work. Because it was a stint from 6pm until 11pm I haven’t had chance to fully explore it yet, but hopefully it gets people thinking about ‘write once, run anywhere’ testing.

The code I checked in was commit b16caa90a for turing-test.js and commit a7be08a5 for turing.js.

References

• Git Book – Submodules
• Pro Git – Submodules

Welcome to the Node Roundup. Send in your apps and libraries using our contact form or @dailyjs.

We’re now on Facebook. If you like the page you’ll see DailyJS news, and you can share stuff you’ve made on our wall. You can also like our posts so your friends know you’re an awesome JavaScript hacker.

Crankshaft

Crankshaft, a new compilation infrastructure for V8, was announced on the Chromium blog. The authors claim this improves the performance of compute-intensive JavaScript applications. These changes include a runtime profiler and an optimizing compiler.

The implications for Node projects aren’t yet clear, but Arnout Kazemier has already run Node benchmarks (brief analysis by Dean Landolt).

Portland NodeConf 2011

Portland NodeConf has been announced for May 5th, 2011. The site doesn’t have much more information than that, however, but there’s a Hacker News thread on it NodeConf May 5th 2011 Portland, OR.

htracr

htracr by Mark Nottingham (GitHub: htracr) is a HTTP sniffing and visualization application written with Node. He’s just released it through npm as well, so it can be quickly installed with npm install htracr. The blog post is an interesting read because Nottingham explores how different browsers behave at the TCP and HTTP level — for example, Opera 10.63 was doing less pipelining than Firefox 3.6 (with pipelining turned on). That kind of analysis could be used to better optimise your sites and servers.

It uses libpcap to sniff HTTP traffic, which means you might need to run it with sudo:

npm install htracr
sudo htracr 8000
# Open http://localhost:8000/ in your browser and press the 'Start' button

Nottingham’s got a huge list of interesting ideas in the project’s TODO list, but it’s already a fascinating use of Node.

Welcome to the jQuery roundup 35. You can send your plugins and articles in for review through our contact form or @dailyjs.

Head JS

Head JS by Tero Piirainen (GitHub) is a parallel script loader. The project has the byline “the only script in your <head>”. It also includes some other features:

• onReady handling based on when scripts have been loaded with head.ready
• CSS modernizer and HTML5 backwards compatibility
• Dynamic CSS loading based on conditions
• Browser detection
• JavaScript feature detection

Which makes this library a flexible meta-library that sits above your favourite frameworks like jQuery.

The story behind Head JS is interesting because the project was being quietly developed by the author… that is until it was discovered on GitHub and apparently went viral:

This project was never announced. git push and it was all viral.

Unlike most projects out there, the Piirainen made a good site with detailed documentation early on, so it just goes to show how packaging your projects can help them succeed. Dumping jQuery plugins on free hosting with a poor website and little or no documentation isn’t a good way to get your work noticed.

TumblrPostMap

TumblrPostMap by Cláudio Gil (demo, documentation) is another well-documented project that aims to enable Tumblr blogs to show graphs with as little work as possible. It looks at posts tagged with geo for markup like this:

<p class="geo">Uluru, NT, Austrália: 
    <span class="latitude">-25.345</span>, <span class="longitude">131.036111</span>
</p>

I like the fact he’s using the Geo microformat to structure the markup. The Tumblr demo blog doesn’t seem to work right now, I’m not sure if this is due to Tumblr’s recent downtime or the fact it’s being actively worked on. Anyway, kudos for the documentation!

Fixed Center

Fixed Center (demo) by David Tang is a simple plugin that centres a div on a page. It’s the kind of thing a client wants you to do that you’d rather not have to spend much time on. He recently updated it with performance enhancements, so it feels fairly lightweight.

Packaging Your Plugins

I’ve said it many times before, but please package your jQuery projects with the following:

• A README with basic information
• A license
• Documentation
• Tests, however limited
• A corresponding website and demo

GitHub even hosts pages for free, so you’ve got no excuse for not having a site or documentation. If you’re going to spend time building a plugin, why not package it in a way that makes it easy for us to use? Even better, your work will be more likely to be featured on sites like this!

Welcome to part 5 of Let’s Make a Web App, a tutorial series about building a web app with Node. This series will walk you through the major areas you’ll need to face when building your own applications. These tutorials are tagged with lmawa.

Previous tutorials:

• Part 1: Introduction
• Part 2: Installation and Skeleton App, source code commit: 4ea936b
• Part 3: RESTful Methods and Testing, source code commit: 39e66cb
• Part 4: Templates, Partials, Creating and Editing Documents, source code commit: f66fdb

Before starting this tutorial remember to start up a mongo daemon if your computer doesn’t run one automatically.

Authentication

We’ve built a serviceable app, but it’s not that useful without some kind of authentication system. Most production systems and client projects require authentication, and even though there are interesting initiatives like OpenID and OAuth, most commercial projects are likely to have their own login system.

This is usually achieved with a session:

• A user fills out a form with their username and password
• The password is encrypted with a hashing algorithm and salt
• This value is compared with the user’s record in the database
• If it matches, a session key is generated that identifies the user

We need the following things to manage users and sessions:

• Users in the database
• Sessions that can store the logged-in user ID
• Password encryption
• A way of restricting access to routes that require a logged-in user

Sessions in Express

Express relies on Connect’s session middleware, which is backed by a data storage mechanism. There’s a memory-based store, and third-party stores including connect-redis and connect-mongodb. An alternative option is cookie-sessions which stores session data inside the user’s cookie.

A session can be configured like this:

app.use(express.cookieDecoder());
app.use(express.session());

It’s very important to place these configuration options in the right place during configuration, else the session variable won’t appear in request objects. I’ve put it after bodyDecoder and before methodOverride. Refer to the full source on GitHub to see this in context.

Now our HTTP responders will have access to req.session:

app.get('/item', function(req, res) {
  req.session.message = 'Hello World';
});

MongoDB Sessions

Install connect-mongodb with npm install connect-mongodb.

connect-mongodb works like any other session store. During application configuration we need to specify our connection details:

app.configure('development', function() {
  app.set('db-uri', 'mongodb://localhost/nodepad-development');
});

var db = mongoose.connect(app.set('db-uri'));

function mongoStoreConnectionArgs() {
  return { dbname: db.db.databaseName,
           host: db.db.serverConfig.host,
           port: db.db.serverConfig.port,
           username: db.uri.username,
           password: db.uri.password };
}

app.use(express.session({
  store: mongoStore(mongoStoreConnectionArgs())
}));

Most of this code wouldn’t be required if the API authors could decide on a standard format for connection options. I’ve written a function that extracts the connection details from Mongoose. In this example, db holds a Mongoose connection instance. Mongoose expects connection details to be provided through URIs, which I like because it’s easy to remember the format. I’ve stored environment-specific connection strings using app.set.

When writing Express applications it’s a good idea to use app.set('name', 'value'). Just remember that app.set('name') is used to access the setting, rather than app.get.

Running db.sessions.find() in the mongo console will now return any sessions that have been created.

Access Control Middleware

Express provides an elegant way of restricting access to logged-in users. When HTTP handlers are defined, an optional route middleware parameter can be specified:

function loadUser(req, res, next) {
  if (req.session.user_id) {
    User.findById(req.session.user_id, function(user) {
      if (user) {
        req.currentUser = user;
        next();
      } else {
        res.redirect('/sessions/new');
      }
    });
  } else {
    res.redirect('/sessions/new');
  }
}

app.get('/documents.:format?', loadUser, function(req, res) {
  // ...
});

Now every route that requires a logged-in user can be handled by adding loadUser to the route. The middleware itself gets the usual route parameters, and also next — this can be used to run the route handler based on arbitrary logic. In our project a user is loaded using a user_id in the session; if the user cannot be found next simply isn’t called and the browser is redirected to the login screen.

RESTful Session Modelling

I’ve modelled sessions in a similar way to documents. There are new, create, and delete routes:

// Sessions
app.get('/sessions/new', function(req, res) {
  res.render('sessions/new.jade', {
    locals: { user: new User() }
  });
});

app.post('/sessions', function(req, res) {
  // Find the user and set the currentUser session variable
});

app.del('/sessions', loadUser, function(req, res) {
  // Remove the session
  if (req.session) {
    req.session.destroy(function() {});
  }
  res.redirect('/sessions/new');
});

User Model

The User model is more complicated than the Document model because it has to contain authentication-related code. The strategy I’ve used, which you’ve probably seen before in OO web frameworks, is:

• Passwords are stored in an encrypted format with a salt
• Authentication can be performed by comparing an encrypted plain text password with the password in the database for a given user
• A ‘virtual’ password property exposes a plain text password for the convenience of registration and login forms
• This property has a setter which automatically encrypts the password before saving
• A unique index is used to ensure there’s only one user for each email address

The password encryption uses Node’s standard crypto library:

var crypto = require('crypto');

mongoose.model('User', {
  methods: {
    encryptPassword: function(password) {
      return crypto.createHmac('sha1', this.salt).update(password).digest('hex');
    }
  }
});

encryptPassword is an instance method which returns a sha1-hashed password with a salt. The salt is generated before encrypting the password in the password setter:

mongoose.model('User', {
  // ...

  setters: {
    password: function(password) {
      this._password = password;
      this.salt = this.makeSalt();
      this.hashed_password = this.encryptPassword(password);
    }
  },

  methods: {
    authenticate: function(plainText) {
      return this.encryptPassword(plainText) === this.hashed_password;
    },

    makeSalt: function() {
      return Math.round((new Date().valueOf() * Math.random())) + '';
    },

    // ...

The salt could be anything you like, I’ve generated a fairly random string here.

Saving Users and Registration

Mongoose makes it easy to do things when records are saved by overriding the save method:

mongoose.model('User', {
  // ...
  methods: {
    // ...

    save: function(okFn, failedFn) {
      if (this.isValid()) {
        this.__super__(okFn);
      } else {
        failedFn();
      }
    }

    // ...

I’ve overridden save to allow for a failed save method. This makes the failed registration handling simpler:

app.post('/users.:format?', function(req, res) {
  var user = new User(req.body.user);

  function userSaved() {
    switch (req.params.format) {
      case 'json':
        res.send(user.__doc);
      break;

      default:
        req.session.user_id = user.id;
        res.redirect('/documents');
    }
  }

  function userSaveFailed() {
    res.render('users/new.jade', {
      locals: { user: user }
    });
  }

  user.save(userSaved, userSaveFailed);
});

No error messages are displayed right now; I’ll get to that in another tutorial.

Even though this validation is pretty dumb, the index is critical to this application:

mongoose.model('User', {
  // ...

  indexes: [
    [{ email: 1 }, { unique: true }]
  ],

  // ...
});

This will prevent duplicate users from being saved. The format is the same as MongoDB’s ensureIndex.

Conclusion

As of commit 03fe9b2 we now have:

• MongoDB sessions
• User model, with support for sha1 password encryption
• Routing middleware for controlling access to documents
• User registration and login
• Session management

I’ve updated the Jade templates to include a basic login form.

There are a few things missing from this version though:

• Documents don’t take into account the owner
• The tests don’t work properly because I’m having trouble figuring out session handling in Expresso tests

I’ll get to these things in future parts of this tutorial series.