Microsoft’s WebGL Concerns

Microsoft’s Security Research & Defense blog recently published a post about serious concerns over WebGL security, with the rather damning title WebGL Considered Harmful.

Our analysis has led us to conclude that Microsoft products supporting WebGL would have difficulty passing Microsoft’s Security Development Lifecycle requirements.

They cite low-level hardware access as a serious issue, along with reliance on third-party security and “problematic DoS scenarios”:

Modern operating systems and graphics infrastructure were never designed to fully defend against attacker-supplied shaders and geometry.

As a fan of WebGL I balked at these claims, but there’s definitely something to them. However, I don’t think refusing to support WebGL is the right choice; couldn’t a compromise be reached through the use of UAC or other Windows/IE security features?

Meanwhile, Chris Marrin (a WebKit reviewer) had this to say about iOS 5 WebGL support:

WebGL will not be publicly available in iOS 5. It will only be available to iAd developers.

This makes sense from Apple’s perspective because they’ll be able to review content before it hits browsers. Sadly it’s more frustration for those of us who enjoy working with WebGL and iOS devices.

csonv.js

If the world was made of JSON and CSS selectors we’d be the most powerful wizards in the universe! Anyway… csonv.js (GitHub: archan937 / csonv.js, License: MIT) by Paul Engel is a library that fetches CSV files and translates it to JSON. I always find extracting CSV headers to be extremely tedious, so getting more convenient JSON data sets may appeal to those of us who have to work with CSV.

One particularly useful feature of this library is the ability to process relational data by nesting it. Paul includes an example of how this works based on books and authors.

Same Game Gravity

Same Game Gravity is a HTML5/CSS3/etc. game that uses CSS transforms to rotate a Canvas that contains a tile-matching game. The game itself is similar to something like Bejeweled, except whenever the board is rotated the tiles fall down and rearrange themselves.

Over the last two weeks I’ve been looking at how web frameworks implement promises, and I’ve built a little Promise class along the way.

• Promises: Part 1
• Promises: Part 2

Building on the Promise Class

To build on last week’s example we first need to make it sit alongside the rest of the framework code. I’ve created turing.promise.js, and test/promise_test.js, then added code comments for Turing’s annotated source code.

Here’s what the annotated source looks like now:

/*!
 * Turing Promise
 * Copyright (C) 2010-2011 Alex R. Young
 * MIT Licensed
 */

/**
 * The Turing Promise module.
 */
(function() {

  /**
   * The Promise class.
   */
  function Promise() {
    var self = this;
    this.pending = [];

    /**
     * Resolves a promise.
     *
     * @param {Object} A value
     */
    this.resolve = function(result) {
      self.complete('resolve', result);
    },

This is what part of the unit test looks like:

'test then': function() {
  function delay(ms) {
    var p = new turing.Promise();
    setTimeout(p.resolve, ms);
    return p;
  }

  function timeout(duration, limit) {
    var p = new turing.Promise();
    setTimeout(p.resolve, duration);
    setTimeout(p.reject, limit);
    return p;
  }

  delay(1000).then(function() {
    assert.ok('Delay completed');
  });

Framework Integration

One of the reasons I was interested in using promises was to make animation code more accessible. The following example should be possible:

turing().delay(1000).then(function() {
  // A second has passed!
});

This could be tested by looking at the current time:

'test chain': function() {
  var start = (new Date()).valueOf();

  // Don't match on exactly 1000 because setTimeout isn't accurate
  turing().delay(1010).then(function() {
    assert.ok((new Date()).valueOf() - start >= 1000);
  });
}

Implementation

A few months ago we added “middleware” to Turing which allows features to be injected based on the parameters passed to turing(). Satisfying the previous test requires no parameters. In future, the Promise module should be integrated with the animation module and supporting chain hooks can be added there:

var chain = {};

turing.init(function() {
  if (arguments.length === 0)
    return chain;
});

chain.delay = function(ms) {
  var p = new turing.Promise();
  setTimeout(p.resolve, ms);
  return p;
};

Now turing().delay(1000).then(callback); is entirely possible!

Conclusion

I’ve updated Turing’s repository with all of this, in commit d64f0cf.

Common Node

Common Node (GitHub: olegp / common-node) by Oleg Podsechin uses node-fibers to build CommonJS proposals. This means it’s now possible in many cases to write asynchronous code with synchronous logic.

The CommonJS proposals implemented by Common Node are:

• assert
• binary
• fs
• httpclient
• io
• jsgi
• system
• test

This fragment is from the https.js example:

new HttpClient({url: 'http://google.com'}).finish().body

The finish() method calls HttpClient.prototype.read which uses a fiber:

var req = protocols[uri.protocol].request(options, function(response) {
  fiber.run(response);
});

As Oleg says in his documentation:

Fibers are used to emulate multi-threading within a single process, allowing one to use a synchronous programming style and as a result.

NodeCamp.eu Talks

I saw Ryan Dahl tweeted a link to this huge list of nodecamp.eu talk slides: A collection of talks and presentations held at nodecamp.eu 2011.

using libuv and http-parser to build a webserver from ry on Vimeo.

The Understanding V8 talk includes some real JavaScript examples, and Ryan Dahl’s talk is available as a screencast.

Ryan’s talk is about using libuv" and http-parser to build a web server using C. It’s not as scary as it sounds, libuv really looks very accessible.

Hook.io

Hook.io by Marak Squires is a distributed EventEmitter:

You create custom i/o scenarios by picking and choosing from an extensive library of tiny, independent, autonomous “hooks” that seamlessly work together.

It has a big list of interesting features:

• Create hooks on ANY device that supports JavaScript
• Interprocess Message Publishing and Subscribing done through EventEmitter2 and dnode
• Spawning and Daemonizing of processes handled with Forever

Fathom.js

Fathom.js (GitHub: markdalgleish / fathom, License: MIT) by Mark Dalgleish is a plugin for building presentations.

Given this HTML:

<div id="presentation">
<div class="slide"> <h1>My Presentation</h1> </div>
<div class="slide"> <h2>My Dot Points</h2> <ul> <li>First dot point</li> <li>Second dot point</li> <li>Third dot point</li> </ul> </div>
</div>

It just needs $('#presentation').fathom(); to display the content as a slide show. It comes bundled with CSS and supports keyboard shortcuts, so it’s easy to get started and use for real presentations.

jQuery-News-Ticker

jQuery-News-Ticker (GitHub: rhodimus / jQuery-News-Ticker, License: GPL v2) by Rhodri George is a news ticker plugin, similar to the one featured on the BBC News site. Before you scoff, this is exactly the kind of thing clients nag me to add to their sites, and this plugin is very well presented.

With some suitable markup, $('#js-news').ticker(); will do its magic:

<div id="ticker-wrapper" class="no-js">
  <ul id="js-news" class="js-hidden">
    <li class="news-item"><a href="#">This is the 1st latest news item.</a></li>
    <li class="news-item"><a href="#">This is the 2nd latest news item.</a></li>
    <li class="news-item"><a href="#">This is the 3rd latest news item.</a></li>
    <li class="news-item"><a href="#">This is the 4th latest news item.</a></li>
  </ul>
</div>

jVectorMap

jVectorMap is a nice little plugin for rendering maps with various attributes. The basic useage is just $('#map').vectorMap();, and this will even show zoom controls.

And, there’s no need to worry about IE!

It uses SVG in all modern browsers like Firefox 3 or 4, Safari, Chrome, Opera, IE9, while legacy support for older versions of IE from 6 to 8 is provided with VML.

I’m a fan of Raphaël, so I thought it was high time we took a look at its innards to see how it really works. Raphaël simplifies working with vector graphics in browsers by providing an easy API for working with SVG, and it also transparently generates VML for Microsoft’s browsers.

Drawing with vector graphics on the web is extremely convenient, because server-side image generation can be relatively CPU-intensive. Would you rather generate images for potentially thousands of people, or make their browsers do the work instead? Because Raphaël works well with Microsoft’s browsers, it’s safe to use it to generate things like graphs in a corporate environment.

About Raphaël

Raphaël (GitHub: DmitryBaranovskiy / raphael, License: MIT) by Dmitry Baranovskiy (and now part of Sencha) can be included on a page, then drawing is as simple as this:

var paper = Raphael(10, 50, 320, 200);
paper.circle(50, 40, 10);

Great! The website has some more involved demos. I like these curves which can be dragged around. There’s also the obligatory tiger vector art.

Usage

Raphaël can be set up with a container element where it’ll add a Canvas and start drawing. It comes with a lot of drawing primitives: circle, rect, ellipse, image, text, and path. There are also transformations like scale and rotate.

One other useful feature is the ability to get a DOM object for event handling (or anything else you can do with DOM objects):

var c = paper.circle(10, 10, 10);
c.node.onclick = function () {
    c.attr('fill', 'red');
};

Structure

Raphaël comes with a simple integration test, some plugins for additional functionality that falls outside of the core library, and a reference document.

All of the library code is inside raphael.js. It doesn’t look like it’s split into separate files — there’s no build script or anything like that. Everything is wrapped in an anonymous wrapper, which “exports” the Raphael object to window.

Most of the library’s classes are implemented as simple prototype classes, and there’s a lot of prototype aliasing to make referencing shorter. Even Math functions are aliased.

There are some high-level tricks that I’ve seen all over jQuery, like using strings for lists and splitting on spaces to make arrays (event names are listed this way in both libraries). There’s also some duplication of functionality from libraries like jQuery to provide event handling. Did the author find managing events was inexorably linked with drawing, or did he want to avoid depending on another library?

Another point is there are very few local (effectively private) functions. There are a few that are clearly implementation details (like createUUID), but not as many as I’d expect to see. This isn’t a bad thing, in fact it probably makes Raphaël relatively easy to test.

Drawing

Most of the low-level drawing operations are split between VML and SVG. On line 1084 if (R.svg) is used to branch off for SVG support, then later on line 1824, if (R.vml) branches the code into two sections. The VML code is almost 1000 lines of code, then a comment says // rest and we see the paperproto methods for SVG. Some simple user-agent-based detection is used to handle browser issues.

Some of the paperproto methods are defined twice, whereas others call a different internal function that is set in the VML/SVG sections.

As an example, this is how a circle is drawn for SVG:

var theCircle = function (svg, x, y, r) {
  var el = $("circle");
  svg.canvas && svg.canvas[appendChild](el);
  var res = new Element(el, svg);
  res.attrs = {cx: x, cy: y, r: r, fill: "none", stroke: "#000"};
  res.type = "circle";
  $(el, res.attrs);
  return res;
}

Then we see the IE version later on:

theCircle = function (vml, x, y, r) {
  var g = createNode("group"),
      o = createNode("oval"),
      ol = o.style;
  g.style.cssText = "position:absolute;left:0;top:0;width:" + vml.width + "px;height:" + vml.height + "px";
  g.coordsize = coordsize;
  g.coordorigin = vml.coordorigin;
  g[appendChild](o);
  var res = new Element(o, g, vml);
  res.type = "circle";
  setFillAndStroke(res, {stroke: "#000", fill: "none"});
  res.attrs.cx = x;
  res.attrs.cy = y;
  res.attrs.r = r;
  res.setBox({x: x - r, y: y - r, width: r * 2, height: r * 2});
  vml.canvas[appendChild](g);
  return res;
};

The dollar method is a shortcut for creating SVG elements:

var $ = function (el, attr) {
  if (attr) {
    for (var key in attr) {
      if (attr[has](key)) {
        el[setAttribute](key, Str(attr[key]));
      }
    }
  } else {
    el = doc.createElementNS(paperproto.svgns, el);
    el.style.webkitTapHighlightColor = "rgba(0,0,0,0)";
    return el;
  }
};

Code Style

The code is written in a very consistent style. The author avoids verbose naming, but I still find it relatively easy to navigate through the library. There aren’t many comments, just a few one-line suggestions here and there.

Conclusion

Raphaël is like four libraries in one:

• SVG drawing
• VML drawing
• Animations
• UI events

Supporting VML doesn’t impair the readability, but it does cause a huge duplication of effort. Providing a consistent experience for both VML and SVG looks like an enormous amount of work, which probably explains why Raphaël is still the reigning champ of browser graphics libraries.

Rutt-Etra-Izer

Rutt-Etra-Izer (WebGL app) by Felix Turner is an implementation of a video synthesizer. Simply drag and drop an image file to the browser, and it’ll render a 3D version using lines.

Different images have varying effects, so it’s fun to experiment. Rutt-Etra-Izer is built with three.js.

GLOW

GLOW (GitHub: empaempa / GLOW, License: MIT) by Mikael Emtinger is a WebGL API that focuses on shaders. The authors also worked on the impressive ROME – Three Dreams of Black project.

There are tutorials on the GLOW site, including one which shows how it can work alongside three.js: Hello Three.js.

Chrysaora

Chrysaora is a WebGL demo by Aleksandar Rodic. There’s a lot of background material about the project on Aleksandar’s blog.

Last week’s post introduced the concept of promises, which are defined by the CommonJS Promises/A specification.

Implementation Concepts

It’s important to realise that implementing promises requires some coupling or wrapping between the promise library and code that will use it. For example, in jQuery, deferred.js contains the promise API implementation. That doesn’t mean Ajax requests can now simply be manipulated to work with a promise, like this:

// This won't work:
promiseLibrary($.get('/example')).then(function() {
  // Ajax complete!
});

There’s no way for promiseLibrary to be signalled by a given method in a generic way.

This next example is from q for Node. It wraps promises around a file system call:

var FS = require('fs');

function list(path) {
  path = String(path);
  var result = Q.defer();
  FS.readdir(path, function(error, list) {
    if (error)
      return result.reject(error);
    else
      result.resolve(list);
  });
  return result.promise;
}

list('/path/name');

Here Q is used inside the callback usually called by the fs module. This new list() method will return a promise, allowing subsequent use of q’s when primitive to build potentially more expressive file system queries.

What all of this means is jQuery’s ajax.js is coupled to the promise implementation. This allows jQuery to offer an alternative API based around promises:

$.get('test').then(
  function() { alert('$.get succeeded'); },
  function() { alert('$.get failed!'); }
);

Despite this coupling between the Promise and Ajax libraries, injecting jQuery’s Deferred objects into the Ajax library isn’t particularly taxing. Here’s a core piece of this implementation:

// Success/Error
if ( isSuccess ) {
  deferred.resolveWith( callbackContext, [ success, statusText, jqXHR ] );
} else {
  deferred.rejectWith( callbackContext, [ jqXHR, statusText, error ] );
}

jQuery’s public APIs place a veneer between jQuery’s other functionality and the Deferred object’s API. Using an Ajax call looks a lot like CommonJS Promises/A’s suggestion, but in reality methods like resolveWith and rejectWith are being used underneath to provide the logic required to maintain asynchronous states.

An Example

I’m going to introduce promises gently, because it’s the kind of thing that can easily end up a confusing mess. The example in this article is intentionally limited so anyone should be able to follow it.

Developing a promise API needs a straightforward example to get the basic features nailed down. We need to be able to create a delay that can trigger the promise API in a predictable way. When experimenting with anything asynchronous, I always find good old setTimeout to be a big help, so let’s use that:

function delay(ms) {
  var d = new Promise();
  setTimeout(d.resolve, ms);
  return d;
}

This is based on q’s examples, and it just wraps setTimeout with a promise. Ideally, running the following in an interpreter should take a second, then display the message:

delay(1000).then(function() {
  console.log('Delay complete');
});

However… once a promise has fired, it shouldn’t be able to get triggered again. Imagine if someone had implemented a timeout system like this:

function timeout(duration, limit) {
  var d = new Promise();
  setTimeout(d.resolve, duration);

  // If the duration was over the limit then ideally the promise should fail
  setTimeout(d.reject, limit);
  return d;
}

Should both callbacks fire? Should an exception be thrown? Well, jQuery is designed to simply clear out the then callbacks as it rattles through them:

// resolve with given context and args
resolveWith: function( context, args ) {
  if ( !cancelled && !fired && !firing ) {
    // make sure args are available (#8421)
    args = args || [];
    firing = 1;
    try {
      while( callbacks[ 0 ] ) {
        callbacks.shift().apply( context, args );
      }
      // ...

See that use of shift() there? That’s effectively looping through each callback, running it, then removing it.

A Promise Object

Of course, our promise API isn’t just a stack of callbacks. We also need to store state, and allow callbacks to be added. To me this implies a class with a simple API:

function Promise() {
  // Pending callbacks
  this.pending = [];
}

Promise.prototype = {
  // Called when something finished successfully
  resolve: function(result) {
    // This is where we'll loop through callbacks
  },

  // Called when something broke
  reject: function(result) {
  },

  // I think you'll agree that implementing then is pretty easy
  // This uses a handy object that follows along with our API nicely
  then: function(success, failure) {
    this.pending.push({ resolve: success, reject: failure });
    return this;
  }
};

That’s looking good, but I want to use setTimeout(promise.resolve, 100) and that will cause resolve to have the wrong this. These methods could easily be moved to the constructor to make the resolve and reject methods behave more naturally:

function Promise() {
  var self = this;
  this.pending = [];

  this.resolve = function(result) {
    self.complete('resolve', result);
  },

  this.reject = function(result) {
    self.complete('reject', result);
  }
}

Promise.prototype = {
  then: function(success, failure) {
    this.pending.push({ resolve: success, reject: failure });
    return this;
  },

  complete: function(type, result) {
    while (this.pending[0]) {
      this.pending.shift()[type](result);
    }
  }
};

I also started writing some tests for this:

function delay(ms) {
  var p = new Promise();
  setTimeout(p.resolve, ms);
  return p;
}

function timeout(duration, limit) {
  var p = new Promise();
  setTimeout(p.resolve, duration);
  setTimeout(p.reject, limit);
  return p;
}

delay(1000).then(function() {
  console.log('Delay complete');
  assert.ok('Delay completed');
});

timeout(10, 100).then(
  function() {
    console.log('Timeout 1 OK');
    assert.ok('10ms is under 100ms');
  },
  function() {
    assert.fail();
  }
);

timeout(100, 10).then(
  function() {
    assert.fail();
  },
  function() {
    console.log('Timeout 2 OK');
    assert.ok('100ms is over 10ms');
  }
);

Conclusion

This is a very naive implementation of what CommonJS Promises/A describes. Ultimately, I’d like to have something that can be used to provide an elegant API for Ajax callbacks and animations, which are both areas in client-side JavaScript where asynchronous code is important.

natural

I was recently doing some work that involved stemming. It wasn’t very fun. Anyway, I found natural by Chris Umbel (License) which offers tools for stemming, classification, phonetics and inflection:

var natural = require('natural'),
    classifier = new natural.BayesClassifier();

natural.PorterStemmer.stem('reading');
// 'read'

classifier.train(
  [{classification: 'happy', text: 'I love pizza'},
   {classification: 'happy', text: 'javascript is awesome'},
   {classification: 'sad', text: 'I hate tax'}
]);

classifier.classify('how does alex feel about pizza?');
// 'happy'
classifier.classify('how does alex feel about tax?');
// 'sad'

node-beatport

node-beatport is a client library for the Beatport API. Beatport is a service for buying and downloading electronic music, so the API provides a way of querying track, artist, chart or genre details.

var Beatport = require('beatport')

// initialize client
var bp = Beatport()

// resources (i.e: featured/releases) as methods (camelCased, i.e: featuredReleases)
bp.releases({
  facets: [ 'genreName:Trance', 'performerName:Above&Beyond' ]
, perPage: 5
, page: 1
}, function(err, results, metadata) {
  // do something
})

bp.labelDetail({ id: 804 }, function(err, results, metadata) {
  // do something
})

I found it surprising that Beatport has an API — I don’t think I’ve seen other music retailers offering this kind of service.

node-schema-org

node-schema-org by Charlie Robbins is a library for parsing the microdata schemas found on schema.org.

At the moment it will just output all of the scemas as JSON files:

$ npm install -g schema-org
$ read-schema-org
warn:   Removing all schemas in /usr/local/lib/node_modules/schema-org/schemas
info:   Spawning: node /usr/local/lib/node_modules/schema-org/list-schemas.js
info:   Contacting: http://schema.org/docs/full.html

This actually spawns lots of background jobs and totally owned my machine, so be wary of using it in its current state.

jQuery Color

jQuery Color (GitHub: jquery / jquery-color, License: MIT, GPL) version 2 beta 1 has been released. This version has a new API, RGBA, HSLA, and partial colour animation support. This is the backbone of jQuery’s colour animations, but only now is it being exposed as a public object.

This new object, $.Color has a pretty simple API:

// Create a red Color object:
var red = $.Color('rgba(255,0,0,1)');

// Create a red Color object, then make orange:
var orange = $.Color('#FF0000').green( 153 );

// Get the color halfway between red and blue:
var between = $.Color([255, 0, 0]).transition('blue', 0.5);

// Desaturate the background of this element
elem.animate({
  backgroundColor: $.Color({ saturation: 0 })
}, 1000);

Anyone who has worked with CSS colour animations will know how useful this is. It’s also going to come in handy when building interfaces for things like colour picker widgets. I imagine the jQuery UI project will benefit from this new API.

animate-textshadow

animate-textshadow (GitHub: alexpeattie / animate-textshadow, License: MIT) by Alex Peattie is a little plugin for creating text-shadow animations.

This will make a shadow that slowly moves away from text as if it’s being lifted off the screen:

$('#slow').hover(function() { 
	$(this).animate({textShadow: '#000 -10px -10px 30px'}, 1000);
}, function() { 
	$(this).animate({textShadow: '#259 5px 5px 5px'}, 1000);
});

jQuery.forrst

jQuery.forrst (License: GPL) by Jason Gerfen is a plugin for the Forrst API. It currently supports accessing user info, posts, and interestingly offers AES encryption for local storage.

I initially thought it might be a handy way of creating Forrst widgets for blogs, but it seems more like something that could be used to build a Forrst client.

In last week’s code review I looked at CoffeeScript, which is built with the excellent Jison parser generator. I wanted to contrast this to Jade. Although Jade is a very different language to CoffeeScript, I wanted to compare TJ’s hand-crafted parser and lexer.

About Jade

Jade (GitHub: visionmedia / jade, License: MIT, npm: jade) by TJ Holowaychuk is a Haml-inspired template language. It started off as a Node library, but it’s been ported to other languages too.

It has some interesting time saving features that make HTML feel like hard work. Rather than writing div all the time, it’s possible to just write classes or IDs with CSS selector syntax:

#main
  .post
    h3 #{post.title}

There are even shortcuts for conditional comments. This will insert <!--[if IE]>:

body
  /if IE
    a(href='http://www.mozilla.com/en-US/firefox/') Get Firefox

Installation

Jade can be downloaded from GitHub or simply installed with npm:

npm install jade

Usage

Most people use Jade as part of Express apps. It’s possible to use it outside of Express:

var jade = require('jade'),
    options = {};

jade.render('#main', options);
jade.renderFile('file.jade', options, function(err, html) {
  // Callback receives HTML
});

// Compile a function
var fn = jade.compile('#main', options);
fn.call(scope, locals);

The project also comes bundled with bin/jade:

$ jade -h

  Usage: jade [options]
              [path ...]
              < in.jade > out.jade  
  Options:
    -o, --options <str>  JavaScript options object passed
    -h, --help           Output help information
    -w, --watch          Watch file(s) or folder(s) for changes and re-compile
    -v, --version        Output jade version
    --out <dir>          Output the compiled html to <dir>

Structure

Jade itself is pretty self-contained. There are developer dependencies for testing and benchmarking — parsing performance is very important to the author. The build script is a simple, clean Makefile. This is excellent because as far as I know every machine I use has Make installed.

After checking out the project’s git submodules, I was able to successfully benchmark it:

$ make benchmark

benchmarking 2000 times

  - jade compilation: 1006 ms
  - jade execution: 340 ms
  - jade compile(): 329 ms

benchmarking 2000 times

  - jade self compilation: 713 ms
  - jade self execution: 53 ms
  - jade compile(): 53 ms

benchmarking 2000 times

  - haml-js compilation: 841 ms
  - haml-js execution: 145 ms

benchmarking 2000 times

  - haml compilation: 474 ms
  - haml execution: 75 ms

benchmarking 2000 times

  - ejs compilation: 264 ms
  - ejs execution: 89 ms

It was that simple!

The overall structure of the code is easy to follow:

• Main groups of functionality are separate files under lib/ — the parser and lexer are the main ones we’ll look at
• lib/jade.js provides the public API
• Each feature of the Jade language is represented by a “node”, which is an empty object

Parsing

Let’s look at a route through Jade as it parses input and generates output. The exports.render = function(str, options) method renders a string of Jade into HTML. The input is wrapped in a String so it’ll accept buffer objects. An internal line number position is kept inside an object called meta to help generate readable error messages with rethrow(err, str, filename, meta.lineno).

It then generates a function based on the output of parse:

fn = new Function('locals', parse(str, options));

This is a minor point in the scope of everything Jade is doing here, but if you’re not aware of the Function object it’s worth reading up on. It can be a useful tool for metaprogramming and escaping sticky situations while creating the veneer of a simple API.

The fact that Function uses the output of parse(str, options) as the function body illustrates a central concept to Jade’s compilation stage — it effectively generates JavaScript that gets executed using Function. I’ve used this technique myself to create metaprogrammed code that’s more explicit than a dirty big eval, but my assumption here is TJ went down this route for performance reasons.

Once parse has instantiated a Parser, the output of the parser is passed to a Compiler object. The parser is responsible for driving a Lexer object, which has a simple API.

The lexer consumes the input and generates tokens using regular expressions and very simple, readable JavaScript. Remember how Jade handles element classes? Well, consider this:

className: function() {
  return this.scan(/^\.([\w-]+)/, 'class');
},

The lexer advances when a token is matched:

next: function() {
  return this.deferred()
    || this.eos()
    || this.pipelessText()
    || this.doctype()
    || this.tag()
    || this.filter()
    || this.each()
    || this.code()
    || this.id()
    || this.className()
    || this.attrs()
    || this.indent()
    || this.comment()
    || this.blockComment()
    || this.colon()
    || this.text();
}

The parser uses these tokens to build a tree of Node objects. The parser’s parse* methods are mostly co-ordinated by parseBlock and parseExpr according to what constitutes valid Jade. The abstract representations of nodes and implied tree structure form what’s known as an Abstract Syntax Tree (AST).

In some cases lookahead is used, which is supported by the peek method, and expect is also used to check for invalid syntax. This will throw an exception which can be used to generate useful error messages as we saw earlier.

It’s interesting to note that the parser houses the formal grammar rules — although this was true of CoffeeScript, CoffeeScript has Jison representations of the grammar which are used to generate the parser. The end result is superficially similar, but separating grammar from the parser may afford some advantages.

The missing piece of the puzzle here is the Compiler. This compiles the parse tree into JavaScript, which we saw being executed by the parse function in lib/jade.js.

Each node is “visited” by the compiler:

visitNode: function(node){
  var name = node.constructor.name
    || node.constructor.toString().match(/function ([^(\s]+)()/)[1];
  return this['visit' + name](node);
},

The Compiler class contains methods that begin with visit and will be matched here. This is another handy and extremely simple way of effectively metaprogramming without any fuss or cartoon foxes.

Code Style

TJ uses many conventions that make his code easy to read. A small one is commas at the start of lines:

var nodes = require('./nodes')
  , filters = require('./filters')
  , doctypes = require('./doctypes')
  , selfClosing = require('./self-closing')
  , utils = require('./utils');

I’ve never been able to get used to writing code this way, but it makes it easy to spot missing commas. Another point is everything is well-named; he uses established computer science concepts where appropriate (specifically referencing ASTs for example), and writes simple but fast community-friendly JavaScript.

Conclusion

In comparing CoffeeScript and Jade, the most interesting thing that sticks in my mind is the separation of grammar from the parser that Jison makes possible. However, even though I’ve rushed through a brief analysis of Jade, I think you’ll agree that it’s very easy to follow what the parser is doing.

TJ’s packaged the project with tests, benchmarks, and all the libraries required to get started as a developer for the project. There are also helpful code comments throughout.

The Makefile makes managing the project simple and immediate on any platform I can think of. I’m starting to advocate this approach myself — there are now more language-specific versions of make than I can be bothered to learn, so I’ve given up and gone back to make too.

I feel like many Node developers can learn a lot from this project, and what’s more TJ has made it easy to contribute if you’re in a position to help out.

Processing.js 1.2

Processing.js 1.2 has been released. This version improves WebGL performance, and there’s now a guide that shows how Canvas and WebGL relate to Processing’s rendering modes: Understanding Rendering Modes in Processing.js.

There’s a full changelog in the blog post. And, the image above comes from Spring and Processing.js.

Canvas and WebGL Benchmarks

Ilmari Heikkinen posted some interesting jsPerf Canvas and WebGL benchmarks to his blog, which includes a text sources benchmark.

Running the benchmarks will show a comparison against other browsers using Browserscope.

What is WebGL?

What is WebGL? was a talk by Neil Trevett given at the WebGL Meetup group. It’s fairly high-level, but Neil does a good job of explaining how the stack of technologies required for WebGL come together to make it all possible.

It also mentions a project by Google called ANGLE which attempts to translate OpenGL ES 2.0 API calls to DirectX 9 for Windows machines that don’t have OpenGL 2.0 drivers. Interesting stuff!

Promises are something you may have used without realising it. jQuery manages them using Deferred Object, and Dojo has dojo.Deferred which is similar. There are also a growing number of Node modules that offer APIs for working with asynchronous promises, some of which can work in browsers as well.

It’s easiest to understand promises with an example:

$.get('test').then(
  function() { alert('$.get succeeded'); },
  function() { alert('$.get failed!'); }
);

This is from jQuery’s documentation. When the Ajax request succeeds, the first callback will be run, else the second is run. The most important thing is the Ajax method returns an object with a method called then with this signature: then(fulfilledHandler, errorHandler, progressHandler). It might not sound particularly exciting, but it can lead to surprisingly readable and expressive code.

CommonJS

There is a CommonJS proposal for Promises, known as Promises/A. It defines the then method, and also get and call. These methods form what it calls an “interactive-promise”.

In an interactive-promise, the get(propertyName) method will return a promise that requests the given property from the target of the current promise. Meanwhile, call(functionName, arg1, arg2, ...) will request to call the function from the target of the current promise and returns a promise to provide the resulting value.

Notice how this syntax is eminently chainable — various levels of asynchronous operations could be stacked and manipulated.

Practicality

While promises sound like a great way to interact with asynchronous code, the practicality isn’t immediately obvious in a web framework, outside of Ajax calls. However, they can be used to build more advanced functionality within the framework. People don’t necessarily need to fully understand promises to be able to take advantage of $.get('/url').then(function() { }). And promises seem like a good way of modelling animation behaviour without adding a complex library for animation scheduling.

Next

Next week I’ll take a look at building promises in browser-friendly JavaScript. If you’re interested in reading more, take a look at:

• Promises/A
• jQuery Deferred
• q: Defer/when style promise API for JavaScript

Sequelize 1.0.0

Sequelize (GitHub: sdepold / sequelize, License, npm: sequelize) by Sascha Depold has just been updated to version 1.0.0. Sascha has written about the release in Changes in Sequelize 1.0.0.

There’s a nice new API for chained queries:

new Sequelize.Utils.QueryChainer
  .add(person.save())
  .add(pet.save())
  .run()
  .on('success', function() { });

And changes to the way models are instantiated:

// Before
new Model({attr1: 1, attr2: 2}).save()

// Now
Model.build({}).save()
// to first create the instance and save it afterwards or…

Model.create({})
// to create it and directly write it to the database

Other changes include:

• An event-based architecture instead of callbacks
• More convenient syncing
• A count method for models
• New model schema options: unique, primaryKey, autoIncrement

And plenty more!

If you’re interested in Sequelize but haven’t tried it before, www.sequelizejs.com has great documentation to get you started.

node-shell

node-shell (npm: shell) by David Worms is a module for building shell scripts. It’s interesting because it lifts a lot of concepts straight from Express, particular “pluggable” modules. It’s very similar to working with Express.

This is a cut-down example from the author’s documentation:

var shell = require('shell');
var app = new shell.Shell();

app.configure(function() {
    app.use(shell.history({
        shell: app
    }));

    // ... snip
});

// Command registration
app.cmd('redis keys :pattern', 'Find keys', function(req, res, next){
    if(!app.client){
        app.client = require('redis').createClient();
    }
    app.client.keys(req.params.pattern, function(err, keys){
        if(err){ return res.styles.red(err.message), next(); }
        res.cyan(keys.join('\n')||'no keys');
        res.prompt();
    });
});

// Event notification
app.on('redis quit', function(){
    if(app.client){
        app.client.quit();
    }
});

If you’re a web app developer who hasn’t worked with the console before, node-shell looks like it might be a good way to leapfrog a lot of tediousness.

Formaline

Formaline (License: MIT, npm: formaline) by Guglielmo Ferri helps handle form requests and file uploads, complete with connect middleware. It has several interesting features:

• Progress tracking
• HTML5 multiple file upload support
• Speaks JSON
• Tested against malicious and incorrect HTTP headers
• Handles filename collisions
• The SHA1 data checksum of received files can be returned

Basic usage looks like this:

var formaline = require('formaline'),
    form = new formaline({});

form.on('filereceived', function(json) {

});

The resulting json object will contain these properties:

json = { 
  hashname: '..',           // <-- 40 HEX SHA1 STRING ( IT IS THE (SHA1) RESULTING HASH OF FILENAME )
  name: '..',               // <-- FILE ORIGINAL NAME  
  path: '..',               // <-- FILE PATH       
  type: '..',               // <-- MIME TYPE
  size: 217,                // <-- BYTES 
  fieldname: '..',          // <-- FILE FIELD NAME 
  datasha1sum: '..',        // <-- 40 HEX SHA1 STRING  ( IT IS THE (SHA1) RESULTING HASH OF THE FILE'S DATA )
  lastModifiedDate: '..'    // <-- FILE MTIME       
}

Path to Hybrid

Alex Kessinger has posted a video of his Path to Hybrid talk originally given at jQuery Conference 2011. He discusses how to create app-like mobile web apps, and explores the various pitfalls and what jQuery Mobile offers to solve these issues, including “fastclick” — you can read about this in the jQuery Mobile Alpha 4 blog post.

Formly

Formly (GitHub: ThrivingKings / Formly) by Daniel Raftery is a “form glamorizer” for jQuery. It includes CSS and JavaScript necessary for quickly building modern-looking forms, AND validation!

Giving Formly suitable markup will cause it to interpret attributes on each input to add appropriate validation. This HTML fragment:

<form id="ContactInfo" width="600px" title="Member sign up"><br/>
  <input type="password" name="pword" require="true" label="Password" place="Password" /> 
  <input type="password" name="pwordm" match="pword" label="Password" place="Re-type password" /><br/> 

Just requires this JavaScript to kick things off:

$('#ContactInfo').formly({'theme':'Dark'}, function(e) {
  $('.callback').html(e);
});

I feel like this could work better with data attributes and more semantic markup, I don’t think it would take too much effort to make it work this way.

jQuery WormHole

jQuery WormHole (GitHub: JonMidhir / jQuery-Worm-Hole-Plugin, License: Don’t be a Dick (DD)) by Jon Hope is the kind of thing I’m going to need one day and be unable to craft the right set of search terms to find it. It creates an invisible link between elements on a page, enabling jQuery UI’s draggables to disappear in one place and appear in another. There’s a demo on the site that illustrates how it works.

WormHole was built for the Rotify project to allow overnight shifts to be scheduled easily and intuitively. It’ll fit snugly into any similar situation, where the user needs to share child objects amongst discrete containers. The codebase is minimal and it is easily customisable.

This week’s code review is on CoffeeScript.

Why? Well, I really wanted to compare the source of CoffeeScript to Jade because I know TJ Holowaychuk wrote Jade pretty much from scratch whereas CoffeeScript is built with Jison. I find Jade’s source very readable and clean, so I wanted to see what value Jison brought to CoffeeScript.

About CoffeeScript

CoffeeScript (GitHub: jashkenas / coffee-script, npm: coffee-script, License: MIT License) by Jeremy Ashkenas is a language that compiles to JavaScript prior to execution, so there’s no compilation at runtime.

The compiled output is readable and pretty-printed, passes through JavaScript Lint without warnings, will work in every JavaScript implementation, and tends to run as fast or faster than the equivalent handwritten JavaScript.

Installation

Either check out the source from GitHub, download a tarball, or use npm:

npm install -g coffee-script

Usage

CoffeeScript is basically a command-line tool, which can be run with coffee:

alex@b ~$ coffee -h

Usage: coffee [options] path/to/script.coffee

  -c, --compile      compile to JavaScript and save as .js files
  -i, --interactive  run an interactive CoffeeScript REPL
  -o, --output       set the directory for compiled JavaScript
  -j, --join         concatenate the scripts before compiling
  -w, --watch        watch scripts for changes, and recompile
  -p, --print        print the compiled JavaScript to stdout
  -l, --lint         pipe the compiled JavaScript through JavaScript Lint
  -s, --stdio        listen for and compile scripts over stdio
  -e, --eval         compile a string from the command line
  -r, --require      require a library before executing your script
  -b, --bare         compile without the top-level function wrapper
  -t, --tokens       print the tokens that the lexer produces
  -n, --nodes        print the parse tree that Jison produces
      --nodejs       pass options through to the "node" binary
  -v, --version      display CoffeeScript version
  -h, --help         display this help message

Structure

CoffeeScript’s source in lib/ is actually generated from CoffeeScript, found in src/.

Everything stems from bin/coffee which loads lib/command.js to parse command-line options and then uses lib/coffee-script.js to do the real work. This file loads lib/parser.js and lib/lexer.js. The lexer generates a sequence of tokens from the CoffeeScript input, then the Jison parser parses these tokens:

return (parser.parse(lexer.tokenize(code))).compile(options);

The parser is generated by Jison (GitHub: zaach / jison, npm: jison, License: MIT X) by Zach Carter, which is a JavaScript parser generator. Jison is based on Bison, which can generate parsers based on annotated context-free grammars.

Why not just use Bison? Well, Jison provides several things that make it easier for those fluent in JavaScript to work with. Consider this example from Zach’s documentation:

var Parser = require('jison').Parser;
var grammar = {
  "lex": {
    "rules": [
     ["\\s+", "/* skip whitespace */"],
     ["[a-f0-9]+", "return 'HEX';"]
    ]
  },

  "bnf": {
    "hex_strings" :[ "hex_strings HEX", "HEX" ]
  }
};

var parser = new Parser(grammar);

// generate source, ready to be written to disk
var parserSource = parser.generate();

// you can also use the parser directly from memory

// returns true
parser.parse("adfe34bc e82a");

This example uses a parser to determine if the input contains a hex string. BNF refers to Backus-Naur Form which is a notation for describing context-free grammars — ideal for describing the syntax of the input language. Notice that the grammar is represented in JSON — it’s almost trivial for us to understand and write this.

This is all documented in docs/grammar.html. The grammar.coffee file is the real core of the project. Once the grammar has been described, Jison can interpret and parse it.

Grammar Examples

The grammar file provides a top-down view of how CoffeeScript works internally. One of the interesting features of CoffeeScript is the function arrow, which reads like this:

square = (x) -> x * x

race = (winner, runners...) ->
  print winner, runners

It’s defined in the grammar like this:

  # The **Code** node is the function literal. It's defined by an indented block
  # of **Block** preceded by a function arrow, with an optional parameter
  # list.
  Code: [
    o 'PARAM_START ParamList PARAM_END FuncGlyph Block', -> new Code $2, $5, $4
    o 'FuncGlyph Block',                        -> new Code [], $2, $1
  ]

  # CoffeeScript has two different symbols for functions. `->` is for ordinary
  # functions, and `=>` is for functions bound to the current value of *this*.
  FuncGlyph: [
    o '->',                                     -> 'func'
    o '=>',                                     -> 'boundfunc'
  ]

  # An optional, trailing comma.
  OptComma: [
    o ''
    o ','
  ]

  # The list of parameters that a function accepts can be of any length.
  ParamList: [
    o '',                                       -> []
    o 'Param',                                  -> [$1]
    o 'ParamList , Param',                      -> $1.concat $3
  ]

  # A single parameter in a function definition can be ordinary, or a splat
  # that hoovers up the remaining arguments.
  Param: [
    o 'ParamVar',                               -> new Param $1
    o 'ParamVar ...',                           -> new Param $1, null, on
    o 'ParamVar = Expression',                  -> new Param $1, $3
  ]

  ParamVar: [
    o 'Identifier'
    o 'ThisProperty'
    o 'Array'
    o 'Object'
  ]

These grammar actions are used to generate nodes, found in src/nodes.coffee:

# A function definition. This is the only node that creates a new Scope.
# When for the purposes of walking the contents of a function body, the Code
# has no *children* -- they're within the inner scope.
exports.Code = class Code extends Base
  constructor: (params, body, tag) ->
    @params  = params or []
    @body    = body or new Block
    @bound   = tag is 'boundfunc'
    @context = 'this' if @bound

All nodes in the syntax tree descend from a class called Base. The most important method in this class is compile:

compile: (o, lvl) ->
  o        = extend {}, o
  o.level  = lvl if lvl
  node     = @unfoldSoak(o) or this
  node.tab = o.indent
  if o.level is LEVEL_TOP or not node.isStatement(o)
    node.compileNode o
  else
    node.compileClosure o

Each subclass implements compileNode which is used to compile a given node to JavaScript. For the function literal example, this gets run:

code  = 'function'
code  += ' ' + @name if @ctor
code  += '(' + vars.join(', ') + ') {'
code  += "\n#{ @body.compileWithDeclarations o }\n#{@tab}" unless @body.isEmpty()
code  += '}'

This should look familiar!

Conclusion

Back in the Let’s Make a Framework series, I created a mini CSS parser which included a lexer. I was reminded of this here, and wondered if using Jison for CSS parsing might be an interesting exercise.

The basic process at work in CoffeeScript is very simple, although it can look daunting at first:

• The lexer uses regular expressions to generate tokens that can be fed into a parser
• The parser is generated from a Jison grammar
• The parser then builds a representation of the tokens using nodes

If you wanted to build your own JavaScript-powered language using Jison, you’d need to implement your own lexer, Jison grammar, and something to interpret the output of the Jison parser.

If you want to dig further into the CoffeeScript source, look at the CoffeeScript annotated source in the menu on the homepage. As a whole the project has extremely readable comments.

Next week I’ll have a look at Jade so we can see what a parser written in JavaScript without Jison looks like.

London WebGL

Mozilla Labs has a good post up about the London WebGL Meetup in a post entitled London WebGL Meetup Yesterday. The event was organised by Giles Thomas who writes Learning WebGL (featured previously on DailyJS here: JavaScript 3D).

When asked for “How many of you know what the Demoscene is?” 90% raised their hands. Record!

Mr. Doob was there too, it sounded like a great event. I’d love to attend the next one!

JSONSelect

JSONSelect (GitHub: lloyd / JSONSelect, License, npm: JSONSelect) by Lloyd Hilaiel is a selector language for JSON. There’s an interactive demo on the project’s site which shows the supported selectors.

The author writes about JSONSelect more like a language than a convenient Node parser — in fact he calls the JavaScript code a reference implementation.

Here’s an example of the npm package in action:

var jsonselect = require('jsonselect');

jsonselect.match('.name', { people: [ { name: 'Alex' }, { name: 'Jones' } ] });
// [ 'Alex', 'Jones' ]

Do you remember when I reviewed TJ Holowaychuk’s finance app in the Finance Code Review? Well, TJ included a little file-based database using JSON. JSONSelect could be the perfect partner for that approach, particularly for small open source Express web apps.

Welcome to part 64 of Let’s Make a Framework, the ongoing series about building a JavaScript framework.

These articles are tagged with lmaf. The project we’re creating is called Turing. Documentation is available at turingjs.com.

For the past three weeks I’ve been looking at accessing element attributes:

• Element Attributes
• Element Attributes Part 2
• Element Attributes Part 3

When we talk about attributes we’re referring to the actual HTML attributes. These values can differ from the DOM element’s properties. The distinction is somewhat confusing, and many frameworks attempt to iron out the differences behind the scenes. jQuery ended up providing access to both attributes and properties.

Properties vs. Attributes

The most obvious case that demonstrates the difference between properties and values is checkboxes. Accessing a checkbox’s checked attribute should return 'checked', whereas the property will return true.

Perhaps that isn’t completely obvious, but if you play around with this you’ll see what I mean:

<input id="check" type="checkbox" checked="checked" />

Tests

To test if these checkboxes really behaved this way, I wrote these tests for the prop method:

'test getting properties': function() {
  var checkbox = turing.dom.get('#checkbox')[0];

  assert.equal(turing.dom.prop(checkbox, 'checked'), true);
  assert.equal(turing('#checkbox').prop('checked'), true);
},

'test setting properties': function() {
  var checkbox = turing.dom.get('#checkbox')[0];
  
  turing.dom.prop(checkbox, 'checked', false);
  assert.equal(turing.dom.prop(checkbox, 'checked'), false);
},

'test removing properties': function() {
  var checkbox = turing.dom.get('#checkbox')[0];

  turing.dom.removeProp(checkbox, 'checked');
  assert.eqyal(turing.dom.prop(checkbox, 'checked'), undefined);
}

Implementation

This is a basic implementation that just access an element’s properties, applying propertyFix from the previous tutorials when required:

/**
 * Get or set properties.
 *
 * @param {Object} element A DOM element
 * @param {String} attribute The property name
 * @param {String|Number|Boolean} value The property value
 */
dom.prop = function(element, property, value) {
  if (propertyFix[property])
    property = propertyFix[property];
  if (typeof value === 'undefined') {
    return element[property];
  } else {
    if (value === null) {
      return dom.removeProperty(element, property);
    } else {
      return element[property] = value;
    }
  }
};

And this is the corresponding DOM chain method:

/**
 * Get or set a property.
 *
 * @param {String} property The property name
 * @param {String} value The property value
 * @returns {String} The property value
 */
prop: function(property, value) {
  if (this.elements.length > 0) {
    return dom.prop(this[0], property, value);
  }
},

It accesses the first element, if one has been found.

Attribute Implementation

This will actually fail in IE:

assert.equal(turing.dom.attr(checkbox, 'checked'), 'checked');

To get around this, we need to test to see if an attribute is a boolean attribute. A boolean attribute should return its own name when it’s set, which means IE’s getAttribute code should test for this and return the expected value:

booleanAttributes = {
  'selected': true,
  'readonly': true,
  'checked':  true
};

function getAttribute(element, name) {
  if (propertyFix[name]) {
    name = propertyFix[name];
  }

  if (getAttributeParamFix[name]) {
    return element.getAttribute(name, 2);
  }

  if (name === 'value' && element.nodeName === 'BUTTON') {
    return element.getAttributeNode(name).nodeValue;
  } else if (booleanAttributes[name]) {
    return element[name] ? name : undefined;
  }

  return element.getAttribute(name);
}

I’ll have to add more booleanAttributes later on.

Removing Properties

When I tried to remove the checked attribute in IE it raised an exception which said Object doesn’t support this action. I decided to give up and catch the exception:

dom.removeProp = function(element, property) {
  if (propertyFix[property])
    property = propertyFix[property];
  try {
    element[property] = undefined;
    delete element[property];
  } catch (e) {
  }
};

Conclusion

Property manipulation built on what we’d already achieved with attributes, but has a few browser quirks of its own to deal with. A naive implementation can be built easily, but supporting a wide range of browsers is where the real effort comes in.

This week’s code is commit f385e09.

Dialect

Dialect (License: MIT, npm: dialect) by Pau Ramon Revilla is an internationalisation API for Node that stores translations in memory. The translations are periodically synced to MongoDB, so it’s fast and easy to serve multiple clients with.

The author’s example demonstrates this:

var dialect = require('dialect').dialect({current_locale: 'es', store: {mongodb: {}}});

// connects to the store
dialect.connect(function () {

  // syncs the memory dictionaries with the store
  dialect.sync({interval:3600}, function (err, foo) {
    d.get('Hello World!'); // => Hola mundo
  });
});

The mongodb property expects an object with database, host, and other connection options. I’d prefer to see a connection URI here, but it’s fairly easy to get going with MongoDB or SQLite.

The API has a lot of features that I’ve come to expect from a good i18n library — plurals, contexts, and interpolation are all supported.

The best thing about Dialect, however, is dialect-http (License: MIT, npm: dialect-http). This is a nice little web app that translators can use to find missing translations and fill them in. It’s one of the better open source examples of this type of tool that I’ve seen.

I’m having some trouble running dialect-http; I think the author might need to try running it on a fresh npm install with a blank database.

node-jus-i18n

node-jus-i18n (npm: jus-i18n) by Nicolas Chambrier also supports plurals, context, and interpolation. It comes with support for various data stores: module, file, gettext, and he’s also working on various databases. The documentation is still a little bit confusing, but he’s written a lot of examples and future plans.

There’s integration with Express, which means req objects get an i18n object:

app.configure(function() {
  i18n.enableForApp(app, { // options (all are optional, you can pass {} or undefined
    "locale": "en",          // default locale
    "catalogue": "messages", // catalogue to load
    "locales": undefined,    // locales to load
  }, function(err) { // called when i18n has loaded messages
    ...
  });
});

req.i18n.translate(...)

And there are helpers for use in templates:

_('Hello, {name}', {name: userName});
plural('You have {n} messages', nbMessages);

Polyglot

Polyglot (npm: polyglot) by Ricardo Tomasi is aimed squarely at Express apps. Ricardo suggests some helper aliases in the README:

app.helpers({
    __: i18n.translate
  , languages: i18n.languages
  , n: i18n.plural // optional
})

As you can see, it does basic translation lookups and plurals. I’m not sure what’s going on with the README, because it refers to express- and express-voyage as package names, but I installed it with polyglot as suggested by the package.json file.

jQuery UI 1.9 Milestone 5 – Tabs Redesign

The jQuery UI blog covered the most recent milestone release in jQuery UI 1.9 Milestone 5 – Tabs Redesign. This version updates the Tabs API to make it more consistent with other jQuery UI widgets.

Details of the changes are available in this post: Tabs API Redesign.

jquery-floating-widget

I was trying to vertically float an advert using CSS so it was always visible on the page, but only scrolled up to a certain point. It was hard to get it to behave properly with the particular layout I was using, so I ended up writing some JavaScript to do it.

If only I had jquery-floating-widget! This plugin, by Takeru Suzuki, does exactly what I needed. Just drop it in a page with $('.floating-widget').floatingWidget(); and some suitable CSS (detailed in this blog post) and you should get something like this: jquery-floating-widget demo.

Ninja UI

Ninja UI (GitHub: ninja / ui, License: Apache 2.0) by Jamie Hoover and Faisal Jawdat is a plugin that includes lots of UI widgets and icons.

The API looks a bit like jQuery UI:

/* Tabs */
var $tabs = $.ninja().tabs({
  choices: [{
    html: 'One',
    select: function () {
      console.log('Local tab function called.');
    }
  },
  {
    html: 'Two',
  },
  {
    html: 'Three',
  }],
  choice: 2
}).select(function (event) {
  console.log('Global tab function called returning: ' + event.html);
});
$examplesBody.append($title.clone().text('Tabs'), $tabs);

I pulled this from the examples.js file on the project’s site.

This week’s code review is on Calipso by Clifton Cunningham. I only found out about Calipso today, but so many people seemed to want to learn more about Express apps that I thought it would be worth checking it out.

About Calipso

Calipso (GitHub: cliftonc / calipso, License: MIT) is a content management system, inspired by popular tools like WordPress and Drupal. It already has some cool features:

• Create themes with the Express-compatible view rendering engines and Stylus
• Background scheduler for importing content from feeds
• Modular structure
• Internationalisation

Installation

You’ll need Node, npm, and MongoDB to use Calipso.

Check the code out with Git, then set up the packages with npm:

$ git clone git@github.com:cliftonc/calipso.git
$ cd calipso
$ npm install
$ node app

Then visit http://localhost:3000/ to see a suspiciously empty site.

Usage

A fresh Calipso install makes a lot more sense when logged in, so log in with admin/password first. There’s a settings section under Admin which allows modules to be turned on. Content can be created using the navigation under Content, and the Content Type section is used to manage the types of pages a site will use.

App Structure

Calipso has a structure that is deceptively similar to most Express apps. The app.js file is fairly light — it just deals with booting the application. The configuration files can be found in conf/ which are split into environments (development, production, test). There’s also a file for managing settings which saves them to the database.

There aren’t view or controller directories because this part of the app is split into themes and modules.

Loading Settings

Settings are loaded from a similar pattern to something I use to separate out potentially lengthy Express settings:

module.exports = function(app,express) {
  // Database connection
  app.set('db-uri', 'mongodb://localhost/calipso-dev');

  // ...
}

The environment configuration files are then loaded by the configuration manager:

var NODE_ENV = global.process.env.NODE_ENV || 'development';
app.configure(NODE_ENV, function() {
  require("./"+NODE_ENV+".js")(app, express);
  loadConfig(app, defaultConfig, function(err, config) {
    app.set('config', config);
    next(err);
  });
});

Database

Mongoose is the database library, so there are a lot of schemas kicking around:

• AppConfigSchema: Stores the site’s various settings
• AppModule: Module status, embedded within AppConfigSchema
• User and Role: Used by the user module for managing users
• Tag: Used to create tag clouds
• TaxonomyMenu: Used by the taxonomy module for creating menus
• ScheduledJob: Stores background tasks that are run by cron
• Media and MediaGallery: Used to manage uploads
• Content and ContentType: Generic content management

The Mongoose library is passed around inside modules:

var MediaGallery = new calipso.lib.mongoose.Schema({
  name: {type: String, required: true},
  description: {type: String, required: true},
  author: {type: String, required: true},
  ispublic: {type: Boolean, required: true, default: false},
  created: {type: Date, default: Date.now},
  updated: {type: Date, default: Date.now}
});

This is true for other shared libraries as well. The calipso module is passed around everywhere so modules don’t need to keep loading the same libraries.

From what I can tell, the author has used sensible data types and embedded documents where they make sense.

Calipso Modules

Calipso abstracts Express away from modules, and makes heavy use of Step to get parallel execution for some boot time and reloading speed improvements.

Modules are defined like this:

exports = module.exports = {
  init: init,
  route: route,
  install: install,
  reload: reload,
  disable: disable,
  about: {
    description: 'Sample content to test themes.',
    author: 'cliftonc',
    version: '0.2.0',
    home: 'http://github.com/cliftonc/calipso'
  }
};

Every property except about is a function. Some have signatures with req, res like Express actions, others have extra values from Calipso, and init takes module, app, next.

Routing

Calipso’s routing is Express middleware, which has allowed the author to distance Calipso’s concept of modules from Express’s API.

Logging

Calipso logs a lot of stuff, so the author has decided to use winston which is an asynchronous logging library. This is actually a very flexible library — it can log to the console, files, Loggly, or potentially anywhere.

Forms

Forms are generated using CalipsoForm, which takes readable JSON input and produces HTML. This is used inside modules, and there’s a good example in modules/community/sample-content/:

function renderSampleContentPage(req, res, template, block, next) {
  calipso.form.process(req, function(incomingForm) {
    var sampleForm = {
      id: 'sample-content-form',
      cls: 'sample-content-form',
      title: 'Sample Content Form',
    // ...

    calipso.form.render(sampleForm, incomingForm, req, function(form) {
      calipso.theme.renderItem(req, res, template, block, {
        form: form
      });
      next();
    });

Internationalisation

I was extremely pleased to see some effort towards internationalisation in Calipso. It has its own translation code which is exposed as Connect middleware. That means the translation module can access the session to determine what language is selected.

The translation module keeps a cache around for lookups, which are just a case of looking up the string based on the English string, then replacing any embedded values. Curly braces are used for interpolation: "{name} Profile":"{name} Profile".

There are a few modules that provide similar functionality, available through npm. I’ve been looking at node-jus-i18n and Polyglot.

Conclusion

Calipso builds on Express, but it feels very different to a typical Express app because of the module loading system. Many apps would benefit from using the settings loading approach, and I’m definitely going to look at using winston in my next project.

Passing around the shared calipso object is interesting as well, it reduces the need to keep loading key libraries.

There’s a lot of Mongoose code here as well, so if you’re looking to learn more about that it might be worth digging in. And, if you’re interested in learning how to write Calipso modules, Clifton writes extremely detailed code comments so it shouldn’t be too hard to get started with his template module.