WebGL Materials: Skin

This three.js skin demo by AlteredQualia (@alteredq) uses a skin shader based on a GDC 2007 presentation from Nvidia and GPU Gems 3 Chapter 14. Advanced Techniques for Realistic Real-Time Skin Rendering. It seems to perform very well (I tested it on a fairly lightweight laptop and my desktop).

Mr.doob also tweeted about this three.js facial rigging demo. It’s interesting to see techniques from games development cropping up in work by three.js developers!

Swarming Example

This jsFiddle swarming demo is interesting because it shows everything used to create the WebGL demo, complete with a panel containing the 3D Canvas results. I did some digging to find the author and found him on Reddit, here: Swarms battle.

It was meant as an attempt to learn 3d projection, and I just threw in the swarming for fun, and then some lasers, and then some explosions.

He says he adapted 2D swarming code and searched around online for the appropriate specs to get the drawing code working.

Paladin

Paladin (GitHub: alankligman / paladin, License) by Alan Kligman and Mozilla is “Mozilla’s movement to provide open source gaming technology for the web”:

Paladin sits at the intersection of 3D gaming, JavaScript framework and library development, and the browser. We’re tied into the bits of the web that are up-and-coming, and are working to weaponize them for gaming. Where the web is missing critical gaming support, we aim to fill those gaps by adding new browser APIs, enhancing existing ones, and building technologies on top of the web..

This includes libraries for 3D graphics, sound, and user input APIs for joysticks and mice. Actually, they’d probably be better off referencing “gamepads” seeing as the 360 pad seems to popular amongst modern PC gamers. Even though the project is at an early stage there’s already a lot of code, complete with qunit tests.

There’s also a project called RescueFox which is a demo game that they hope will inspire other developers to build things with Paladin. And, before I forget, it seems like “Paladin” is only the work-in-progress name, so keep an eye on the Mozilla Paladin Wiki to keep track of future developments.

I was impressed by the Superagent HTTP library, so I decided to see what improvements I could make to turing.net based on it. Along the way I found some IE compatibility issues that I hadn’t spotted before, and improved the functional testing script.

Response and Requests

Both Express and Superagent pass abstracted response and request objects back to callbacks. I found the need for this arose when I noticed IE6 didn’t like me adding properties to the ActiveXObject it uses to support XMLHttpRequest. To fix this I decided to take some inspiration from these projects and provide an abstracted response object to callbacks:

var response = {};

response.status = request.status;
response.responseText = request.responseText;
if (/json/.test(contentType)) {
  response.responseJSON = net.parseJSON(request.responseText);
} else if (/xml/.test(contentType)) {
  response.responseXML = net.parseXML(request.responseText);
}

if (successfulRequest(request)) {
  if (options.success) options.success(response, request);
  if (promise) promise.resolve(response, request);
} else {
  if (options.error) options.error(response, request);
  if (promise) promise.reject(response, request);
}

Now the callbacks get a more friendly response object, as well as the original XMLHttpRequest request object. The JSON and XML tests were also added here — previously only JSON was parsed, but I added support for XML as well.

Parsing XML

In Parsing and serializing XML on MDN, the following fragment is suggested for parsing XML:

var theString='<a id="a"><b id="b">hey!</b></a>';
var parser = new DOMParser();
var dom = parser.parseFromString(theString, "text/xml");
// print the name of the root element or error message
dump(dom.documentElement.nodeName == "parsererror" ? "error while parsing" : dom.documentElement.nodeName);

The resulting XML isn’t parsed into a plain JavaScript Object but a Document instead, which means methods and properties like nodeName are available.

Microsoft’s approach is again to use ActiveXObject:

// Instantiate a DOM object at run time.
var dom = new ActiveXObject("msxml2.DOMDocument.6.0");
dom.async = false;
dom.resolveExternals = false;
dom.loadXML("<a>A</a>");

This is from InstantiateDOM.js at MSDN.

I decided to define a parseXML method once based on browser support:

  /**
    * Parses XML represented as a string.
    *
    * @param {String} string The original string
    * @returns {Object} A JavaScript object
    */
  if (window.DOMParser) {
    net.parseXML = function(text) {
      return new DOMParser().parseFromString(text, 'text/xml');
    };
  } else {
    net.parseXML = function(text) {
      var xml = new ActiveXObject('Microsoft.XMLDOM');
      xml.async = 'false';
      xml.loadXML(text);
      return xml;
    };
  }

To test this, I wrote the following:

'test xml parsing': function() {
  $t.post('/give-me-xml', {
    contentType: 'application/xml',
    success: function(r) {
      assert.equal('key', r.responseXML.documentElement.nodeName);
    }
  });
}

This runs through an Express app, in test/functional/ajax.js.

Promises

As I mentioned, IE doesn’t like modifying the XMLHttpRequest object it provides through ActiveX. I was setting a then property on request objects to support promises which was only used because the XMLHttpRequest was being returned from the network-related methods. To get around the IE issue, I decided just to return an empty object with a then property, so this is still possible:

$t.get('/get-test').then(
  function(r) { assert.equal('Sample text', r.responseText); },
  function(r) { assert.ok(false); }
);

I made the documentation slightly ambiguous about what the network methods return because I suspect returning the current turing object to allow other chaining might be more useful. It currently reads @returns {Object} An object for further chaining with promises, which is what I intended it to do in the first place.

Conclusion

After all that, I never really got to use any of TJ’s ideas from Superagent, other than abstracting response objects. The most important thing to remember when creating cross-browser code is to be careful about extending native objects. That’s a good rule of thumb for JavaScript in general, and I feel doubly embarrassed about doing it in the first place because I’ve been preaching this for a while!

You can get this week’s code in commit 93a5d75.

Node Knockout Winners

The Node Knockout Winners have been announced!

• Overall winner: Observer by Speedo
• Popularity: Driv.in by Go Horse Brazil
• Utility / Fun: Doodle or Die by opower
• Design: ACROnode.com by rochester-js
• Innovation: Blue GPU Lava by Minimason
• Completeness: Chess@home by Joshfire

The entries this year were extremely diverse and creative. It’s inspiring to see so many entries that are nothing to do with business software or social networks.

readabilitySAX

readabilitySAX (npm: readabilitySAX) by Felix Böhm is a port of Readability that attempts to improve on the direct ports of Readability’s JavaScript to Node. The author claims it’s faster (the source includes benchmarks), and uses a SAX parser to handle the required HTML parsing.

The easiest way to see how it works is by looking at this readabilitySAX jsFiddle example.

And the README includes some benchmark information:

Using a (jsdom cleaned) package of 620 pages from CleanEval, readabilitySAX processed all of them in 10874ms, that’s an average of 17.5387ms per page. The benchmark was done using benchmark.js and is probably far from perfect.

Browserling: 90 Open Source Node Modules

I enjoyed perusing this list of 90 Node modules written by Browserling that includes lots of cool stuff by Peteris Krumins and James Halliday. If writing Node modules is addictive, then James should definitely see someone! Or at least take a vacation somewhere sunny.

jQuery 1.6.3

jQuery 1.6.3 has been released, not long after RC1. This version fixes a much discussed XSS attack, animation browser tab switchiing issues, and better handling of HTML5 attribute names.

lccache

lccache (License: Apache 2.0) by Pamela Fox and John Munsch emulates memcache for client-side caching. John Munsch’s port uses Lawnchair instead of HTML5 Local Storage, which means it should work in older browsers.

The library has three methods: set, get and remove:

//          key,        value,          time/expiration
lccache.set('greeting', 'Hello World!', 2);

lccache.get('greeting');
// Returns 'Hello World!'

// Objects can also be stored
lccache.set('data', { 'greeting': 'Hello World!' }, 2);
lccache.get('data').greeting;

Pamela’s real world examples included developing against JSON APIs on unreliable networks. The most obvious example is a mobile web app. I’d also consider it for single page apps, because it makes knowing when to refresh data that might change on the server a lot easier.

Storagify

Storagify (GitHub: ekdevdes / Storagify, License: MIT/GPL) by Ethan Kramer combines HTML5 contenteditable with Local Storage to make potentially any element editable. Calling $('selector').storagify('storageKey'); makes the element editable, and changes will be visible after a page refresh.

Now imagine combining this with a simple JSON API and you’ve got yourself a CMS in minutes!

Search by TJ Holowaychuk is a small ack inspired utility for searching source code. I picked this project out in particular because it showcases TJ’s nifty Commander.js library and demonstrates how easy it is to build fast command line utilities with Node.

Installation and Usage

Search can be installed with npm install -g search. This clashes with my Sphinx binaries, so I installed it in ~/ and set up an alias in my shell.

To use it, navigate to a directory with lots of code and run search text, where text is a case insensitive regular expression.

Structure

Like TJ’s other projects, this is distributed with a README, Makefile, history, and all of the code is in bin/search. I’d argue against putting all of the code in bin/search so I could require core modules elsewhere (potentially making testing easier), but the binary itself could be tested as well.

Set Up

As I mentioned, this project uses Commander.js, which makes setting everything up a breeze:

var program = require('commander')
  , path = require('path')
  , join = path.join
  , fs = require('fs');

// options

program
  .version('0.0.4')
  .usage('[options] <query> [path ...]')
  .option('-H, --hidden', 'search hidden files and directories')
  .parse(process.argv);

// no args

if (!program.args.length) {
  process.stdout.write(program.helpInformation());
  process.exit(0);
}

The chainable API allows everything that describes the program to be described in a concise manner.

Next, the paths and query are processed. The regular expression is instantiated once for performance reasons:

var query = program.args.shift()
  , paths = program.args
  , pending = paths.length
  , re = new RegExp('(' + query + ')', 'ig');

Searching

The main searching code is similar to code I’ve written a few times for my Node apps, which is one of the reasons I picked this for a code review. It uses Node’s asynchronous file system APIs to recursively walk over each path and their children. However, the slight twist here is TJ uses Array.prototype.(forEach|filter|map) rather than for loops. A lot of people use for loops over iterators for performance reasons, reducing scoping complexity, or browser support. It’s worth considering this counter example in terms of readability.

I’ve added some comments to explain how it works:

function search(path) {
  // Does this file exist?
  fs.stat(path, function(err, stat){
    if (err) throw err;
     
    // Is it a directory?
    if (stat.isDirectory()) {
      // If it's a directory, remove hidden files using the hidden() function (defined below),
      // then generate a list of file names with the current path, then run search() again on the resulting paths
      fs.readdir(path, function(err, files){
        if (err) throw err;
        files.filter(hidden).map(function(file){
          return join(path, file);
        }).forEach(search);
      });

The next part reads through each file and searches each line for the regular expression. Output is printed directly with console.log, and colour codes are inserted to make the matches easier to spot.

    } else if (stat.isFile()) {
      var lines = [];
      fs.readFile(path, 'utf8', function(err, str){
        if (err) throw err;
        str.split('\n').forEach(function(line, i){
          if (!re.test(line)) return;
          lines.push([i, line]);
        });

        if (lines.length) {
          console.log('\n  \033[36m%s\033[0m', path);
          lines.forEach(function(line){
            var i = line[0]
              , line = line[1];
            line = line.replace(re, '\033[37;43m$1\033[0;90m');
            console.log('  \033[90m%d: %s\033[0m', i+1, line);
          });
        }
      });

Conclusion

TJ makes writing command line apps look easy. That’s partly because it actually is! The next time you’re itching to solve an interesting console-based problem, try scripting something with Node. There are lots of projects similar to Search that you can reference to get a head start.

Contracts.coffee

Contracts.coffee (GitHub: disnet / contracts.coffee) by Tim Disney is a new dialect of CoffeeScript that adds contracts.

Contracts let you clearly express how your code behaves, and free you from writing tons of boilerplate, defensive code.

// id is a function that should always be called with a number and return a number
id :: (Num) -> Num
id = (x) -> x

When I first looked at this I thought I was looking at Haskell, and the author points out the similarity:

It looks a lot like types (in fact the syntax looks a lot like Haskell) but unlike types, contracts are enforced at runtime in pure JavaScript.

The Contracts.coffee site includes lots of details on how to get started using the dialect. I don’t usually write CoffeeScript, but this twist on the language is definitely extremely interesting.

TJ Holowaychuk on Game Prototyping

In Game prototyping with JavaScript & CSS3, TJ talks about game prototyping with CSS3, HTML5, and move.js (GitHub: visionmedia / move.js, License: MIT). TJ discusses how he started writing a game for iOS, moved to Canvas, and then discovered the performance gains of CSS3 and HTML.

He’s posted a game-prototype to GitHub, and I hope to see more (particularly as he mentions Grim Fandango in his blog post).

Persistence.js

Jacob Mumm emailed us about his experiences using persistence.js, an asynchronous ORM mapper by Zef Hemel. In particular, persistence.js comes with persistence.sync.js, a remote server synchronisation plugin:

persystence.sync.js is a persistence.js plug-in that adds data synchronization with remote servers. It comes with a client-side component (persistence.sync.js) and a sample server-side component (persistence.sync.server.js) for use with node.js. It should be fairly easy to implement server-components using other languages, any contributions there are welcome.

I’d like to write more on mobile sync in the future, as I’ve done a lot of work on native app syncing (mostly Objective-C), and I’ve also recently been working with single page client-side apps that sync against remote APIs. If you’re interested in this area, give persistence.sync.js a look.

Last week we had a very interesting discussion about optimising the deceptively simple hasClass function that I wrote for Turing’s DOM module. Not only is optimisation difficult, once you bring browsers into the mix it can seem like a dark art. Particularly when cross-browser issues are taken into account, which is why I’ve covered things like cached browser feature detection in previous tutorials.

I mentioned that we really needed client-side benchmarks to talk confidently about performance, because my benchmark example script was just intended to be used in Node. As I already used Benchmark.js (GitHub: bestiejs / benchmark.js, License: MIT, npm: benchmark) I’ve used it again for browser benchmarks. And guess what? It even works in IE6!

Writing Browser Benchmarks

I’ve added "benchmark": "latest" to the devDependencies in the package.json file. Then, at the bottom of a HTML test harness file, I added a script tag to load Benchmark.js.

    <script src="../../node_modules/benchmark/benchmark.js" type="text/javascript"></script>
  </body>
</html>

Next I wrote a pure JavaScript file for the DOM-related benchmarks and added it to the other script tags:

var suite = new Benchmark.Suite,
    div = $t('#test-div')[0],
    cache = {};

function log(text) {
  $t('#results').append('<li>' + text + '</li>');
}

function hasClassRegExp(element, className) {
  if (element.className && element.className.length) {
    return new RegExp('(^|\\s)' + className + '($|\\s)').test(element.className);
  } else {
    return false;
  }
};

function hasClassCachedRegExp(element, className) {
  if (!cache[className]) {
    cache[className] = new RegExp('(^|\\s)' + className + '($|\\s)');
  }
  if (element.className && element.className.length) {
    return cache[className].test(element.className);
  } else {
    return false;
  }
};

suite.add('hasClassRegExp', function() {
  hasClassRegExp(div, 'example1');
  hasClassRegExp(div, 'unknown');
})
.add('hasClassCachedRegExp', function() {
  hasClassCachedRegExp(div, 'example1');
  hasClassCachedRegExp(div, 'unknown');
})
.add('built-in', function() {
  turing.dom.hasClass(div, 'example1');
  turing.dom.hasClass(div, 'unknown');
})
.on('cycle', function(event, bench) {
  log(String(bench));
})
.on('complete', function() {
  log('Fastest is ' + this.filter('fastest').pluck('name'));
  $t('#notice').text('Done');
})
.run(true);

Benchmark.js uses callbacks and events to organise benchmarks. That means you need to instantiate a suite using var suite = new Benchmark.Suite, then add benchmarks using suite.add('name', function() {}). It allows chaining, so as you can see I’ve added a few benchmarks and then watched for two events, cycle and complete. The cycle event will run after each benchmark. Easy!

I’m using the $t Turing alias to do some simple DOM manipulation for displaying results. The log function could actually be placed in a benchmark helpers file once more benchmarks have been added. Just out of interest, I kept the old simple hasClass functions and also included the one currently implemented in turing.dom.hasClass.

This benchmark also includes hasClassCachedRegExp. I noticed that Zepto caches regexes, and it turns out this performs extremely well in Firefox and Chrome, but not so well in IE6. However, remember that when comparing the built-in function, you might be looking at element.classList depending on the browser. In Firefox, Ryan Cannon’s String.prototype.indexOf solution performs better than element.classList.

Given that each browser appears to have different performance characteristics, should we use different functions? I’d probably never do this, unless I was targeting a specific browser. This might sound unusual, but plenty of people are developing games that can only run in WebKit mobile browsers (and Zepto specifically targets WebKit).

Results

Chrome 13, Mac:

Firefox 6, Mac:

Internet Explorer 6, Windows XP, VirtualBoxVM:

Conclusion

If you’re working on client-side code, it doesn’t take much work to be scientific about benchmarks. And, using Node and npm to manage your tools can make it quick to set things up. When writing optimised code, don’t champion a given solution — be scientific, experiment, and try to discover the solution most suited to the task at hand. In the interest of science, benchmarks like these should be run on a wide range of machines (not just virtual machines, but I use those purely for convenience).

This code can be found in commit 095a229.

Buildr, Query-Engine, DocPad

Benjamin Arthur Lupton sent us three packages to look at:

• Buildr (License: MIT, npm: buildr) — tools for building and merging CoffeeScript, Less, and CSS
• Query-Engine (License: MIT, npm: query-engine) — a NoSQL query engine
• DocPad (License: MIT, npm: docpad) — a static site generator

All of the projects have installation and usage guides in the README files. Buildr supports bundling, compression, and client-side loading. Query-Engine can manage collections of objects, but also optionally extends Object to work with other objects. DocPad’s templating engine has complete access to the DOM, which means it can be used a bit like a novel CMS.

Search

Search (License: MIT, npm: search) by TJ Holowaychuk is a simple file searching utility that works a bit like ack (my tool of choice, which I use with vim). It’s built with TJ’s new library commander.js, and has deliciously colourised output:

Is it faster than ack? On my machine it performs slightly slower than ack, but not noticeably (there’s a pause after all the results have been displayed). Given how simple the source is, it seems like an extremely hackable version of ack.

Node Knockout

Node Knockout has had a ridiculous 183 entries this year! I’ve been playing around with the entries and some of them seriously give the startups I’ve seen on Y Combinator a run for their money. Check them out, show your support, and get ready for the winners on the 6th.

jQuery 1.6.3 RC1

jQuery 1.6.3 RC1 is out, which means the team is looking for feedback. The preferred way to submit bug reports is jsFiddle using the jQuery (edge) option.

The requestAnimationFrame has been removed to avoid problems caused when browsers schedule animations for hidden tabs. An XSS attack vector has been fixed by stopping HTML passed to $() from being evaluated if a # precedes the expression. HTML5 attribute support has also been improved.

Spin.js

Spin.js (GitHub: fgnass / spin.js, License: MIT) by Felix Gnass is a small JavaScript library with no dependencies that creates spinners using CSS3 (or VML for Internet Explorer). I seem to remember a similar thing appearing on 37signals’ blog a year ago, but this library makes it easy for anyone to drop spinners into a page with little trouble.

The author also has a suggested jQuery plugin wrapper in the documentation:

$.fn.spin = function(opts) {
  this.each(function() {
    var $this = $(this),
        spinner = $this.data('spinner');

    if (spinner) spinner.stop();
    if (opts !== false) {
      opts = $.extend({color: $this.css('color')}, opts);
      spinner = new Spinner(opts).spin(this);
      $this.data('spinner', spinner);
    }
  });
  return this;
};

jQuery imagesLoaded

jQuery imagesLoaded (GitHub: desandro / imagesloaded) by David DeSandro (and based on a script by Paul Irish) is a small plugin for tracking when images have loaded inside a parent element. You get a callback which receives an array of images:

$('selector').imagesLoaded(function($images) {
  // Callback
});

When I sign in to GitHub I keep noticing new watchers appear on the Nodepad project (it’s currently at 207 watchers). Although the Let’s Make a Web App tutorials have already noticably aged (particularly as Mongoose changed drastically in the middle of the series), I thought I’d give the Nodepad followers something to look at!

I updated the libraries by editing package.json:

• Express went from 2.2.2 to 2.4.x with no major API changes
• Mongoose has gone from 1.2.0 to 2.0.3 without much trouble
• Jade, Stylus and connect-mongodb have also been updated. Again, no major API changes affected Nodepad

Upgrading modules is a common task when maintaining production Node apps. Keeping tight versions in package.json can feel like a pain at times, but it’s really important for keeping production and staging environments sane and working with other people.

Backbone

I’ve updated Backbone.js to 0.5.3. This version has API changes, but the only one that affected us was a naming change from refresh to reset:

We’ve taken the opportunity to clarify some naming with the 0.5.0 release. Controller is now Router, and refresh is now reset. The previous saveLocation and setLocation functions have been replaced by navigate. Backbone.sync’s method signature has changed to allow the passing of arbitrary options to jQuery.ajax. Be sure to opt-in to pushState support, if you want to use it.

While I was updating the client-side code I improved the ‘empty state’ handling. Now when first logging in, Nodepad will behave much better. There’s no-longer a different add document view when creating the first document, in fact it’s now possible to just press ‘Save’ right away (an empty document will be created, but it makes more sense than the old version).

Summary

The main Nodepad resurrection commit was 56f7554:

• Client-side sorting uses toLowerCase
• Most persistent views are now on AppView
• Nobody keeps document model references around unless they need them directly
• Added selectedDocument to track the currently viewed document
• Changed refresh to reset to support Backbone 0.5.0+
• Improved new account handling by handling an empty document list correctly

And I followed that up with the library updates. See the full history here: Nodepad commit history.

Hopefully this will make Nodepad a little less confusing for people just discovering the tutorials and app!

CAMDUG Summer of Open Source

Like an idiot I forgot to post this sooner (I’ve been snowed under by cool JavaScript links lately!) The CAMDUG Summer of Open Source is tomorrow (27th August) in Cambridge UK, and has a Node open day, with speakers and coding sessions. Oleg Podsechin will be speaking, and he also recently gave a talk entitled The Future of Server-Side JavaScript at FrOSCon.

SunCalc

SunCalc (GitHub: mourner / suncalc, License: BSD) by Vladimir Agafonkin is a small library for calculating the Sun’s position, and sunlight phases.

This example will fetch the sunlight times for London:

var times = SunCalc.getTimes(new Date(), 51.5, -0.1);

It’ll also fetch the position of the Sun, returning the azimuth and altitude:

var sunrisePos = SunCalc.getSunPosition(times.sunrise, 51.5, -0.1);

It’s a nicely designed library with a lot of configuration options (detailed in the README).

BubblesJS

BubblesJS by Pantelis Kalogiros can be used to manage captions and subtitles for HTML5 videos. It’ll even read .srt files:

var bubbles = new Bubbles.video('video-srt');
bubbles.subtitles(false, {
  'English': {
    language: 'English',
    file: 'subs/english.srt'
  },
  'Greek': {
    language: 'Greek',
    file: 'subs/greek.srt'
  }
});

The way it works is actually fairly simple. Progress can be tracked using element.currentTime, and it includes a built-in srt parser. The first parameter to subtitles is the type of parser, false makes it use the built-in one but remote parsers can also be used.

The author notes there isn’t currently a standard usable way of managing HTML5 subtitles, and he said he’s going to put the source on GitHub soon too.

Last week I explained how a simple hasClass implementation might work for detecting CSS classes, complete with tests and suitable documentation. Henrik Lindqvist wrote a comment with some code that he claimed was faster. If you’re building your own open source project it’s likely that people may post their own performance suggestions and patches. This should be managed with care, because overly aggressive optimisation can potentially lead to confusing code or unexpected bugs.

In this tutorial I’m going to walk through Henrik’s code as I would any optimisation suggestion, using a little bit of science in the form of tests and benchmarks.

The Original hasClass

The original code is based around a regular expression. Like most of my tutorial code, I’ve attempted to make it extremely explicit and easy to follow:

dom.hasClass = function(element, className) {
  if (!className || typeof className !== 'string') return false;
  if (element.nodeType !== nodeTypes.ELEMENT_NODE) return false;
  if (element.className && element.className.length) {
    return new RegExp('(^|\\s)' + className + '($|\\s)').test(element.className);
  } else {
    return false;
  }
};

The Optimised hasClass

This is Henrik’s code:

function hasClassString(e, c) {
  var s = e.className, i = s.indexOf(c);
  return i != -1 && (s.charCodeAt(i - 1) || 32) == 32 && (s.charCodeAt(i + c.length) || 32) == 32;
};

The first line gets the class name from the element and finds the index of the class name that we’re looking for using indexOf.

The second line is longer. The comparison with -1 is done as early as possible to optimise cases where the class name hasn’t been found. The next part checks to see if the character before the match is a space (32 is the character code for space). When the index is outside the string, charCodeAt will return 32 using the || because NaN will be returned:

''.charCodeAt(0)
// NaN

''.charCodeAt(0) || 32
// 32

'a'.charCodeAt(0) || 32
// 97

The same thing is true for the end of the line, which the part after && deals with.

It looks like this code makes sense, but is it really faster?

Benchmarking

To benchmark these functions, I used Benchmark.js to compare the performance of each:

var Benchmark = require('benchmark')
  , suite = new Benchmark.Suite
  , div = { className: 'example1 example2 example3' };

function hasClassString(e, c) {
  var s = e.className, i = s.indexOf(c);
  return i != -1 && (s.charCodeAt(i - 1) || 32) == 32 && (s.charCodeAt(i + c.length) || 32) == 32;
};

function hasClassRegExp(element, className) {
  if (element.className && element.className.length) {
    return new RegExp('(^|\\s)' + className + '($|\\s)').test(element.className);
  } else {
    return false;
  }
};

suite.add('hasClassString', function() {
  hasClassString(div, 'example1');
  hasClassString(div, 'example2');
  hasClassString(div, 'example3');
  hasClassString(div, 'unknown');
})
.add('hasClassRegExp', function() {
  hasClassRegExp(div, 'example1');
  hasClassRegExp(div, 'example2');
  hasClassRegExp(div, 'example3');
  hasClassRegExp(div, 'unknown');
})
.on('cycle', function(event, bench) {
  console.log(String(bench));
})
.on('complete', function() {
  console.log('Fastest is ' + this.filter('fastest').pluck('name'));
})
// run async
.run(true);

The results on my machine look like this:

hasClassString x 2,319,886 ops/sec ±0.27% (81 runs sampled)
hasClassRegExp x 17,348 ops/sec ±0.39% (81 runs sampled)

The indexOf/charCodeAt version appears to perform over 100 times faster!

Testing

I dropped the optimised code into the DOM module:

/**
 * Detects if a class is present, optimised by Henrik Lindqvist.
 *
 * @param {Object} element A DOM element
 * @param {String} className The class name
 * @return {Boolean}
 */
dom.hasClass = function(element, className) {
  if (!className || typeof className !== 'string') return false;
  if (element.nodeType !== nodeTypes.ELEMENT_NODE) return false;
  var s = element.className, i = s.indexOf(className);
  return i != -1 && (s.charCodeAt(i - 1) || 32) == 32 && (s.charCodeAt(i + className.length) || 32) == 32;
};

Then I ran the tests in IE6, 7, 8, Firefox 4, Chrome, Safari, and gave up because it seemed fine. That’s not to say my tests are perfect, however, but it seems Henrik’s code does what we want.

Managing Optimisation

When it comes to optimising code, you’re only as good as your benchmarks and tests. There’s also the question of whether optimisation is really useful. In this case, hasClass is a good candidate for optimisation because it’s likely to be used frequently by client-side developers. There are times where regular expressions won’t perform as well as direct string manipulation, but will yield more succinct code. I found Henrik’s code easy to follow and the performance improvement was huge, so this seems clear cut to me.

The new code should have proper client-side benchmarks, but I’ll save that for another week.

This week’s latest commit was 0ddf1c.

[Update] Reader Feedback

Ryan Cannon suggested this:

dom.hasClass = function(element, className) {
  return (' ' + element.className + ' ').indexOf(' ' + className + ' ') !== -1;
};

It’s slightly slower than Henrik’s suggestion, but it fixes a problem I didn’t spot that Adam Solove pointed out:

Given a div with the classes: “something some” and a query for the class “some”, this code returns false when it should return true. The first match of a substring isn’t followed with a space, but you need to repeatedly look for the same substring in case it occurs by itself later.

I forgot to include coverage of element.classList again, so I’ve added that as well:

if (turing.detect('classList')) {
  dom.hasClass = function(element, className) {
    return element.classList.contains(className);
  };
} else {
  dom.hasClass = function(element, className) {
    return (' ' + element.className + ' ').indexOf(' ' + className + ' ') !== -1;
  };
}

Recall that turing.detect will cache the result and first requires turing.addDetectionTest to work. In this case it’s only called once.

I’ve added the readers that suggested these improvements to the contributor list in Turing’s README.

References

• String.prototype.indexOf
• indexOf Method at MSDN
• Benchmark.js
• element.classList

Windows Performance Boost

I noticed Ryan Dahl post a link to these Node http_simple benchmarks on Twitter:

nodejs v0.5 benchmarks gist.github.com/1166690 (still some work to do on libuv backend but Windows is on par with Linux now!)

- @ryah

This is good news for Windows developers, and again demonstrates the phenomenal progress we’re seeing on libuv.

Bricks.js

Bricks.js (GitHub: JerrySievert / bricks, License: MIT/X11, npm: bricks) by Jerry Sievert is a modular web framework that isn’t built on existing web technology like Connect.

The most basic usage example is a static web server:

var bricks = require('bricks');
var appServer = new bricks.appserver();

appServer.addRoute('/static/.+', appServer.plugins.filehandler, { basedir: './static' });
appServer.addRoute('.+', appServer.plugins.fourohfour);
var server appServer.createServer();

server.listen(3000);

Routes can be strings or regular expressions. Routes are grouped into sections, which are pre, main, post, and final. Each of these sections are explained in the Bricks.js documentation.

When a function is passed to a route, it gets request, response parameters like Express. Routes can also take plugins, which can be used to manage state. The routing and error APIs are event-based. That means it’s possible to set up listeners on events like section completion and errors:

response.on('main.complete', mainCompletion);

// Errors
appserver.addEventHandler('route.fatal', function(error) { console.log('FATAL:', error); });

Bricks.js seems to be quite different to most Node web frameworks, and I like the fact it’s event-based. It seems like the strength of the framework will come from contributed plugins, so hopefully Jerry will start some kind of centralised list of plugins as they emerge.

Commander.js

Commander.js (GitHub: visionmedia / commander.js, License: MIT, npm: commander) by TJ Holowaychuk is a “complete solution” for developing command-line interfaces.

It has a chained API, allowing you to express a command-line application in terms of options, prompts (including a non-echoing password prompt), confirmation, and simple menus. This allows command-line help to be automatically generated.

program
  .version('0.0.1')
  .option('-i, --integer <n>', 'An integer argument', parseInt)
  .option('-f, --float <n>', 'A float argument', parseFloat)
  .option('-r, --range <a>..<b>', 'A range', range)
  .option('-l, --list <items>', 'A list', list)
  .option('-o, --optional [value]', 'An optional value')
  .parse(process.argv);

jStat

jStat (GitHub, License: MIT) is a statistical library that includes general statistical functions and a large set of probability distribution:

• Beta
• Cauchy
• Chi-square
• Exponential
• Gamma
• Kumaraswamy
• Log-normal
• Normal
• Pareto
• Weibull
• Uniform

There are a few more that appear to be work in progress. Each distribution has the following methods: pdf, cdf, inv, mean, median, mode, and variance.

jStat also includes many commonly used functions as static methods, and even includes tools for generating random numbers and manipulating arrays of values.

The reason I’ve diligently waded through the source linking to Wikipedia is to demonstrate the scope of the library, because documentation is currently scant. The source is easy to follow though, so if you’re interested in using the library for something serious don’t be afraid to check it out.

Although not a jQuery library, it’s designed to work in browsers and can work with the jQuery Flot plotting library.

Art Text Light

Art Text Light (License: MIT) will apply CSS to elements over a repeated interval. The author has used this to apply text-shadow, creating an interesting effect. Any CSS or time intervals could be used, so I imagined using it to set up a fast shimmering text effect on a gaming site to direct people to a button. Perhaps.

Portamento

Portamento (GitHub: krisnoble / Portamento, License: GPLv3) by Kris Noble solves a problem that I had with DailyJS’s page design. It allows panels to float vertically, with lots of customisation possibilities. For example, panels can be made to stop at a given position like this:

$('#sidebar').portamento({ wrapper: $('#wrapper') });

It also adapts well to small displays:

Portamento also has sensible behaviour if the user’s viewport is too small to display the whole panel, so you don’t need to worry about users not being able to see your important content.

JavaScript Enlightenment by Cody Lindley is a guide to JavaScript by giving native objects centre stage. It’s self-published and available as a PDF for $15 through Google Checkout and PayPal (bulk licensing is also available).

The book begins with an explanation of objects and properties:

In JavaScript, objects are king: Almost everything is an object or acts like an object. Understand objects and you will understand JavaScript. So let’s examine the creation of objects in JavaScript.

Objects are explored in detail, with full examples of constructors, native constructors, literals, and even how values are stored and copied. Each example has source code with a link to jsFiddle, so it’s easy to play around with examples to attain new levels of JavaScript enlightenment.

Some of the more confusing aspects of JavaScript are covered early on, particularly object comparison, primitive value object wrapping, and prototype chain property reference resolution. The language used is clear and easy to follow:

All object instances have a property that is a secret link (aka proto) to the constructor function that created the instance. This secret link can be leveraged to grab the constructor function, specifically the prototype property of the instance’s constructor function.

Other fundamental JavaScript concepts like using hasOwnProperty to correctly enumerate over objects are also explained. In fact, I really couldn’t find anything missing. If you’ve ever wondered why JSLint is complaining about something, the root cause is likely to be explained somewhere in this book.

The author even references modern libraries like Underscore.js:

JavaScript 1.5 is lacking when it comes time to seriously manipulate and manage objects. If you are running JavaScript in web browser, I would like to be bold here and suggest the usage of Underscore.js when you need more functionality than is provided by JavaScript 1.5.

It’s also encouraging to see solid coverage of Function objects — one of the hidden aspects of JavaScript that can unlock serious power if used carefully. Slightly confusingly, apply and call are only covered briefly here, with more thorough coverage in chapter 6. The split between explaining this and Function felt slightly awkward, which is one of the reasons why this book works better taken as a whole rather than discrete chunks.

The author carefully notes where JavaScript implementations deviate from the ECMA standard, and explains the cleanest way to mitigate this.

Conclusion

If you’ve ever wondered how prototypal inheritance works, or just want to bolster your JavaScript knowledge, this is an extremely potent book. It’s short — readable in a handful of sittings, but it works best read from start to finish.

Although the technical foundation of this book is solid and the writing is clear and concise, it would benefit from an editor with a more scrupulous eye. There are a few typographical and grammatical errors that conspire to undermine the author’s authority. For example, I got extremely tired of repeatedly reading the phrase the take away. Hopefully Cody will keep publishing new editions, so don’t let this stop you from reading one of the most interesting books on JavaScript this year.

Learning Three.js

Learning Three.js by Jerome Etienne is a new blog full of tutorials about Mr.doob’s three.js. There’s already quite a few tutorials, with one covering drawing a cube, skyboxes and tweening.

If you’d like to learn WebGL but you’re afraid it’s only for genius mind wizards, then try reading right from the start where Jerome demonstrates how to install Three.js, then follow on to the cube tutorial. And if you make something cool, send it to us.

FlyJSONP

I noticed that TJ is still actively working on superagent which is shaping up to be a great all-purpose browser-friendly network library. Meanwhile, DailyJS reader Abdulrahman Alotaiba sent in FlyJSONP (GitHub: alotaiba / FlyJSONP, License: GNU GPLv3). This library supports cross-domain GET and POST (passed through YQL).

The author’s example demonstrates cross-domain POST:

FlyJSONP.post({
  url: 'http://storify.com/story/new',
  parameters: {
    username: 'your-username',
    api_key: 'secret-api-key',
    title: 'FlyJSONP',
    description: 'Testing it out'
  },
  success: function(data) {
    console.log(data);
  }
});

By using the YQL service, FlyJSONP has a small footprint and no dependencies. And you can try it out at the FlyJSONP demo page. And it’s good to see that the author has included QUnit tests!

HTML5Sticky

HTML5Sticky (demo, License: MIT) is actually a full-blown application by Sarfraz Ahmed that uses HTML5 trickery (local storage, CSS3, Google Web Fonts) to create a dynamic dashboard of sticky notes. He’s used Modernizr so it supports slightly older browsers like IE8 and Firefox 3.5.

Using Turing’s DOM module quickly reveals the lack of hasClass. This is a handy method that most frameworks provide to detect if a node has a given class name. We’ve already implemented the class manipulation functionality in Part 60: CSS Classes.

jQuery hasClass

jQuery uses a combination of a regular expression and indexOf to detect if classes are present:

hasClass: function( selector ) {
  var className = " " + selector + " ";
  for ( var i = 0, l = this.length; i < l; i++ ) {
    if ( this[i].nodeType === 1 && (" " + this[i].className + " ").replace(rclass, " ").indexOf( className ) > -1 ) {
      return true;
    }
  }

  return false;
},

A nice touch here is every element in the current internal stack is searched for the class name. If a result is found the method will return straight away.

Zepto

I also looked at how Zepto does this, because Zepto is usually very concise and easy to follow. The hasClass implementation is in zepto.js. Zepto includes caching, but the real work is a regular expression:

function classRE(name){
  return name in classCache ?
    classCache[name] : (classCache[name] = new RegExp('(^|\\s)' + name + '(\\s|$)'));
}

I actually thought of just using new RegExp('\\b' + className + '\\b').test(element.className), but looking at Zepto made me realise that expression might not match all valid class names.

Regular Expression Matching

The reason using \b isn’t sufficient is because CSS identifiers can contain hyphen and underscore (from CSS Level 2: Characters and case), while \b would only match [a-zA-Z0-9_] (which are considered word characters by the regular expression engine). Zepto’s regular expression uses \s and the line ending characters.

Tests

To make sure Turing’s implementation worked along these lines, I wrote some tests:

'test hasClass': function() {
  assert.ok(turing('#attr-test').hasClass('example'));
  assert.ok(turing('#attr-test').hasClass('example-2'));
  assert.ok(turing('#attr-test').hasClass('example_3'));
  assert.ok(!turing('#attr-test').hasClass('example_'));
},

'test nested hasClass': function() {
  assert.ok(turing('#nested-hasClass-test div').hasClass('find-me'));
  assert.ok(!turing('#nested-hasClass-test div').hasClass('aaa'));
},

I wrote this feature test first — this approach has always worked well for me when researching and writing this series. Notice that I’ve also included a “nested test” which is intended to test Turing’s chained hasClass behaviour which searches through every matching element just like jQuery.

Implementation

I used a similar regular expression to Zepto and added some sanity checking for node type and inputs:

  /**
   * Detects if a class is present.
   *
   * @param {Object} element A DOM element
   * @param {String} className The class name
   * @return {Boolean}
   */
  dom.hasClass = function(element, className) {
    if (!className || typeof className !== 'string') return false;
    if (element.nodeType !== nodeTypes.ELEMENT_NODE) return false;
    if (element.className && element.className.length) {
      return new RegExp('(^|\\s)' + className + '($|\\s)').test(element.className);
    } else {
      return false;
    }
  };

To make this work through a chain, it just needs to be put in a loop:

/**
 * Detects if a class is present.
 *
 * @param {String} className A class name
 * @returns {Boolean}
 */
hasClass: function(className) {
  for (var i = 0; i < this.length; i++) {
    if (dom.hasClass(this[i], className)) {
      return true;
    }
  }
  return false;
},

This passes the tests in IE6, Firefox, Chrome, Safari, etc.

To get the version of Turing in this tutorial, checkout commit b169bd4.

References

• CSS Level 2: Characters and case
• zepto.js

Node 0.5.4

Node 0.5.4 was released last week. This version introduces yet more Windows improvements through libuv. It can now be built on Microsoft Visual Studio via GYP. There are also bug fixes, and V8 has been updated to 3.5.4.

Multimeter

Multimeter (npm: multimeter) is another interesting project by James Halliday. This one generates ANSI progress bars, suitable for use in your next great console-based application.

var multimeter = require('multimeter')
  , multi = multimeter(process);

// Drop creates a new progress bar at the cursor
multi.drop(function(bar) {
  var iv = setInterval(function() {
    var p = bar.percent();

    bar.percent(p + 1); 

    if (p >= 100) clearInterval(iv);
  }, 25);
});

Videos for Node Distributed Web Architectures Meetup

Joyent have been kind enough to publish videos for the Distributed Web Architectures meetup that was held on the 4th of August.

Curtis Chambers from Uber discusses how Uber’s real-time mobile system has been made more efficient with Node. Dave Pacheco from Joyent explores behind the scenes of the analytics and visualisation tools in their SmartDataCenter software. And, Matt Ranney explains how Node has helped reduce latency in Voxer’s real-time voice application.

jquery.terminal

jquery.terminal (GitHub: jcubic / jquery.terminal, License: LGPL 3) by Jakub Jankiewicz helps create client-side command-line applications. With one line of code it’ll display an interpreter for JSON-RPC services, and even supports authentication. Keyboard shortcuts like ctrl+d are supported, and history can be saved to local storage.

The source that runs the demo shows how simple it is to get something going with jquery.terminal:

jQuery(function($, undefined) {
    $('#term_demo').terminal(function(command, term) {
        var result = window.eval(command);
        if (result != undefined) {
            term.echo(String(result));
        }
    }, {
        greetings: 'Javascript Interpreter',
        name: 'js_demo',
        height: 200,
        width: 450,
        prompt: 'js>'});
});

jquery-inputs

jquery-inputs (License: MIT) by Denny Shimkoski allows getting and setting of form inputs using hierarchical JSON data structures. Given a suitable form, the following JSON will be interpreted and inserted into the correct fields:

$('form.user').inputs('set', {
  user: {
    name: 'Alex',
    description: 'JavaScript fan',
    password: 'All your base'
  }
});

In this example, an input with a name of user_name found within form.user will be set to 'Alex'. This seems like an extremely useful addition to the growing suite of jQuery templating tools.

MobilyMap, MobilySlider, MobilyBlocks

Marcin Dziewulski sent in three plugins. MobilyMap (Demo, License: MIT) generates Google Maps-inspired draggable interfaces from a regular bitmap image. It supports markers, captions, controls, and it’ll even save the last position in a cookie.

MobilySlider (Demo, License: MIT) is yet another jQuery slideshow plugin. It includes controls for pagination and next/previous scrolling, and will pause on hover.

MobilySelect (Demo, License: MIT) can replace one set of items for another. The author’s example usage is switching between groups of images.

MobilyBlocks (Demo, License: MIT) displays unordered lists as animated circles. It’s a very interesting effect (pictured above).

I thought this was a great collection of plugins, but I can’t seem to find uncompressed source code. Hopefully Marcin will post the original source somewhere like GitHub (or his favourite code sharing site) so we can contribute!

About LABjs

LABjs (GitHub: getify / LABjs, License: MIT) by Kyle Simpson is an asynchronous script loader. Although there are many similar projects now on the scene, LABjs is the first that I used heavily.

LABjs by default will load (and execute) all scripts in parallel as fast as the browser will allow. However, you can easily specify which scripts have execution order dependencies and LABjs will ensure proper execution order. This makes LABjs safe to use for virtually any JavaScript resource, whether you control/host it or not, and whether it is standalone or part of a larger dependency tree of resources.

It has a nice little chained API:

$LAB.script('https://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js').wait()
    .script('/javascripts/example1.js').wait()
    .script('/javascripts/example2.js');

Inserting wait() into the chain can be used to express a dependency — in this case example2.js won’t run until the other scripts have been fully loaded.

Structure

LABjs is distributed as a single script that currently weighs in at 513 lines of code. The project also comes with tests and a minimised file. This project is religiously indented with tabs, and it looks best with a tabstop of 4. This project doesn’t ship with its build scripts, so I can’t duplicate the minification process.

As we’ve seen many times in this series, the self-executing anonymous function is used to contain the entire library:

(function(global){
  var _$LAB = global.$LAB,
  
  // ...

  global.$LAB = create_sandbox();
})(this);

Presumably the author has chosen to export the library as $LAB to avoid collisions with LAB, but I can’t imagine using that name for anything anyway.

One thing I noticed early on was this debug style:

/*!START_DEBUG*/
  // console.log() and console.error() wrappers
  log_msg = function(){}, 
  log_error = log_msg,
/*!END_DEBUG*/

This explains the presence of LAB-debug.min.js and LAB.min.js. It would be useful if the build process was present in the form of a build script so I could build my own versions of these scripts.

Feature Sniffing

Generally, LABjs opts to use feature sniffing to gather browser capabilities:

// feature sniffs (yay!)
test_script_elem = document.createElement("script"),
explicit_preloading = typeof test_script_elem.preload == "boolean", // http://wiki.whatwg.org/wiki/Script_Execution_Control#Proposal_1_.28Nicholas_Zakas.29
real_preloading = explicit_preloading || (test_script_elem.readyState && test_script_elem.readyState == "uninitialized"), // will a script preload with `src` set before DOM append?
script_ordered_async = !real_preloading && test_script_elem.async === true, // http://wiki.whatwg.org/wiki/Dynamic_Script_Execution_Order

// XHR preloading (same-domain) and cache-preloading (remote-domain) are the fallbacks (for some browsers)
xhr_or_cache_preloading = !real_preloading && !script_ordered_async && !opera_or_gecko

This is preferable to branching based on the user agent string. I wondered if test_script_elem needs to be discaded to avoid leaking a DOM element, but I’m fairly sure even older IEs will garbage collect it because no properties are changed and there aren’t any circular references.

Support Functions

LABjs is developed to work without any particular client-side framework, which means it includes a few commonly found helper functions: is_func, is_array, and merge_objs jumped out at me. Is it time for someone to make a “native JavaScript helper function” library?

Chained API

When using LABjs, we’re actually interacting with the create_sandbox function. This contains several variables to manage state, functions for managing the chained API and requests, and it returns an object that forms a public API. Here we can see how wait works:

wait:function(){
  return create_chain().wait.apply(null,arguments);
}

It actually creates new chains, then calls the wait method on the chained API. Following this through illustrates how requests and dependencies are managed:

// force LABjs to pause in execution at this point in the chain, until the execution thus far finishes, before proceeding
wait:function(){
  if (arguments.length > 0) {
    for (var i=0; i<arguments.length; i++) {
      chain.push(arguments[i]);
    }
    group = chain[chain.length-1];
  }
  else group = false;
  
  advance_exec_cursor();
  
  return chainedAPI;
}

This is a fairly typical way of managing a chained API. Afterwards, advance_exec_cursor is called and the current chained API object is returned so subsequent calls to script() can be performed. The trick to making a natural chained API is working out where code can be executed or deferred. In this case, advance_exec_cursor is used to execute parts of the chain in the right sequence.

Ultimately, do_script will be called by script, which just sets up the request. A registry of loaded scripts will be built up, and it’ll optionally insert parameters to force caches to be ignored.

If a script requires pre-loading, setTimeout is used to immediately call the ready listener.

request_script, called by do_script will attempt to load the script using the most suitable method.

Request Handling

Inside request_script, setTimeout is used to run the body of the function. This is apparently to avoid race conditions in older browsers. Another setTimeout is used to wait until the element where the script tags is pasted is ready. The options provided by do_script in script_obj are used to create a script tag and insert it into the desired element.

Once the script tag has been set up, the real cross-browser code kicks in that has to manage the callbacks needed to monitor the script’s network events and ready state. Like many mature client-side projects, this requires some careful handling of browser kinks.

Conclusion

LABjs isn’t a trivial library, but it can make web pages feel fast without concatenating all of the client-side JavaScript into monolithic files. On reviewing the code I found several areas that had interested notes about cross-browser support and proposals for more modern solutions.

Some of the core methods in the library have very high arity. Using option objects might make these functions clearer. It would also be nice to keep the build scripts in the repository.