Meincraft, Virtual Joystick, Filer.js

30 Dec 2011 | By Alex Young | Comments | Tags html5 filesystem libraries webgl

Meincraft

Meincraft screenshot

Meincraft (GitHub: mitsuhiko / webgl-meincraft, License: BSD) by Armin Ronacher a WebGL demo that generates terrain and allows navigation with its own camera implementation.

It’s currently a simple personal project rather than a fully-fledged web-based Minecraft engine, similar to the three.js Minecraft demo, but it’s an interesting start and different to the other WebGL Minecraft clones that I’ve found so far.

Virtual Joystick

Let’s Make a 3D Game: Virtual Joystick by Jerome Etienne is a tutorial on his virtualjoystick.js project which provides a joystick suitable for use with touchscreens. It works a lot like the joysticks seen in many iOS and Android games.

The tutorial post includes a demo, and it even works with a mouse. The basic API is simply var joystick = new VirtualJoystick().

Filer.js

Filer.js (GitHub: ebidel / filer.js, License: Apache 2.0) by Eric Bidelman is a friendly API for the HTML5 FileSystem API based around Unix commands. Most commands are asynchronous and expect a callback:

var filer = new Filer();
filer.init({size: 1024 * 1024}, onInit.bind(filer), onError);

function onInit(fs) {
  filer.ls('/', function(entries) {
    // entries is an Array of file/directories in the root folder.
  }, onError);
}

function onError(e) { ... }

That example lists a directory. Other familiar commands include cd, create, mkdir, rm, cp, mv, open, and write. Each API method is documented in the project’s README file in the repository.

Retrofitting AMD

29 Dec 2011 | By Alex Young | Comments | Tags frameworks tutorials lmaf amd

Let's Make a Framework is an ongoing series about building a JavaScript framework from the ground up.

These articles are tagged with lmaf. The project we're creating is called Turing. Documentation is available at turingjs.com.

Last week I looked at how AMD (Asynchronous Module Definition) works and how projects use it. This week I’m going to show how to retrofit it to an existing project, in this case our Turing framework, and hopefully you’ll be able to apply this to your own reusable client-side JavaScript.

Redefining Modules

Turing’s modules were structured around Immediately-Invoked Function Expressions (IIFEs), with CommonJS module loading support. AMD supports CommonJS modules, and Node can work with AMD too.

The core Turing module can be defined as an AMD module like this:

define('turing.core', [], function() {
  var turing = function() {};

  // Core functionality added here

  return turing;
});

A module loader is now needed to use this code in a browser. RequireJS works like this:

require(['turing.core'], function(turing) {
});

The methods in turing.core.js will be available once RequireJS has loaded the script.

Dependencies

A good reason for using AMD is dependencies can be specified and resolved by loaders. In Turing, every module depends on turing.core. That means the DOM module would have to be updated to look like this:

define('turing.dom', ['turing.core'], function(turing) {
  var dom = {};
  // DOM module defined here
  return dom;
});

Previously, expressing dependencies between modules wasn’t possible outside of code comments. Dependencies make it easier to break down functionality into smaller reusable chunks and loaded when required.

 Loading the Entire Framework at Once

When using frameworks like jQuery, many projects need a broad selection of functionality and therefore load the entire library rather than parts of it. Rather than forcing users to use require or define invocations with a large amount of dependencies, it’s possible to offer a broad selection in one file.

I’ve created a turing.js file that contains the following:

define('turing', ['turing.core', 'turing.dom', 'turing.anim'], function(core, dom, anim) {
  core.dom = dom;
  core.anim = anim;
  return core;
});

These are the modules that I’ve ported to work with AMD so far, but I’ll add all of them under turing.js once I’ve finished.

Building

What about building a monolithic, minified version of Turing? RequireJS addresses this by including the RequireJS Optimizer. This basically boils down to:

> npm install requirejs
> r.js -o app.build.js

Conclusion

I’ve started adapting Turing to work with AMD, and my early tests work with RequireJS. However, I’m finding it difficult to get builds to generate with r.js (it seems like the current release has bugs).

Although using AMD’s well-defined module pattern and dependencies solves certain problems when developing reusable code, it makes it difficult to satisfy users who simply want to include a library using a script tag without a module loader like RequireJS. I’ve been experimenting with creating a small define function that is used when a module loader isn’t available, and I’ll cover that in next week’s tutorial.

Node Roundup: Clarinet, Sandbox, actionHero

28 Dec 2011 | By Alex Young | Comments | Tags node modules frameworks parsing json
You can send in your Node projects for review through our contact form or @dailyjs.

Clarinet

Clarinet (GitHub: dscape / clarinet, License: Apache 2.0/MIT, npm: clarinet) by Nuno Job is a streaming JSON parser. The parser is event-based, and since it’s streaming it makes dealing with huge files possible. The author still recommends using JSON.parse for most tasks, however.

The announcement blog post linked to above has detailed performance analysis and more background on the project.

Sandbox

Sandbox (GitHub: gf3 / sandbox, npm: sandbox) by Gianni Chiappetta is a sandbox for Node. That means untrusted code can be run in a relatively safe environment. It comes with some useful features, including timeouts and error handling.

To execute code in a sandbox, the run method on a Sandbox instance must be used:

var s = new Sandbox()
s.run( '1 + 1 + " apples"', function( output ) {
  // output.result == "2 apples"
})

actionHero

actionHero (License: license.txt, npm: actionHero) by Evan Tahler is a transactional API framework for sockets and HTTP clients. Actions are defined for GET and POST requests, and they’re constructed from simple objects decorated with metadata for documentation:

var action = {};
action.name = "randomNumber";

action.description = "I am an API method which will generate a random number";
action.inputs = {
  "required" : [],
  "optional" : []
};
action.outputExample = {
  randomNumber: 123
}

action.run = function(api, connection, next){
  connection.response.randomNumber = Math.random();
  next(connection, true);
};

exports.action = action;

The framework comes with MySQL support, Models are implemented using the Sequelize MySQL ORM library.

actionHero also supports “tasks”, which are periodic actions performed by the server. Other features and example code can be found in the project’s repository.

jQuery Roundup: jQuery-inlog, -prefix-free, jquery.animate-enhanced

27 Dec 2011 | By Alex Young | Comments | Tags jquery plugins css3 animation
Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

jQuery-inlog

jQuery-inlog (GitHub: Prinzhorn / jquery-inlog, License: MIT) by Alexander Prinzhorn is a passive plugin that injects console.log calls into jQuery core. It can help debug selectors and chained calls.

This plugin has several options that can be changed at any time:

$.inlog({
  enabled: false,    // Shortcut: $.inlog(true|false);
  thisValue: false,  // Output this-value or not
  returnValue: true, // Output return-value or not
  indent: true       // Indent nested calls or not
});

-prefix-free

-prefix-free (GitHub: LeaVerou / prefixfree, License: MIT) by Lea Verou makes it possible to use CSS properties without vendor strings, adding the required prefixes only when they’re required. Every stylesheet in <link> or <style> tags will be processed, and jQuery’s $.css() method can be used to get or set CSS properties without prefixes.

The browsers supported are IE9+, Opera 10+, Firefox 3.5+, Safari 4+ and Chrome:

In older browsers like IE8, nothing will break, just properties won’t get prefixed. Which wouldn’t be useful anyway as IE8 doesn’t support much CSS3.

There are a few limitations that the author has documented on the project’s site: @import stylesheets aren’t supported, and neither are cross-origin stylesheets.

The jQuery plugin part of -prefix-free is here: prefixfree.jquery.js. This plugin adds the required changes to allow $.css() to work transparently.

jquery.animate-enhanced

jquery.animate-enhanced (GitHub: benbarnett / jQuery-Animate-Enhanced, License: MIT) by Ben Barnett is an increasingly popular plugin for extending $.animate() to detect CSS transitions for modern browsers. It’s tested with jQuery 1.3.2 to 1.6.1 and is even compatible with IE6+.

The plugin will analyse the properties you’re animating on, and select the most appropriate method for the browser in use. This means your transitions on left, top and opacity will convert to a CSS3 transition on Webkit & Mozilla agents that support it, and Opera 10.50+. If the user is on a browser that has no CSS3 transitions, this plugin knows about it and won’t get involved.

By including this plugin on a page, jQuery animations should use CSS3 translate where available. Outside of this basic usage, there are also three new options for $.animate():

  • avoidTransforms: Avoid using -webkit-transform or similar to aid hardware acceleration
  • useTranslate3d: Use 3D translations, the author recommends this for iPhone-focused development
  • leaveTransforms: Once transitions are complete, remove them and convert positional values back to the real values

3D support is enabled by default, and can be toggled by using $.toggle3DByDefault().

Backbone Fundamentals, Three.js Boilerplate, zoom.js

26 Dec 2011 | By Alex Young | Comments | Tags webgl backbone.js books dom animation

Backbone Fundamentals

Backbone Fundamentals (GitHub addyosmani / backbone-fundamentals) by Addy Osmani is a new book that introduces Backbone to beginners, helping to piece together the collection of tools and knowledge required to use it effectively.

It’s still being worked on, but Addy has already written a huge amount that can be read in the GitHub repository’s Backbone.js Fundamentals README.md file.

There’s also a Backbone Fundamentals wishlist for those wishing to add their own suggestions for the book, or perhaps even help through contributions.

Boilerplate for Three.js

Boilerplate for Three.js (GitHub jeromeetienne / threejsboilerplate) by Jerome Etienne is a template to get started with three.js development.

It renders WebGL if available, else it’ll fall back to a 2D Canvas. It’ll also create a scene with camera controls, touch event support, fullscreen, window resize, and screenshots.

 zoom.js

zoom.js (GitHub: hakimel / zoom.js) by Hakim El Hattab is a small JavaScript API for zooming into a specific point on a page or a DOM element. The author states that it’s an experimental proof-of-concept, but the effect worked well on the browsers I tested it with.

Some browsers will have a more noticeable visual glitch as anti-aliasing is applied, but the effect uses animation easing so it feels relatively gentle considering what’s actually happening behind the scenes.

randexp.js, ECMAScript 5 Presentation, Online Editor for Sparks.js

23 Dec 2011 | By Alex Young | Comments | Tags webgl slides ecmascript standards

randexp.js

randexp.js (GitHub: fent / randexp.js, License: MIT, npm: randexp) by Roly Fentanes is a library for creating random strings based on a regular expression. The resulting string will match the expression provided:

require('randexp');
/hello+ (world|to you)/.gen
// => hellooooooooooooooooooo world

A more useful example might be generating a random date:

new RegExp((January|February|March|April|May|June|July|August|September|October|November|December) ([1-9]|[12][0-9]|3[01]), (19|20)[0-9][0-9]).gen
// March 4, 2038

This could be a useful way to generate test data in unit tests.

The way the RegExp object is modified by this library goes against the grain a little bit, and using a getter to return values seems a little bit odd, so I’ve asked the author about the API design through GitHub.

ECMAScript 5 Presentation

I don’t usually post slides, because without the corresponding talk they’re usually too hard to follow. ECMAScript 5 by Damian Wielgosik is a 150 slide presentation that introduces ECMAScript 5, and is relatively easy to follow if you don’t mind pressing the right arrow key 150 times.

The majority of the slides are concerned with new methods on Object, and some patterns that make use of these new methods are explained. Other areas covered include Function.prototype.bind, new Array.prototype methods, and strict mode.

Particles: Online Editor for Sparks.js

sparkseditor

Particles: Online Editor for Sparks.js is an introductory tutorial for sparkseditor (GitHub: jeromeetienne / sparkseditor, License: MIT) by Jerome Etienne which is a GUI editor for Sparks.js, a 3D JavaScript particle engine.

Press the “code” button to show the code used to generate the particle system. The code can be edited, and the changes will be displayed in real time. To share the results, press the “export” button.

A wide range of effects can be created with Sparks.js. There are examples available at jabtunes.com/labs/arabesque/.

The How and Why of AMD

22 Dec 2011 | By Alex Young | Comments | Tags frameworks tutorials lmaf amd

Let’s Make a Framework is an ongoing series about building a JavaScript framework from the ground up.

These articles are tagged with lmaf. The project we’re creating is called Turing. Documentation is available at turingjs.com.

In Improving Client-Side Modularity I talked about better ways to load modules, and showed how jQuery supports the Asynchronous Module Definition API. As open source developers, supporting this specification makes it easier for others to reuse our code. It also helps resource loading frameworks like RequireJS.

In the Let’s Make a Framework series, we’ve already covered Common JS Modules and seen how they can make our code accessible for Node developers. However, implementing a convincing client-side version of this is difficult because require is generally implemented as a synchronous statement. There are projects to wrap this for browser-based development, but AMD offers a native solution that’s easy to work with. This is where AMD steps in as a “transport format”.

Conditional AMD Support

jQuery provides conditional AMD support. When writing smaller, tightly focused libraries, this is a useful approach because it allows AMD to be supported where available. jQuery checks for the existence of define, and this approach can be reused:

var myModule = {
  awesome: 'not really'
};

if (typeof define === 'function' && define.amd) {
  define('myModule', [], function() {
    return myModule;
  });
}

The amd property is explained by the AMD specification:

To allow a clear indicator that a global define function (as needed for script src browser loading) conforms to the AMD API, any global define function SHOULD have a property called “amd” whose value is an object. This helps avoid conflict with any other existing JavaScript code that could have defined a define() function that does not conform to the AMD API.

We need to check for both the define function, and that it conforms to the specification.

Internal AMD Use

Another approach is to build an entire project around AMD. Dojo was restructured to work this way. This comment is from Dojo’s source:

This function defines an AMD-compliant loader that can be configured to operate in either synchronous or asynchronous modes.

Then modules are wrapped with define:

define(["./_base/kernel", "./_base/lang", "./_base/Color", "./_base/array"], function(dojo, lang, Color, ArrayUtil) {

});

Notice how dependencies in AMD are the same order as the callback (or “factory function”, as the AMD specification refers to it). This is explained in the specification:

[…] the resolved values should be passed as arguments to the factory function with argument positions corresponding to indexes in the dependencies array.

As I mentioned in Improving Client-Side Modularity, Kris Zyp documented this move to AMD in Asynchronous Modules Come to Dojo 1.6.

CommonJS Wrapping

The AMD specification also addresses CommonJS modules with this example:

define(function(require, exports, module) {
  var a = require('a'),
      b = require('b');

  exports.action = function() {};
});

The RequireJS documentation points out that this can help with cases where there are a lot of dependencies:

define([ 'require', 'jquery', 'blade/object', 'blade/fn', 'rdapi',
         'oauth', 'blade/jig', 'blade/url', 'dispatch', 'accounts',
         'storage', 'services', 'widgets/AccountPanel', 'widgets/TabButton',
         'widgets/AddAccount', 'less', 'osTheme', 'jquery-ui-1.8.7.min',
         'jquery.textOverflow'], // ...

This is difficult to read. The RequireJS documentation goes on to say that using require is potentially easier to follow. An AMD module loader has to parse out the require calls and insert the define dependencies transparently.

Since the authors of RequireJS have experience implementing such things, they’re all too aware of browser inconsistencies and limitations:

Not all browsers give a usable Function.prototype.toString() results. As of October 2011, the PS 3 and older Opera Mobile browsers do not. Those browsers are more likely to need an optimized build of the modules for network/device limitations, so just do a build with an optimizer that knows how to convert these files to the normalized dependency array form, like the RequireJS optimizer.

This means a pre-compilation step may be necessary to support a broad range of browsers using this approach.

Conclusion

If you’ve followed our previous tutorials on building an asynchronous script loader, you’ll know that writing JavaScript APIs for loading scripts isn’t trivial. Supporting CommonJS isn’t necessarily difficult, but supporting features like parsing require calls is where things get tricky.

This leads us to an important point that must be considered when building our own libraries and frameworks: should we ship our own script loading solution, or conditionally support AMD? Given the complexity of building a script loader, it may be better to conditionally support AMD as jQuery does.

This doesn’t quite satisfy our needs for Turing, however. Turing does have one or two modules that are dependent on each other, and define provides a way to express this programatically. It may be better to use define to wrap modules, as we already do using an IIFE (Immediately-Invoked Function Expression), and effectively no-op it if it isn’t present.

Next week I’ll look at adding AMD to Turing, then test it out with RequireJS.

References

Node Roundup: 0.6.6, asyncblock, Introduction to Node Modules, Review19

21 Dec 2011 | By Alex Young | Comments | Tags node modules async fibers apps

Node 0.6.6

Node 0.6.6 is out, which updates npm, fixes some bugs, and adds pause/resume semantics to stdin. Node also has a new website that is greener than ever, but remains clean and features a new corporate-friendly edge.

There’s also a post on the official Node blog entitled Growing up by Ryan Dahl that discusses Windows Azure support:

The overarching goal of the port was to expand our user base to the largest number of developers. Happily, this has paid off in the form of being a first class citizen on Azure.

asyncblock

asyncblock (License: MIT, npm: asyncblock) is a fork of node-green-light. It works like a flow control library combined with node-fibers.

The author compares a ‘pure Node’ example with his own library’s code:

function example(callback){
    var finishedCount = 0;
    var fileContents = [];

    var continuation = function(){
        if(finishedCount < 2){
            return;
        }

        fs.writeFile('path3', fileContents[0], function(err) {
            if(err) {
                throw new Error(err);
            }

            fs.readFile('path3', 'utf8', function(err, data){ 
                console.log(data);
                console.log('all done');
            });
        });
    };

    fs.readFile('path1', 'utf8', function(err, data) {
        if(err) {
            throw new Error(err);
        }

        fnishedCount++;
        fileContents[0] = data;

        continuation();
    });

    fs.readFile('path2', 'utf8', function(err, data) {
        if(err) {
            throw new Error(err);
        }

        fnishedCount++;
        fileContents[1] = data;

        continuation();
    });
}

This is the equivalent source using asyncblock and fibers:

var asyncblock = require('asyncblock');

asyncblock(function(flow){
    fs.readFile('path1', 'utf8', flow.add('first'));
    fs.readFile('path2', 'utf8', flow.add('second'));

    var fileContents = flow.wait();

    fs.writeFile('path3', fileContents.first, flow.add());
    flow.wait();

    fs.readFile('path3', 'utf8', flow.add());
    var data = flow.wait();

    console.log(data);
    console.log('all done');
});

Error handling is addressed by including flow.errorCallback. There are examples of how to use this in the project’s documentation.

Introduction to Node Modules

Robert Kuzelj has been writing a series of posts about Node modules:

He demonstrates how to make packages, and how to take advantage of npm’s major features.

Review19

In other Node-related DailyJS articles I’ve invited readers to share their own apps with the community. Review19 by Srirangan is one such project: a kanban and scrum project management tool.

It’s built with Socket.IO to provide a real-time view of a project, and uses simple agile-inspired records that should be familiar to users of Jira or Pivotal Tracker.

Sign up is free, so give it a try. The author has a Google Group to support the community at Review19 Community.

jQuery Roundup: AMD-Utils, jquery.mentionsInput, Echo Nest API Plugin, i18next

20 Dec 2011 | By Alex Young | Comments | Tags jquery plugins music amd ui translation

Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

AMD-Utils

AMD-Utils (GitHub: millermedeiros / amd-utils, License: MIT) by Miller Medeiros is a set of modular utilities written using the AMD API.

All code is library agnostic and consist mostly of helper methods that aren’t directly related with the DOM, the purpose of this library isn’t to replace Dojo, jQuery, YUI, Mootools, etc, but to provide modular solutions for common problems that aren’t solved by most of them.

The modules written so far include some of the things covered on my Let’s Make a Framework tutorial series, like a functional style Array module, and other commonly required number and string utility functions often added by large frameworks.

jquery.mentionsInput

jquery.mentionsInput (GitHub: podio / jquery-mentions-input, License: MIT) by Kenneth Auchenberg and the Podio Dev Team is a UI component for handling @mentions. It’ll display an autocomplete list for matching names and allow one to be selected. Once a name is selected, it’ll change colour in the text box, as shown above.

The authors have packaged it with CSS, so it’s easy to get started right away. It does expect some data, rather than automatically searching an API like Twitter’s, so the expected format looks like this:

$('textarea.mention').mentionsInput({
  onDataRequest:function (mode, query, callback) {
    var data = [
      { id:1, name:'Kenneth Auchenberg', 'avatar':'http://cdn0.4dots.com/i/customavatars/avatar7112_1.gif', 'type':'contact' },
      { id:2, name:'Jon Froda', 'avatar':'http://cdn0.4dots.com/i/customavatars/avatar7112_1.gif', 'type':'contact' },
      { id:3, name:'Anders Pollas', 'avatar':'http://cdn0.4dots.com/i/customavatars/avatar7112_1.gif', 'type':'contact' },
      { id:4, name:'Kasper Hulthin', 'avatar':'http://cdn0.4dots.com/i/customavatars/avatar7112_1.gif', 'type':'contact' },
      { id:5, name:'Andreas Haugstrup', 'avatar':'http://cdn0.4dots.com/i/customavatars/avatar7112_1.gif', 'type':'contact' },
      { id:6, name:'Pete Lacey', 'avatar':'http://cdn0.4dots.com/i/customavatars/avatar7112_1.gif', 'type':'contact' }
    ];

    data = _.filter(data, function(item) { return item.name.toLowerCase().indexOf(query.toLowerCase()) > -1 });

    callback.call(this, data);
  }
});

Echo Nest API Plugin

Echonest-jQuery-Plugin by Samuel Richardson is a plugin for The Echo Nest which is a real-time API for accessing music data. Song data and audio fingerprinting are just some of the cool things that this API provides.

Let’s say I needed to get a set of album images. All I’d have to do is this:

var echonest = new EchoNest('YOUR_API_KEY');
echonest.artist('Radiohead').images(function(imageCollection) {
  $('body').prepend(imageCollection.to_html('<img src="${url}">'));
});

Combined with a templating system, this makes working with music data extremely convenient. The only issue with this approach is the API key is exposed. Echo Nest uses the API key for enforcing rate limits, so it’s not safe to expose it publicly. As it stands, I’d probably use client-side Echo Nest API code as a way of rapidly prototyping a music service built on this platform, then create my own server-side bridge to obscure the API key.

i18next

i18next (GitHub: jamuhl / i18next, License: MIT) by Jan Mühlemann is a client-side translation plugin with lots of features, including: plurals, localStorage, namespaces, and variables. JSON resource files can be used to store translations, and then i18next will load them as required.

Given a suitable file at /locales/en-US/translation.json:

{
  "app": {
    "name": "i18n"
  },
  "creator": {
    "firstname": "Jan",
    "lastname": "Mühlemann"
  }
}

Then $.i18n.init can be used to load the resource and access translations:

$.i18n.init({}, function(t) { // will init i18n with default settings and set language from navigator
  var appName = t('app.name'); // -> i18n
  var creator = t('creator.firstname') + ' ' + t('creator.lastname'); // -> Jan Mühlemann
});

The i18next documentation contains more examples and shows how to change languages, organise translations with nesting, and use variables.

Using SilkJS ORM and ExtJS Grids Together

19 Dec 2011 | By Mike Schwartz | Comments | Tags extjs sencha v8 tutorials
Mike Schwartz has been developing software since the early 1970s. He's proficient in JavaScript, PHP, C/C++, Java, C#, Perl, and Assembler language. He founded Best Internet Communications, the SF Bay Area's largest ISP through the late 1990s. He currently works for PINT, Inc, the longest-established San Diego firm specializing in Web site design and Web application development. His focus for the past several years has been in writing front facing and back end JavaScript, especially ExtJS and Helma. He recently released SilkJS, a very fast WWW server written almost entirely in JavaScript.

SilkJS is a very fast JavaScript based HTTP server that was designed to work with ExtJS. All code for the server is written in JavaScript, including almost the entirety of the server itself. You can check out the server’s GitHub repository at mschwartz / SilkJS and the repository for this demo at mschwartz / SilkJS-ext.

The SilkJS Install.js script creates a /usr/share/SilkJS directory with the HTTP server, library files, and default documentRoot. It also installs the main HTTP server in /usr/local/bin/httpd-silk.js.

Let’s examine the SilkJS-ext repository first. This is config.js:

Config.mysql = {
  host: 'localhost',
  user: 'mschwartz',
  passwd: '',
  db: 'ext3'
};
Config.documentRoot = 'docroot';

This file overrides the default Config object, specifying MySQL connection information, and a documentRoot for this example.

The actual Server-Side JavaScript code is implemented in under 60 lines of code, in Server.js:

SQL = new MySQL();
SQL.connect();

Schema.add({
  name: 'Users',
  fields: [
    { name: 'userId', type: 'int', autoIncrement: true, defaultValue: 0 },
    { name: 'username', type: 'varchar', size: 64, header: 'User Name', width: 128, autoExpand: true, editable: true },
    { name: 'password', type: 'varchar', size: 64, header: 'Password', serverOnly: true, editable: true },
    { name: 'email', type: 'varchar', size: 128, header: 'E-Mail Address', width: 128, editable: true },
    { name: 'created', type: 'int', defaultValue: function() { return parseInt(new Date().getTime() / 1000, 10); }, header: 'Created', width: 150, format: 'DateTime', editable: false }
  ],
  primaryKey: 'userId',
  indexes: [
    'username',
    'email'
  ]
});


function main_action() {
  res.write([
    '<!doctype html>',
    '<html>',
    ' <head>',
    '   <title>Schema / ExtJS Demo</title>',
    '   <link rel="stylesheet" type="text/css" href="/ext-3.4.0/resources/css/ext-all.css" />',
    ' </head>',
    ' <body>',
    ' <script type="text/javascript" src="/ext-3.4.0/adapter/ext/ext-base.js"></script>',
    ' <script type="text/javascript" src="/ext-3.4.0/ext-all-debug.js"></script>',
    ' <script type="text/javascript" src="/client/Ext.ux.SchemaGrid.js"></script>',
    ' <script type="text/javascript" src="/client/ViewPort.js"></script>',
    ' <script type="text/javascript">',
    '   Schemas = ' + Json.encode(Schema.getSchemaExtJs()) + ';',
    ' </script>',
    ' </body>',
    '</html>'
  ].join('\n'));
  res.stop();
}
function Server_action() {
  switch (req.data.method) {
    case 'listUsers':
      Json.success(Schema.list('Users', {}, function(o) {
        o = Schema.clean(o);
      }));
    case 'editUser':
      var example = Json.decode(req.data.example);
      example.userId = example.userId || 0;
      Schema.putOne('Users', example);
      Json.success();
    case 'deleteUsers':
      var examples = Json.decode(req.data.examples);
      forEach(examples, function(example) {
        Schema.remove('Users',example);
      });
      Json.success();
  }
}

The first thing it does is create a global MySQL object, named SQL. This SQL object is used by the ORM to generate queries. With the ORM, you rarely have to generate any queries yourself, and even then, the ORM does most of the work.

The Schema.add() method is called to install a “Schema” in the ORM. The Schema is defined as a JavaScript object. The definition above shows only a part of what’s possible. The name member is the name of the table in the database. The fields array is an array of field definitions. The primaryKey member is the primary key of the table, and the indexes array is an array of additional indexes to be created for the table.

For the fields array, each item is an object with a few required members and some optional ones. The required ones are name, and type. If type is 'varchar', then size is also required. Only one of the fields may be autoIncrement: true, and is typically the primaryKey of the table as well. This autoIncrement field is also typically the ID field of ExtJS DataStores on the client-side.

The defaultValue member may be an expression or a function. The defaultValue member is used to set a default value for the field when creating a new entry in the database. As you can see in the example above, we have a ‘created’ field of type 'int' (a Unix-style timestamp) that has a defaultValue of a function that creates a Unix timestamp for the current time.

One additional members may be specified. Note the password field is marked serverOnly: true – this is to specify that the value of this field should not be sent to the client (we’ll see this later).

Currently, every other member of a field definition is ignored by the ORM, and can be used for your application for whatever purposes you like. In this case, I specify information that will help us generate ExtJS Grid columns, and form fields for editing records. For example, username is autoExpand: true, and fields with header: 'some string' will show in the grids.

The SilkJS HTTP server will call main_action() (if it is defined) if / is the specified URL. Here we simply write a typical HTML document to the browser. The output HTML looks like this:

<!doctype html>
<html>
  <head>
    <title>Schema / ExtJS Demo</title>
    <link rel="stylesheet" type="text/css" href="/ext-3.4.0/resources/css/ext-all.css" />
  </head>
  <body>
    <script type="text/javascript" src="/ext-3.4.0/adapter/ext/ext-base.js"></script>
    <script type="text/javascript" src="/ext-3.4.0/ext-all-debug.js"></script>
    <script type="text/javascript" src="/client/Ext.ux.SchemaGrid.js"></script>
    <script type="text/javascript" src="/client/ViewPort.js"></script>
    <script type="text/javascript">
    Schemas = {"Users":{"name":"Users","fields":[{"name":"userId","type":"int","autoIncrement":true,"defaultValue":0},{"name":"username","size":64,"header":"User Name","width":128,"autoExpand":true,"editable":true},{"name":"password","size":64,"header":"Password","serverOnly":true,"editable":true},{"name":"email","size":128,"header":"E-Mail Address","width":128,"editable":true},{"name":"created","type":"int","header":"Created","width":150,"format":"DateTime","editable":false}],"primaryKey":"userId"}};
    </script>
  </body>
</html>

All records in SilkJS and ExtJS are effectively just JavaScript objects; each member is a column/value pair from the database. But not every field from the database must exist in these objects! A partial record of this form is what I call an “example.” If you call Schema.find(example), it will do a SELECT query based upon the fields you do specify in the example. For example, Schema.find({ username: '%msch%' }) will return an array of records from the database that have a username with ‘msch’ somewhere in it. Schema.putOne({ username: 'mschwartz' }) will store a new record in the database with defaultValues for the other fields and username set to ‘mschwartz’. The only way I can explain examples is with examples!

So editUser expects the client to build an example and send it over. If editing an existing record, the example will have a userId, and the record will be updated by Schema.putOne(). If adding a new record, the example will have no userId or a userId set to 0.

The deleteUsers method expects an array of examples. It calls Schema.remove(example) on each element of the array, which deletes the specified rows from the database. Schema.remove() can be dangerous if you’re not careful about the examples you give it. In this case, the array of examples would be some multiple selections on the grid on the client side.

The client-side ExtJS code is in docroot/client and accessed via /client/whatever paths in the HTML. In this case, we have just two files, Viewport.js and Ext.ux.SchemaGrid.js.

ViewPort.js is rather simple:

Ext.onReady(function() {
  var items = [];
  for (var i in Schemas) {
    var schema = Schemas[i];
    items.push({
      xtype: 'ext-ux-schemagrid',
      title: i,
      schema: schema,
      method: 'list'+i
    });
  }


  new Ext.Viewport({
    layout: 'fit',
    items: [
      {
        xtype: 'tabpanel',
        activeTab: 0,
        items: items
      }
    ]
  })
});

It loops through the global Schemas and creates an items array of Panel configs, one Schema Grid per panel. The Ext.ViewPort is simply an Ext.TabPanel with one tab per panel.

The real work on the client is done by Ext.ux.SchemaGrid.js.

I don’t want to go into great detail about how this custom component works, but I will explain some of the things it does. Basic ExtJS knowledge is assumed here.

SchemaGrid extends Ext.grid.GridPanel and takes as additional configuration options a Schema, and a method. Those are set in the ViewPort.js code.

In the initComponent method, the URL of the JsonStore is set to /Server (so Server_action() is called on the SilkJS side), and baseParams method is set to the SchemaGrid’s config.method, listUsers in our case. The JsonStore’s fields are directly used from the Schema’s fields array (no muss, no fuss).

After setting up the SelectionModel, the code loops through the Schema’s fields and generates column definitions for the grid. Those fields that have a header member and are not serverOnly will have columns displayed. The renderer for each column is a single method in the SchemaGrid class. The default action for our renderer is to render the created/timestamp in human readable format. An autoExpand column may be specified in the Schema definition, and it is handled automatically by this code.

When the JsonStore is loaded or the user clicks on items in the grid, the updateToolbar() method is called to conditionally enable and disable the edit and delete buttons. Basically, you can only edit if one row is selected, and you can’t delete unless at least one row is selected.

The editRecord() method dynamically creates a form in a modal dialog window. If a record is passed in, the dialog title says “Edit record” otherwise “Add record.” The dialog/form can be used to add new records (click the add button) or edit existing records (select a record, click edit button).

When the OK button is clicked on the dialog, the record (EXAMPLE) is updated with the form fields’ values and the Ext.Ajax.request() made to call editUser to update the database.

The deleteRecords() method is called to delete one or more selected records in the grid. It creates the array of examples and calls Ext.Ajax.request() to invoke deleteUsers on the server side.

So let’s see it in action!

Screenshots

This is the application when it first starts:

SilkJS/ExtJS application starts

Now the add button has been clicked:

SilkJS/ExtJS button click

Here I’ve added a few records:

SilkJS/ExtJS records added

This is what happens when a record is opened for editing:

SilkJS/ExtJS editing record

Advantages

So, what’s neat about this?

If you edit the Schema definition at the top of Server.js, those changes will immediately be reflected in the database when you either restart the server or the browser makes a new request (reload the browser, click the reload button in the paging toolbar).

What do I mean? If you change the size of password field to 128, the ORM will submit the appropriate ALTER TABLE statements to MySQL to change the size of the field. If you add a field to the fields array, it will be added to the table in the database as well. If you delete a field, it will be deleted in the database. If you rename a field, it will be renamed in the database.

If you add a second Schema definition at the top of Server.js, the table will be created next access. If you reload your browser, you will see two tabs, one for each table, and the CRUD operations apply to both.

Outside of the Schema definitions, which are compact in their own right, and the main_action() method that spews out the HTML for the app, the entire CRUD operations on the server side consist of under 20 lines of code.

The entire demo took me under 1 hour to create, all but 2-3 minutes was spent writing the client-side code.

Kanso Repository, Recipes with Backbone, Appify UI

16 Dec 2011 | By Alex Young | Comments | Tags couchdb mac backbone.js

Kanso Package Repository

Kanso (GitHub: kanso ) has been changed to work more like a generic build system for CouchApps. It’s more framework agnostic, and includes a package repository, which is available at kan.so/packages. From Caolan McMahon in the New Kanso release and change of direction thread:

Essentially, all you need to add to your project is a kanso.json file describing how it should be built. Once your app is in this format, it’s possible to merge it with other design docs written in the same way. Using this basic principle, we’ve developed a library of JavaScript modules, build-steps, and more, all of which can be easily shared between projects.

If you’re working with CouchDB and want a relatively easy way to run JavaScript apps, then it’s worth giving Kanso a try. The community on the Kanso Google Group is still relatively small and helpful.

Recipes with Backbone

Recipes with Backbone (price: $24) is an ebook by Nick Gauthier and Chris Strom that covers an introduction to Backbone.js, namespacing, view templates, collection views, actions, animations, non-REST models, routes, and more. The full table of contents is here: Recipes with Backbone and excerpts are available for Collection View and Changes Feed.

I haven’t yet bought a copy for review, but the sample content looks promising. If you buy it, please let everyone know what you think in the comments!

Appify UI

Appify UI by Thomas Aylott helps package web apps as Mac OS X bundles. The thing that makes it slightly special is Node apps can be bundled just as easily as anything else. There’s a Node demo app, and instructions on how to make them in the project’s documentation.

JavaScript Developer Survey 2011 Results

15 Dec 2011 | By Alex Young | Comments | Tags community surveys

Note: Send feedback about our surveys by using the contact form or @dailyjs.

The results are in! Thanks to everyone who took part, your answers are now the property of the community.

The raw result set is here: JavaScript Developer Survey 2011, Data.
A summary of the results generated by Google Docs can be downloaded here: JavaScript Developer Survey 2011, PDF.

I got the feeling that client-side developers felt a little underrepresented by the survey, so in future I can run additional client-side, server-side, or even Node-specific surveys if enough people are interested.

The purpose of this survey is purely to gather information that will help people develop software and services for JavaScript developers. For example, if you’re looking to develop an open source project and want to see what areas are lacking, then maybe this data might be useful. Download the data, and use it!

Environment

Almost everyone who took part said they do browser development (98%). Server-side web app development is also popular, at 45%. This is up 10% from last year. I find it encouraging to see an strong interesting in Unix scripting (14%).

The preferred server-side interpreter was Node, at 71%. 12% selected Other, and there were some strongly-worded comments about including more environments. Also, clarification will be applied in future surveys to state whether the question relates to browser, server, or both, and in this case it wasn’t clear to some people.

Testing, Static Analysis, Benchmarks

58% of people don’t write tests. This is actually down from 68% last year. I appreciate that testing might seem awkward for client-side development, but it’s not uncommon to find open source client-side code bundled with tests, so I’d encourage both browser-based and server-side developers to at least write basic tests if possible.

JSLint is still the king of static analysis (67%), but Google Closure Compiler and YUI Compressor are close by at 25% and 21% respectively.

Now the shocking news: Jasmine (44%) has edged out QUnit (41%)! Vows is also doing well with 13%. Express/Mocha scored 11%, slightly ahead of Nodeunit at 8%.

Another surprising trend is YUI Compressor is beating Google’s Closure Compiler for minifying code, at 41% vs. 31%. JSMIN is down in popularity again, from 25% last year to 20% this year.

The well-known cabal of Node developers with prominent packages on NPM are known for writing benchmarks. However, this survey showed that 81% of respondents use client-side tools for debugging. I suspect the Node benchmark fetishists (and I mean that in the nicest possible way) were represented by the 20% that answered “benchmark using my own code”.

Project Discovery and Hosting

Most people find JavaScript libraries and tools through GitHub (81%). Search engines are the next most popular method (51%). NPM seemed a little bit low to me, at 20% — the website isn’t actually too bad for discovering projects, it supports keywords and has links to dependent projects. When I’m working on a Node project I often find useful libraries through search.npmjs.org.

GitHub is the champion of hosting for personal JavaScript projects (90%). Bitbucket is the second most popular at 10%. I wonder if Bitbucket will grow next year given the generous deal they have on private project hosting?

Google’s CDN is the most popular third-party hosting service (83%).

Debugging

WebKit inspector has edged out Firefox with 66%, compared to 58%. Surprisingly, 525 people (51%) use node --debug, which interested me because a lot of people ask me about Node debugging without even realising it has the --debug option.

131 people said they use the Cloud 9 IDE as part of the debugging question. I’ve tried this service out myself and it does seem like an interesting way to work on client-side code (technically any code, but the debugging for client-side code is impressive).

Languages

PHP remains the most popular language amongst our readers, with 46%, which is the same as last year. I expected (wanted) more people to be interested in Clojure (2%) or Lisp in general (2%), but Ruby, Java, Python, and C#/.Net were the other most popular languages, which is again unchanged from last year.

“Other”

I promised people in the comments that I’d mention some of the “Other” field data. This is by no means an exhaustive look at this data, but some interesting observations.

Some people said they had “no preference” when it came to interpreter. Others seemed upset that I didn’t include ‘V8’, but I meant the question to be purely about server-side JavaScript, so I should have clarified that and just included Node, Ringo, etc.

In terms of unit testing, I found the following mentioned:

  • Ringo’s test module
  • Sinon.JS
  • Evidence.js
  • Around 10 custom or homegrown unit testing frameworks
  • “I must start doing this”, “I CODE EVERYTHING PERFECT ON THE FIRST TRY

Previous Surveys

Node Roundup: 0.6.5, NodeUp, Logme, Consolidate.js, mdoc, Firmata, Nodeflakes

You can send your Node modules and articles in for review through our contact form or @dailyjs.

Node 0.6.5

Node 0.6.5 is out, which upgrades V8 to 3.6.6.11 and potentially avoids national firewall-related issues.

NodeUp

NodeUp is a podcast all about Node. In NodeUp Nine features Isaac Z. Schlueter, Matt Ranney, Dominic Tarr, and James Halliday discussing scaling, deployment, and Erlang. Other topics include migrating to Node 0.6 and hosting on Joyent with Solaris.

Subscribe with either iTunes or RSS:

Logme

Logme (License: MIT, npm: logme) by Veselin Todorov is a slick little logging library. Basic usage looks like this:

var logme = require('logme');
logme.critical('Error');

It has a lot of colour themes, which are set by instantiating a Logme object:

var Logme = require('logme').Logme;
  , logme = new Logme({ theme: 'smile' });

logme.error('This is now themed.');

The only thing I miss is the ability to quickly append values with spaces, like this: console.log('Value:', value);.

It’s worth remembering that console also comes with console.error, which will write to stderr.

Consolidate.js

Consolidate.js (License: MIT, npm: consolidate) by TJ Holowaychuk is a wrapper that gives template engines the same API. TJ’s documentation mentions that Express 3.x uses the signature (path[, locals], callback) for template rendering, which is the same as Consolidate.js.

This library adds a thin layer that makes it easier to switch between different template engines.

mdoc

mdoc (License: MIT, npm: mdoc) by Miller Medeiros is a markdown-based documentation generator. It’ll create HTML files for each markdown file in a directory, and a table of contents.

One of the author’s examples is unofficial Node mdoc-generated documentation. He’s only been working on it for a few weeks, yet the style of the documentation it generates is readable and suits larger projects.

Some projects benefit handcrafted documentation rather than JSDoc-style API documentation. Miller wrote a post about this topic called Inline Documentation: Why I’m Ditching it.

Firmata

Firmata (GitHub: jgautier / firmata, License: MIT, npm: firmata) by Julian Gautier is a library for interacting with Arduino boards running the firmata protocol.

If you’re a Node developer you may find this more appealing than typical Arduino code:

var firmata = require('firmata')
  , board;

board = new firmata.Board('path to usb', function() {
  // Arduino is ready to communicate
});  

Nodeflakes

Nick Payne sent in Nodeflakes (GitHub: makeusabrew / nodeflakes), a Twitter Streaming API experiment that displays CSS3 unicode snowflakes that display tweets. The architecture is actually multiprocess, using a producer/consumer architecture. The client-side code is in nodeflakes-client.

Despite appearing to be a holiday-themed gimmick, Nodeflakes is very educational. The author has written a detailed blog post about the project: Nodeflakes – real time tweet powered snowflakes.

jQuery Roundup: Create, jQuery/Switch, Matteo Bicocchi's Plugins, nanoScroller.js, Mobi Pick, jQuery Geo, Brequire

13 Dec 2011 | By Alex Young | Comments | Tags jquery plugins geolocation ui scrollbars

Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

Create

Create (GitHub: bergie / create, License: MIT) by Henri Bergius is built on jQuery UI, Backbone.js, Modernizr, and VIE. It provides a dynamic editing interface suited to browser-based HTML5 content management.

One relatively unique feature of Create is the use of RDFa annotations to mark up editable content. This reminded me of a CMS I built in the early 2000s that used XSLT to transform arbitrary data into editable data, but this seems far easier to use. To read more about the RDFa portion of this project, see VIE by the same author.

jQuery/Switch

jQuery/Switch (GitHub: rawnet / jquery-switch, License: MIT) from Rawnet is an iOS-inspired slide and toggle widget:

The plugin is accessed by using the switchify method. The HTML for this plugin can be marked up by using a select element with suitable on/off values.

The documentation for this plugin is excellent, with fully interactive examples for lots of different configuration options.

Matteo Bicocchi’s Plugins

mb.jQuery Components (License: MIT and GPL) by Matteo Bicocchi is a collection of 16 plugins focused on media and UI. There’s a tree menu, draggable windows, sliders, and even a background YouTube movie player.

All of the plugins are presented with documentation and demos.

nanoScroller.js

nanoScroller.js (GitHub: jamesflorentino / nanoScrollerJS, License: MIT) by James Florentino is a Lion-style scrollbar implementation. It requires a little bit of CSS, but usage basically boils down to this:

$('selector').nanoScroller();

The author has tested it with IE8+, Firefox, Chrome, Safari, and iOS 5. Older browsers like IE7 will degrade to the native scrollbar.

Mobi Pick

Mobi Pick (GitHub: sustainablepace / mobipick, License: MIT) by Christoph Baudson is a rather cute Android-inspired date picker. It uses the impressive XDate library, and features progressive enhancement through Modernizr.

Mobi Pick has implemented localisation through XDate’s locale support, which means a localised picker can be displayed simply by specifying a language:

page.find('selector').mobipick({
  locale: 'es'
});

jQuery Geo

jQuery Geo (GitHub: AppGeo / geo) from Applied Geographics is an open source mapping project that provides a friendly JavaScript API for services like Open Street Map, WMS and Esri ArcGIS Server.

The simplest jQuery Geo demo shows how easy it is to get started with the plugin:

$(function () {
  $('#map').geomap();
});

Centring and zooming is also easy:

$('#map').geomap({
  center: [-71.037598, 42.363281],
  zoom: 10
});

The project also includes support for geometry and user interaction with geometric objects.

Brequire

We’ve been covering asynchronous script loaders a lot recently, more by chance than design, and so Jonah Fox sent in his latest updates for Brequire. I’ve mentioned this library before on DailyJS, but the recent updates add some interesting asynchronous functionality through the optional require.async.js module:

require('./app', function(app) {
  // do stuff with app
});

Also, in commit c9aba57 the author added AMD support.

Flickr's Asynchronous Script Loading Part 2

12 Dec 2011 | By Alex Kessinger | Comments | Tags events dom code-review flickr-async
Alex Kessinger is a programmer who lives in the Bay Area. He strives to make websites, cook, and write a little bit better each day. You can find more from Alex at his blog, on Twitter, and Google+.

Part One: Flickr’s Asynchronous Script Loading

This is part two of Production Teardown: Flickr’s Asynchronous Script Loading. To recap, if we use an async JS loader, or even if we just put our script tags at the bottom of our pages we have a problem. There is a small window where it’s possible to interact with the page before the code has loaded.

Flickr uses something they call an actionQueue, and their code is very tight and isolated. With little modification I think we could use that exact same piece of code for other things. We are going to build a simple page to put this to the test. To start, we need to make a few changes to the original code so that we can load multiple modules instead of just one.

Note: I am using QUnit to run the tests.

Adapting actionQueue.js

The code: actionQueue.js.

I had to make a couple of changes to the stock code for the purposes of generalizing the library, and to make it more testable. First, the original module is hard to test because its state is private. I wouldn’t expect anything less, the state is only used by the actionQueue function. However, during testing we need to reset the internal state of actionQueue.

One possible way to do this is to write an actionQueue factory. This is a function that creates a brand new actionQueue every time it’s called. Then it can be assigned to A.actionQueue as required. This has a lot of advantages but seems like overkill for our requirements. An alternative solution is to simply create a new function that can reset the internal state of the module. I’ve called this function refresh_module.

The second thing I did was to use [JSLint](http://www.jslint.com/) to help clean up the original code. For example, there were cases when a variable was assigned in a `for` loop:
for (var i in an_array) {

}
This is usually a bad idea because it's re-initializing `i` every iteration of the loop. A better form suggested by JSLint is this:
var i;
for (i in an_array) {

}
I expect the engineers who wrote this code knew about this. I don't even think everything needs to pass JSLint, it's just a rubric that one can use to begin the process of evaluating code. What I think happened here is that they used the first version to save space. In the second version `i` is repeated twice, it's not as small as the first version. Also, they were also looping over a very small array. They probably optimized their code for the most pressing priority which was code size. Given that this code is going to show up in many, many pages on Flickr, which is an extremely popular site, then the size argument makes sense.

I got this wrong. As some people in the comments have pointed out the variable declaration is hoisted. I have corrected the error, and welcome any additional criticism. To be clear though, I would rather have an error in everything I have ever written, than to not write at all. We can’t walk through the world constantly second guessing everything we say, or we wouldn’t say anything. The best we can do is welcome being wrong, and fix it.

With testability improved and static analysers applied, I made one more major functional modification. When I started looking at Flickr’s code I could see that there was an internal flag called queueing. When true, which is its default, interim functions get executed. When false, nothing happens; everything just passes through to the queue_click function.

What’s curious about queueing is that it’s global. actionQueue was clearly written with the idea of multiple modules, but apparently that part ended up not being needed. When the first module gets loaded queueing is set to false. That means all queue_click calls now pass through, even for modules that haven’t yet called module_load. I have a feeling that Flickr loads all of its modules in a monolithic file. They aren’t seeing any repercussions in this code because if one module has loaded, they have all loaded. I don’t think that this will be the story for everyone who might want to use this code. So, I modified it so that each module has its own queueing state.

I’ve added the code on line 38 to set queueing to true for any new modules that get added by the register function. Then as module_loaded gets called, it will only turn queueing off for that module. You can see all my tests, and how I tested the multiple module section in actionQueue tests.

Building a Like Button

To demonstrate this code, we are going to build a simple “like” button. There will be two steps. When clicked it will change state, and then there will be an action for when the module loads. The compete example is here: actionQueue in practice.

This simple markup creates a like button with two states. The visual state is controlled by flipping a class name. Then I set up some initial code building our global container, and include the actionQueue code.

Line 20 is where we start to see how the like button is going work. This is comprised of two steps. In the interim step the like button will get a new class. This will flip the visual state. In the cleanup state, which will fire after the module, like-handler loads and the like button is set to Done!.

The like button itself is implemented as HTML. It’s important to note that the anchor tag has an onclick. I know, this looks dirty, but really it’s the easiest way to handle events before all of the code has loaded. The arguments behind this were covered in the previous tutorial. So, when a user clicks the like button an event gets fired into the action queue which will first fire the interim step.

Later I setup a link that will ‘load’ the module. This will then fire the cleanup function.

Conclusion

By first researching Flickr’s code, then adapting it to be easier to test, we’ve created something genuinely useful and reusable. And although using onclick might be a little bit controversial, the previous part had a great comment from Ross Harmes that explained the thinking behind it:

I can speak to why we use onclick in this situation. For performance reasons, we want to load as little JS as possible until the page content is fully loaded. We also don’t want to litter the page with script blocks just to attach these interim events. These two constraints mean that we can’t load a library that normalizes event handling across browsers, nor can we attach event listeners in the standard way, using JS only. onclick is the only approach left to us, but it isn’t a bad one. Like you said, we try to not be too dogmatic about these things.

Amber.js, HelsinkiJS, csg.js, SilkJS

09 Dec 2011 | By Alex Young | Comments | Tags languages webgl events

Amber

Note: Amber has now been renamed to Ember.js.

Amber.js (GitHub: amberjs / amber.js, License: MIT) by Yehuda Katz started life as SproutCore 2.0 but has been renamed to Amber.js to avoid potential confusion, as it’s actually a very different framework. While Katz calls SproutCore a “native-style widget library”, Amber.js is closer to a client-side MVC framework. Katz essentially describes Amber.js as lighter than SproutCore, and appealing to Backbone.js users.

In fact, Amber.js is surprisingly similar to Backbone.js, aiming to solve the problem of client-side data storage and synchronisation with view code. It uses the Handlebars templating library and some of its own modules to achieve this. There’s an example app available at todos20.strobeapp.com.

This seems like an interesting departure from SproutCore, but a few things concern me as a server-side (and client-side!) JavaScript developer. The project doesn’t ship with a package.json or use npm for distribution. The build process is based on Ruby, and tests are run with Ruby. Although it’s obvious that this project is aimed at any server-side language, since it’s a JavaScript-centric project I’d much prefer to see it using the tools I use for other JavaScript projects.

Also, from my brief experimentation with Amber.js, it doesn’t seem to address script loading. I feel like a project like this should include something like RequireJS and be developed using AMD-friendly modules.

Despite these complaints, I do like the minimal amount of boilerplate required compared to similar frameworks, and the project is already edging up to 1000 GitHub followers. Given the level of interest it has a promising future ahead.

HelsinkiJS

HelsinkiJS is being held on December 12th from 7pm to 9pm at Hietalahdenkatu 8 A 22, 5th floor. Two speakers are currently billed: Henri Bergius and Oleg Podsechin.

If you want to track future events, following @helsinkijs on Twitter should do the trick.

csg.js

csg.js (GitHub: evanw / csg.js, License: MIT) by Evan Wallace is a Constructive Solid Geometry (CSG) library that implements “operations on meshes elegantly and concisely using BSP trees”. This might sound specialist 3D graphics talk, but the examples on the site do a good job of explaining it. There’s even an interactive example.

This makes a little 3D olive:

var a = CSG.cylinder({ radius: 1.35 });
var b = CSG.sphere({ radius: 1.35 });
a.setColor(1, 0, 0);
b.setColor(0, 1, 0);
return a.intersect(b);

SilkJS

SilkJS (GitHub: mschwartz / SilkJS) by Michael Schwartz is a C++ program that uses V8 to provide access to OS-level functions and libraries. There’s an ncurses.js example that demonstrates this.

SilkJS scripts expect a main() function. System libraries are linked to SilkJS itself, so the ncurses JavaScript example is made possible by the fact SilkJS includes a binding to ncurses.

The SilkJS Wiki contains more information on the project, including comparisons with Node and tips on building the project.

Testing with Mocha

08 Dec 2011 | By Alex Young | Comments | Tags node modules tutorials testing

Mocha (GitHub: visionmedia / mocha, npm: mocha, License: MIT) by TJ Holowaychuk is a new test library that does just about everything a JavaScript developer needs, and yet remains customisable enough to support both behaviour-driven and test-driven development styles.

Mocha presents itself as an unassuming and lightweight library, but it actually goes far beyond the test frameworks I’m familiar with in Node. I’ve written literally hundreds of tests for various Node apps, and the experience hasn’t always been positive. Even popular test libraries lack functionality that asynchronous, database-driven web apps require for sane testing. My recommendation is to take Mocha seriously and give it a try — it’s surprisingly intuitive.

The Basics

Mocha is available through npm, but I install it by adding it to my package.json files. Make a package.json file like this one:

{
    "name": "async-testing-tutorial"
  , "version": "0.0.1"
  , "description": "A tutorial for Mocha"
  , "keywords": ["test", "tutorial"]
  , "author": "Alex R. Young <info@dailyjs.com>"
  , "main": "index"
  , "engines": { "node": ">= 0.4.x < 0.7.0" }
  , "scripts": {
    "test": "make test"
  }
  , "devDependencies": {
    "mocha": "0.3.x"
  }
}

Then run npm install and you’re almost ready to use Mocha. The Mocha script will be in ./node_modules/.bin/mocha, so rather than typing this every time add a simple Makefile to your project:

test:
	@./node_modules/.bin/mocha

.PHONY: test

While you’re at it, add a test/ directory:

mkdir test

Now typing make test will run any tests in test/!

Any options for Mocha can now be added to the Makefile:

test:
	@./node_modules/.bin/mocha -u tdd

.PHONY: test

This says we want to run tests in the TDD style:

The “BDD” interface provides describe(), it(), before(), after(), beforeEach(), and afterEach(). The “TDD” interface provides suite(), test(), setup(), and teardown().

Testing Synchronously

We’d better write something! Add this to index.js:

function nextPrime(n) {
  var smaller;
  n = Math.floor(n);

  if (n >= 2) {
    smaller = 1;
    while (smaller * smaller <= n) {
      n++;
      smaller = 2;
      while ((n % smaller > 0) && (smaller * smaller <= n)) {
        smaller++;
      }
    }
    return n;
  } else {
    return 2;
  }
}

module.exports.nextPrime = nextPrime;

This is just an excuse to do something, it’s not meant to be the world’s greatest prime number generator. One reason I like this function is large numbers will make the interpreter really slow, which can be useful for demonstrating certain run-loop related issues.

Now create a new file called test/test.next-prime.js:

var assert = require('assert')
  , nextPrime = require('./../index').nextPrime
  , asyncPrime = require('./../index').asyncPrime;

suite('nextPrime', function() {
  test('nextPrime should return the next prime number', function() {
    assert.equal(11, nextPrime(7));
  });

  test('zero and one are not prime numbers', function() {
    assert.equal(2, nextPrime(0));
    assert.equal(2, nextPrime(1));
  });
});

And run make test in the terminal. The test should pass:

Why is this simple test interesting? Well, Mocha explicitly supports asynchronous testing, yet we didn’t have to tell it that the test had finished running. To do that, we’d write a test like this:

var assert = require('assert')
  , nextPrime = require('./../index').nextPrime;

suite('nextPrime', function() {
  test('nextPrime should return the next prime number', function(done) {
    assert.equal(11, nextPrime(7));

    // Tell Mocha the test has finished
    done();
  });
});

How is Mocha able to achieve this amazing feat of introspection? It’s actually just checking how many arguments the callback has by using the length property on Function. This makes the API able to cope with both synchronous and asynchronous tests without any extra boilerplate coding. If you want to read more about this, MDN: Function length has examples.

Asynchronous Tests

Expanding on nextPrime, I’ve added a contrived asynchronous version:

function nextPrime(n) {
  var smaller;
  n = Math.floor(n);

  if (n >= 2) {
    smaller = 1;
    while (smaller * smaller <= n) {
      n++;
      smaller = 2;
      while ((n % smaller > 0) && (smaller * smaller <= n)) {
        smaller++;
      }
    }
    return n;
  } else {
    return 2;
  }
}

function asyncPrime(n, fn) {
  setTimeout(function() {
    fn(nextPrime(n));
  }, 10);
}

module.exports.nextPrime = nextPrime;
module.exports.asyncPrime = asyncPrime;

The asyncPrime method just wraps the old function with setTimeout to simulate an asynchronous call. Now we need to use the done function provided by Mocha to signal the end of the test:

suite('asyncPrime', function() {
  test('asyncPrime should return the next prime number', function(done) {
    asyncPrime(128, function(n) {
      assert.equal(131, n, 'Wrong number');
      done();
    });
  });
});

Mocha Tricks

Mocha comes packed with lots of test reporter format options, including TAP and JSON, which provide opportunities for easy integration with other tools like Continuous Integration (CI) services. TJ uses Travis and has a little build status indicator on Mocha’s README page.

Mocha also works with CoffeeScript and web browsers.

Sample Project

If you need help getting started with Mocha, I’ve put the code for this tutorial on GitHub at alexyoung / async-testing-tutorial, and TJ’s projects use Mocha, which means popular libraries like Express have plenty of examples to explore.

Node Roundup: 0.6.4, Node Web Development, Node.js in Action, read, XJST

07 Dec 2011 | By Alex Young | Comments | Tags node modules parsing books unix

You can send your Node modules and articles in for review through our contact form or @dailyjs.

Node 0.6.4

Node 0.6.4 has been released. One interesting thing I noticed was this:

sys: deprecate module some more, print stack trace if NODE_DEBUG=sys

It can be a little bit frustrating when working with dependencies that still use this module, so at least these stack traces will make it easier to find out who to blame.

Node Web Development

Node Web Development (Price: $22.49, Ebook: $12.74, Publisher: Packt) by David Herron assumes rudimentary JavaScript knowledge and introduces Node, covering the basic theory behind asynchronous and event-driven development, performance, installation, CPU utilisation, modules, Express, EventEmitter, SQLite3, and even Mongoose.

If you’re a web developer struggling to understand Node, then Node Web Development may help you get up to speed quickly. For more information, take a look at the Node Web Development table of contents and Chapter 3: Node Modules (PDF).

Node.js in Action

Node.js in Action (Price: $39.99, Ebook: $31.99, Publisher: Manning) by Mike Cantelon and TJ Holowaychuk is currently available as part of Manning’s Early Access Program, which means you can subscribe to read the book as chapters are finished.

The book is split into three parts, which cover the fundamentals, web development, and “going further with Node” which appears to cover “stuff that isn’t web development”. The first chapter is actually a well-tempered argument on why we need Node, so it’s worth considering for boss-convincing ammunition. The second chapter explains how to install Node for Windows, Mac OS X, and Linux, then goes on to make some fun scripts like a little TCP/IP server accessed with telnet.

The authors waste no time diving right into asynchronous development in chapter 3. This chapter is a solid treatment of asynchronous programming, with EventEmitter taking centre stage, and coverage of control flow techniques and tools.

Node.js in Action’s estimated publishing date is April 2012, with 300 pages in the finished version. It’s currently around 130 pages, so solid progress has been made so far. Chapter 1: Why the Web needs Node (PDF) is available as a free sample chapter.

read

read (npm: read) by Isaac Z. Schlueter is billed as “read(1) for Node”, and is intended to read lines from the standard input.

var read = require('read');

read({ prompt: 'echo: ' }, function(err, res) {
  if (err) {
    console.error(err);
  } else {
    console.log(res);
  }
});

Documentation is available in the project’s README. Isaac notes that care must be taken to manage process.stdin in certain versions of Node:

In node 0.6.0 through 0.6.5, you must explicitly call process.stdin.destroy() or process.exit() when you know that your program is done reading, or else it will keep the event loop running forever.

XJST

XJST (GitHub: veged / xjst, npm: xjst) by Sergey Berezhnoy is a Node and browser-compatible DSL for creating data transformations. Suggested applications include HTTP request routing, template engines, and parsers.

The authors have written some XJST examples to help people get the hang of the language. To use XJST, an input file must be created using XJST’s DSL. It reads like JavaScript, and XJST will compile it into JavaScript. The simple.xjst example is a good one for getting started.

If you’re writing a parser of some kind but feel put off by something like Jison, then XJST might help you get started without too much effort.

jQuery Plugins Site Alternatives

06 Dec 2011 | By Alex Young | Comments | Tags jquery plugins

Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

The jQuery plugins site is currently down. The intention is to replace it with a better site. However, is a replacement even necessary? When researching our weekly jQuery Roundup articles I rarely use the jQuery plugins site.

Fortunately, there are alternatives, and they’re arguably more friendly to front-end developers.

Hosting Code

Guess what? I don’t want to download a zip file from your website.

Most plugins I encounter are found through GitHub. With a free GitHub account, open source projects get version control, issue tracking, and a wiki. Another useful service for plugin developers is GitHub Pages which can be used to host site for your plugin, including any documentation and code examples. GitHub is essentially a complete service for plugin developers.

And believe it or not, GitHub recently announced a client-side editor in Edit like an Ace. Even if you’re a less technically-minded client-side developer, you can now edit code through GitHub without even learning how git works. Although, GitHub has solid documentation, I’d still recommend learning the basics.

If a web-based editor appeals to you, then why not give Cloud9 IDE a try? It can integrate with GitHub, and allows free public project creation which is perfect for open source projects. Old fashioned Vim/Emacs die-hard like me might be suspicious of services like Cloud9, but if it works for you then use it!

There are GitHub alternatives, including Bitbucket and Google Code.

Issue Tracking

Allow people to get help when things go wrong. Writing client-side code is an art, and it’s likely that no matter how well-tested your plugin is, it’ll have a few bugs.

If you’re looking for issue tracking outside of what the version control services offer, then Lighthouse is a good choice because it allows free open source issue tracking. I’ve also noticed open source developers using Pivotal Tracker.

Promotion

There are curated jQuery plugin lists out there. The jQuery Handpicked Plugin Repository is one such list, which features a plugin demo browser. Writing to the authors of such lists, or sites like DailyJS, is a perfectly acceptable way to promote your plugin. In fact, if you don’t send me your plugins, it’s unlikely that I’ll ever find them!

Writers like myself will look at several indicators to determine if our readership will be interested in a given plugin, and if it’s a new plugin that’s poorly packaged then I’m unlikely to cover it. I also quickly download plugins and try them out, but a demo page is preferred because it saves me the effort of downloading a plugin and creating a suitable mock HTML page to run it.

Another useful way to host demos is to use jsFiddle. It’s a convenient way of bundling and running HTML, JavaScript, and CSS. A project README file that links to a demo on jsFiddle is perfectly acceptable.

It’s also possible that you could write a tutorial for your plugin and syndicate it. Lots of blogs and dedicated tutorial sites are looking for content like this. Editors generally prefer it if the content is unique, so write to your favourite blogs with a tutorial pitch before committing to writing anything. If you don’t mind putting the effort in and not getting published, it only takes a few minutes to set up a free blog and post your own tutorials and news.

Open source projects were traditionally promoted through receptive mailing lists. This is still common in some circles — I often look for [announce] posts on popular JavaScript mailing lists. The equivalent for jQuery is the Using jQuery Forum, according to the jQuery site:

Using jQuery: […] if you’ve built a site that uses jQuery, or would like to announce a new plugin, this is the place to do it.

Discovery

Beyond curated plugin lists and blogs, a good way to discover new plugins is social bookmarking sites. The pinboard.in/t:jquery link is the first autocomplete item in my browser address bar when I type ‘pi’. The reason for this is Pinboard has a good signal to noise ratio.

Example: Chosen

Chosen (GitHub: harvesthq / chosen, License: MIT) is a solid example of a beautifully presented and well-marketed jQuery plugin. It hits everything on my checklist:

  • Includes a README with useful information about jQuery version support and how to contribute
  • Tracks issues through GitHub
  • Has a great site on GitHub Pages with full demos, example code, FAQs, credits
  • Includes a license
  • Includes the build scripts the authors use to generate releases

I think the only thing that’s missing here is unit tests, unless they’re hidden somewhere in the source that I didn’t expect.

Echo JS, brid.js, ql.io

05 Dec 2011 | By Alex Young | Comments | Tags modules libraries social parsing

Echo JS

Echo JS (GitHub: antirez / lamernews) is a social news site that focuses on JavaScript. I’ve already found some interesting libraries through it, and I basically read about JavaScript every day so it gets my vote.

It’s built with “Lamer News” which is a Ruby-based Hacker News implementation. Can’t someone write them a slick real-time Node version with the cutting edge server-side and client-side libraries that we all know and love?

If you prefer Twitter for consuming news, there’s also @echojs. It’s currently getting around five posts a day which seems promising. Judging by the Twitter account, Echo JS was launched on December 1st, so hopefully posts will increase in regularity rather than petering out.

brid.js

I’m not sure what to make of brid.js (License: MIT, Demo: brid.js demo). It’s a small Lisp implementation that uses JSON instead of S-expressions. On one hand, it’s a fun and interesting project, and on the other it seems like affront against nature.

This is a Hello World example in brid.js:

["println", ["quote", "hello world"]]

Adding numbers looks closer to Lisp:

["+", 1, 1]
// Lisp: (+ 1 1)

Lisp’s parenthesized syntax is analogous to the JSON representation, but the quoting required to create valid JSON is a little bit hard to get the hang of.

If you’re interested in parsing in JavaScript, then you might enjoy looking at the source. We’ve covered Jison on DailyJS before, but context-free grammar isn’t the only way to build parsers, and hand-coded parsers like this are just as useful to learn from.

ql.io

ql.io (GitHub: ql-io / ql.io, License: Apache 2.0) from eBay is a “declarative, evented, data-retrieval and aggregation gateway for HTTP APIs”. It has a DSL inspired by JSON and SQL for declaring API calls and their interdependencies.

The code itself is built on Node, and is comprised of several packages:

  • ql.io-engine: This is the core ql.io engine
  • ql.io-ecv: Checks servers are responding with JSON
  • ql.io-mon: Statistics on the ql.io runtime
  • ql.io-uri-template: A small URI template processor
  • ql.io-app: An app template for building sites using ql.io
  • ql.io-compiler: A PEG.js-based parser with a compiler (ql.peg contains the main grammar)
  • ql.io-console: Provides the HTTP interface for ql.io
  • ql.io-mutable-uri A utility for manipulating URIs

This is a diverse and complicated project, employing some interesting techniques. I’m particularly intrigued by the use of PEGs (parsing expression grammar) over context-free grammar, as Jison seems like the easiest way to write parsers in JavaScript at the moment. PEG.js (GitHub: dmajda / pegjs, npm: pegjs) is available for Node and browsers.

Announcing ql.io at the eBay Tech Blog has more details on the design and architecture of the project.