New Streaming API for Node, Components Tutorial, Holler, GruntStart

21 Dec 2012 | By Alex Young | Comments | Tags components grunt node streams

New Streaming API for Node

In New Streaming API for Node, Isaac Schlueter outlines the issues found in Node’s current streaming API, and presents the new Node 0.10 API that addresses these problems. Although the documentation makes it clear that the stream API is “unstable”, it’s notable for two reasons: it’s built on EventEmitter, and many of Node’s core modules have stream interfaces. Streams are an integral part of Node, and it’s becoming clear that many problems are best solved with streams.

The 0.10 version of streams will cause backward compatibility issues, but this has been considered and mitigated:

For backwards compatibility with older Node programs, Readable streams switch into “old mode” when a 'data' event handler is added, or when the pause() or resume() methods are called.

Isaac has included the new stream API’s documentation in the post, and if you’re working with Node at all it’s worth reading. If you’re struggling to understand the motivation behind the changes, it boils down to pause() not really pausing, and the potential for 'data' events to be dropped before they’re ready to be consumed. I liked Isaac’s practical example of this issue:

'data' events come right away (whether you’re ready or not). This makes it unreasonably difficult to do common tasks like load a user’s session before deciding how to handle their request.

So, read A New Streaming API for Node v0.10 carefully even if you’re not working with streams directly. I’m of the opinion that you should be thinking about streams when designing Node programs, and if Node’s core developers can get them right for version 0.10 it’ll be a huge win for the platform.

Components Tutorial

The date picker from the tutorial

TJ Holowaychuk has written up a detailed tutorial on Components, about building a date picker. I’m a believer of the Components idea, and at the moment good tutorials are lacking in this area, so it’s good to see TJ writing up detailed examples like this.

There are also some screencasts available: Creating components, and Web Components - Introduction.

Holler.js

Holler.js (GitHub: bitpshr / holler, License: WTFPL, npm: holler) by Paul Bouchon is a real-time notification service. It’s distributed as a Node module with client-side code that builds on alertifyjs.

The real-time communication is handled by Faye. Paul has made a brief demonstration video to show how it works.

GruntStart

GruntStart

GruntStart (GitHub: tsvensen / GruntStart, License: MIT, GPL) by Tim Svensen is a Grunt task for building optimised websites using HTML5 Boiler Plate, jQuery, Modernizr and Respond.js. It includes a task that watches for file changes and automatically generates optimised assets.

The client-side scripts are all included, so all you need to do to get started is download GruntStart from GitHub.

Backbone.js Tutorial: Backbone.sync

20 Dec 2012 | By Alex Young | Comments | Tags backbone.js mvc node backgoog

Preparation

Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit c1d5a2e7cc
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone git@github.com:alexyoung/dailyjs-backbone-tutorial.git
cd dailyjs-backbone-tutorial
git reset --hard c1d5a2e7cc

Google’s Tasks API

To recap: the point of this tutorial series is to build a Backbone.js single page application that uses client-side JavaScript to communicate with Google’s authentication and to-do list APIs. Got that? Good!

Google provides access to our to-do lists through two APIs:

When loading Google’s JavaScript, the browser is bestowed with a global called gapi that provides access to various objects and methods. In the last part, I quietly included a call to gapi.client.load that loads the tasks API:

gapi.client.load('tasks', 'v1', function() { /* Loaded */ });

This can be found in app/js/gapi.js. The remaining challenge before building the interface is to implement a new Backbone.sync method that uses gapi to communicate with the Tasks and Tasklists APIs.

Backbone.sync Structure

I’ve already talked about the overall structure of Backbone.sync in part 2. The pattern I’ll use in these tutorials is fairly generic, so you could use the same approach to communicate with something other than Google’s APIs.

The sync method itself takes three arguments, the first of which is the method (create, update, delete, and read). We need to map method to something Google’s API can understand.

This is what we’ve got so far:

Backbone.sync = function(method, model, options) {
  options || (options = {});

  switch (method) {
    case 'create':
    break;

    case 'update':
    break;

    case 'delete':
    break;

    case 'read':
    break;
  }
};

Google’s Tasks API methods map to the Backbone method argument like this:

Google Tasks APIBackbone.sync MethodDescription
insertcreateCreate a new task.
updateupdateUpdate an existing task.
deletedeleteDelete a task.
listreadGet a list of tasks.

Even though Google’s API doesn’t look like the Rails 3-based RESTful API that Backbone.js is designed for out of the box, it’s still very close.

Making Requests with gapi

The gapi object makes requests using this pattern:

  • Call one of the gapi.client.tasks methods with the request content to get a request object
  • Call request.execute with a callback to send the request
  • The callback receives a response object, much like a standard Ajax request

Here’s what this looks like in reality:

var requestContent = {}
  , request
  , gapiResource;

gapiResource = 'tasks';
requestContent['tasklist'] = tasklistId; // Assuming we have one
requestContent['resource'] = model.toJSON();

// 'insert' is for creating new tasks
request = gapi.client.tasks[gapiResource].insert(requestContent);

// Send the request to the API
request.execute(function(res) {
  // Handle the response
});

Looking at this, it’s clear that we need two models: Task and TaskList. There also need to be two corresponding collections: Tasks and TaskLists.

Backbone models and collections have URLs – these are used for making API requests. Similarly, Google’s APIs have URLs: tasks and tasklists, so by using the model URL Backbone.sync can determine which API resource is required for a given request.

Models

Create a new directory called app/js/models and add task.js:

define(function() {
  var Task = Backbone.Model.extend({
    url: 'tasks'
  });

  return Task;
});

You’ll also want to create a app/js/models/tasklist.js:

define(function() {
  var TaskList = Backbone.Model.extend({
    url: 'tasklists'
  });

  return TaskList;
});

Collections

Create another new directory called app/js/collections and add tasklists.js:

define(['models/tasklist'], function(TaskList) {
  var TaskLists = Backbone.Collection.extend({
    model: TaskList
  , url: 'tasklists'
  });

  return TaskLists;
});

We’re going to use the TaskList collection later on to load your task lists.

Making API Requests

Open up app/js/gapi.js and add a new line after line 36:

app.views.auth.$el.hide();
$('#signed-in-container').show();
self.trigger('ready'); // This one

This 'ready' event will be used to signify that authentication was successful, and the Tasks API is ready for use. Next, add the following two lines to Backbone.sync, inside the read switch case:

case 'read':
  var request = gapi.client.tasks[model.url].list(options.data);
  Backbone.gapiRequest(request, method, model, options);
break;

This code creates a request, and then Backbone.gapiRequest will execute it and delegate the response.

Here’s the most basic Backbone.gapiRequest implementation:

Backbone.gapiRequest = function(request, method, model, options) {
  var result;
  request.execute(function(res) {
    if (res.error) {
      if (options.error) options.error(res);
    } else if (options.success) {
      result = res.items;
      options.success(result, true, request);
    }
  });
};

All it does is run request.execute, which is provided by Google, and then maps the result to be compatible with Backbone’s API by running the success and error callbacks.

Just so you can see something is really happening, open app/js/app.js and make it load the TaskLists collection by changing the define invocation at the top:

define([
  'gapi'
, 'views/app'
, 'views/auth'
, 'collections/tasklists'
],

function(ApiManager, AppView, AuthView, TaskLists) {

Now add this to the connectGapi method:

this.apiManager.on('ready', function() {
  self.collections.lists.fetch({ data: { userId: '@me' }, success: function(res) {
    _.each(res.models, function(model) {
      console.log(model.get('title'));
    });
  }});
});

That code uses Underscore’s each method to iterate over each “model” returned by Backbone.sync, which is called by the TaskList collection.

Run the server with npm start, and visit http://localhost:8080. If you run it in a browser that supports console, then you should see your task lists printed out.

My task lists.

If you’ve got this working then you’re not far off building a real world Backbone.js app that communicates with Google’s APIs. The same concepts can be applied to other Google JavaScript APIs.

Summary

The full source for this tutorial can be found in alexyoung / dailyjs-backbone-tutorial, commit fcd653ec6.

Substack's Musical Node Modules

19 Dec 2012 | By Alex Young | Comments | Tags node modules audio music
Prerequisites: Install gnuplot and SoX to follow along.

James Halliday, otherwise known as “substack”, has been making what he calls computer generated beepstep using two new modules: baudio (npm: baudio, License: MIT) and plucky (npm: plucky, License: MIT).

The baudio module returns a readable stream that generates raw audio data. It requires SoX to play or record audio, which should be installable from your package manager (Debian has it, and so does Homebrew).

The callback passed to the baudio function receives two arguments: t and i – the time in seconds, and a counter. The callback will be run using process.nextTick to generate a stream of audio data. The audio data will be passed to SoX for playback or recording using child_process.spawn.

The audio data itself is where things get interesting. The baudio stream is sent directly into SoX through SoX’s “pipeline” mode, in which audio data is read from standard input, using the “s16” format – it’s streams all the way down! Internally, baudio converts the float values returned from the callback into integers, which are written to a buffer using Node’s buf.writeInt16LE method.

Your callback should generate floating point values between -1.0 and 1.0. By default, baudio is set to use a frequency of 44 kHz – that’s 44,000 values a second. This is close to CD quality (44.1 kHz).

Unless you’re well-versed in audio programming, generating sounds with baudio is going to be hard work. To help you understand what’s going on, I’ve written a small example that uses gnuplot to visualise the output.

First, install baudio:

npm install baudio

And then create a file called baudio-simple.js:

var baudio = require('baudio')
  , out = process.stdout
  , note = 1.0
  ;

var b = baudio(function(t, i) {
  var value = Math.sin(Math.PI * t * 261.626);
  out.write(value.toString() + '\n');
  return value;
});

setTimeout(function() {
  b.end();
}, 1000);

b.play();

This file can be run with node baudio-simple.js > audio.dat (the redirection is important for generating the graphs), and if you’ve got SoX installed you’ll get a sound.

Now you’re going to write a bit of gnuplot. Create a file called baudio-plot with this script:

#!/usr/bin/env gnuplot

set terminal png size 530,420
set output "baudio.png"
plot "audio.dat" using 0:1 with lines

Now make it executable, and run it:

chmod 700 baudio-plot
./baudio-plot

It should generate a file called baudio.png with a second’s worth of audio data plotted.

A graph of baudio's output using one second of audio.

If you’re looking at Math.sin in the example code and wondering why there isn’t a beautiful sweeping sine wave, then the reason is simple: there’s too much data. Let’s try ending the output earlier:

var baudio = require('baudio')
  , out = process.stdout
  , note = 1.0
  ;

var b = baudio(function(t, i) {
  if (i > 100) b.end();
  var value = Math.sin(Math.PI * t * 261.626);
  out.write(value.toString() + '\n');
  return value;
});

b.play();

Now your graph should look like a wave, but you won’t hear much sound:

A much shorter sample shows the output really is a sine wave.

Wave Period and Amplitude

The simple example I’ve used above is focused on controlling the “wave period”, or the pitch of the output. To demonstrate this, try changing 261.626 to 60.0:

Some sub-bass.

Now the output is a lower pitch, and the graph makes this clear because you can see there are less cycles in the same amount of time. So we’ve mastered pitch, but what about volume?

It’s actually easy once you know what the wave equation is doing. The generalised equation is A sin(t - K) + b (from Amplitude on Wikipedia). In programmer-speak, this equation can be written as A * sin(t - K) + b, where A is the “peak amplitude of the wave”, t is time (which we already know baudio gives us), and K and b are offsets for the wave (which I’m not going to talk about here).

That gives rise to the following example that allows volume to be controlled by adding a variable, vol, for A:

var baudio = require('baudio')
  , out = process.stdout
  , note = 1.0
  , vol = 0.1
  ;

var b = baudio(function(t, i) {
  if (i > 100) b.end();
  var value = vol * Math.sin(Math.PI * t * 261.626);
  out.write(value.toString() + '\n');
  return value;
});

b.play();

The graph is now appropriately smaller:

A quieter audio sample, plotted with set yrange [-1.0:1.0] to correct the axis.

More Fun Stuff

If you’ve managed to get gnuplot and SoX installed and played around with these examples, then there’s more! First, try taking a look at plucky, which can be used to make “arrangements” of callbacks that generate different channels of audio. Also, James wrote beepstep.js which is a much more involved example than anything I’ve talked about here.

And, James tweets about this stuff, so follow @substack if you’re into streams and audio hacking.

jQuery Roundup: 1.9, UI 1.10, Maskew

18 Dec 2012 | By Alex Young | Comments | Tags jquery plugins mobile effects
Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

jQuery 1.9 Beta 1

jQuery 1.9 Beta 1 has been released, and this version removes previously deprecated features so you’ll want to pay attention. Fortunately, Dave Methvin has been working on jquery / jquery-migrate, which is a plugin for figuring out which deprecated features are being used in your projects. The development version shows console warning messages, and these are stored in jQuery.migrateWarnings for browsers that don’t support console. There’s a full list of the warnings in jQuery Migrate Plugin - Warning Messages.

The focus of 1.9 has been API cleanup, and there’s also a jQuery 1.9 Upgrade Guide, where these API changes have been documented.

jQuery UI 1.10 Beta

jQuery UI 1.10 Beta is out. This version features a new API for the Dialog widget, and a redesigned Progressbar API.

There are major API changes in 1.10, so if you haven’t updated to 1.9 yet then you might want to read through the 1.9 upgrade guide.

Maskew

Get skewed.

Maskew (GitHub: dmotz / maskew, License: MIT) by Dan Motzenbecker is a small plugin that can skew elements along a specified angle. It has an optional jQuery plugin, but also has a simple prototype class API. It has support for touchscreen events, specified by passing { touch: true } as an argument.

Dan has included the build script and some tests.

JavaScript Developer Survey 2012

17 Dec 2012 | By Alex Young | Comments | Tags community surveys
Note: This survey has now closed. The results are here.

Welcome to the JavaScript Developer Survey 2012. To complete the survey, visit JavaScript Developer Survey 2012 or fill out the embedded form in this post.

I’ve includes some question suggestions from readers Todd Bashor, Tyler Larson, and Adam Alexander.

If you can share this survey on social networking sites, or your favourite news site, then I’d appreciate it. As always this survey is open in the sense that responses are public, so you can use the data to develop related open source or commercial projects.

This survey has closed.

Continuum, Analytics.js, MaskJS

14 Dec 2012 | By Alex Young | Comments | Tags libraries templates browser

Continuum

Continuum (GitHub: Benvie / continuum, License: MIT, npm: continuum) by Brandon Benvie is an ES6 virtual machine written in ES3. By using Esprima to parse a given source file, it generates bytecode which is then executed within an ES6 runtime environment.

If this doesn’t sound enough like science fiction to you, then think about it like this: this project will run your ECMAScript 6 code inside its own space-time continuum that allows legacy browsers to support ECMAScript 6 features.

Brandon has already implemented a slew of ES6 features, but hasn’t yet added support for array comprehensions, tail call optimisation, or the binary data API. What you do get, however, is destructuring assignment, spread in arguments and array initializers, “rest” parameters, classes and super, arrow functions, and even Map, Set, and WeakMap.

If you want to get an idea of how this project works, take a look at the engine/assembler.js and engine/runtime.js files to see the opcodes and how the stack is used.

Analytics.js

Analytics.js (GitHub: segmentio / analytics.js, License: MIT) from Segment.io is a simplified API for supporting various web analytics services, including Google Analtyics, KISSmetrics, Mixpanel, and Chartbeat.

It provides a single API for defining events that should be tracked, allowing you to focus on getting the data out of activities on your site without becoming bogged down in a given service’s implementation peculiarities.

Segment.io also made Socrates.io, which seems to be very popular. I recommend keeping an eye on the GitHub Segment.io account for more cool projects.

MaskJS

MaskJS (GitHub: tenbits / MaskJS, License: MIT) by Alexander Kit is a template engine for Node and browsers. The author has built it with performance and mobile devices in mind, and has a jsperf benchmark that compares it against other template engines.

It supports custom tags, which are designed to help encapsulate markup into reusable components, and the author has created some for two-way data binding. The best way to get a feel for the markup style and API is to check out the MaskJS examples.

Backbone.js Tutorial: Authenticating with OAuth2

13 Dec 2012 | By Alex Young | Comments | Tags backbone.js mvc node backgoog

In Part 2: Google’s APIs, I laid the groundwork for talking to Google’s JavaScript APIs. Now you’re in a position to start talking to the todos API, but first a user account is required.

Preparation

Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit 9d09a66b1f
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone git@github.com:alexyoung/dailyjs-backbone-tutorial.git
cd dailyjs-backbone-tutorial
git reset --hard 9d09a66b1f

Google’s OAuth 2.0 Client-side API

Open app/js/gapi.js and take a look at lines 11 to 25. There’s a method, provided by Google, called gapi.auth.authorize. This uses the “Client ID” and some scopes to attempt to authenticate. I’ve already set the scopes in app/js/config.js:

config.scopes = 'https://www.googleapis.com/auth/tasks https://www.googleapis.com/auth/userinfo.profile';

This tells the authentication system that our application would like to access the user’s profile and Gmail tasks. Everything is almost ready to work, but two things are missing: an implementation for handleAuthResult and an interface.

Templates

RequireJS can load templates by using the text plugin. Download text.js from GitHub and save it to app/js/lib/text.js.

This is my preferred technique for handling templates. Although this application could easily fit into a monolithic index.html file, breaking up projects into smaller templates is more manageable in the long run, so it’s a good idea to get used to doing this.

Now open app/js/main.js and add the text plugin to the paths property of the RequireJS configuration:

paths: {
  text: 'lib/text'
},

Finally, add this to app/js/config.js:

_.templateSettings = {
  interpolate: /\{\{(.+?)\}\}/g
};

This tells Underscore’s templating system to use double curly braces for inserting values, otherwise known as interpolation.

The app needs some directories to store template-related things:

  • app/js/views – This is for Backbone.js views
  • app/js/templates – Plain HTML templates, to be loaded by the views
  • app/css

The app/index.html file needs to load the CSS, so add a link tag to it:

<link rel="stylesheet" href="css/app.css">

And create a suitable CSS file in app/css/app.css:

#sign-in-container, #signed-in-container { display: none }

The application will start up by hiding both the sign-in button and the main content. The oauth API will be queried for existing user credentials – if the user has already logged in recently their details will be stored in a cookie, so the views need to be configured appropriately.

Templates

The templates aren’t particularly remarkable at this stage, just dump this into app/js/templates/app.html:

<div class="row-fluid">
  <div class="span2 main-left-col" id="lists-panel">
    <h1>bTask</h1>
    <div class="left-nav"></div>
  </div>
  <div class="main-right-col">
    <small class="pull-right" id="profile-container"></small>
    <div>
      <div id="sign-in-container"></div>
      <div id="signed-in-container">
        <p>You're signed in!</p>
      </div>
    </div>
  </div>
</div>

This template shows some things that we won’t be using yet, just ignore it for now and focus on sign-in-container and signed-in-container.

Next, paste the following into app/js/templates/auth.html:

<a href="#" id="authorize-button" class="btn btn-primary">Sign In with Google</a>

The auth.html template will be inserted into sign-in-container. It’s very simple at the moment, I only really included it for an excuse to create extra Backbone.js views so you can see how it’s done.

Backbone Views

These templates need corresponding Backbone.js views to manage them. This part demonstrates how to load templates with RequireJS and render them. Create a file called app/js/views/app.js:

define([
  'text!templates/app.html'
],

function(template) {
  var AppView = Backbone.View.extend({
    id: 'main',
    tagName: 'div',
    className: 'container-fluid',
    el: 'body',
    template: _.template(template),

    events: {
    },

    initialize: function() {
    },

    render: function() {
      this.$el.html(this.template());
      return this;
    }
  });

  return AppView;
});

The AppView class doesn’t have any events yet, but it does bind to an element, body, and load a template: define(['text!templates/app.html']. The text! directive is provided by the RequireJS “text” plugin we added earlier. The template itself is just a string that contains the corresponding HTML. It’s rendered by binding it to the Backbone.View, and then calling jQuery’s html() method which replaces the HTML within an element: this.$el.html(this.template());.

The AuthView is a bit different. Create a file called app/js/views/auth.js:

define(['text!templates/auth.html'], function(template) {
  var AuthView = Backbone.View.extend({
    el: '#sign-in-container',
    template: _.template(template),

    events: {
      'click #authorize-button': 'auth'
    },

    initialize: function(app) {
      this.app = app;
    },

    render: function() {
      this.$el.html(this.template());
      return this;
    },

    auth: function() {
      this.app.apiManager.checkAuth();
      return false;
    }
  });

  return AuthView;
});

The app object is passed to initialize when AuthView is instantiated (with new AuthView(this) later on). The reason I’ve done this is to allow the view to call the required authentication code from ApiManager. This could also be handled with events, or many other ways – I just wanted to show that we can initialise views with values just like any other class.

App Core

The views need to be instantiated and rendered. Open app/js/app.js and change it to load the views using RequireJS:

define([
  'gapi'
, 'views/app'
, 'views/auth'
],

function(ApiManager, AppView, AuthView) {
  var App = function() {
    this.views.app = new AppView();
    this.views.app.render();

    this.views.auth = new AuthView(this);
    this.views.auth.render();

    this.connectGapi();
  }

The rest of the file can remain the same. Notice that the order these views are rendered is important – AuthView won’t work unless it has some of AppView’s tags available. A better way of modeling this might be to move AuthView inside AppView so the dependency is reflected. You can try this yourself if you want to experiment.

Authentication Implementation

The app/js/gapi.js file still doesn’t have the handleAuthResult function, so nothing will work yet. Here’s the code to handle authentication:

function handleAuthResult(authResult) {
  var authTimeout;

  if (authResult && !authResult.error) {
    // Schedule a check when the authentication token expires
    if (authResult.expires_in) {
      authTimeout = (authResult.expires_in - 5 * 60) * 1000;
      setTimeout(checkAuth, authTimeout);
    }

    app.views.auth.$el.hide();
    $('#signed-in-container').show();
  } else {
    if (authResult && authResult.error) {
      // TODO: Show error
      console.error('Unable to sign in:', authResult.error);
    }

    app.views.auth.$el.show();
  }
}

this.checkAuth = function() {
  gapi.auth.authorize({ client_id: config.clientId, scope: config.scopes, immediate: false }, handleAuthResult);
};

The trick to a smooth sign-in flow is to determine when the user is already signed in. If so, then authentication should be handled transparently, otherwise the user should be prompted.

The handleAuthResult function is called by gapi.auth.authorize from the checkAuth function, which isn’t displayed here (it’s before handleAuthResult in the source file if you want to check it). The this.checkAuth method is different – this is a public method that calls gapi.auth.authorize with immediate set to false, while the other invocation calls it with true.

The immediate option is important because it determines whether a popup will be displayed or not. I’ve used it to check if the user is already signed in, otherwise it’s called again with immediate: false and will display a suitable popup so the user can see what permissions the app wants to use:

Authentication process

I designed it this way based on the Google APIs Client Library for JavaScript documentation:

“The standard authorize() method always shows a popup, which can be a little jarring if you are just trying to refresh an OAuth 2.0 token. Google’s OAuth 2.0 implementation also supports “immediate” mode, which refreshes a token without a popup. To use immediate mode, just add “immediate: true” to the login config as in the example above.”

I’ve also changed the ApiManager class to store a reference to App:

// Near the top of gapi.js
var app;

function ApiManager(_app) {
  app = _app;
  this.loadGapi();
}

Summary

In this tutorial you’ve seen how to use Google’s APIs to sign into an app you’ve previously registered with the Google API Console (documented in part 2). It might seem like a lot of work to get RequireJS, Backbone.js, and Google OAuth working together, but think about what this has achieved: 100% client-side scripting that can authenticate with existing Google accounts.

If I’ve missed anything here, you can check out the full source from commit c1d5a2e7c.

Node Roundup: thin-orm, node-tar.gz, connect-bruteforce

12 Dec 2012 | By Alex Young | Comments | Tags node modules middleware express compression databases
You can send in your Node projects for review through our contact form or @dailyjs.

thin-orm

thin-orm (License: MIT, npm: thin-orm) by Steve Hurlbut is a lightweight ORM module for SQL databases with a MongoDB-inspired API:

var orm = require('thin-orm');

orm.table('users')
   .columns('id', 'login', 'firstName', 'lastName', 'createdAt');

It’s designed to be used with existing libraries, like pg and sqlite3, so you’ll need one of those modules installed to use it.

thin-orm currently supports the following features:

  • Filtering
  • Sorting
  • Pagination
  • Joins
  • Optional camelCase property-to-field mapping
  • SQL injection protection

Steve has included Nodeunit tests that cover the basic functionality, and some integration tests for PostgreSQL and SQLite.

node-tar.gz

node-tar.gz (License: MIT, npm: tar.gz) by Alan Hoffmeister is a tar helper module and command-line utility, built with Node’s zlib module, tar, and commander.

The module can be used to easily tar and compress a folder, and it will install a targz script that supports the zxvf flags. There are also Vows tests.

connect-bruteforce

connect-bruteforce (License: GPLv2, npm: connect-bruteforce) by Pedro Narciso García Revington provides middleware that can help prevent bruteforce attacks. It will add a small delay to requests when an attack is detected.

The author has written a useful example that requires captcha validation after a successive number of validation failures: express-recaptcha.

For a simpler example, see express-hello-world.

The project includes Mocha tests.

jQuery Roundup: SocialCount, Literally Canvas, Socrates

11 Dec 2012 | By Alex Young | Comments | Tags jquery plugins social markdown apps images
Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

SocialCount

Social buttons can slow down page loading times due to a variety of reasons. One solution is to lazy load them, and this is exactly what the SocialCount (GitHub: filamentgroup / SocialCount, License: MIT) from Filament Group does. It can show “like” style buttons, counts, and lazy load the social network’s original buttons.

It’s designed using progressive enhancement techniques, and is tested against IE 7+, as well as the other major browsers, and various touchscreen devices. It also includes QUnit tests.

Literally Canvas

Drawing with a trackpad is tricky business.

Literally Canvas (GitHub: literallycanvas / literallycanvas, License: BSD) by Stephen Johnson and Cameron Paul is a drawing widget built with jQuery and Underscore.js. It has some basic drawing tools and can upload images to imgur.

The plugin accepts an options object that can be used to enable or disable certain features:

$('.literally').literallycanvas({
  backgroundColor: 'rgb(255, 0, 0)', // default rgb(230, 230, 230)
  keyboardShortcuts: false,          // default true
  sizeToContainer: false,            // default true
  toolClasses: [LC.Pencil]           // see coffee/tools.coffee
});

Socrates

Real-time Markdown editing with Socrates.

Socrates (GitHub: segmentio / socrates, License: MIT) by Ilya Volodarsky and Ian Storm Taylor is a Markdown editor and previewer. It’s built with jQuery, Backbone.js, and a client-side Markdown parser by Christopher Jeffrey.

The data is stored in Firebase, so you’ll need an account with Firebase to install your own instance of Socrates.

Extender, Gridy.js, grunt-reduce

10 Dec 2012 | By Alex Young | Comments | Tags libraries build grunt tv browser

Extender

Extender (GitHub: doug-martin / extender, npm: extender, License: MIT) by Doug Martin is a library for making chainable APIs. It works as a Node module or with RequireJS.

Extender has a define method that accepts a function and an object with methods that will form the API:

function isString(obj) {
  return !isUndefinedOrNull(obj) && (typeof obj === "string" || obj instanceof String);
}

var myExtender =
  .define(isString, {
    multiply: function(str, times) {
      var ret = str, i;
      for (i = 1; i < times; i++) {
        ret += str;
      }
      return ret;
    },
    toArray: function(str, delim) {
      delim = delim || '';
      return str.split(delim);
    }
  });

myExtender('hello').multiply(2).value(); // hellohello

The author has included tests and lots of examples. Although making chainable APIs is pretty straightforward, Extender might be a more explicit and testable way to do it.

Gridy.js

Gridy.js

In the UK only one of my favourite shows is on Netflix. The situation might be better in the US with Hulu, but if you’re a cult TV fan outside of the US finding content can be challenging. Even with a dearth of legitimate content sources, I’ll always prefer hacking my TV to using locked down devices – I had loads of fun this weekend with a Raspberry Pi and open source media projects.

One thing that’s often lacking is cool web interfaces for media software. Igor Alpert sent in Gridy.js (GitHub: ialpert / gridy.js), which is a library designed for building media browser interfaces. It includes tools for carousels, grids, and sliders.

Igor said he’s tested it with the Samsung SDK, Opera TV, and Google TV for the LG and Vizio platforms.

grunt-reduce

grunt-reduce by Peter Müller is a Grunt task for AssetGraph and other related projects from One.com. AssetGraph is a Node-based module for optimising web pages. By adding grunt-reduce to your project, you can bundle and minify assets, rename assets to MD5-sums of their content, optimise images, and even generate automatic CSS sprites.

Although AssetGraph doesn’t currently work with AngularJS, Peter notes that this is being addressed: #84: Support AngularJS templates.

JavaScript Survey 2012, Gitgraph, ES6 Proxies

07 Dec 2012 | By Alex Young | Comments | Tags ES6 git graphs Canvas survey community

JavaScript Survey 2012: RFC

I’m currently researching the next JavaScript Developer Survey. I’d like feedback on questions. If there’s anything you’d strongly like to see in the survey, please contact me and I’ll see if I can incorporate it.

Previous surveys can be found here:

In general, the surveys try to determine:

  • How many readers are client-side or server-side developers
  • Whether or not readers write tests
  • Other languages used (C#, Java, Objective-C, PHP, Ruby, Python, etc.)

It’s not necessarily used to design content for DailyJS – the results are shared with the community benefit everyone.

Gitgraph

Gitgraph (GitHub: bitpshr / Gitgraph, License: WTFPL) by Paul Bouchon is a Canvas-based GitHub participation graph library. It’s based around a constructor function that accepts arguments for things like GitHub username, width, height, and colours:

var graph = new Gitgraph({ 
    user        : 'nex3',                // any github username
    repo        : 'sass',                // name of repo
    domNode     : document.body,         // (optional) DOM node to attach to 
    width       : 800,                   // (optional) graph width
    height      : 300,                   // (optional) graph height
    allColor    : "rgb(202, 202, 202)",  // (optional) color of user's participation
    userColor   : "rgb(51, 102, 153)",   // (optional) color of total participation
    background  : "white",               // (optional) background styles
    showName    : true                   // (optional) show or hide name of user / repo
});

The author wrote some background on it in GitHub Graphs Fo’ Errbody, because he had to wrap missing API functionality with a proxy.

Multiple Inheritance in ES6 with Proxies

Multiple Inheritance in ES6 with Proxies is an introduction to ES6 proxies by Jussi Kalliokoski. The author’s example uses EventEmitter, which I find useful because multiple inheritance with EventEmitter is something I’ve seen typically implemented using a for loop to copy properties.

The Proxy solution isn’t far off that approach and requires more code, but it’s worth reading if you’re struggling to understand proxies.

Backbone.js Tutorial: Google's APIs and RequireJS

06 Dec 2012 | By Alex Young | Comments | Tags backbone.js mvc node backgoog

In Part 1: Build Environment, I explained how to set up a simple Node server to host your Backbone.js app and test suite. Something that confused people was the way I used relative paths, which meant the tests could fail if you didn’t visit /test/ (/test won’t work). There was a reason for this: I developed the original version to run on Dropbox, so I wanted to use relative paths. It’s probably safer to use absolute paths, so I should have made this clearer.

In this part you’ll learn the following:

  • How Backbone.sync works
  • How to load Backbone.js and Underscore.js with RequireJS
  • How to get started with Google’s APIs

The Backbone.sync Method

Network access in Backbone.js is nicely abstracted through a single method which has the following signature:

Backbone.sync = function(method, model, options) {
};

The method argument contains a string that can be one of the following values:

  • create
  • update
  • delete
  • read

Internally, Backbone.js maps these method names to HTTP verbs:

var methodMap = {
  'create': 'POST',
  'update': 'PUT',
  'delete': 'DELETE',
  'read':   'GET'
};

If you’re familiar with that particular flavour of RESTful API then this should all look familiar.

The second argument, model, is a Backbone.Model or Backbone.Collection – collections are used when reading multiple values.

The final argument, options, is an object that contains success and error callbacks. It’s ultimately handed off to the jQuery Ajax API.

To work with Google’s APIs we need to write our own Backbone.sync method. In general terms our implementation should be structured like this:

Backbone.sync = function(method, model, options) {
  switch (method) {
    case 'create':
      googleAPI.create(model, options);
    break;

    case 'update':
      googleAPI.update(model, options);
    break;

    case 'delete':
      googleAPI.destroy(model, options);
    break;

    case 'read':
      // The model value is a collection in this case
      googleAPI.list(model, options);
    break;

    default:
      // Something probably went wrong
      console.error('Unknown method:', method);
    break;
  }
};

The googleAPI object is fictitious, but this is basically how Backbone.sync is usually extended – a lightweight wrapper that maps the method names and models to another API. Using a lightweight wrapper means the underlying target API can be easily used outside of a Backbone.js.

In our case, Google actually provides a JavaScript API – there will be a gapi.client object available once the Google APIs have been loaded.

Setting Up A Google API Account

The main page for Google’s developer documentation is at developers.google.com, but what we’re interested in is the Google Tasks API which can be found under Application APIs.

Google’s Application APIs are designed to work well with both server-side scripting and client-side JavaScript. To work with the Google Tasks API you’ll need three things:

  1. A Google account (an existing one is fine)
  2. Google API Console access (you may have already enabled it if you work with Google’s services)
  3. An API key

To set up your account to work with the Google API Console, visit code.google.com/apis/console. Once you’ve enabled it, scroll down to Tasks API:

Google Tasks API switch

Then switch the button to on, and accept the terms (if you agree to them). Next, click API Access in the left-hand navigation bar, and look under Simple API Access for the API key. This “browser apps” key is what we need. Make a note of it for later.

OAuth 2.0 for Client-side Applications

Still in the API Access section of the console, click the button to create an OAuth 2.0 project. Enter “bTask” (or whatever you want) for the product name, then http://localhost:8080 for the URL. In the next dialog, make sure http:// is selected instead of https://, then enter localhost:8080 and click “Create client ID”.

You’ll now see a set of values under “Client ID for web applications”. The field that says “Client ID” is important – make a note of this one as well.

You should now have an API key and a Client ID. These will be used to load Google’s APIs and allow us to use an OAuth 2.0 service from within the browser – we won’t need to write our own server-side code to authenticate users.

Follow Along

If you want to check out the source from Part 1 so you can follow along, you can use Git to get the exact revision from last week:

git clone git@github.com:alexyoung/dailyjs-backbone-tutorial.git
cd dailyjs-backbone-tutorial
git reset --hard 2a8517e

Required Libraries

Before progressing, download the following libraries to app/js/lib/:

Open app/js/main.js and edit the shim property under requirejs.config to load Underscore.js and Backbone.js:

requirejs.config({
  baseUrl: 'js',

  paths: {
  },

  shim: {
    'lib/underscore-min': {
      exports: '_'
    },
    'lib/backbone-min': {
      deps: ['lib/underscore-min']
    , exports: 'Backbone'
    },
    'app': {
      deps: ['lib/underscore-min', 'lib/backbone-min']
    }
  }
});

require([
  'app'
],

function(App) {
  window.bTask = new App();
});

This looks weird, but remember we’re using RequireJS to load scripts as modules. RequireJS “shims” allow dependencies to be expressed for libraries that aren’t implemented using AMD.

Loading the Tasks API

The basic pattern for loading the Google Tasks API is:

  1. Load the Google API client library: https://apis.google.com/js/client.js
  2. Call gapi.client.load to load the “tasks” API
  3. Set the API key using gapi.client.setApiKey()

To implement this, you’ll need a place to put the necessary credentials. Create a file called app/js/config.js and add the API key and Client ID to it:

define([], function() {
  var config = {};
  config.apiKey = 'your API key';
  config.scopes = 'https://www.googleapis.com/auth/tasks https://www.googleapis.com/auth/userinfo.profile';
  config.clientId = 'your client ID';
  return config;
});

This file can be loaded by our custom Google Tasks API/Backbone.sync implementation.

Now create a new file called app/gapi.js:

define(['config'], function(config) {
  function ApiManager() {
    this.loadGapi();
  }

  _.extend(ApiManager.prototype, Backbone.Events);

  ApiManager.prototype.init = function() {
  };

  ApiManager.prototype.loadGapi = function() {
  };

  Backbone.sync = function(method, model, options) {
    options || (options = {});

    switch (method) {
      case 'create':
      break;

      case 'update':
      break;

      case 'delete':
      break;

      case 'read':
      break;
    }
  };

  return ApiManager;
});

This skeleton module shows the overall layout of our Google Tasks API loader and Backbone.sync implementation. The ApiManager is a standard constructor, and I’ve used Underscore.js to inherit from Backbone.Events. This code will be asynchronous, so event handling will be useful later.

The loadGapi method loads Google’s JavaScript using RequireJS. Once the gapi global object has been found, it will do the rest of the configuration by calling the init method:

ApiManager.prototype.loadGapi = function() {
  var self = this;

  // Don't load gapi if it's already present
  if (typeof gapi !== 'undefined') {
    return this.init();
  }

  require(['https://apis.google.com/js/client.js?onload=define'], function() {
    // Poll until gapi is ready
    function checkGAPI() {
      if (gapi && gapi.client) {
        self.init();
      } else {
        setTimeout(checkGAPI, 100);
      }
    }
    
    checkGAPI();
  });
});

All the init method needs to do is load the Tasks API with gapi.client.load:

ApiManager.prototype.init = function() {
  var self = this;

  gapi.client.load('tasks', 'v1', function() { /* Loaded */ });

  function handleClientLoad() {
    gapi.client.setApiKey(config.apiKey);
    window.setTimeout(checkAuth, 100);
  }

  function checkAuth() {
    gapi.auth.authorize({ client_id: config.clientId, scope: config.scopes, immediate: true }, handleAuthResult);
  }

  function handleAuthResult(authResult) {
  }

  handleClientLoad();
};

The config variable was one of the dependencies for this file, and contains the credentials required by Google’s API.

Loading the API Manager

Now open app/js/app.js and make it depend on gapi, then create an instance of ApiManager:

define([
  'gapi'
],

function(ApiManager) {
  var App = function() {
    this.connectGapi();
  };

  App.prototype = {
    connectGapi: function() {
      this.apiManager = new ApiManager();
    }
  };

  return App;
});

If you want to check this works by running the tests, you’ll need to change test/setup.js to flag gapi as a global:

var assert = chai.assert;

mocha.setup({
  ui: 'tdd'
, globals: ['bTask', 'gapi', '___jsl']
});

However, I don’t intend to load the API remotely during testing – this will effectively be mocked. I’ll come onto that in a later tutorial.

Results

gapi loaded

If you run the app or tests and open a JavaScript console, a gapi global object should be available. Using Google’s APIs with RequireJS and Backbone.js seems like a lot of work, but most of this stuff is effectively just configuration, and once it’s done it should work solidly enough, allowing you to focus on the app design and development side of things.

Full Source Code

Commit 9d09a6.

References

Node Roundup: pkgcloud, rewire, ssh2

05 Dec 2012 | By Alex Young | Comments | Tags node modules ssh cloud testing
You can send in your Node projects for review through our contact form or @dailyjs.

pkgcloud

pkgcloud (GitHub: nodejitsu / pkgcloud, License: MIT, npm: pkgcloud) from Nodejitsu is a module for scripting interactions with cloud service providers. It supports various services from Joyent, Microsoft, Rackspace, and several database providers like MongoHQ and RedisToGo. The authors have attempted to unify the vocabulary used by each provider – for example, pkgcloud uses the term ‘Server’ to refer to Joyent’s “machines” and Amazon’s “instances”.

Services can be introspected and resources can be fetched. The API is naturally asynchronous, with callback arguments using the standard error-first pattern.

The roadmap promises support for more services in the future, including CDN and DNS.

rewire

rewire (License: MIT, npm: rewire by Johannes Ewald is a dependency injection implementation that can be used to inject mocks into other modules and access private variables.

As an example, consider a module within your project that uses the standard fs module to read a file. When writing tests for this module, it would be entirely possible to use rewire to modify the fs module to mock the readFile method:

var rewire = require('rewire')
  , exampleModule = rewire('./exampleModule')
  ;

exampleModule.__set__('fs', {
  readFile: function(path, encoding, cb) {
    cb(null, 'Success!');
  }
});

// Tests would follow...

Notice that rewire was used instead of require – rewire itself works by appending special getters and setters to modules rather than using an eval-based solution.

SSH2

SSH2 (License: MIT, npm: ssh2) by Brian White is an SSH2 client written in pure JavaScript. It’s built with the standard Node modules – streams, buffers, events, and lots of prototype objects and regular expressions.

It supports several authentication methods, including keys, bidirectional port forwarding, execution of remote commands, interactive sessions, and SFTP. Brian has provided some detailed examples of how to use the library’s event-based API.

jQuery Roundup: jquery.snipe, Photobooth.js, jHERE

04 Dec 2012 | By Alex Young | Comments | Tags jquery images plugins maps
Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

jquery.snipe

jquery.snipe showing two images, one black and white.

jquery.snipe (GitHub: RayFranco / jquery.snipe, License: Apache 2.0, Component: RayFranco/jquery.snipe) by Franco Bouly shows a pleasing lens effect over images that follows the mouse. The “zoom” image is separate, so effects can be created – Franco’s examples have black and white images with a colour image for the zoom.

Note that Franco’s example image is slightly risqué – it’s perfectly safe for work in the DailyJS office but it might not be elsewhere.

Photobooth.js

Photobooth.js (GitHub: WolframHempel / photobooth-js, License: BSD) by Wolfram Hempel is an API for working with webcams. It currently works in recent versions of Chrome, Firefox, and Opera – browsers that support navigator.getUserMedia. It can be used like a class or through jQuery. It supports hue, saturation, and brightness adjustments and image resizing.

jHERE

jHERE displaying a route using KML.

jHERE (GitHub: mmarcon / jhere, License: MIT) by Massimiliano Marcon is a jQuery and Zepto API for working with maps. It supports markers, KML (Keyhole Markup Language), and heatmaps.

The map service used is Here from Nokia.

JS101: Deep Equal

03 Dec 2012 | By Alex Young | Comments | Tags js101 tutorials language beginner testing

Back in JS101: Equality I wrote about the difference between == and ===. This is one area of the language that quite clearly causes issues for beginners. In addition, there is another equality concept that can come in handy when writing tests: deep equal. It also illustrates some of the underlying mechanics of the language. As an intermediate JavaScript developer, you should have at least a passing familiarity with deepEqual and how it works.

Unit Testing/1.0

Deep equality is defined in CommonJS Unit Testing/1.0, under subsection 7. The algorithm assumes two arguments: expected and actual. The purpose of the algorithm is to determine if the values are equivalent. It supports both primitive values and objects.

  1. Strict equals (===) means the values are equivalent
  2. Compare dates using the getTime method
  3. If values are not objects, compare with ==
  4. Otherwise, compare each object’s size, keys, and values

The fourth point is probably what you would assume deep equality actually means. The other stages reveal things about the way JavaScript works – the third stage means values that are not objects can easily be compared with == because they’re primitive values (Undefined, Null, Boolean, Number, or String).

The second step works because getTime is the most convenient way of comparing dates:

var assert = require('assert')
  , a = new Date(2012, 1, 1)
  , b = new Date(2012, 1, 1)
  ;

assert.ok(a !== b);
assert.ok(a != b);
assert.ok(a.getTime() == b.getTime());
assert.deepEqual(a, b);

This script can be run in Node, or with a suitable CommonJS assertion library. It illustrates the point that dates are not considered equal using the equality or strict equality operators – the easiest way to compare them is with getTime.

Object comparison implies recursion), as some values may also be objects. Also, key comparison isn’t as simple as it might seem: real implementations sort keys, compare length, then compare each value.

Bugs

Bugs have been found in the Unit Testing/1.0 specification since it originally appeared. Two have been flagged up on the main Unit Testing page. The Node assert module addresses these points. For example, regular expressions are a special case in the deepEqual implementation:

return actual.source === expected.source &&
       actual.global === expected.global &&
       actual.multiline === expected.multiline &&
       actual.lastIndex === expected.lastIndex &&
       actual.ignoreCase === expected.ignoreCase;

The source property has a string that represents the original regular expression, and then each flag has to be compared.

Object Comparison

The next time you’re writing a test, or even just comparing objects, remember that == will only work for “shallow” comparisons. Testing other values like arrays, dates, regular expressions, and objects requires a little bit more effort.

Viewport Component, grunt-saucelabs, FastClick

30 Nov 2012 | By Alex Young | Comments | Tags grunt mobile components libraries

Viewport Component

Viewport Component (License: MIT, component: pazguille/viewport) by Guille Paz can be used to get information about a browser’s viewport. This includes the width and height as well as the vertical and horizontal scroll positions.

It also emits events for scrolling, resizing, and when the top or bottom has been reached during scrolling.

var Viewport = require('viewport')
  , viewport = new Viewport()
  ;

viewport.on('scroll', function() {
  console.log(this.scrollY);
});

viewport.on('top', function() {
  console.log('Top');
});

viewport.on('bottom', function() {
  console.log('Bottom');
});

grunt-saucelabs

grunt-saucelabs (License: MIT, npm: grunt-saucelabs) by Parashuram N (axemclion) is a Grunt task for qunit and jasmine tests using Sauce Labs’ Cloudified Browsers. This is similar to the built-in qunit Grunt task, but uses the remote service provided by Sauce Labs instead.

Sauce Connect can be used to create a tunnel for testing pages that aren’t accessible on the Internet.

FastClick

FastClick (License: MIT, npm: fastclick, component: ftlabs/fastclick) from the Financial Times helps remove the delay in mobile browsers that occurs between a tap and the trigger of click events.

The authors have included some simple tests, documentation, and examples. The project is extremely well packaged, including support for npm, component, AMD, and Google Closure Compiler’s ADVANCED_OPTIMIZATIONS.

Internally, FastClick works by binding events to a “layer” and binding several touch event handlers. These handlers use internal properties to determine how elements are being interacted with. If certain conditions are met, then a click event will be generated, and several attributes will be added to the event to allow further tracking.

The event handlers can be easily removed using FastClick.prototype.destroy, and the project has a wide range of special cases for handling divergent behaviour in iOS and Android.

Backbone.js Tutorial: Build Environment

29 Nov 2012 | By Alex Young | Comments | Tags backbone.js mvc node backgoog

This new Backbone.js tutorial series will walk you through building a single page web application that has a customised Backbone.sync implementation. I started building the application that these tutorials are based on back in August, and it’s been running smoothly for a few months now so I thought it was safe enough to unleash it.

Gmail to-do lists: not cool enough!

The application itself was built to solve a need of mine: a more usable Google Mail to-do list. The Gmail-based interface rubs me the wrong way to put it mildly, so I wrote a Backbone.sync method that works with Google’s APIs and stuck a little Bootstrap interface on top. As part of these tutorials I’ll also make a few suggestions on how to customise Bootstrap – there’s no excuse for releasing vanilla Bootstrap sites!

The app we’ll be making won’t feature everything that Google’s to-do lists support: I haven’t yet added support for indenting items for example. However, it serves my needs very well so hopefully it’ll be something you’ll actually want to use.

Roadmap

Over the next few weeks I’ll cover the following topics:

  • Creating a new Node project for building the single page app
  • Using RequireJS with Backbone.js
  • Google’s APIs
  • Writing and running tests
  • Creating the Backbone.js app itself
  • Techniques for customising Bootstrap
  • Deploying to Dropbox, Amazon S3, and potentially other services

Creating a Build Environment

If your focus is on client-side scripting, then I think this will be useful to you. Our goal is to create a development environment that can do the following:

  • Allow the client-side code to be written as separate files
  • Combine separate files into something suitable for deployment
  • Run the app locally using separate files (to make development and debugging easier)
  • Manage supporting Node modules
  • Run tests
  • Support Unix and Windows

To do this we’ll need a few tools and libraries:

Make sure your system has Node installed. The easiest way to install it is by using one of the Node packages for your system.

Step 1: Installing the Node Modules

Create a new directory for this project, and create a new file inside it called package.json that contains this JSON:

{
  "name": "btask"
, "version": "0.0.1"
, "private": true
, "dependencies": {
    "requirejs": "latest"
  , "connect": "2.7.0"
  }
, "devDependencies": {
    "mocha": "latest"
  , "chai": "latest"
  , "grunt": "latest"
  , "grunt-exec": "latest"
  }
, "scripts": {
    "grunt": "node_modules/.bin/grunt"
  }
}

Run npm install. These modules along with their dependencies will be installed in ./node_modules.

The private property prevents accidentally publicly releasing this module. This is useful for closed source commercial projects, or projects that aren’t suitable for release through npm.

Even if you’re not a server-side developer, managing dependencies with npm is useful because it makes it easier for other developers to work on your project. When a new developer joins your project, they can just type npm install instead of figuring out what downloads to grab.

Step 2: Local Web Server

Create a directory called app and a file called app/index.html:

<!DOCTYPE html>
<head>
  <meta charset="utf-8">
  <title>bTask</title>
  <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
  <script type="text/javascript" src="js/lib/require.js"></script>
</head>
<body>
</body>
</html>

Once you’ve done that, create a file called server.js in the top-level directory:

var connect = require('connect')
  , http = require('http')
  , app
  ;

app = connect()
  .use(connect.static('app'))
  .use('/js/lib/', connect.static('node_modules/requirejs/'))
  .use('/node_modules', connect.static('node_modules'))
  ;

http.createServer(app).listen(8080, function() {
  console.log('Running on http://localhost:8080');
});

This file uses the Connect middleware framework to act as a small web server for serving the files in app/. You can add new paths to it by copying the .use(connect.static('app')) line and changing app to something else.

Notice how I’ve mapped the web path for /js/lib/ to node_modules/requirejs/ on the file system – rather than copying RequireJS to where the client-side scripts are stored we can map it using Connect. Later on the build scripts will copy node_modules/requirejs/require.js to build/js/lib so the index.html file won’t have to change. This will enable the project to run on a suitable web server, or a hosting service like Amazon S3 for static sites.

To run this Node server, type npm start (or node server.js) and visit http://localhost:8080. It should display an empty page with no client-side errors.

Step 3: Configuring RequireJS

This project will consist of modules written in the AMD format. Each Backbone collection, model, view, and so on will exist in its own file, with a list of dependencies so RequireJS can load them as needed.

RequireJS projects that work this way are usually structured around a “main” file that loads the necessary dependencies to boot up the app. Create a file called app/js/main.js that contains the following skeleton RequireJS config:

requirejs.config({
  baseUrl: 'js',

  paths: {
  },

  shim: {
  }
});

require(['app'],

function(App) {
  window.bTask = new App();
});

The part that reads require(['app'] will load app/js/app.js. Create this file with the following contents:

define([], function() {
  var App = function() {
  };

  App.prototype = {
  };

  return App;
});

This is a module written in the AMD format – the define function is provided by RequireJS and in future will contain all of the internal dependencies for the project.

To finish off this step, the main.js should be loaded. Add some suitable script tags near the bottom of app/index.html, before the </body> tag.

<script type="text/javascript" src="js/main.js"></script>

If you refresh http://localhost:8080 in your browser and open the JavaScript console, you should see that bTask has been instantiated.

window.bTask

Step 4: Testing

Everything you’ve learned in the previous three steps can be reused to create a unit testing suite. Mocha has already been installed by npm, so let’s create a suitable test harness.

Create a new directory called test/ (next to the ‘app/’ directory) that contains a file called index.html:

<html>
<head>
  <meta charset="utf-8">
  <title>bTask Tests</title>
  <link rel="stylesheet" href="/node_modules/mocha/mocha.css" />
  <style>
.toast-message, #main { display: none }
  </style>
</head>
<body>
  <div id="mocha"></div>
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
  <script src="/node_modules/chai/chai.js"></script>
  <script src="/node_modules/mocha/mocha.js"></script>
  <script src="/js/lib/require.js"></script>
  <script src="/js/main.js"></script>
  <script src="setup.js"></script>
  <script src="app.test.js"></script>
  <script>require(['app'], function() { mocha.run(); });</script>
</body>
</html>

The require near the end just makes sure mocha.run only runs when /js/app.js has been loaded.

Create another file called test/setup.js:

var assert = chai.assert;

mocha.setup({
  ui: 'tdd'
, globals: ['bTask']
});

This file makes Chai’s assertions available as assert, which is how I usually write my tests. I’ve also told Mocha that bTask is an expected global variable.

With all this in place we can write a quick test. This file goes in test/app.test.js:

suite('App', function() {
  test('Should be present', function() {
    assert.ok(window.bTask);
  });
});

All it does is checks window.bTask has been defined – it proves RequireJS has loaded the app.

Finally we need to update where Connect looks for files to serve. Modify ‘server.js’ to look like this:

var connect = require('connect')
  , http = require('http')
  , app
  ;

app = connect()
  .use(connect.static('app'))
  .use('/js/lib/', connect.static('node_modules/requirejs/'))
  .use('/node_modules', connect.static('node_modules'))
  .use('/test', connect.static('test/'))
  .use('/test', connect.static('app'))
  ;

http.createServer(app).listen(8080, function() {
  console.log('Running on http://localhost:8080');
});

Restart the web server (from step 2), and visit http://localhost:8080/test/ (the last slash is important). Mocha should display that a single test has passed.

Step 5: Making Builds

Create a file called grunt.js for our “gruntfile”:

module.exports = function(grunt) {
  grunt.loadNpmTasks('grunt-exec');

  grunt.initConfig({
    exec: {
      build: {
        command: 'node node_modules/requirejs/bin/r.js -o require-config.js'
      }
    }
  });

  grunt.registerTask('copy-require', function() {
    grunt.file.mkdir('build/js/lib');
    grunt.file.copy('node_modules/requirejs/require.js', 'build/js/lib/require.js');
  });

  grunt.registerTask('default', 'exec copy-require');
};

This file uses the grunt-exec plugin by Jake Harding to run the RequireJS command that generates a build of everything in the app/ directory. To tell RequireJS what to build, create a file called require-config.js:

({
  appDir: 'app/'
, baseUrl: 'js'
, paths: {}
, dir: 'build/'
, modules: [{ name: 'main' }]
})

RequireJS will minimize and concatenate the necessary files. The other Grunt task copies the RequireJS client-side code to build/js/lib/require.js, because our local Connect server was mapping this for us. Why bother? Well, it means whenever we update RequireJS through npm the app and builds will automatically get the latest version.

To run Grunt, type npm run-script grunt. The npm command run-script is used to invoke scripts that have been added to the package.json file. The package.json created in step 1 contained "grunt": "node_modules/.bin/grunt", which makes this work. I prefer this to installing Grunt globally.

I wouldn’t usually use Grunt for my own projects because I prefer Makefiles. In fact, a Makefile for the above would be very simple. However, this makes things more awkward for Windows-based developers, so I’ve included Grunt in an effort to support Windows. Also, if you typically work as a client-side developer, you might find Grunt easier to understand than learning GNU Make or writing the equivalent Node code (Node has a good file system module).

Summary

In this tutorial you’ve created a Grunt and RequireJS build environment for Backbone.js projects that use Mocha for testing. You’ve also seen how to use Connect to provide a convenient local web server.

This is basically how I build and manage all of my Backbone.js single page web applications. Although we haven’t written much code yet, as you’ll see over the coming weeks this approach works well for using Backbone.js and RequireJS together.

The code for this project can be found here: dailyjs-backbone-tutorial (2a8517).

Contributions

Node Roundup: 0.8.15, JSPath, Strider

28 Nov 2012 | By Alex Young | Comments | Tags node modules apps express json
You can send in your Node projects for review through our contact form or @dailyjs.

Node 0.8.15

Node 0.8.15 is out, which updates npm to 1.1.66, fixes a net and tls resource leak, and has some miscellaneous fixes for Windows and Unix systems.

JSPath

JSPath (License: MIT/GPL, npm: jspath) by Filatov Dmitry is a DSL for working with JSON. The DSL can be used to select values, and looks a bit like CSS selectors:

// var doc = { "books" : [ { "id" : 1, "title" : "Clean Code", "author" : { "name" : "Robert C. Martin" } ...

JSPath.apply('.books.author', doc);
// [{ name : 'Robert C. Martin' }, ...

It can also be used to apply conditional expressions, like this: .books{.author.name === 'Robert C. Martin'}.title. Other comparison operators are also supported like >= and !==. Logical operators can be used to combine predicates, and the API also supports substitution.

The author has included unit tests, and build scripts for generating a browser-friendly version.

Strider

Strider (GitHub: Strider-CD / strider, License: BSD, npm: strider) by Niall O’Higgins is a continuous deployment solution built with Express and MongoDB. It’s designed to be easy to deploy to Heroku, and can be used to deploy applications to your own servers. It directly supports Node and Python applications, and the author is also working on supporting Rails and JVM languages.

Strider integrates with GitHub, so it can display a list of your repositories and allow them to be deployed as required. It can also test projects, so it can be used for continuous integration if deployment isn’t required.

The documentation includes full details for installing Strider, linking a GitHub account, and then setting it up as a CI/CD server with an example project.

jQuery Roundup: 1.8.3, UI 1.9.2, oolib.js

27 Nov 2012 | By Alex Young | Comments | Tags jquery jquery-ui oo
Note: You can send your plugins and articles in for review through our contact form or @dailyjs.

jQuery 1.8.3

jQuery 1.8.3 and jQuery Color 2.1.1 are out. There are a few interesting bug fixes in this release that you might want to check out:

jQuery UI 1.9.2

jQuery UI 1.9.2 is out:

This update brings bug fixes for Accordion, Autocomplete, Button, Datepicker, Dialog, Menu, Tabs, Tooltip and Widget Factory.

The 1.9.2 changelog contains a full breakdown of the recent changes.

oolib.js

oolib.js (GitHub: idya / oolib, License: MIT) by Zsolt Szloboda is a JavaScript object-oriented library that is conceptually similar to jQuery UI’s Widget Factory. It supports private methods, class inheritance, object initialisation and deinitialisation, super methods, and it’s fairly small (min: 1.6K, gz: 0.7K).

It looks like this in practice:

var MyClass = oo.createClass({
  _create: function(foo) {
    this.myField = foo;
  },

  _myPrivateMethod: function(bar) {
    return this.myField + bar;
  },

  myPublicMethod: function(baz) {
    return this._myPrivateMethod(baz);
  }
});

var MySubClass = oo.createClass(MyClass, {
  _myPrivateMethod: function(bar) {
    return this.myField + bar + 1;
  }
});

JS101: __proto__

26 Nov 2012 | By Alex Young | Comments | Tags js101 tutorials language beginner

When I originally wrote about prototypes in JS101: Prototypes a few people were confused that I didn’t mention the __proto__ property. One reason I didn’t mention it is I was sticking to standard ECMAScript for the most part, using the Annotated ECMAScript 5.1 site as a reference. It’s actually hard to talk about prototypes without referring to __proto__, though, because it serves a very specific and useful purpose.

Recall that objects are created using constructors:

function User() {
}

var user = new User();

The prototype property can be used to add properties to instances of User:

function User() {
}

User.prototype.greet = function() {
  return 'hello';
};

var user = new User();
user.greet();

So far so good. The original constructor can be referenced using the constructor property on an instance:

assert.equal(user.constructor, User);

However, user.prototype is not the same as User.prototype. What if we wanted to get hold of the original prototype where the greet method was defined based on an instance of a User?

That’s where __proto__ comes in. Given that fact, we now know the following two statements to be true:

assert.equal(user.constructor, User);
assert.equal(user.__proto__, User.prototype);

Unfortunately, __proto__ doesn’t appear in ECMAScript 5 – so where does it come from? As noted by the documentation on MDN it’s a non-standard property. Or is it? It’s included in Ecma-262 Edition 6, which means whether it’s standard or not depends on the version of ECMAScript that you’re using.

It follows that an instance’s constructor should contain a reference to the constructor’s prototype. If this is true, then we can test it using these assertions:

assert.equal(user.constructor.prototype, User.prototype);
assert.equal(user.constructor.prototype, user.__proto__);

The standards also define Object.getPrototypeOf – this returns the internal property of an object. That means we can use it to access the constructor’s prototype:

assert.equal(Object.getPrototypeOf(user), User.prototype);

Putting all of this together gives this script which will pass in Node and Chrome (given a suitable assertion library):

var assert = require('assert');

function User() {
}

var user = new User();

assert.equal(user.__proto__, User.prototype);
assert.equal(user.constructor, User);
assert.equal(user.constructor.prototype, User.prototype);
assert.equal(user.constructor.prototype, user.__proto__);
assert.equal(Object.getPrototypeOf(user), User.prototype);

Internal Prototype

The confusion around __proto__ arises because of the term internal prototype:

All objects have an internal property called [[Prototype]]. The value of this property is either null or an object and is used for implementing inheritance.

Internally there has to be a way to access the constructor’s prototype to correctly implement inheritance – whether or not this is available to us is another matter. Why is accessing it useful to us? In the wild you’ll occasionally see people setting an object’s __proto__ property to make objects look like they inherit from another object. This used to be the case in Node’s assertion module, but Node’s util.inherits method is a more idiomatic way to do it:

// Compare to: assert.AssertionError.__proto__ = Error.prototype;
util.inherits(assert.AssertionError, Error);

This was changed in assert: remove unnecessary use of __proto__.

The Constructor’s Prototype

The User example’s internal prototype is set to Function.prototype:

assert.equal(User.__proto__, Function.prototype);

If you’re about to put on your hat, pick up your briefcase, and walk right out the door: hold on a minute. You’re coming to the end of the chain – the prototype chain that is:

assert.equal(User.__proto__, Function.prototype);
assert.equal(Function.prototype.__proto__, Object.prototype);
assert.equal(Object.prototype.__proto__, null);

Remember that the __proto__ property is the internal prototype – this is how JavaScript’s inheritance chain is implemented. Every User inherits from Function.prototype which in turn inherits from Object.prototype, and Object.prototype’s internal prototype is null which allows the inheritance algorithm to know it has reached the end of the chain.

Therefore, adding a method to Object.prototype will make it available to every object. Properties of the Object Prototype Object include toString, valueOf, and hasOwnProperty. That means instances of the User constructor in the previous example will have these methods.

Pithy Closing Remark

JavaScript’s inheritance model is not class-based. Joost Diepenmaat’s post, Constructors considered mildly confusing, summarises this as follows:

In a class-based object system, typically classes inherit from each other, and objects are instances of those classes. … constructors do nothing like this: in fact constructors have their own [[Prototype]] chain completely separate from the [[Prototype]] chain of objects they initialize.

Rather than visualising JavaScript objects as “classes”, try to think in terms of two parallel lines of prototype chains: one for constructors, and one for initialised objects.

References