Discussion: What Do You Want From Components?

22 Feb 2013 | By Alex Young | Comments | Tags component bower

Yogruntbower component!

Assume for a second that TJ Holowaychuk’s Component project isn’t the future. And then let’s say that, due to support from Twitter and Google (through Yeoman), Bower becomes the de facto tool for managing and installing client-side dependencies. Whether you’re using Yeoman or Bower with another build tool, you’re still left with a gap where reusable UI widgets should be.

While Yeoman improves workflow, Component also tackles the notion of sharing “widgets” that contain markup, stylesheets, and code. If you read TJ’s tutorials he pushes the idea of structural markup – stripping away unnecessary markup to leave behind a vanilla slice of templates that can easily be reskinned with CSS.

With more advanced client-side workflows provided by libraries like Backbone.js, RequireJS, and tools like Yeoman and Bower, I feel like moving away from monolithic UI projects is necessary. While I like to jump start projects with jQuery UI, Bootstrap, Closure Library, or perhaps even Dojo’s Dijit and DojoX, these projects are more monolithic than the modular dream promised by the Component project.

I believe there’s a missing library to the Bower/Yeoman future: something to support the notion of reusable widgets. Packaging chunks of markup, styles, and JavaScript is nothing new – but what is being done to solidify this goal outside of the Component project and older monolithic approaches?

Standards

The component idea is being formalised in Introduction to Web Components, and lists Dominic Cooney and Dimitri Glazkov from Google as the editors. Therefore, the concept is being standardised, but this particular vision of components seems very different to what TJ and other developers have envisioned.

Accessibility

Should a widget/component library enforce ARIA, or provide tools for making accessible components? jQuery UI went through many iterations to improve its accessibility, and Dojo has documentation on creating accessible widgets.

Data Binding APIs

What about data binding or MVC style development? How would UI components fit in? If you’re shipping JavaScript inside a component, how would the API provide hooks that can work with Knockout, AngularJS, and other libraries without manually plugging them in?

It feels like we’re settling on binding using data- attributes, so this might be relatively trivial in practice. Perhaps the “ultimate” component library would address this, or perhaps it’s unnecessary.

What Do You Want?

Let’s say you settle on Yeoman as your workflow tool for client-side development. What do you think reusable client-side widgets should look like? What would be the perfect fit alongside Yeoman, Bower, Grunt, and a data binding library?

Cloudinary Tutorial

Cloudinary

Cloudinary

This tutorial introduces Cloudinary, and demonstrates how to build a gallery application using Express and Node. Cloudinary is a service for streamlining asset management – if you’re tired of optimising images and manually uploading them to an asset server or CDN, then Cloudinary might be what you’re looking for.

One reason Cloudinary is useful to us as Node developers is the Cloudinary Node module (GitHub: cloudinary / cloudinary_npm, License: MIT, npm: cloudinary). It can be used to easily generate optimised images, thumbnails, and automatically upload them to Cloudinary. Let’s drop it into an Express application to see what happens!

The full source for this tutorial is available here: alexyoung / dailyjs-cloudinary-gallery.

Step 1: Create a Cloudinary Account

Register for a free account at Cloudinary. That’ll give you 500 MB of storage and a gigabyte of monthly bandwidth. Paid plans start at $39 a month, and that increases the storage to 10 GB and adds 40 GB a month of bandwidth.

Once you’ve created your account, sign in and take a look at the right-hand panel that reads “Account Details”.

The Cloudinary management interface.

To follow this tutorial, you’ll need the api_key and api_secret, so make a note of those.

Step 2: The Express App

Assuming you’ve already installed Node, open a terminal and run npm install -g express. Once that’s done, run express dailyjs-cloudinary-gallery to create a new Express project.

Open package.json and add the cloudinary dependency:

{
  "name": "application-name",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node app"
  },
  "dependencies": {
    "express": "3.0.3",
    "jade": "*",
    "cloudinary": "*"
  }
}

Once you’ve done that, save the file and run npm install. This will install the project’s dependencies, including the Cloudinary module.

Step 3: Image Uploads

Cloudinary can be used for every aspect of an image gallery:

  • Uploading images
  • Creating and serving thumbnails
  • Fetching a list of images to display

Go to the top of app.js and add the following require statements:

var cloudinary = require('cloudinary')
  , fs = require('fs')

Now add a new route to handle uploads:

app.post('/upload', function(req, res){
  var imageStream = fs.createReadStream(req.files.image.path, { encoding: 'binary' })
    , cloudStream = cloudinary.uploader.upload_stream(function() { res.redirect('/'); });

  imageStream.on('data', cloudStream.write).on('end', cloudStream.end);
});

Add your Cloudinary configuration to app.configure:

app.configure('development', function(){
  app.use(express.errorHandler());
  cloudinary.config({ cloud_name: 'yours', api_key: 'yours', api_secret: 'yours' });
});

app.locals.api_key = cloudinary.config().api_key;
app.locals.cloud_name = cloudinary.config().cloud_name;

This allows you to use different Cloudinary accounts for development and production purposes, depending on your requirements.

The last two lines make the values available to the templates. The secret is not meant to be accessible from outside the server, but the api_key and cloud_name options can be used by client-side scripts.

Change views/index.jade to show an upload form:

extends layout

block content
  h1= title
  p Welcome to #{title}

  form(action="/upload", method="post", enctype="multipart/form-data")
    input(type="file", name="image")
    input(type="submit", value="Upload Image")

  - if (images && images.length)
    - images.forEach(function(image){
      img(src=image.url)
    - })

Before you try out the app, change the / route in app.js to load a set of images from Cloudinary:

app.get('/', function(req, res, next){
  cloudinary.api.resources(function(items){
    res.render('index', { images: items.resources, title: 'Gallery' });
  });
});

The cloudinary.api.resources method fetches all of the images in your account. Note that this is rate limited to 500 requests per hour – I’ve used it here to keep the tutorial simple, but in production you should cache the results or store them in a database.

At this point, image uploads should work if you start up the app with npm start and navigate to http://localhost:3000, but the output won’t look great without thumbnails.

Step 4: Thumbnails

Cloudinary supports a wide range of image transformations. The API is based around generating a URL that includes parameters to change the image in some way. All we need for the gallery is a simple crop, which is supported through the cloudinary.url method. Change the / route to pass a reference to the cloudinary object to the template:

app.get('/', function(req, res, next){
  cloudinary.api.resources(function(items){
    res.render('index', { images: items.resources, title: 'Gallery', cloudinary: cloudinary });
  });
});

Now, open views/index.jade so that it calls cloudinary.url with some options to get the desired effect:

- images.forEach(function(image){
  a(href=image.url)
    img(src=cloudinary.url(image.public_id + '.' + image.format, { width: 100, height: 100, crop: 'fill', version: image.version }))
- })

The important part here is this:

cloudinary.url(image.public_id + '.' + image.format, { width: 100, height: 100, crop: 'fill', version: image.version })

The cloudinary.url method generates a URL that includes the width, height, and crop options:

http://res.cloudinary.com/your_account/image/upload/c_fill,h_100,w_100/v1234/file.jpg

Because the API is based around URLs, you could easily use this from browser-based JavaScript, utilising Cloudinary to add behaviour that would typically be associated with server-side web development.

I’ve also included the version property, which is recommended by Cloudinary when overriding the public_id. It’s returned by both the upload API and the admin API.

Step 5: Effects

The previous example can easily be adapted to generate lots of interesting effects. This change makes it generate images that feature a vignette lens effect:

- images.forEach(function(image){
  a(href=image.url)
    img(src=cloudinary.url(image.public_id + '.' + image.format, { width: 100, height: 100, crop: 'fill', effect: 'vignette', version: image.version }))
- })

The effect parameter can be one of the following effects:

  • grayscale
  • blackwhite
  • vignette
  • sepia
  • brightness
  • saturation
  • contrast
  • hue
  • pixelate
  • blur
  • sharpen

Some effects take an argument, and this is simply prefixed with a colon. For example, brightness:40.

As well as face detection, adding rounded corners, overlays, and other transformations are also available. Again, check the image transforms documentation for full details.

The vignette effect applied to several images.

jQuery Uploads

Cloudinary has a CORS API for file uploads which degrades to an iframe in legacy browsers. This means you can do image uploads with no server-side code at all! The cloudinary_js repository has a jQuery plugin, with jQuery UI support, which can be used to upload images.

Download the JavaScript files from the cloudinary/cloudinary_js repository and add them to the public/ folder. Then edit views/layout.jade to load jQuery and the other files in this order:

  script(src='http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js')
  script(src='/jquery.ui.widget.js')
  script(src='/jquery.iframe-transport.js')
  script(src='/jquery.fileupload.js')
  script(src='/jquery.cloudinary.js')
  block scripts

The block scripts part at the end is some Jade jiggery-pokery to allow HTML to be appended to this template from another template. Open views/index.jade and add this markup:

  h2 jQuery Uploads

  .preview

  form(enctype="multipart/form-data")!=cloudinary.uploader.image_upload_tag('image')

block scripts
  script(type="text/javascript")
    // Configure Cloudinary
    $.cloudinary.config({ api_key: '!{api_key}', cloud_name: '!{cloud_name}' });

    $('.cloudinary-fileupload').bind('fileuploadstart', function(e){
      $('.preview').html('Upload started...');
    });

    // Upload finished
    $('.cloudinary-fileupload').bind('cloudinarydone', function(e, data){
      $('.preview').html(
        $.cloudinary.image(data.result.public_id, { format: data.result.format, version: data.result.version, crop: 'scale', width: 100, height: 100 })
      );
      return true;
    });

This displays a form with a file input, generated by the cloudinary.uploader.image_upload_tag helper. That keeps the markup lightweight by doing all of the signing and other things Cloudinary’s API needs behind the scenes.

The client-side JavaScript at the end of the template will display a message when an image is being uploaded, and then display it once it’s finished uploading. The other event which I haven’t used here is fileuploadfail, which is, of course, useful for displaying errors when file uploads fail.

If you want to read more about Cloudinary and jQuery, check out this article: Direct image uploads from the browser to the cloud with jQuery, Upload Images: Remote Uploads.

Summary

The completed gallery.

In this tutorial you’ve seen how to integrate both Node and client-side projects with Cloudinary. If you’d like more details on the service, visit Cloudinary.com.

This gallery example could be easily expanded using features from Cloudinary’s API to do a lot of practical and cool stuff:

  • Pagination could be added
  • The effects API could be used for editing photos
  • Face detection could be used to tag people in photos

The full source for my example Express app is available here: https://github.com/alexyoung/dailyjs-cloudinary-gallery.

Node Roundup: 0.8.20, 0.9.10, continuation.js, selenium-node-webdriver

20 Feb 2013 | By Alex Young | Comments | Tags node modules testing functional
You can send in your Node projects for review through our contact form.

Node 0.8.20, 0.9.10

Node 0.8.20 was released last week. The most significant updates in this version are fixes for the HTTP core module, so if you’re on 0.8.19 then I can’t see any reason not to upgrade.

Node 0.9.10 meanwhile has several stream-related updates. The default options for WriteStream have been updated to improve performance, and empty strings and buffers no longer signal EOF.

continuation.js

continuation.js (GitHub: dai-shi / continuation.js, License: BSD, npm: continuation.js) by Daishi Kato automatically adds tail call optimisation to modules loaded with require. It’s written using esprima and escodegen to parse and generate a new version of existing code. It does this by using trampolined functions, which is also how tail recursive functions are implemented in functional languages like Lisp.

The author has included benchmarks that show where the module improves performance. There are cases where it won’t be faster due to how trampolining is handled – there are also some interesting posts by Guillaume Lathoud about implementing tail call optimisation without trampolining.

selenium-node-webdriver

selenium-node-webdriver (GitHub: WaterfallEngineering / selenium-node-webdriver, License: Apache 2, npm: selenium-node-webdriver) by Lon Ingram packages a prebuilt WebDriver client so it’s easier to get started writing tests that use WebDriver. Lon notes that it was designed to work with PhantomJS, but it could be used with any WebDriver server.

jQuery Roundup: Durandal, Version.js, Navi.js

19 Feb 2013 | By Alex Young | Comments | Tags jquery plugins frameworks libraries testing navigation
Note: You can send your plugins and articles in for review through our contact form.

Durandal

Durandal (GitHub: BlueSpire / Durandal, License: MIT) combines jQuery, Knockout, and RequireJS with some of its own code to create a framework for developing single page applications. Durandal apps are built using AMD-based modules, and it also supports the notion of a widget.

One interesting feature is application-wide messaging – the main app object can handle events, so it can be used as a universal message bus to help keep functionality nicely decoupled.

The project includes Jasmine/PhantomJS tests in the test/ directory, but the documentation itself doesn’t mention tests and the application skeletons don’t include them either. That seems like an oversight to me, given that the project claims to be “single page apps done right”.

Version.js

Justin Stayton sent in Version.js (jstayton / version.js, License: MIT, bower: version.js), which he developed while testing scripts against multiple versions of jQuery. It works by using attributes to specify the required versions of libraries:

<script src="version.js" data-url="google" data-lib="jquery" data-ver="1.7.2"></script>

This will cause jQuery 1.7.2 to be loaded from Google’s CDN as the default. If another version is required, the versionjs GET parameter can be used. This makes it easy to switch between versions of a dependency, which might be useful in tests or during local development.

Navi.js (GitHub: tgrant54 / Navi.js, License: MIT) by Tyler Grant makes single pages behave like a full site using hash routing. It has breadcrumb support, and can be called multiple times. The jQuery plugin method takes a hash option so you could embed multiple menus on a page, each using a different hash to distinguish between them:

$('#naviMenu').navi({
  hash: '#!/'
, content: $('#naviContent')
});

The project’s homepage has markup samples and demos.

memdiff, numerizerJS, Obfuscate.js

18 Feb 2013 | By Alex Young | Comments | Tags testing debugging memory node modules parsing text

memdiff

memdiff (GitHub: azer / memdiff, License: WTFPL, npm: memdiff) by Azer Koculu is a BDD-style memory leak tool based on memwatch. It can either be used by writing scripts with describe and it, and then running them with memdiff:

function SimpleClass(){}
var leaks = [];

describe('SimpleClass', function() {
  it('is leaking', function() {
    leaks.push(new SimpleClass);
  });

  it('is not leaking', function() {
    new SimpleClass;
  });
});

Or by loading memdiff with require and passing a callback to memdiff. The memwatch module itself has an event-based API, and includes a platform-independent native module – so both of these projects are tied to Node and won’t work in a browser.

numerizerJS

numerizerJS (GitHub: bolgovr / numerizerJS, License: MIT, npm: numerizer) by Roman Bolgov is a library for parsing English language string representations of numbers:

var numerizer = require('numerizer');
numerizer('forty two'); // '42'

It’s currently very simple, and doesn’t support browsers out of the box, but I like the fact the author has included Mocha tests. It’d work well alongside other libraries like Moment.js for providing intuitive text-based interfaces.

Obfuscate.js

Obfuscate.js (GitHub: miohtama / obfuscate.js, License: MIT) by Mikko Ohtamaa is a client-side script for replacing text on pages with nonsense that may be more desirable than private information. Mikko suggests this might be useful for making screenshots, so post-processing isn’t required to blur out personal information. The obfuscate function takes an optional selector, so either the entire body of a document can be obfuscated, or just the contents of a given selector.

It walks through each child node looking for text nodes, so it’s lightweight and doesn’t have any dependencies. It also tries to make the text look similar (at a glance) to the original text.

HHHHold, w2ui, Event Spy

15 Feb 2013 | By Alex Young | Comments | Tags node testing ui events jquery

HHHHold

HHHHold (GitHub: ThisIsJohnBrown / hhhhold-js, License: MIT) is a library for faking user generated content with hhhhold!:

Drop hhhhold URLs into your code for quick access to safe-for-work, attributed images from ffffound. Simulate real user content in your project.

It can be included as a client-side script for automatically generating random images whenever an image element has hhhhold.js/ in the src attribute. This allows various parameters to be passed to hhhhold, like the size of the image, or other options such as image brightness.

w2ui

w2ui

w2ui (GitHub: vitmalina / w2ui, License: MIT) by Vitali Malinouski is a UI library that is designed to be used with jQuery. The site has demos which use Bootstrap, but it doesn’t actually depend on Bootstrap as such – the project’s CSS files have been designed to work alongside other CSS libraries. There’s also a w2ui demo page that shows what the various widgets look like without Bootstrap.

So, what’s included? There are some widgets I find myself needing for a lot of projects that don’t come with Bootstrap, like sidebars and the data grid. There are also utility functions for validating values base64 encoding and decoding.

The JavaScript is all namespaced in w2utils and w2ui, and the CSS styles all have a w2ui- prefix, so it should be easy to drop it into a project to see what the widgets look like alongside existing functionality.

Event Spy

Event Spy (Google Code: event-spy, License: New BSD, Chrome Web Store: Event Spy) by Johan Laursen is a Chrome extension that adds event tracking to the developer tools. Only events with a listener will be displayed, and the target will be highlighted on the page.

The Chrome Web Store page for the project has a video that demonstrates the plugin in action, along with screenshots.

Backbone.js Tutorial: Testing with Mocks

14 Feb 2013 | By Alex Young | Comments | Tags backbone.js mvc node backgoog testing

Preparation

Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit 5b0a529
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone git@github.com:alexyoung/dailyjs-backbone-tutorial.git
cd dailyjs-backbone-tutorial
git reset --hard 5b0a529

Mocks

Last week I wrote about testing a custom Backbone.sync implementation using Sinon’s spies. This worked well in our situation where the transport layer isn’t necessarily pinned down – Sinon includes Fake XMLHttpRequest, but this won’t work with Google’s API as far as I know. This week I want to introduce another testing concept that Sinon provides: mocks.

Mocks are fake methods that allow expectations to be registered. Historically, you’ll find mocks being used in unit tests where I/O occurs. If you’re testing business logic you don’t need to check if a file was written or a network call was made, it’s often preferable to attach an expectation to make sure the appropriate API would have been called.

In Sinon, creating a mock returns an object that can be decorated with expectations. The API is chainable, so it’s low on boilerplate and high on readability. What you’re aiming to do is state “whenever this method is called, ensure it was called with these parameters”. This can be done through mocks by setting up expectations using matchers.

Matchers are similar to assertions – they can be used to check that arguments are everything from primitive types to instances of a constructor, or even literal values.

Last week we used spies to ensure Google’s API was accessed in the expected way. Mocks could be used for this as well. We don’t really care about the request so much as the fact a particular CRUD operation was requested. The signature for Backbone.gapiRequest is request, method, model, options – the method argument is generally what we’re interested in. Therefore, to set up an expectation that saving an existing task caused update to fire, we can use a mock with sinon.match.object:

var mock = sinon.mock(Backbone);
mock.expects('gapiRequest').once().withArgs(sinon.match.object, 'update');

// Do UI stuff to cause the task to be edited and the form to be submitted
mock.verify();

Mocks Compared to Spies

The previous example looked a lot like last week’s spies, and using spies for the same thing used less code. So, when should we use mocks, and when should we use spies? Mocks give you a fine-grained control over the order and behaviour of method calls. Spies have a different API which focuses on checking how callbacks or methods are used. If you were testing a method that accepts a callback, you could pass in a spy to see how the callback gets used. With a mock, the callback would be from the system under test, and you’d set up expectations on it.

When it comes to UI testing – triggering interface actions to invoke code, I find it’s easier to treat the entire Backbone stack as a whole and use spies to ensure the expected behaviour occurs. Rather than writing a test for each model, view, and collection, it makes more sense to drive the UI and hook into model or sync operations to verify the outcome.

In last week’s tests where lists were being tested, I probably wouldn’t use mocks because mocks should have a closer relationship to a given method under test. The kinds of tests we’re writing involve more than one method, so spies and assertions on the DOM make more sense.

Mock Example

A good place to use mocks is for testing app/js/gapi.js. Let’s say we’re interested in making sure gapiRequest gets called by Backbone.sync. We could use mocks:

test('gapiRequest is called by Backbone.sync', function() {
  var mock = sinon.mock(Backbone);
  mock.expects('gapiRequest').once();
  Backbone.sync('update', model, {});
  mock.verify();
});

This calls Backbone.sync to cause gapiRequest to be called once. This test doesn’t verify the behaviour of gapiRequest itself, just the fact it gets called.

One quirk of the custom Backbone.sync API is Task.prototype.get is called twice: once to fetch task’s ID, and another to get the list’s ID. We could test this with mocks if it was deemed important:

test('Ensure Task.prototype.get is called twice', function() {
  var mock = sinon.mock(model);

  mock.expects('get').twice().returns(model.id);
  Backbone.sync('update', model);
  mock.verify();
});

This uses the twice expectation with another mock.

Hopefully you’re starting to understand how mocks and spies differ. There’s another major part of Sinon, though, and that’s the stub API.

Stubs

Digging further into Backbone.gapiRequest, requests are expected to have an execute method which gets called to send data to Google’s API. Both spies and stubs can be used to test this using the yieldsTo method:

test('gapiRequest causes the execute callback to fire', function() {
  var spy = sinon.spy();
  sinon.stub(Backbone, 'gapiRequest').yieldsTo('execute', spy);
  Backbone.sync('update', model);

  assert.ok(spy.calledOnce);
  Backbone.gapiRequest.restore();
});

This test causes the following chain of events to occur:

  1. Backbone.sync calls Backbone.gapiRequest
  2. Backbone.gapiRequest receives an object with an execute property, which we’ve replaced with a spy
  3. Backbone.gapiRequest calls this execute method, therefore satisfying assert.ok(spy.calledOnce)

Putting these ideas together can be used to make sure the right success or error callbacks are triggered after a request has completed:

test('Errors get called', function() {
  var spy = sinon.spy()
    , options = { error: spy }
    ;

  // Stub the internal update method that would usually come from Google
  sinon.stub(gapi.client.tasks.tasks, 'update').returns({
    execute: sinon.stub().yields(options)
  });

  // Invoke a sync with a fake model and the options with the error callback
  Backbone.sync('update', model, options);

  assert.ok(spy.calledOnce);
  gapi.client.tasks.tasks.update.restore();
});

This test makes sure error gets called by using a spy, and it also stubs out gapi.client.tasks.tasks.update with our own object. This object has an execute property which causes the callback inside gapiRequest to run, and ultimately call error.

Clearing Up

I’ve written a test suite for tasks. It’s based on last week’s tests so there isn’t really anything new, apart from the teardown method:

setup(function() {
  /// ...

  spyUpdate = sinon.spy(gapi.client.tasks.tasks, 'update')
  spyCreate = sinon.spy(gapi.client.tasks.tasks, 'insert')
  spyDelete = sinon.spy(gapi.client.tasks.tasks, 'delete')

  // ...
});

teardown(function() {
  gapi.client.tasks.tasks.update.restore();
  gapi.client.tasks.tasks.insert.restore();
  gapi.client.tasks.tasks.delete.restore();
});

I’ve found this pattern is better than calling reset, because it’s easy to attempt to wrap objects more than once when multiple test files are loaded.

Writing Good Tests with Sinon

Sinon might look like a small library that you can drop into Mocha, Jasmine, or QUnit, but there’s an art to writing good Sinon tests. Sinon’s documentation has some explanation of when exactly spies, mocks, and stubs are useful, but there is a subjective factor at play particularly when it comes to deciding whether a test is best written with a mock or a stub.

A few tips I’ve found useful are:

  • Spies are great for the times when you want to test the entire application, rather than a specific class or method
  • Stubs come in handy when there are methods you don’t want run or want to force execution down a given path
  • Mocks are good for testing specific methods
  • A single mock per test case should be used
  • You should use restore() after using spies and stubs, it’s easy to forget and causes “double wrap” errors

Summary

Stylistically spies, stubs, and mocks are very different, but they’re vexingly similar until you’ve had some practice with Sinon. There have been mock vs. stub discussions on the Sinon.JS Google Group, so it’s probably best to ask Christian on that group if you’re struggling get Sinon to do what you want.

The source for this tutorial can be found in alexyoung / dailyjs-backbone-tutorial, commit 0c6de32.

Node Roundup: 0.8.19, 0.9.9, Peer Dependencies, hapi, node-treap

13 Feb 2013 | By Alex Young | Comments | Tags node modules frameworks web
You can send in your Node projects for review through our contact form.

Node 0.8.19, 0.9.9, and Peer Dependencies

Node 0.8.19 was released last week and this version includes an update for npm that supports peer dependencies. I’m excited about this feature, and I’ll be interested to see how it pans out over time. Basically, you can now specify dependencies for “plugins”. Think jQuery plugins, or in a Node project Grunt plugins are a good example.

This will require plugin authors to update their package.json files with a peerDependencies property, but it should make managing things like Express middleware and Grunt easier in the future. I already find npm’s dependency management relatively stress-free, and this seems like a step in the right direction.

Also, Node 0.9.9 was also released last week, which now features a streams2-powered tls module.

hapi

hapi

hapi (GitHub: walmartlabs / hapi, License: LICENSE, npm: hapi) from Walmart Labs is a framework for building RESTful API services. There are already a few solid RESTful API modules for Node (like restify), so hapi looks to be building on that concept rather than being an MVC web framework.

There’s a basic example that provides an overview of the API:

var Hapi = require('hapi');

// Create a server with a host, port, and options
var server = new Hapi.Server('localhost', 8080);

// Define the route
var hello = {
  handler: function(request) {
    request.reply({ greeting: 'hello world' });
  }
};

// Add the route
server.addRoute({
  method : 'GET',
  path : '/hello',
  config : hello
});

// Start the server
server.start();

Gone is the req, res pattern (which comes from Node’s core modules, not Connect). The hapi API documentation is extremely detailed, and includes examples for the main features. The routing API does seem extremely flexible, but it’s hard to judge it without seeing a large hapi application.

hapi, a Prologue by Eran Hammer is a detailed post that compares hapi to Express, which is useful if you’re familiar with Express. Eran writes:

We also had some bad experience with Express’ lack of true extensibility model. Express was a pleasure and easy to use 2 years ago with a limited set of middleware and very little interdependencies among them. But with a long list of chained middleware, we found hard to debug problems when we simply changed the order in which middleware modules were being loaded.

I’ve always thought the answer to this was to make smaller, interconnected services. Rather than a large Express application with complex middleware, shouldn’t we be using multiple Express applications that communicate with each other? Technically Express could be a veneer on top of a more complex architecture.

Eran brings up other points as well, but it’s difficult to say how well hapi or Express satisfy the goals of building large production web applications because people building things at that level don’t release gory details about their solutions to architectural problems. I occasionally run into a friend who works on a Rails project with thousands of models. It doesn’t work very well. But where are concrete details on real solutions to the problem of scaling business logic?

node-treap

node-treap (GitHub: brenden / node-treap, License: MIT, npm: treap) by Brenden Kokoszka is a Treap implementation. A treap is a self-balanced binary search tree. Once a tree has been created, keys can be added with data objects:

var treap = require('treap');
var t = treap.create();

// Insert some keys, augmented with data fields
t.insert(5, { foo: 12 });
t.insert(2, { foo: 8 });
t.insert(7, { foo: 1000 });

Then elements can be fetched and removed:

var a = t.find(5);
t.remove(a); // by reference to node
t.remove(7); // by key

Brenden has included tests, and each API method has documentation in the readme. He’s included some notes on what treaps are, so you don’t need to be fresh off the back of a computer science degree to figure out what the module does.

jQuery Roundup: Formwin, Three Sixty Slider, slideToucher

12 Feb 2013 | By Alex Young | Comments | Tags jquery plugins animations forms ui
Note: You can send your plugins and articles in for review through our contact form.

Formwin

Formwin (GitHub: rocco / formwin, License: MIT) by Rocco Georgi started as a fork of Uniform, but is now very different. It removes legacy browser support (IE8+) and relies on CSS for things like rounded corners.

The required markup is documented in the project’s readme. In general it relies on labels and spans:

<label class="formwintexts">
  <span>Label Text</span>
  <input type="text" name="yourinput">
</label>

It must be initialised to be used on a page, either with $.formwin.init(); or by setting $.formwinSettings. The init method accepts several options for configuring which CSS classes get used for things like active elements and hovering. This is similar to Uniform, and makes it extremely easy to drop into an existing project.

Three Sixty Slider

360

Three Sixty Slider (GitHub: creativeaura / threesixty-slider, License: MIT/GPL) by Gaurav Jassal allows multiple images to be displayed to give the illusion of multiple viewing angles. It features smooth animations, mouse and touchscreen support, and has a lot of tweakable options.

Basic usage is like this:

$('.product1').ThreeSixty({
  totalFrames: 72, // Total no. of image you have for 360 slider
  endFrame: 72, // end frame for the auto spin animation
  currentFrame: 1, // This the start frame for auto spin
  imgList: '.threesixty_images', // selector for image list
  progress: '.spinner', // selector to show the loading progress
  imagePath:'/assets/product1/', // path of the image assets
  filePrefix: 'ipod-', // file prefix if any
  ext: '.jpg', // extension for the assets
  height: 265,
  width: 400,
  navigation: true
});

It’ll figure out the image names based on the settings, so the markup doesn’t need to include lots of img tags. There’s a live demo on creativeaura.github.com/threesixty-slider/.

slideToucher

slideToucher (GitHub: Yuripetusko / slideToucher, License: MIT) by Yuri Petusko is a swipe gesture plugin that is designed to be high performance. It supports horizontal and vertical swipes, and uses translate3d to produce smooth animations where available.

It expects markup with the slide and row classes, and is invoked with $(selector).slideToucher({ vertical: true, horizontal: true });. The author has posted a demo here: yuripetusko.github.com/slideToucher/.

Numeric JavaScript, howler.js, depot.js

11 Feb 2013 | By Alex Young | Comments | Tags localStorage html5 mathematics audio

Numeric JavaScript

Numeric JavaScript (GitHub: sloisel / numeric, License: MIT) by Sébastien Loisel is a library that provides tools for matrix and vector calculations, convex optimisation, and linear programming. This library was sent in by Emil Bay, who uses it for computationally intensive tasks like genetic programming and AI. Emil says it’s extremely fast, and the Numeric author has some detailed benchmarks of Numeric with comparisons against Closure and Sylvester.

howler.js

howler.js

howler.js (GitHub: goldfire / howler.js, License: MIT) by James Simpson and GoldFire Studios is an audio library that works with Web Audio and HTML5 Audio. Like similar libraries, it can automatically load the right file format for a given browser, but also comes with a bevy of other features as well. It has an event-based API, and methods like fadeIn for handling some of the basic tasks you’ll face when working with audio.

It implements a cache pool and automatically fetches the audio files, which explains why it seemed so fast when I played around with the examples. It’s implemented without any dependencies, and I noticed the source was consistently formatted and easy to follow.

depot.js

depot.js (GitHub: mkuklis / depot.js, License: MIT, bower: depot) by Michal Kuklis is a localStorage wrapper that can be used with CommonJS or AMD, but also works with plain-old script tags. To use it, define a store and then call methods on the store’s instance:

var todoStore = depot('todos');

todoStore.save({ title: 'todo1' });
todoStore.updateAll({ completed: false });

// Fetch all:
todoStore.all();

It comes with Mocha tests which can be run with PhantomJS.

voxel.js, holla, Blitz, OneJS 2.0

08 Feb 2013 | By Alex Young | Comments | Tags graphics games webrtc WebSocket frameworks modules

voxel.js

voxel.js

When I was at BathCamp this week, Andrew Nesbitt mentioned voxel.js – a collection of projects for building browser-based 3D games. The core components were written by Max Ogden and James Halliday, take a look at voxel-engine (GitHub: maxogden / voxel-engine, License: BSD, npm: voxel-engine) if you want to see some code examples.

There are lots of demos on the voxel.js site, at the moment most of them support simple world traversal and the removal of blocks just like Minecraft. The project also has add-ons which includes voxel-creature for adding NPCs and player-physics. A huge amount of effort has already gone into the project, and it was apparently inspired by the awesome 0 FPS blog posts about voxels.

holla

holla (GitHub: wearefractal / holla, License: MIT, npm: holla) from Fractal is a module for WebRTC signalling. The author calls it “WebRTC sugar” – compared to the underlying API the library’s use of methods like .pipe make it a lot easier to get the hang of.

It has some helpers for creating audio and video streams, and there’s a demo up at holla.jit.su that accesses your webcam and microphone.

Blitz

Blitz (GitHub: Blitz, License: Modified MIT) by Eli Snow can help safely extend objects, overload functions based on types and arguments, and provides some native type recognition across global contexts:

Unlike other frameworks that have one generic wrapper for every object, Blitz creates unique wrappers for every prototype. So, for example, instead of having one method replace that works only with Arrays we can have a replace method for Arrays another for HTMLElements and/or any other object type.

Some of the functionality is accessible through a chainable API, so you can do things like this:

// [35, 16]
blitz([35, 16, 21, 9]).length(2).value;

Function overloading works using blitz.overload, which accepts an object that lists types alongside target functions.

OneJS 2.0

Azer Koculu has updated OneJS to version 2.0. OneJS converts CommonJS modules to standalone, browser-compatible files. It now supports splitting bundles into multiple files, and loading them asynchronously. It also has a more flexible build system: you can use it from the command-line, package.json, or from within a Node script.

Backbone.js Tutorial: Spies, Stubs, and Mocks

07 Feb 2013 | By Alex Young | Comments | Tags backbone.js mvc node backgoog testing

Preparation

Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit 9691fc1
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone git@github.com:alexyoung/dailyjs-backbone-tutorial.git
cd dailyjs-backbone-tutorial
git reset --hard 9691fc1

Testing Custom Backbone.Sync APIs

The goal of this tutorial is to demonstrate how to write tests so they don’t need to use live APIs. The way this is usually done is through a technique known as mocking – when the application attempts to talk to the API, it will communicate with a special object that we can control.

When testing applications that revolve around a custom Backbone.sync implementation, you need to cleanly separate application testing from testing the remote API. We just want to test what we’re responsible for, and run tests without an Internet connection! If you look at the way app/js/gapi.js works, it relies on gapi.client which is provided by Google. This is an easy target for mocking – we can replace Google’s library with something that returns sample data instead.

Sinon.JS does all of this, and more. Sinon makes it easy to “spy” on the methods that would usually result from trying to connect to the remote server, and these spies can easily be plugged into suitable assertions.

The basic approach I employ for testing Backbone applications with Sinon is as follows:

  • Create spies for CRUD operations
  • Script the DOM to trigger the things I want to test
  • Ensure the spies have seen the expected calls
  • Check that the UI has been updated accordingly

Here’s an example in our application:

  • The user clicks ‘Add List’ and enters a list title
  • The form is submitted
  • Assert that gapi.client.tasks.tasklists has been called to insert the new list
  • Assert that a new list item has been added to the UI

In Mocha/Sinon, that could be expressed like this:

suite('Lists', function() {
  var spyUpdate = sinon.spy(gapi.client.tasks.tasklists, 'update')
    , spyCreate = sinon.spy(gapi.client.tasks.tasklists, 'insert')
    , spyDelete = sinon.spy(gapi.client.tasks.tasklists, 'delete')
    ;

  setup(function() {
    spyUpdate.reset();
    spyCreate.reset();
  });

  test('Creating a list', function() {
    // TODO: Do the DOM stuff for creating a list
    assert.equal(1, spyCreate.callCount);
  });

  test('Editing a list', function() {
    // TODO: Do the DOM stuff for editing a list
    assert.equal(1, spyUpdate.callCount);
  });

  // Example: Abstraction for testing
  test('Deleting a list', function() {
    // TODO: Do the DOM stuff for deleting a list
  });
});

Save this as test/lists.test.js and add lists.test.js as a script tag to test/index.html (after app.test.js).

Sinon spies are created with sinon.spy(). In this example I’ve provided two arguments to sinon.spy, an object and a method. This causes Sinon to replace the method with a wrapped version that can count the number of times it gets called. It behaves like the original method, but makes testing things easier.

Spies can be invoked in other ways, but in this tutorial I’m just going to focus on this particular pattern. In general this is the part of Sinon that I find myself using the most when it comes to testing Backbone applications.

Housekeeping

In my quest to keep these tutorials clear and simple, I noticed I made some changes to the app which caused the tests to break. That means you’ll need to do a bit of housekeeping to get the tests to work correctly.

First, let’s get the application views loaded inside a container instead of overwriting the entire body. Mocha needs a div to display test results, and the last version of the app caused it to be overwritten when the tests urn.

Add a new div to app/index.html and test/index.html just after the body opening tag:

<div id="todo-app"></div>

You can hide this div in the tests if you like, setting display: none won’t break anything. This change also requires that app/js/views/app.js is updated to use #todo-app instead of body for the el property (near the top of the file).

You’ll also need to add some new script tags to test/index.html:

<script src="lib/sinon.js"></script>
<script src="fixtures/gapi.js"></script>

There’s one thing left to do in test/index.html – change the way the Mocha tests are invoked at the bottom of the file:

<script>
  // Only run the tests when the application is ready
  require(['app'], function() {
    bTask.apiManager.on('ready', function() { mocha.run(); });
  });
</script>

This is a good way to make sure the tests run only when the application’s dependencies have all loaded and have been evaluated.

Download Sinon.JS

This is the version of Sinon.JS that I’ve used: sinon-1.5.2.js. Save it to test/lib and create the lib/ directory if it doesn’t already exist.

Mocking Google’s API

To mock Google’s API, I’ve simply overwritten the library it provides with my own object that runs the expected callbacks with suitable test data. I created this test data by using the app and looking at the Network tab in WebKit Inspector, so it’s based on a subset of my actual to-do lists and tasks.

You can use my test data if you want to follow this tutorial instead of checking out the full source from GitHub: alexyoung/4730178 (gapi.js).

Save it to test/fixtures/gapi.js, creating the directory if required.

Now when app/js/gapi.js calls gapi.client.load and logs in with OAuth2, it’ll actually receive fake user data. I’ve used similar data structures to real values, but I’ve removed a few things like etags to make it easier to read.

Testing Lists

Now test/lists.test.js can be finished off. Here’s one way to test that lists are created correctly:

test('Creating a list', function() {
  var $el = bTask.views.app.$el
    , listName = 'Example list';

  // Show the add list form
  $el.find('#add-list-button').click();

  // Fill out a value for the new list's title
  $el.find('#list_title').val(listName);
  $el.find('#list_title').parents('form').first().submit();

  // Make sure the spy has seen a call for a list being created
  assert.equal(1, spyCreate.callCount);

  // Ensure the expected UI element has been added
  assert.equal(listName, $('.list-menu-item:last').text().trim());
});

This test uses jQuery to click the button that opens the list add/edit form, then fills out a title, and subsequently submits the form. That will cause Backbone.sync to run, and call an insert operation from Google’s API. Because I’ve replaced gapi it will call the fixture instead, and since Sinon is spying on it we’ll get an incremented call count. Boom!

Editing a list is practically the same:

test('Editing a list', function() {
  var $el = bTask.views.app.$el;

  // Show the edit list form
  $el.find('.list-menu-item:first').click();
  $el.find('#edit-list-button').click();

  $el.find('#list_title').val('Edited list');
  $el.find('#list_title').parents('form').first().submit();

  assert.equal(1, spyUpdate.callCount);
  assert.equal('Edited list', $('.list-menu-item:first').text().trim());
});

This time an update API call will fire rather than an insert.

There is one curious wrinkle in these tests – handling delete. I’ve used a confirm dialog instead of a fancy modern modal widget, which means the tests will cause a confirm dialog to appear. It’s possible to stop the dialog from appearing by replacing confirm:

test('Deleting a list', function() {
  var $el = bTask.views.app.$el;

  // Automatically accept the confirmation
  window.confirm = function() { return true; };

  // Show the edit list form
  $el.find('#edit-list-button').click();

  // Click the list delete button
  $el.find('.delete-list').click();

  assert.equal(1, spyDelete.callCount);
});

The problem with this is it’ll cause Mocha to see a “global leak”. You can add confirm to test/setup.js to prevent that error. However, ideally we shouldn’t need to do this – coupling tests to implementation details like this is usually a bad idea. It would be better to design the application to allow the dialog to be turned off to better support testing. I don’t usually mind adapting applications to make them easier to test, it can be beneficial in the long run.

To run these tests, launch the Node app with npm start and then visit http://localhost:8080/test/ in your browser. Make sure you type the URL correctly or the tests won’t work due to the use of relative paths.

Summary

In this part you’ve seen how to write tests for an application based around a custom Backbone.sync implementation. Sinon spies provide the perfect way to wrap internal parts of a Backbone application to write succinct test suites.

This particular type of testing focuses on the business logic represented by Backbone models, collections, and views. This ultimately helps in maintaining your client-side applications as they grow and change over time.

The majority of the source for this tutorial can be found in alexyoung / dailyjs-backbone-tutorial, commit 5b0a529. The final commit was 45dd59.

Node Roundup: GNOME, fs, procjs

06 Feb 2013 | By Alex Young | Comments | Tags node modules gnome desktop bindings browser
You can send in your Node projects for review through our contact form.

JavaScript and GNOME

GNOME now recommends JavaScript for authoring GNOME applications. For information on what this means for the near future of GNOME desktop development, see JavaScript in GNOME. Although it looks like they’re using SpiderMonkey rather than Node, Jérémy Lal sent in an email detailing his positive experiences with node-gir (GitHub: creationix / node-gir, npm: gir) by Tim Caswell which provides bindings for GObject Introspection.

These bindings can be used to make dynamic calls to any library that has GI annotations installed – Jérémy said he was using it to generate PDFs from HTML.

Component: fs

fs (GitHub: matthewp / fs, License: MIT, component: matthewp/fs) by Matthew Phillips is a component that brings Node’s fs module to the browser. It’s designed to be cross-browser, with the FileSystem API for Chrome and IndexedDB for Firefox and Internet Explorer.

procjs

procjs (GitHub: vzaccaria / procjs, License: MIT, npm: procjs) by Vittorio Zaccaria is a set of command-line utilities for getting JSON representations from the output of ps. It also comes with a REST server that provides a JSON API for the same data.

The project is built with LiveScript, and can be invoked with jsps along with several arguments.

jQuery Roundup: 1.9.1, jui_datagrid, jQuery Waiting, jquery.defer

05 Feb 2013 | By Alex Young | Comments | Tags jquery plugins animations database backbone.js
Note: You can send your plugins and articles in for review through our contact form.

jQuery 1.9.1

jQuery 1.9.1 has been released:

Whether you’re using 1.9.0 or using an older version, these are the droids you’re looking for.

There are bug fixes for Chrome, IE, and Safari, and a few small enhancements like #13150: Be able to determine if $.Callback() has functions.

jui_datagrid

jui_datagrid

jui_datagrid (GitHub: pontikis / jui_datagrid, License: MIT) by Christos Pontikis is a one of those “rich table” plugins that makes tabular data sortable, editable, and so on. It has a specific focus on editing server-side data, and will work with JSON data out of the box. It supports multiple instances on the same page, jQuery UI themes, localisation, and a modular design that makes adding new data filters easier.

There is a a demo of jui_datagrid that shows the major features.

jQuery Waiting

jQuery Waiting (GitHub: trentrichardson / jQuery-Waiting, License: MIT/GPL) by Trent Richardson is a plugin for displaying spinners that’s designed to be cross-browser. Instead of relying on modern CSS animations, it simply switches CSS classes on sets of elements. It has a namespaced event-based API, so you can see when the control is enabled, starts playing, and so on:

// Initialise
$el.waiting({});

// Play
$el.waiting('play');

// Event example
$el.bind('play.waiting', function(e){});

jquery.defer

jquery.defer/jquery.undefer (GitHub: wheresrhys / jquery.defer, License: MIT) by Rhys Evans are a pair of utility methods for making an object’s methods wait until a deferred object has resolved. The example Rhys provides of this in action is lazy loading Google Maps:

$.defer(GoogleMaps.prototype, _mapsLoaded, {exclude: 'init'});

Rhys also sent in Backbone Namespaced Events (GitHub: wheresrhys / backbone.namespaced-events, License: MIT), which uses the syntax of namespaced events for Backbone’s custom events implementation. To use namespaced events, call Backbone.extend(obj, Backbone.NamespacedEvents) on a Backbone object instance. Alternatively, Backbone.NamespacedEvents.overwriteNativeEvents() can be called to use it everywhere.

Meet the New Stack, Same as the Old Stack

04 Feb 2013 | By Alex Young | Comments | Tags components twitter google jquery

Five years ago, if you asked any client-side developer which library or framework to use the most likely answer would have been jQuery. Since then, client-side development has become far more complex. A friendly wrapper for DOM programming and a browser compatibility layer isn’t enough to help us write modern applications.

Starting a project today involves selecting a package manager, module system, build system, templating language, data binding library, a sync layer, a widget library, and a test framework. And it doesn’t end there: many of us also regularly use a dynamic CSS language, and mobile SDKs.

I wouldn’t say this is a negative trend, but with large companies backing or offering solutions to these problems, making an informed decision on each of these technologies is difficult. And soon the major players will offer a complete set of tools and libraries to satisfy each requirement: we’ll be back to something that looks like the original monolithic frameworks.

Until recently, starting a client-side web application might have looked like this:

  • Module system: RequireJS, AMD
  • Build system: RequireJS (r.js)
  • Templates: text.js
  • Data binding: Backbone.js
  • Sync: Backbone.js
  • Widgets: Bootstrap
  • Test framework: QUnit

There are other popular choices for each of these bullets, of course: Zurb Foundation is a popular front-end framework, and I’ve used Mocha instead of QUnit since Mocha appeared. I also like Knockout for data binding, because the two-way declarative bindings are easy to get the hang of.

These libraries are not interchangeable once a project has been started – Bootstrap uses different CSS classes to jQuery UI, for example. The major difficulty is keeping libraries up to date, particularly if they have a lot of dependencies.

And that’s when you need a package manager. Using a package manager can make the choices even more fine grained, because managing each library and its dependencies is easier. Switching to something like Component is one option, which can lead to a totally new stack:

  • Package manager: Component
  • Module system: CommonJS
  • Build system: Component
  • Templates: Take your pick
  • Data binding: Reactive or Rivets (you could easily use Knockout or Backbone though)
  • Sync: component/model can communicate with JSON APIs
  • Widgets: Componentised UI widgets are popular
  • Test framework: test/assert, Mocha

Bootstrap and Zurb Foundation can be provided as components, there are projects on GitHub to do this. I’ve tried to design projects 100% around components without these larger libraries, and it was a huge amount of work. It may get easier with time, or once the right balance of functionality is found. I’ve noticed there are some “meta packages” that exist to group commonly used dependencies together.

You’ll notice I haven’t mentioned AngularJS yet. The reason for that is AngularJS is now compatible with Closure Library, which makes it possible to use an almost 100% Google-powered development stack:

  • Package manager: None (to my knowledge)
  • Module system: Closure Library modules
  • Build system: ClosureBuilder
  • Templates: AngularJS
  • Data binding: AngularJS
  • Sync: AngularJS services
  • Widgets: Closure Library
  • Test framework: Closure Library testing

While Closure Library is more like the “last generation” monolithic frameworks, each module can be loaded separately, so you don’t need to use the whole thing. You could make a project with Closure Library, ClosureBuilder, Backbone.js, and Boostrap if you wanted. You could also go the other way: deploy a Go/Python/Java app to App Engine that’s built on Closure Library and AngularJS. Google effectively provides the entire stack, including server-side development, data storage, user authentication, billing, and client-side development.

Recently we’ve also seen a huge amount of open source projects coming out of Twitter. A pure Twitter stack looks like this:

  • Package manager: Bower
  • Module system: Flight/AMD
  • Build system:
  • Templates: Hogan.js
  • Data binding: Flight
  • Sync:
  • Widgets: Bootstrap
  • Test framework:

Using components through Flight could satisfy the other dependencies as well. It wouldn’t be difficult to use Flight as a test runner with a suitable assertion library, although the authors use Jasmine and PhantomJS at the moment.

Let’s not forget, however, the incredible features provided by Dojo. Compare Google and Twitter’s client-side stacks to this:

  • Package manager: Dojo Packages, CPM
  • Module system: AMD
  • Build system: dojoBuild
  • Templates: Dijit templates
  • Data binding: dojo.store.Observable
  • Sync: Dojo Object Store
  • Widgets: Dijit
  • Test framework: D.O.H.

Dojo has the entire client-side stack covered, and also includes many more features that I haven’t mentioned here. YUI is comparable:

On the surface, it seems like Twitter’s developers aim to create something more like Component, where each piece of the client-side puzzle can be picked à la carte. Closure Library is more like the older, monolithic model used by YUI, where a custom module system is used, and modules are harder to reuse without the base framework.

The question is, can these larger companies support modern, “componentised” client-side development, or is it easier to offer a consolidated monolithic framework? Projects like AngularJS and Flight suggest to me that developers within Twitter and Google want to promote that approach, while how that fits into the wider organisation remains to be seen. Will we see AngularJS alongside Closure Library in the Python, Java, and Go App Engine documentation, or will it remain a useful library that exists outside of Google’s core developer offering?

An interesting third option is Yeoman. This is another project from Google that provides a selection of smaller libraries to kick-start development:

  • Package manager: yeoman
  • Module system: RequireJS, AMD
  • Build system: Grunt
  • Widgets: Bootstrap

Yeoman generates application templates by helping you select various options, and then gives you a build system so you can easily generate something deployable. It doesn’t enforce decisions like template language or CoffeeScript, but provides a harness for installing packages, and building and testing your application.

Conclusion

Client-side development is changing. It’s no-longer enough to learn a small amount of jQuery – an entire client-side stack is becoming the norm. The libraries that plugged the gaps in jQuery and its clones now have serious competition from tech giants.

Flight, ComponentJS

01 Feb 2013 | By Alex Young | Comments | Tags components twitter

Flight

Flight by Twitter

What’s the most sensible thing to do when there’s an established project called Component that aims to make client-side development more modular through reusable components? Invent something else that yet again overloads this increasingly overused term! If you weren’t already confused about components, then get ready to unlearn everything you’ve learned.

I was expecting this, however. After seeing Bower, I felt like it really needed another project to further abstract client-side development, and that appears to have been satisfied by the release of Flight.

Flight (GitHub: twitter / flight, License: MIT, bower: flight) from Twitter is a new framework that maps “behaviour” to DOM nodes. To do this it brings together a few things you’re probably already familiar with: multiple inheritance (mixins), the AMD pattern, and the observer pattern. The supplied Flight Mail example app is built using RequireJS, so you can see why AMD is relevant here.

Flight apps look a bit like Backbone/RequireJS apps. They both use AMD and rely on events for communication. However, Flight is deceptively different. Which isn’t necessarily a bad thing, because we’ve argued about MVC/MVVC/MV* for a long time at this point.

The authors define a component in Flight as a constructor with properties mixed in. Each component has event handling, and is eventually attached to a DOM node. They can’t refer to each other directly – communication is purely through events.

That last point is potentially important, because it demonstrates the authors have a good sense of one of the major issues with client-side development: writing reusable, decoupled code. Given that Angus Croll and Dan Webb are contributors to this project, it’s not surprising that there’s a focus on composition.

Although Flight already has a bazillion stars on GitHub (Twitter™!), it’s going to face the same hurdles in adoption as Backbone did: the learning curve. Casual JavaScript developers are going to see AMD, mixin, and “advice”, and struggle to understand how to relate these concepts to their existing development practices.

However, the project has some great ideas and has serious pedigree behind it. It’s different enough to Backbone, AngularJS, and Knockout, so I’ve welcomed it to my rapidly growing client-side development toy box.

ComponentJS

ComponentJS (GitHub: rse / componentjs, License: MPL) by Ralf S. Engelschall is a totally unrelated project that just happens to involve the term “component”. This library is aimed at creating dialog-based hierarchical interfaces for single page apps. It has an OO library, a graphical debugger, UI toolkit, state transitions, and communication using events.

According to the author, this project has been developed since 2009 (although the domain name was created a year ago), so the naming clash is purely coincidental from what I can tell. It has extremely detailed API documentation, and a demo with browsable source code.

Backbone.js Tutorial: Oh No, Not More Tasks

31 Jan 2013 | By Alex Young | Comments | Tags backbone.js mvc node backgoog

Preparation

Before starting this tutorial, you’ll need the following:

  • alexyoung / dailyjs-backbone-tutorial at commit 0491ad
  • The API key from part 2
  • The “Client ID” key from part 2
  • Update app/js/config.js with your keys (if you’ve checked out my source)

To check out the source, run the following commands (or use a suitable Git GUI tool):

git clone git@github.com:alexyoung/dailyjs-backbone-tutorial.git
cd dailyjs-backbone-tutorial
git reset --hard 0491ad

Tasks Continued

A preview of today's tutorial.

The application is now displaying lists of tasks, but you can’t yet interact with them. This tutorial will cover:

  • Adding tasks
  • Editing tasks
  • Deleting tasks
  • Toggling tasks

Most of this content builds on what we did with lists, but it’ll be good practice if you’re looking for more experience with Backbone.

Adding Tasks

Open app/js/views/tasks/index.js and ensure there’s an event binding for addTask:

events: {
  'submit .add-task': 'addTask'
},

In the initialize method, add a listener to this class’s collection:

initialize: function() {
  this.children = [];
  this.collection.on('add', this.renderTask, this);
},

This will make it automatically render new tasks when they’re added to the collection.

The addTask method should call Task.prototype.save to persist the task using Google’s API, after instantiating it with a reference to the current list. It should also render the task once it’s been saved. I’ve passed in { at: 0} because Google Tasks places new tasks at the top of a list. Notice that I prefer to only add tasks once they’ve been successfully saved – that makes this application always require an Internet connection. It may be preferable to save to a local database and sync with Google later, but we’re not going to do that here.

addTask: function() {
  var $input = this.$el.find('input[name="title"]')
    , task = new this.collection.model({ tasklist: this.model.get('id') })
    , self = this
    ;

  task.save({ title: $input.val() }, {
    success: function() {
      self.collection.add(task, { at: 0 });
    }
  });
  $input.val('');

  return false;
},

renderTask: function(task, list, options) {
  var item = new TaskView({ model: task, parentView: this })
    , $el = this.$el.find('#task-list');
  if (options && options.at === 0) {
    $el.prepend(item.render().el);
  } else {
    $el.append(item.render().el);
  }
  this.children.push(item);
},

The renderTask method will receive the options argument, and it uses it to determine how to add the task to the list. The reason I don’t just prepend new tasks is the render method can now be refactored to use this method:

render: function() {
  this.$el.html(this.template());

  var $el = this.$el.find('#task-list')
    , self = this;

  this.collection.fetch({ data: { tasklist: this.model.get('id') }, success: function() {
    self.collection.each(function(task) {
      task.set('tasklist', self.model.get('id'));
      self.renderTask(task);
    });
  }});
}

Open app/js/views/lists/menuitem.js and make it pass in a Tasks collection in the open method where it instantiates bTask.views.tasksIndexView:

bTask.views.tasksIndexView = new TasksIndexView({ collection: new Tasks({ tasklist: this.model.get('id') }), model: this.model });

You’ll need to change the define statement at the top of the file to include the Tasks collection:

define(['text!templates/lists/menuitem.html', 'views/tasks/index', 'collections/tasks'], function(template, TasksIndexView, Tasks) {

Due to how Google’s API works, you’ll need to make a small change to app/js/gapi.js to insert a tasklist ID into the requestContent payload:

Backbone.sync = function(method, model, options) {
  var requestContent = {};
  options || (options = {});

  switch (model.url) {
    case 'tasks':
      requestContent.task = model.get('id');
      requestContent.tasklist = model.get('tasklist');
    break;

Adding tasks should now work – there’s no need for a new template because it’s already been added as part of last week’s tutorial.

Editing Tasks

To edit tasks, a few things are needed:

  • A suitable form template
  • A Backbone.View
  • Event handling for saving the task

Here’s the template, which should be saved to app/js/templates/tasks/edit.html:

<fieldset>
  <legend>
    Task Properties
    <a href="#" data-task-id="" class="pull-right delete-task btn"><i class="icon-trash"></i></a>
  </legend>
  <div class="control-group">
    <label for="task_title">Title</label>
    <input type="text" class="input-block-level" name="title" id="task_title" value="" placeholder="The task's title">
  </div>
  <div class="control-group">
    <label class="radio"><input type="radio" name="status" value="needsAction" > Needs action</label>
    <label class="radio"><input type="radio" name="status" value="completed" > Complete</label>
  </div>
  </div>
  <div class="control-group">
    <label for="task_notes">Notes</label>
    <textarea class="input-block-level" name="notes" id="task_notes" placeholder="Notes about this task"></textarea>
  </div>
</fieldset>
<div class="form-actions">
  <button type="submit" class="btn btn-primary">Save Changes</button>
  <button class="cancel btn">Close</button>
</div>

I’ve included all of the usual Bootstrap classes and markup in this form fragment so it will look nice when it’s rendered.

The corresponding view (app/js/views/tasks/edit.js) should look like this:

define(['text!templates/tasks/edit.html'], function(template) {
  var TaskEditView = Backbone.View.extend({
    tagName: 'form',
    className: 'well edit-task',
    template: _.template(template),

    events: {
      'submit': 'submit'
    , 'click .cancel': 'cancel'
    },

    initialize: function() {
      this.model.on('change', this.render, this);
    },

    render: function() {
      this.$el.html(this.template(this.model.toJSON()));
      return this;
    },

    submit: function() {
      var title = this.$el.find('input[name="title"]').val()
        , notes = this.$el.find('textarea[name="notes"]').val()
        , status = this.$el.find('input[name="status"]:checked').val()
        ;

      this.model.set('title', title);
      this.model.set('notes', notes);

      if (status !== this.model.get('status')) {
        this.model.set('status', status);
        if (status === 'needsAction') {
          this.model.set('completed', null);
        }
      }

      this.model.save();
      return false;
    },

    cancel: function() {
      this.remove();
      return false;
    }
  });

  return TaskEditView;
});

This sets up a submit event for catching the form submission, and also an event for closing the form, which is what the cancel method is for.

Now add a method to app/js/views/tasks/index.js that invokes TaskEditView:

editTask: function(task) {
  if (this.taskEditView) {
    this.taskEditView.remove();
  }
  this.taskEditView = new TaskEditView({ model: task });
  this.$el.find('#selected-task').append(this.taskEditView.render().el);
}

And make sure it loads TaskEditView:

define(['text!templates/tasks/index.html', 'views/tasks/task', 'views/tasks/edit', 'collections/tasks'], function(template, TaskView, TaskEditView, Tasks) {

This needs to be called by an individual task, so open app/js/views/tasks/task.js and add this to the open method:

this.parentView.editTask(this.model);

These two views have a lot of coupling between them, which makes TaskView difficult to reuse. However, is it likely that it’ll make sense to use it without TasksIndexView? That’s the kind of question you’ll ask yourself a lot when trying to write maintainable Backbone code.

Deleting Tasks

Add a destroy method to app/js/views/tasks/edit.js:

destroy: function() {
  this.model.destroy();
  return false;
}

Then bind the method to the event on the trash can icon (.delete-task) and also bind an event to the model being deleted:

events: {
  'submit': 'submit'
, 'click .cancel': 'cancel'
, 'click .delete-task': 'destroy'
},

initialize: function() {
  this.model.on('change', this.render, this);
  this.model.on('destroy', this.remove, this);
},

Toggling Tasks

Here’s the icing on the cake, toggling the task status! With this change, the app will really start to feel like a real to-do list app. Open app/js/views/tasks/task.js and add an event binding for the checkboxes in the list – a change event is required for this:

events: {
  'click': 'open'
, 'change .check-task': 'toggle'
},

Then the toggle method just needs to toggle the status attribute based on the checkbox’s state:

toggle: function() {
  var id = this.model.get('id')
    , $el = this.$el.find('.check-task')
    ;

  this.model.set('status', $el.attr('checked') ? 'completed' : 'needsAction');
  if (this.model.get('status') === 'needsAction') {
    this.model.set('completed', null);
  }

  this.model.save();
  return false;
}

Google’s nomenclature for the task state is completed and needsAction, which takes a bit of digging in the documentation to find out.

Summary

It’s taken a while to get this far, but working with unfamiliar APIs with their idiosyncrasies can take a lot of patience. And if you try running the code from this project, make sure you actually have some tasks and lists in Gmail already – it doesn’t work tell well without any. I’ll fix it later!

The full source for this tutorial can be found in alexyoung / dailyjs-backbone-tutorial, commit 9691fc1.

Node Roundup: 0.9.8, Queen, AssetViz

30 Jan 2013 | By Alex Young | Comments | Tags node modules graphics
You can send in your Node projects for review through our contact form.

0.9.8

Node 0.9.8 is out. This release includes an interesting patch from Jake Verbaten to support arbitrary objects in streams. Internally, streams now switch to objectMode when objects are detected. The unit tests illustrate how this works in practice:

test('can read objects from stream', function(t) {
  var r = fromArray([{ one: '1'}, { two: '2' }]);
  var v1 = r.read();
  var v2 = r.read();
  var v3 = r.read();

  assert.deepEqual(v1, { one: '1' });
  assert.deepEqual(v2, { two: '2' });
  assert.deepEqual(v3, null);
  t.end();
});

Notice how each read causes an object to be returned. Jake has been heavily involved with streams over the last year or two, with plenty of notable modules in his Raynos GitHub account.

Queen

Queen

Last week I wrote about Ozan Turgut’s Thrill project. The core component, which people seemed to find more interesting, was Queen (GitHub: turn / queen, License: Apache v2, npm: queen). Queen is a server that can run scripts on multiple browsers. This could be used for anything, not just for running tests which is what Thrill does.

Queen clients and servers have bidirectional communication, and Queen will detect and recover unresponsive browsers. It can target browsers by type, version, and OS, and run scripts via the command-line.

AssetViz

AssetViz on DailyJS

AssetViz (GitHub: Munter / assetviz, License: MIT, npm: assetviz) by Peter Müller is a command-line web application source code visualisation tool. It generates self-contained HTML files that show a visualisation of the site using D3.js.

The nodes that make up the visualisation can be dragged and will spring back into place, and you can also zoom using the mousewheel.

jQuery Roundup: QUnit 1.11, Knockout-jQueryUI, Tab Override, FilteredPaste.js

29 Jan 2013 | By Alex Young | Comments | Tags jquery plugins testing knockout keyboard paste
Note: You can send your plugins and articles in for review through our contact form.

QUnit 1.11

There’s a good post on the official jQuery blog about a change in direction to the QUnit project: QUnit 1.11 Release: A Look Back (and Forth). It discusses some of the history behind QUnit, and includes a survey where you can share how you’re using it.

Knockout-jQueryUI

Trying to integrate libraries like Knockout is sometimes confusing when you’re already using jQuery UI. To address this, Vas Gábor has created Knockout-jQueryUI (GitHub: gvas / knockout-jqueryui, License: MIT), which is a collection of Knockout bindings for jQuery UI widgets.

It’s small, comes with a build script and unit tests, and the author has provided full documentation on the project’s homepage.

Tab Override

Tab Override (GitHub: wjbryant / taboverride, License: MIT, bower: taboverride) by Bill Bryant allows the tab key to insert tabs in textarea elements. It also supports auto indent and multi-line tab insertion.

There’s also a jQuery Tab Override plugin, and both scripts are AMD-compatible. Bill has included QUnit tests, which actually simulate key presses to test the script’s various features.

FilteredPaste.js

FilteredPaste.js (GitHub: willemmulder / FilteredPaste.js, License: CC BY-SA 3.0) by Willem Mulder can be used to filter text when it’s pasted into textarea elements, or anything with the contenteditable attribute. Why is this useful? Well, Willem got tired of dealing with support requests when text pasted from Word into his CMS carried across unwanted formatting.

I find myself always using Paste and Match Style and wondering why this isn’t the default for the paste keyboard shortcut. The only time I’ve ever wanted to include formatting when pasting is when I make slides in Keynote/PowerPoint/Google Drive and want to include syntax highlighting in my examples. And that seems like an edge case if ever there was one!

Confused About Components

28 Jan 2013 | By Alex Young | Comments | Tags component.json package-management

Client-side development has been shifting away from monolithic libraries. While jQuery is still hugely popular, building projects from smaller libraries is increasingly common. To make this easier to deal with, projects have appeared to manage dependencies. These tools may be simple package managers, or a combination of a build tool and a package manager.

Recent History

Client-side package managers have been around for a few years now. An earlier example is CPM, which is a CommonJS package manager. Last year the subject got newfound attention from Node developers. There was a very public debate between Isaac Schlueter and TJ Holowaychuk in which several issues were covered:

  • Should client-side libraries use CommonJS modules or AMD?
  • Should npm or a new registry be used to store client-side libraries?
  • What should be included within client-side libraries?

“A component can be not just JavaScript, but CSS, images, fonts or other resources, or a component may be any combination of these. This is the main idea that I want to sell, we need to broaden modularity beyond just JavaScript source.”

TJ Holowaychuk, Components

“However, they should have the same level of modularity and order that we’ve been able to get with JavaScript packages using npm and Node.js. It’s a much stickier problem, because the deployment environment is so much more hostile.”

Isaac’s response

Shortly after this discussion, TJ released Component, which is a component manager. Components may include JavaScript, styles, and markup. Markup and styles are encouraged to be “structural”, allowing users to easily skin components. Examples of components include anything from generic JavaScript libraries – like a small mathematical helper function, to a fully-blown UI widget, like a date picker.

Component Workflows

What isn’t immediately obvious about Component is it builds client-side scripts by adding a CommonJS Modules/1.1 implementation – components are loaded using var lib = require('lib'). The upshot of this is there’s a learning curve to writing components, and also understanding how to make this work inside existing projects. Component is specifically designed to work regardless of server-side architecture, so you could use it with Django, Express, or Rails projects without special handling. Discovering how to use components inside these frameworks is the main point of friction when switching to a component-based workflow – do you structure your entire project around components? Or just use Component as a way of managing dependencies?

Contrast this to Bower. It’s more like earlier projects, which include Jam and Volo, and therefore offers less in terms of treating entire slices of the UI as reusable chunks. This is by design – Bower is meant to work with build tools rather than replace them.

Then there’s Ender, which provides an API for loading modules (CommonJS is supported), and is built on top of npm. Jam encourages the use of AMD and depends on RequireJS. Volo doesn’t depend on AMD modules, but encourages them to be used.

Bower could have effectively been: curl http://wherever-bower-repos-are.com | grep underscore | xargs git clone

TJ Holowaychuk, component Google Group

That’s how I initially felt when I first saw Bower, but Bower’s simplicity is a result of it being designed to work alongside client-side build tools. That means Bower will work with RequireJS, LoadBuilder, or even Sprockets. If you’re using a large server-side framework like Rails, then it might be easier to use Bower with the framework’s existing asset management tools.

Build Tools and Package Managers

As TJ hinted at above, the biggest problem facing client-side package managers is anybody can make one. It doesn’t take much effort to write a script that can fetch libraries from GitHub and then pipe them through UglifyJS.

Bower’s separation between package manager and build tools is insightful, but I also think the modular client-side development being pushed by Component is extremely important and makes client-side development better.

The State of Affairs

We’re now at the point where library authors are increasingly including a component.json file. How do you know if this is a Bower or Component manifest file? Let’s say I want to install pjax. It has a component.json, so I run component install defunkt/jquery-pjax. The result is an error:

error : Error: invalid component name "jquery"

It turns out pjax is designed to be used with Bower.

This situation is made more confusing because neither project addresses this fact. It’s almost like they don’t want to acknowledge each other. There’s a discussion about Bower in the component Google Group, but neither project is particularly helpful with regard to the differences between manifest files, given the decision to use the same name.

Is Bower run by evil Twitter imperialists hell-bent on wrestling component.json away from the Component project? Well, no, because Bower can be configured to use a different file name, and the projects have different goals. However, the clash is unfortunate, particularly for client-side developers who work closer to the design end of the design/coding spectrum, and may not even be aware that both projects exist.

It would be nice to see this issue addressed clearly in Bower and Component’s documentation. The fact Twitter has its logo on Bower doesn’t make it the best or correct project to use – I encourage you to try out both projects. Client-side development is more than just JavaScript files, so I feel like Component has a chance to reach critical mass if enough people get behind it.

Jargon

  • Package Manager: Fetch scripts from online sources like GitHub, taking dependencies into account.
  • Build Tool: Combine scripts and other assets together into something usable by browsers.
  • Module System: A way to programmatically load scripts when they’re needed.

Comparison Table

  Bower Component Ender Jam Volo
License MIT MIT MIT MIT MIT/New BSD
Appeared Sep 2012 Aug 2012 Apr 2011 May 2012 Oct 2010
Package Registry Server Wiki npm Server GitHub
Module Pattern   CommonJS CommonJS AMD AMD/CommonJS compatible