Asynchronous Resource Loading Part 5

2011-10-27 00:00:00 +0100 by Alex R. Young
*Let's Make a Framework* is an ongoing series about building a JavaScript framework from the ground up. These articles are tagged with [lmaf](http://dailyjs.com/tags.html#lmaf). The project we're creating is called [Turing](http://github.com/alexyoung/turing.js). Documentation is available at [turingjs.com](http://turingjs.com/).

Previous parts:

Creating a Preloading API

Last week I fleshed out turing.request so it can handle
preloading with XMLHttpRequest. It's time to look at how to
build an API that will allow scripts to load based on the required

I thought a lot about this and decided that an array is a good way to
model dependencies between scripts. Given this array:

  ['plugin.js', 'plugin2.js'],

then the following actions should occur:

These scripts can be loaded at any time, as long as they're executed in
the specified order. If the scripts are local, they can be preloaded
with XMLHttpRequest.

Preloading Test

To break this down into something implementable, I created a test first:

'test async queue loading': function() {
    ['/load-me.js?test4=4', '/load-me.js?test5=5'],
    ['/load-me.js?test6=6', '/load-me.js?test7=7'],
  ]).on('complete', function() {
  }).on('loaded', function(item) {
    if (item.src === '/load-me.js?test9=9') {
      assert.equal(typeof test4, 'undefined');

    if (item.src === '/load-me.js?test4=4') {
      assert.equal(test9, 9);
      assert.equal(test4, 4);
      assert.equal(typeof test6, 'undefined');
      assert.equal(typeof test8, 'undefined');

    if (item.src === '/load-me.js?test6=6') {
      assert.equal(test9, 9);
      assert.equal(test4, 4);
      assert.equal(test6, 6);
      assert.equal(typeof test8, 'undefined');

I originally wrote this with a callback like the old
require signature implied, but I realised that different
types of callbacks are required, so it felt natural to use something
based on events.

Using this event-based approach should make it easier to build
internally, but also makes a pretty rich API.

Modifying require

To work with this new signature (and event-based API), last week's
require implementation will have to be updated. Recall that
a setTimeout method was used to delay loading until the
head part of the document is ready. That makes returning a
value -- and thus chaining off require -- impossible.

To get around this, I extracted the setTimeout part and
made it into a method:

function runWhenReady(fn) {
  setTimeout(function() {
    if ('item' in appendTo) {
      if (!appendTo[0]) {
        return setTimeout(arguments.callee, 25);

      appendTo = appendTo[0];


When an array is passed to require we can safely assume
queuing is necessary. The old method signature is still available. In
the case when an array has been passed, a Queue object can
be returned to enable chaining:

turing.require = function(scriptSrc, options, fn) {
  options = options || {};
  fn = fn || function() {};

  if (turing.isArray(scriptSrc)) {
    return new Queue(scriptSrc);

  runWhenReady(function() {
    // Last week's code follows

The Queue Class

The Queue class is based on
turing.events.Emitter which is a simple event management
class, created in Let's Make a Framework: Custom
. The basic
algorithm works like this:

  1. Queue.prototype.parseQueue: Iterate over each item in
    the array to work out which scripts require preloading, and group them together based on the array groupings. Single script items will be added to a group that contains one item, so the array of arrays becomes normalised into something easy to manage.
  2. Queue.prototype.enqueue: Whenever
    parseQueue finds a script, call enqueue to add it to a sequentially indexed object.
  3. runQueue: Iterate over each group and preload each
    local script using XMLHttpRequest. Add an event emitter to the XMLHttpRequest callback to signal a preload event has completed.
  4. preload event: When this event is fired, mark the group
    item as 'preloaded' and start executing scripts if the whole group has finished preloading.
  5. complete event: This event is fired when all items
    have executed.

I'll explain these methods and events below.

Initialization Queue and Maintaining the Chain

To allow the on methods to be called after
turing.require, Queue has to use
runWhenReady itself, and proxy calls to
turing.events.Emitter and return this:

function Queue(sources) {
  this.sources = sources;
  this.events = new turing.events.Emitter();
  this.queue = [];
  this.currentGroup = 0;
  this.groups = {};
  this.groupKeys = [];
  this.parseQueue(this.sources, false, 0);

  this.pointer = 0;

  var self = this;
  runWhenReady(function() {

Queue.prototype = {
  on: function() {
    this.events.on.apply(this.events, arguments);
    return this;

  emit: function() {
    this.events.emit.apply(this.events, arguments);
    return this;

  // ...

Parsing the Queue

The array of script sources must be parsed into something that's easy to
execute sequentially. The enqueue method helps sort items
into groups, and flags if they're suitable for preloading:

enqueue: function(source, async) {
  var preload = isSameOrigin(source),

  options = {
    src: source,
    preload: preload,
    async: async,
    group: this.currentGroup

  if (!this.groups[this.currentGroup]) {
    this.groups[this.currentGroup] = [];


The actual job of queuing is fairly simple, but care must be taken to
increment the currentGroup counter as groups are added:

parseQueue: function(sources, async, level) {
  var i, source;
  for (i = 0; i < sources.length; i++) {
    source = sources[i];
    if (turing.isArray(source)) {
      this.parseQueue(source, true, level + 1);
    } else {
      if (level === 0) {
      this.enqueue(source, async);

The reason the level variable is used is to differentiate
between grouped items and single scripts.

Running the Queue

Each script item is iterated over in sequence, and
XMLHttpRequest is used to preload scripts:

runQueue: function() {
  var i, g, group, item, self = this;

  for (g = 0; g < this.groupKeys.length; g++) {
    group = this.groups[this.groupKeys[g]];

    for (i = 0; i < group.length; i++ ) {
      item = group[i];

      if (item.preload) {
        (function(groupItem) {
          requireWithXMLHttpRequest(groupItem.src, {}, function(script) {
            self.emit('preloaded', groupItem, script);

The anonymous function wrapper helps keep the right item
around (because I've used for loops instead of a
callback-based iterator).

As each request comes back and preloaded is fired, the
event handler will check to see if the entire group has been loaded, and
if so, execute each item.


When I had the idea to use events to manage script loading, I thought I
was onto something and the code would be very simple. However, the
Queue class I wrote for this tutorial ended up becoming
quite complex, and it only handles local scripts at this stage.

I'll attempt to add support for remote preloading as well (where
available), and also add support for script tag insertion as a last

This week's code is in commit
If you've got any feedback, post a comment (or fork) and I'll see if I
can incorporate it.