Previous parts:

• Part 1: Introduction, library review
• Part 2: Script Insertion
• Part 3: HTML5
• Part 4: Preloading with XMLHttpRequest

Creating a Preloading API

Last week I fleshed out turing.request so it can handle preloading with XMLHttpRequest. It’s time to look at how to build an API that will allow scripts to load based on the required order.

I thought a lot about this and decided that an array is a good way to model dependencies between scripts. Given this array:

[
  'framework.js',
  ['plugin.js', 'plugin2.js'],
  'app.js']
]

then the following actions should occur:

• framework.js is downloaded and executed
• plugin.js and plugin2.js are downloaded asynchronously, but only executed when framework.js is ready
• Finally, app.js is executed

These scripts can be loaded at any time, as long as they’re executed in the specified order. If the scripts are local, they can be preloaded with XMLHttpRequest.

Preloading Test

To break this down into something implementable, I created a test first:

'test async queue loading': function() {
  $t.require([
    '/load-me.js?test9=9',
    ['/load-me.js?test4=4', '/load-me.js?test5=5'],
    ['/load-me.js?test6=6', '/load-me.js?test7=7'],
    '/load-me.js?test8=8'
  ]).on('complete', function() {
    assert.ok(true);
  }).on('loaded', function(item) {
    if (item.src === '/load-me.js?test9=9') {
      assert.equal(typeof test4, 'undefined');
    }

    if (item.src === '/load-me.js?test4=4') {
      assert.equal(test9, 9);
      assert.equal(test4, 4);
      assert.equal(typeof test6, 'undefined');
      assert.equal(typeof test8, 'undefined');
    }

    if (item.src === '/load-me.js?test6=6') {
      assert.equal(test9, 9);
      assert.equal(test4, 4);
      assert.equal(test6, 6);
      assert.equal(typeof test8, 'undefined');
    }
  });

I originally wrote this with a callback like the old require signature implied, but I realised that different types of callbacks are required, so it felt natural to use something based on events.

Using this event-based approach should make it easier to build internally, but also makes a pretty rich API.

Modifying require

To work with this new signature (and event-based API), last week’s require implementation will have to be updated. Recall that a setTimeout method was used to delay loading until the head part of the document is ready. That makes returning a value — and thus chaining off require — impossible.

To get around this, I extracted the setTimeout part and made it into a method:

function runWhenReady(fn) {
  setTimeout(function() {
    if ('item' in appendTo) {
      if (!appendTo[0]) {
        return setTimeout(arguments.callee, 25);
      }

      appendTo = appendTo[0];
    }

    fn();
  });
}

When an array is passed to require we can safely assume queuing is necessary. The old method signature is still available. In the case when an array has been passed, a Queue object can be returned to enable chaining:

turing.require = function(scriptSrc, options, fn) {
  options = options || {};
  fn = fn || function() {};

  if (turing.isArray(scriptSrc)) {
    return new Queue(scriptSrc);
  }

  runWhenReady(function() {
    // Last week's code follows

The Queue Class

The Queue class is based on turing.events.Emitter which is a simple event management class, created in Let’s Make a Framework: Custom Events. The basic algorithm works like this:

1. Queue.prototype.parseQueue: Iterate over each item in the array to work out which scripts require preloading, and group them together based on the array groupings. Single script items will be added to a group that contains one item, so the array of arrays becomes normalised into something easy to manage.
2. Queue.prototype.enqueue: Whenever parseQueue finds a script, call enqueue to add it to a sequentially indexed object.
3. runQueue: Iterate over each group and preload each local script using XMLHttpRequest. Add an event emitter to the XMLHttpRequest callback to signal a preload event has completed.
4. preload event: When this event is fired, mark the group item as ‘preloaded’ and start executing scripts if the whole group has finished preloading.
5. complete event: This event is fired when all items have executed.

I’ll explain these methods and events below.

Initialization Queue and Maintaining the Chain

To allow the on methods to be called after turing.require, Queue has to use runWhenReady itself, and proxy calls to turing.events.Emitter and return this:

function Queue(sources) {
  this.sources = sources;
  this.events = new turing.events.Emitter();
  this.queue = [];
  this.currentGroup = 0;
  this.groups = {};
  this.groupKeys = [];
  this.parseQueue(this.sources, false, 0);

  this.installEventHandlers();
  this.pointer = 0;

  var self = this;
  runWhenReady(function() {
    self.runQueue();
  });
}

Queue.prototype = {
  on: function() {
    this.events.on.apply(this.events, arguments);
    return this;
  },

  emit: function() {
    this.events.emit.apply(this.events, arguments);
    return this;
  },

  // ...

Parsing the Queue

The array of script sources must be parsed into something that’s easy to execute sequentially. The enqueue method helps sort items into groups, and flags if they’re suitable for preloading:

enqueue: function(source, async) {
  var preload = isSameOrigin(source),
      options;

  options = {
    src: source,
    preload: preload,
    async: async,
    group: this.currentGroup
  };

  if (!this.groups[this.currentGroup]) {
    this.groups[this.currentGroup] = [];
    this.groupKeys.push(this.currentGroup);
  }

  this.groups[this.currentGroup].push(options);
},

The actual job of queuing is fairly simple, but care must be taken to increment the currentGroup counter as groups are added:

parseQueue: function(sources, async, level) {
  var i, source;
  for (i = 0; i < sources.length; i++) {
    source = sources[i];
    if (turing.isArray(source)) {
      this.currentGroup++;
      this.parseQueue(source, true, level + 1);
    } else {
      if (level === 0) {
        this.currentGroup++;
      }
      this.enqueue(source, async);
    }
  }
},

The reason the level variable is used is to differentiate between grouped items and single scripts.

Running the Queue

Each script item is iterated over in sequence, and XMLHttpRequest is used to preload scripts:

runQueue: function() {
  var i, g, group, item, self = this;

  for (g = 0; g < this.groupKeys.length; g++) {
    group = this.groups[this.groupKeys[g]];
    
    for (i = 0; i < group.length; i++ ) {
      item = group[i];

      if (item.preload) {
        (function(groupItem) {
          requireWithXMLHttpRequest(groupItem.src, {}, function(script) {
            self.emit('preloaded', groupItem, script);
          })
        }(item));
      }
    }
  }
}

The anonymous function wrapper helps keep the right item around (because I’ve used for loops instead of a callback-based iterator).

As each request comes back and preloaded is fired, the event handler will check to see if the entire group has been loaded, and if so, execute each item.

Conclusion

When I had the idea to use events to manage script loading, I thought I was onto something and the code would be very simple. However, the Queue class I wrote for this tutorial ended up becoming quite complex, and it only handles local scripts at this stage.

I’ll attempt to add support for remote preloading as well (where available), and also add support for script tag insertion as a last resort.

This week’s code is in commit 74a0f7f. If you’ve got any feedback, post a comment (or fork) and I’ll see if I can incorporate it.

Other Recent Posts

• 17 May 2013
  Impossible Mission, Full-Text Indexing, Backbone.Projections
• 16 May 2013
  AngularJS: Tests
• 15 May 2013
  Node Roundup: 0.11.2, 0.10.6, subscribe, Omelette