DailyJS

Managing Asynchronous Assertions

Alex R. Young

Subscribe

@dailyjs

Facebook

Google+

tutorials frameworks testing lmaf

Managing Asynchronous Assertions

Posted by Alex R. Young on .
Featured

tutorials frameworks testing lmaf

Managing Asynchronous Assertions

Posted by Alex R. Young on .
*Let's Make a Framework* is an ongoing series about building a JavaScript framework from the ground up. These articles are tagged with [lmaf](http://dailyjs.com/tags.html#lmaf). The project we're creating is called [Turing](http://github.com/alexyoung/turing.js). Documentation is available at [turingjs.com](http://turingjs.com/).

The biggest problem I've had when writing tests for Turing is checking
when asynchronous callbacks complete. For example, when testing the
resource loading feature, I wrote tests with this pattern:

'test queue loading with no local scripts': function() {
  $t.require([
    'https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js'
  ]).on('complete', function() {
    assert.ok(true);
    assert.done();
  }).on('loaded', function(item) {
    if (item.src === 'https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js') {
      assert.ok(jQuery, 'jQuery should be set');
    }
  });
}  

What happens if loaded never fires? Well, the test would
still pass. That's a consequence of the assertion never being run.

One solution is to wrap the assert module with counters
that count how many assertions have been called. Then at the start of a
test we can write assert.expect(2); to say 'raise an error
if anything other than two assertions are run'.

That's fine, but at this point Turing's test framework always runs tests
asynchronously. If the assert module kept a counter, other
tests would increment that value too. The number of assertions would be
the total number for all of the current suite's tests, rather than the
current test.

Counting Assertions

The initial solution I came up with was to wrap every assertion method
with a function that incremented a counter. Tests take this pattern:

'test queue loading with no local scripts': function() {
  var assertExpect = mixinExpect(assert);
  assertExpect.expect(2);

  $t.require([
    'https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js'
  ]).on('complete', function() {
    assertExpect.ok(true);
    assertExpect.done();
  }).on('loaded', function(item) {
    if (item.src === 'https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js') {
      assertExpect.ok(jQuery, 'jQuery should be set');
    }
  });
}

Each test gets its own instance of the assert module.

The code to do this is slightly clumsy, because I aliased the old
methods using a ** prefix:

function mixinExpect(m) {
  var m2 = {}, method;

  for (method in m) {
    if (m.hasOwnProperty(method)) {
      m2['__' + method] = m[method];
      (function(methodName) {
        m2[methodName] = function() {
          m2.mixinExpectAssertionCount++;
          m2['__' + methodName].apply(m2, arguments);
        };
      }(method));
    }
  }

  m2.expect = function(count) {
    m2.mixinExpectAssertionCount = 0;
    m2.mixinExpectExpected = count;
  };

  m2.done = function() {
    if (m2.mixinExpectAssertionCount !== m2.mixinExpectExpected) {
      throw('Expected assertion count was not found, expected: ' + m2.mixinExpectExpected + ', got: ' + m2.mixinExpectAssertionCount); 
    }
  };

  return m2;
}

It does the job, but is there a more elegant way that doesn't require
changing the assert module?

Expectations

I got this idea when I reviewed Tim Caswell's
Step
library:

'test queue loading with no local scripts': function() {
  var expect = new Expect();
  expect.add('jQuery was set');
  expect.add('loaded fired');

  $t.require([
    'https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js'
  ]).on('complete', function() {
    assert.ok(true);
    expect.fulfill('loaded fired');
    expect.done();
  }).on('loaded', function(item) {
    if (item.src === 'https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js') {
      expect.fulfill('jQuery was set');
      assert.ok(jQuery, 'jQuery should be set');
    }
  });
}

Each test creates an instance of Expect and adds a list of
expectations. As the expectations are fulfilled, it marks down this
fact, then at the end of the test when expect.done() is
called it can see if any expectations were unfulfilled.

I made a quick implementation and it's only a few lines of code:

function Expect() {
  this.expectations = {};
}

Expect.prototype = {
  add: function(expectation) {
    this.expectations[expectation] = false;
  },

  fulfill: function(expectation) {
    this.expectations[expectation] = true;
  },

  done: function() {
    for (var expectation in this.expectations) {
      if (!this.expectations[expectation]) {
        throw('Expected assertion was fulfilled , expected: ' + expectation); 
      }
    }
  }
};

Conclusion

Working with asynchronous tests can quickly get confusing -- sometimes
tests appear to be passing but aren't actually running as expected. The
CommonJS Unit Testing spec actually says the following:

The assertions defined above will not be to everyone's taste style wise (or infact behaviour wise.) Authors are free to define their own assertion methods in new modules as desired.

So adding a assert.expect method to your own assertion
module implementations is perfectly acceptable.

This week's code can be found in the ajax tests in commit
dc3b41f
.