☠ Let's Make a Test Framework

2010-11-25 00:00:00 +0000 by Alex R. Young

Welcome to part 40 of Let's Make a Framework, the ongoing series about
building a JavaScript framework.

If you haven't been following along, these articles are tagged with
lmaf. The project we're creating is called Turing.

Last week we continued building
turing-test.js, a unit testing framework. The idea behind this framework is to build something
that follows the CommonJS specifications and works in modern browsers.
This will help us test Turing more effectively.

Previous parts:

Test Output Design

It's time to make the test results easier to read. The test runner runs
each method that is prefixed with test, as per the CommonJS
specifications. I'd like it to display results a little bit like the
jQuery test suit.

The test runner should list each test method that was run. Because we
can write test method names with full text rather than a JavaScript
function name, the output should be easy to follow:

exports['test strictEqual'] = function() {
  assert.strictEqual('1', '1', "'1' should be equal to '1'");
  assert.strictEqual(1, 1, '1 should be equal to 1');

The test runner could display this test as:

[OK] test strictEqual

Another thing to consider is the design of failed tests. The code we
wrote way back at the start of this test framework detour already checks
to see if exceptions have corresponding stack properties:

run: function(testName, obj) {
  var result = new Tests.Result(testName);

  function showException(e) {
    if (!!e.stack) {
    } else {

We should see a full stack trace in the console, but browsers might not
display it. At its most basic, failed tests should look like this:

✕ Assertion failed in: test strictEqual
  AssertionError: '1' should be equal to '1'
  In "===":
    Expected: 2
    Found: 1

The HTML output will require styles to preserve white space for

Colours and Prefixes

In HTML and consoles that support colour, red and green can be used to
indicate pass or fail. We'll also need another visual indicator for
red/green colourblind people. I've opted to use HTML entities and UTF-8
symbols for these indicators:

This is purely superficial, I just thought readers might find it more
interesting than text. There's a switch statement in
lib/test.js that converts the HTML entities to JavaScript UTF-8 codes for the

Colours are also converted, but this time based on the message type. The
message type is used as a CSS class name, and is also used to determine
the console colour:

function messageTypeToColor(messageType) {
  switch (messageType) {
    case 'pass':
      return '32';

    case 'fail':
      return '31';

  return '';

// ...

var col    = colorize ? messageTypeToColor(messageType) : false;
  startCol = col ? '\033[' + col + 'm' : '',
  endCol   = col ? '\033[0m' : '',
console.log(startCol + (prefix ? htmlEntityToUTF(prefix) + ' ' : '') + message + endCol);

All of these functions are embedded within a closure, and they're only
evaluated if they're needed, with this simple pattern:

printMessage = (function() {
  function htmlEntityToUTF(text) {
    // Removed for clarity

  function messageTypeToColor(messageType) {
    // Removed for clarity

  if (typeof window !== 'undefined') {
    return function(message, messageType, prefix) {
      // Display message with some simple DOM code
  } else if (typeof console !== 'undefined') {
    return function(message, messageType, prefix) {
      // Display message with console.log()
  } else {
    return function() {};

After this closure is evaluated, printMessage contains
everything it needs to display messages. Then all that's required is a
helper logger object:

logger = {
  display: function(message, className, prefix) {
    printMessage(message, className || 'trace', prefix || '');

  error: function(message) {
    this.display(message, 'error', '☠');

  pass: function(message) {
    this.display(message, 'pass', '✓');

  fail: function(message) {
    this.display(message, 'fail', '✕');

AssertionError toString

The AssertionError exceptions will need to carefully handle
toString to ensure that exceptions are readable.

The way I like to think of failed assertions is they have a summary
and extended details. The summary is basically the custom message
supplied by the assertion invocation, and the details display the
expected value, actual value, and the assertion operator (from

assert.AssertionError.prototype.summary = function() {
  return this.name + (this.message ? ': ' + this.message : '');

assert.AssertionError.prototype.details = function() {
  return 'In "' + this.operator + '":\n\tExpected: ' + this.expected + '\n\tFound: ' + this.actual;

assert.AssertionError.prototype.toString = function() {
  return this.summary() + '\n' + this.details();

I've based this approach on the test frameworks we looked at in part

The Results

I've made a test fail on purpose here to illustrate the results. Console
tests should look like this:

And a browser is very similar:

This version of turing-test.js is in commit