DailyJS

☠ Let's Make a Test Framework

Alex R. Young

Subscribe

@dailyjs

Facebook

Google+

tutorials frameworks testing lmaf

☠ Let's Make a Test Framework

Posted by Alex R. Young on .
Featured

tutorials frameworks testing lmaf

☠ Let's Make a Test Framework

Posted by Alex R. Young on .

Welcome to part 40 of Let's Make a Framework, the ongoing series about
building a JavaScript framework.

If you haven't been following along, these articles are tagged with
lmaf. The project we're creating is called Turing.

Last week we continued building
turing-test.js, a unit testing framework. The idea behind this framework is to build something
that follows the CommonJS specifications and works in modern browsers.
This will help us test Turing more effectively.

Previous parts:

  • Part 36 - An introduction to how JavaScript frameworks handle testing
  • Part 37 - A review of CommonJS test specifications, and examples of how to run assert module tests in a browser with a simple test runner
  • Part 38 and commit 9a4c33e - Assertions part 1
  • Part 39 and commit 8dd48b5 - Assertions part 2

Test Output Design

It's time to make the test results easier to read. The test runner runs
each method that is prefixed with test, as per the CommonJS
specifications. I'd like it to display results a little bit like the
jQuery test suit.

The test runner should list each test method that was run. Because we
can write test method names with full text rather than a JavaScript
function name, the output should be easy to follow:

exports['test strictEqual'] = function() {
  assert.strictEqual('1', '1', "'1' should be equal to '1'");
  assert.strictEqual(1, 1, '1 should be equal to 1');
};

The test runner could display this test as:

[OK] test strictEqual

Another thing to consider is the design of failed tests. The code we
wrote way back at the start of this test framework detour already checks
to see if exceptions have corresponding stack properties:

run: function(testName, obj) {
  var result = new Tests.Result(testName);

  function showException(e) {
    if (!!e.stack) {
      logger.display(e.stack);
    } else {
      logger.display(e);
    }
  }

We should see a full stack trace in the console, but browsers might not
display it. At its most basic, failed tests should look like this:

✕ Assertion failed in: test strictEqual
  AssertionError: '1' should be equal to '1'
  In "===":
    Expected: 2
    Found: 1

The HTML output will require styles to preserve white space for
stacktraces.

Colours and Prefixes

In HTML and consoles that support colour, red and green can be used to
indicate pass or fail. We'll also need another visual indicator for
red/green colourblind people. I've opted to use HTML entities and UTF-8
symbols for these indicators:

  • Error is a skull and crossbones: ☠
  • Pass is a tick: ✓
  • Fail is a cross: ✕

This is purely superficial, I just thought readers might find it more
interesting than text. There's a switch statement in
lib/test.js that converts the HTML entities to JavaScript UTF-8 codes for the
console.

Colours are also converted, but this time based on the message type. The
message type is used as a CSS class name, and is also used to determine
the console colour:

function messageTypeToColor(messageType) {
  switch (messageType) {
    case 'pass':
      return '32';
    break;

    case 'fail':
      return '31';
    break;
  }

  return '';
}

// ...

var col    = colorize ? messageTypeToColor(messageType) : false;
  startCol = col ? '\033[' + col + 'm' : '',
  endCol   = col ? '\033[0m' : '',
console.log(startCol + (prefix ? htmlEntityToUTF(prefix) + ' ' : '') + message + endCol);

All of these functions are embedded within a closure, and they're only
evaluated if they're needed, with this simple pattern:

printMessage = (function() {
  function htmlEntityToUTF(text) {
    // Removed for clarity
  }

  function messageTypeToColor(messageType) {
    // Removed for clarity
  }

  if (typeof window !== 'undefined') {
    return function(message, messageType, prefix) {
      // Display message with some simple DOM code
    }
  } else if (typeof console !== 'undefined') {
    return function(message, messageType, prefix) {
      // Display message with console.log()
    };
  } else {
    return function() {};
  }
})();

After this closure is evaluated, printMessage contains
everything it needs to display messages. Then all that's required is a
helper logger object:

logger = {
  display: function(message, className, prefix) {
    printMessage(message, className || 'trace', prefix || '');
  },

  error: function(message) {
    this.display(message, 'error', '☠');
  },

  pass: function(message) {
    this.display(message, 'pass', '✓');
  },

  fail: function(message) {
    this.display(message, 'fail', '✕');
  }
};

AssertionError toString

The AssertionError exceptions will need to carefully handle
toString to ensure that exceptions are readable.

The way I like to think of failed assertions is they have a summary
and extended details. The summary is basically the custom message
supplied by the assertion invocation, and the details display the
expected value, actual value, and the assertion operator (from
lib/assert.js):

assert.AssertionError.prototype.summary = function() {
  return this.name + (this.message ? ': ' + this.message : '');
};

assert.AssertionError.prototype.details = function() {
  return 'In "' + this.operator + '":\n\tExpected: ' + this.expected + '\n\tFound: ' + this.actual;
};

assert.AssertionError.prototype.toString = function() {
  return this.summary() + '\n' + this.details();
};

I've based this approach on the test frameworks we looked at in part
36
.

The Results

I've made a test fail on purpose here to illustrate the results. Console
tests should look like this:

And a browser is very similar:

This version of turing-test.js is in commit
5a6cbf61
.

References