DailyJS

Let's Make a Test Framework

Alex R. Young

Subscribe

@dailyjs

Facebook

Google+

tutorials frameworks testing lmaf

Let's Make a Test Framework

Posted by Alex R. Young on .
Featured

tutorials frameworks testing lmaf

Let's Make a Test Framework

Posted by Alex R. Young on .

Welcome to part 38 of Let's Make a Framework, the ongoing series about
building a JavaScript framework.

If you haven't been following along, these articles are tagged with
lmaf. The project we're creating is called Turing.

Last week I started building
turing-test.js -- a basic unit testing framework. The idea behind this framework is to build
something that follows the CommonJS specifications and works in modern
browsers. This will help us test Turing more effectively.

Asserts

Since browsers don't support CommonJS Unit Test
1.0
, we starting
building our own assert module. The architecture of our
module is similar to many other unit testing frameworks -- a
fail function raises an exception whenever the actual
value isn't the same as the expected value.

The assertions defined by the specification are:

assert.ok(guard, message_opt)\
Pure assertion tests whether a value is truthy.

assert.equal(actual, expected, message_opt)\
Shallow, coercive equality with ==.

assert.notEqual(actual, expected, message_opt)\
Tests if two objects are not equal with !=.

assert.deepEqual(actual, expected, message_opt)\
Equivalence, as determined by ===, and special handling for
dates, Object, Array.

assert.notDeepEqual(actual, expected, message_opt)\
The inverse of the above.

assert.strictEqual(actual, expected, message_opt)\
Tests strict equality, as determined by ===

assert.notStrictEqual(actual, expected, message_opt)\
The inverse, using !==

assert.throws(block, Error_opt, message_opt)\
Expects an exception.

For those of you who tl;dr'd that, the CommonJS assertion module defines
methods for equality, deep equality, and strict equality. The
difference between these is important, and if my tutorial fails to
elucidate you please refer to Douglas Crockford's
JavaScript:
The Good
Parts
.

Equality Assertions

I've written about the equality operators on this very blog before, but
let's recap. JavaScript offers two distinct sets of equality operators.
As JSLint will tell you, =</code> and <code>!
are generally what you want. Crockford calls them good, because they
work how we expect: if two values have the same type and value,
=== will result in true.

Meanwhile,
</code> and <code>!=</code> will coerce values if they're not of the same type. This can lead to confusing behaviour, like <code>0
''
equating to true.

That explains coercive equality and strict, but what about deep? The
CommonJS specification defines it deep equality as:

  1. All identical values are equivalent, as determined by
    =</code> # If the expected value is a <code>Date</code> object, the actual value is equivalent if it is also a <code>Date</code> object that refers to the same time # For pairs that do not both pass <code>typeof value "object", equivalence is determined by ==
  2. For all other Object pairs, including
    Array objects, equivalence is determined by having the same number of owned properties (as verified with Object.prototype.hasOwnProperty.call), the same set of keys (although not necessarily the same order), equivalent values for every corresponding key, and an identical "prototype" property. Note: this accounts for both named and indexed properties on an Array.

Research

Let's take a look at some equality code in the wild from an arbitrary
JavaScript test framework,
Jasmine:

jasmine.Env.prototype.equals_ = function(a, b, mismatchKeys, mismatchValues) {
  mismatchKeys = mismatchKeys || [];
  mismatchValues = mismatchValues || [];

  for (var i = 0; i < this.equalityTesters_.length; i++) {
    var equalityTester = this.equalityTesters_[i];
    var result = equalityTester(a, b, this, mismatchKeys, mismatchValues);
    if (result !== jasmine.undefined) return result;
  }

  if (a === b) return true;

  if (a === jasmine.undefined || a === null || b === jasmine.undefined || b === null) {
    return (a == jasmine.undefined && b == jasmine.undefined);
  }

  if (jasmine.isDomNode(a) && jasmine.isDomNode(b)) {
    return a === b;
  }

  if (a instanceof Date && b instanceof Date) {
    return a.getTime() == b.getTime();
  }

  if (a instanceof jasmine.Matchers.Any) {
    return a.matches(b);
  }

  if (b instanceof jasmine.Matchers.Any) {
    return b.matches(a);
  }

  if (jasmine.isString_(a) && jasmine.isString_(b)) {
    return (a == b);
  }

  if (jasmine.isNumber_(a) && jasmine.isNumber_(b)) {
    return (a == b);
  }

  if (typeof a === "object" && typeof b === "object") {
    return this.compareObjects_(a, b, mismatchKeys, mismatchValues);
  }

  //Straight check
  return (a === b);
};

There are some commonalities between this code and the CommonJS
specification. For example, Date comparison using
getTime. It deviates slightly by having specific handling
for DOM nodes, and the type checks for strings or numbers.

Here's what Node does in its assert module. The original source even has
lines from the specification pasted in:

function _deepEqual(actual, expected) {
  // 7.1. All identical values are equivalent, as determined by ===.
  if (actual === expected) {
    return true;

  } else if (Buffer.isBuffer(actual) && Buffer.isBuffer(expected)) {
    if (actual.length != expected.length) return false;

    for (var i = 0; i < actual.length; i++) {
      if (actual[i] !== expected[i]) return false;
    }

    return true;

  // 7.2. If the expected value is a Date object, the actual value is
  // equivalent if it is also a Date object that refers to the same time.
  } else if (actual instanceof Date && expected instanceof Date) {
    return actual.getTime() === expected.getTime();

  // 7.3. Other pairs that do not both pass typeof value == "object",
  // equivalence is determined by ==.
  } else if (typeof actual != 'object' && typeof expected != 'object') {
    return actual == expected;

  // 7.4. For all other Object pairs, including Array objects, equivalence is
  // determined by having the same number of owned properties (as verified
  // with Object.prototype.hasOwnProperty.call), the same set of keys
  // (although not necessarily the same order), equivalent values for every
  // corresponding key, and an identical "prototype" property. Note: this
  // accounts for both named and indexed properties on Arrays.
  } else {
    return objEquiv(actual, expected);
  }
}

function isUndefinedOrNull (value) {
  return value === null || value === undefined;
}

function isArguments (object) {
  return Object.prototype.toString.call(object) == '[object Arguments]';
}

function objEquiv (a, b) {
  if (isUndefinedOrNull(a) || isUndefinedOrNull(b))
    return false;
  // an identical "prototype" property.
  if (a.prototype !== b.prototype) return false;
  //~<del>I've managed to break Object.keys through screwy arguments passing.
  //   Converting to array solves the problem.
  if (isArguments(a)) {
    if (!isArguments(b)) {
      return false;
    }
    a = pSlice.call(a);
    b = pSlice.call(b);
    return _deepEqual(a, b);
  }
  try{
    var ka = Object.keys(a),
      kb = Object.keys(b),
      key, i;
  } catch (e) {//happens when one is a string literal and the other isn't
    return false;
  }
  // having the same number of owned properties (keys incorporates hasOwnProperty)
  if (ka.length != kb.length)
    return false;
  //the same set of keys (although not necessarily the same order),
  ka.sort();
  kb.sort();
  //</del>~cheap key test
  for (i = ka.length - 1; i >= 0; i--) {
    if (ka[i] != kb[i])
      return false;
  }
  //equivalent values for every corresponding key, and
  //~~~possibly expensive deep test
  for (i = ka.length - 1; i >= 0; i--) {
    key = ka[i];
    if (!_deepEqual(a[key], b[key] ))
       return false;
  }
  return true;
}

Node's code includes a few key techniques that are worth remembering:

  • A strict equality check is performed up front
  • getTime is used on the dates again
  • Array.prototype.slice.call is used to transform function arguments into arrays, so they can be tested like arrays
  • Key equivalence is tested to find differences in keys early before having to resort to another deep test on nested objects
  • _deepEqual is called recursively for nested objects

If you take a look at the Turing Test
repository
, I've basically
used Node's implementation with a few modifications.

Conclusion

This is actually the main body of assertions as defined by the CommonJS
specification. There's one more, assert.throws, which I'll
define next week, along with a look at testing the tests. If you'd like
to look at the code for this tutorial, it's in commit
9a4c33e79cbeb0bf2a71

on GitHub.

References