DailyJS: Client-Side Benchmarks

Client-Side Benchmarks

01 Sep 2011 | By Alex Young |

Let’s Make a Framework is an ongoing series about building a JavaScript framework from the ground up.

These articles are tagged with lmaf. The project we’re creating is called Turing. Documentation is available at turingjs.com.

Last week we had a very interesting discussion about optimising the deceptively simple hasClass function that I wrote for Turing’s DOM module. Not only is optimisation difficult, once you bring browsers into the mix it can seem like a dark art. Particularly when cross-browser issues are taken into account, which is why I’ve covered things like cached browser feature detection in previous tutorials.

I mentioned that we really needed client-side benchmarks to talk confidently about performance, because my benchmark example script was just intended to be used in Node. As I already used Benchmark.js (GitHub: bestiejs / benchmark.js, License: MIT, npm: benchmark) I’ve used it again for browser benchmarks. And guess what? It even works in IE6!

Writing Browser Benchmarks

I’ve added "benchmark": "latest" to the devDependencies in the package.json file. Then, at the bottom of a HTML test harness file, I added a script tag to load Benchmark.js.

    <script src="../../node_modules/benchmark/benchmark.js" type="text/javascript"></script>
  </body>
</html>

Next I wrote a pure JavaScript file for the DOM-related benchmarks and added it to the other script tags:

var suite = new Benchmark.Suite,
    div = $t('#test-div')[0],
    cache = {};

function log(text) {
  $t('#results').append('<li>' + text + '</li>');
}

function hasClassRegExp(element, className) {
  if (element.className && element.className.length) {
    return new RegExp('(^|\\s)' + className + '($|\\s)').test(element.className);
  } else {
    return false;
  }
};

function hasClassCachedRegExp(element, className) {
  if (!cache[className]) {
    cache[className] = new RegExp('(^|\\s)' + className + '($|\\s)');
  }
  if (element.className && element.className.length) {
    return cache[className].test(element.className);
  } else {
    return false;
  }
};

suite.add('hasClassRegExp', function() {
  hasClassRegExp(div, 'example1');
  hasClassRegExp(div, 'unknown');
})
.add('hasClassCachedRegExp', function() {
  hasClassCachedRegExp(div, 'example1');
  hasClassCachedRegExp(div, 'unknown');
})
.add('built-in', function() {
  turing.dom.hasClass(div, 'example1');
  turing.dom.hasClass(div, 'unknown');
})
.on('cycle', function(event, bench) {
  log(String(bench));
})
.on('complete', function() {
  log('Fastest is ' + this.filter('fastest').pluck('name'));
  $t('#notice').text('Done');
})
.run(true);

Benchmark.js uses callbacks and events to organise benchmarks. That means you need to instantiate a suite using var suite = new Benchmark.Suite, then add benchmarks using suite.add('name', function() {}). It allows chaining, so as you can see I’ve added a few benchmarks and then watched for two events, cycle and complete. The cycle event will run after each benchmark. Easy!

I’m using the $t Turing alias to do some simple DOM manipulation for displaying results. The log function could actually be placed in a benchmark helpers file once more benchmarks have been added. Just out of interest, I kept the old simple hasClass functions and also included the one currently implemented in turing.dom.hasClass.

This benchmark also includes hasClassCachedRegExp. I noticed that Zepto caches regexes, and it turns out this performs extremely well in Firefox and Chrome, but not so well in IE6. However, remember that when comparing the built-in function, you might be looking at element.classList depending on the browser. In Firefox, Ryan Cannon’s String.prototype.indexOf solution performs better than element.classList.

Given that each browser appears to have different performance characteristics, should we use different functions? I’d probably never do this, unless I was targeting a specific browser. This might sound unusual, but plenty of people are developing games that can only run in WebKit mobile browsers (and Zepto specifically targets WebKit).

Results

Chrome 13, Mac:

Firefox 6, Mac:

Internet Explorer 6, Windows XP, VirtualBoxVM:

Conclusion

If you’re working on client-side code, it doesn’t take much work to be scientific about benchmarks. And, using Node and npm to manage your tools can make it quick to set things up. When writing optimised code, don’t champion a given solution — be scientific, experiment, and try to discover the solution most suited to the task at hand. In the interest of science, benchmarks like these should be run on a wide range of machines (not just virtual machines, but I use those purely for convenience).

This code can be found in commit 095a229.