Node Roundup: Money and npm, Isaac Schlueter Interview, KOAN

02 Apr 2014 | By Alex Young | Comments | Tags node modules realtime npm

Nebulous Profit Meditations

Isaac Schlueter wrote a long article on the npm blog about how npm Inc. will make money. It has some hints about the future of advertising on npm:

In my opinion, a good example of advertising done very well is the hosting page on WordPress.org. The services offered are beneficial to WordPress users, and are offered in such a way as to avoid distracting from the core product. The focused curation increases the value, and provides a strong incentive for the advertised products to maintain their quality or risk losing their position. We will be pursuing similarly focused and curated advertising partnerships on the npm website, in ways that benefit our users as well as our technology partners.

And GitHub’s influence:

When I describe our plans to people, they often nod and say, “Oh, the GitHub model, ok.” I’m sure that “public for free, private costs money” isn’t new with GitHub. However, pursuing that kind of model, while at the same time acknowledging that coding is a social activity, really was a master stroke in the history of software development. I’m very thankful that they’ve helped pave the way for people to recognize this pattern.

Meet the Face Behind npm

The Modulus hosting company blog has an interview with Isaac, with some history prior to Node and npm:

At Yahoo!, I grew increasingly frustrated that I had to switch back and forth between PHP and JavaScript, and Google had just open sourced their V8 engine, so I started trying to seriously get into doing JavaScript on the server. I got involved with the K7 project, and started studying web servers in more detail. There was also SpiderApe, and v8cgi, and a bunch of other projects. Narwhal caught my eye, and I spent a bunch of time messing with that.

I seem to remember making IRC bots with Rhino and Java sockets, then Node came along and changed everything!

KOAN

KOAN (GitHub: soygul / koan) by Teoman Soygul is a full stack boilerplate that uses Koa, AngularJS, Node, and MongoDB. Unlike other similar projects, this one has WebSocket features baked in.

A KOAN app uses JSON-RPC for syncing data with the server, and the readme has details on how to deploy this to Heroku (using labs:enable websockets).

jQuery 1.11.1, 2.1.1, PositionCalculator, YModules

01 Apr 2014 | By Alex Young | Comments | Tags jquery plugins module-loaders

jQuery 1.11.1 Beta 1 and 2.1.1 Beta 1 Released

New jQuery betas are out. These are maintenance releases with bug fixes, including a tweak that helps create builds with custom names, so you can use a different name in AMD loaders like RequireJS.

PositionCalculator

PositionCalculator (GitHub: tlindig / position-calculator, License: MIT) by Tobias Lindig is a jQuery UI Position replacement:

This plugin will not manipulate any CSS settings or DOM-trees, it only calculate values. To be more exact, it calculates the difference of current position to target position and the overflow over boundary. Thus you are all paths open to implement the positioning according to your requirements. You can apply the new position by using top and left or using the css3 transform function or by setting margin. It is up to you.

YModules

Vladimir Varankin sent in YModules (GitHub: ymaps / modules) by Dmitry Filatov is an asynchronous modular loading system that works with Node and browsers.

The syntax looks like this:

modules.define(
  'jquery',
  ['loader'],
  function(provide, loader) {

  loader('//yandex.st/jquery/2.1.0/jquery.min.js', function() {
    provide(jQuery.noConflict(true));
  });
});

modules.define(
  'my-module',
  ['jquery'],
  function(provide, $) {
    function MyModule() { /* do something here */ }
    provide(MyModule);
  });

modules.require('my-module', function(MyModule) {
  // `MyModule` will be available here after
  // Its dependencies will be resolved and its `provide` will fire.
});

It has a feature that allows existing modules to be redeclared, which can help with component-based libraries:

modules.define('button', function(provide) {
    function Button() { /* our button's constructor */ });
    Button.prototype.doAction = function() {};

    provide(Button);
});

// on the end user's space we could redefine the same Button
modules.define('button', ['jquery'], function(provide, $, Button) {
  // We're declaring the same module "button",
  // so we have an access to the module's previous definition
  // through the last define's argument (`Button` here), which
  // is appended automatically and known as "previousModuleDeclaration"
  Button.prototype.doAnOtherAction = function() {};
  provide(Button);
});

modules.require('button', function(Button) {
  Button.doAction();
  Button.doAnOtherAction();
});

Building a Node OAuth2 Server

31 Mar 2014 | By Alex Young | Comments | Tags node oauth2 authentication

Most of you have probably written an OAuth2 client using Node. I’ve used the passport module for Express and found it worked very nicely. But what about creating an OAuth2 server?

Writing a server takes more work, but as long as you’re clear about the authorisation flows that you want to use then it’s certainly possible. Erica Salvaneschi wrote Building a Node OAuth2 server, a post about our experiences building an OAuth2 service for a commercial project we’re working on at Papers:

This article is a walkthrough of Papers’ test-driven implementation of an OAuth server using Node. We decided to go for what’s known as resource owner password flow and chose node-oauth2-server to build our server.

The sample code was written using test-driven development, and depends on Express, Mongoose, bcrypt, and Mocha and SuperAgent for testing. We’ve kept the issues open on the sample code so you can give us feedback, which we’re interested in seeing as we’re new to OAuth2.

13 Steps to AngularJS Modularization, codefront.io (discount inside!)

28 Mar 2014 | By Alex Young | Comments | Tags conferences events angularjs

13 Steps to AngularJS Modularization

Christopher Hiller is writing a three part series about writing modular AngularJS:

AngularJS is not a silver bullet. It doesn’t just magically generate maintainable and extensible apps. You might get lucky with a small app and write it from scratch elegantly and efficiently, using each of AngularJS’ features correctly, and maybe you even have 100% unit test coverage. Your app might do one thing, and do it flawlessly. There are steps you can take to improve your application and get it back in shape for easy development. That means quicker bug fixes, less bugs, and the faster implementation of new features. Taking some time to knock these out will save you from experiencing a lot of pain down the road.

He outlines 13 steps that you can follow to modularise your application. Some might seem obvious, like putting assets in the right directories, but it’s surprising how often people dump files all over the place. This guide might help you if you’ve created your first AngularJS project and want to make it tidier.

codefront.io

codefront.io is a conference taking place in Austria on May 10, 2014. Tickets start at €75 for students, and go up to €150 for tickets that include food and access to the after party.

24 speakers have been listed so far, including Monica Dinculescu from Google, Andy Hume from Twitter, Vitaly Friedman from Smashing Magazine, and Kevin Sawicki from GitHub. I also noticed Lena Reinhard, who is a core member of Hoodie, which is a cool open source project you should check out if you haven’t seen it before.

The organisers sent me a discount code, so if you buy tickets with DAILYJS you may be able to get €10 off your entry fee!

Tetris with WebSockets

27 Mar 2014 | By Alex Young | Comments | Tags games node websocket
Tetris with WebSockets.

A few weeks ago I ran an introductory Node workshop at Makers Academy:

Makers Academy is a highly selective, 12 week full-time programme that teaches web development in London. We accept only the top applicants onto the course for a rigorous programme that culminates with the graduation day when we introduce them to London’s top technology companies looking to hire entry-level developers.

The workshop had a 50 minute talk where I introduced Node, then we set the students a challenge: improve our Tetris game. The challenges started at basic UI improvements and ended at intermediate Node web development.

To make the game, I created a small Tetris game engine – using test-driven development, naturally. The idea behind the game and workshop was to get people thinking about what Node is good at, but we also had an ulterior motive: recruitment. We were struggling to hire a web junior web developer with Node skills, so Makers Academy provided us with a unique opportunity to talk to some enthusiastic new developers.

I’ve written a more detailed post about the workshop on Medium: An introductory Node workshop at Makers Academy. I enjoyed writing the game engine as a Node module, and it made me want to try making a bigger WebSocket powered game… (when my book is finished!)

Node Roundup: npm Search Ranking, shortest-route, prova

26 Mar 2014 | By Alex Young | Comments | Tags node npm modules testing

npm Search Ranking

Improved search results.

npm’s search results are now ranked by popularity:

Packages are now scored by a nifty new algorithm that takes into account the package’s popularity (i.e. number of stars and average weekly downloads over a month). The algorithm also does some proper word parsing (so you can actually find “socket.io” with the search “socket io”) and gives higher weight to things that match the search query in the package name and details.

The interface has been tweaked as well, so you can easily see how many downloads and starts a project has.

I was sceptical about the quality of the results, but I’ve recently been researching material for my book so I’ve been searching npm a lot, and generic searches now seem to get more useful results.

shortest-route

Shortest-route (GitHub: tarun29061990 / shortest-route, License: ISC, npm: shortest-route) by Tarun Chaudhary is a travelling salesman problem solver that you can install with npm.

It calculates the distance between cities using the Google Distance Matrix API, and accepts city descriptions as a pipe-separated list:

var shortestRoute = require('shortest-route');

shortestRoute.getShortPath('A|B|C', function(json) {
  console.log('data='+json);
});

Although you probably won’t need this for a project any time soon, I like the fact a hard problem is installable with npm. For more details, see Tarun’s blog post.

prova

Automatically running tests when files change.

I like tape by Substack – it’s easy to read and produces flexible test output. Azer also likes it, so he wrote prova (GitHub: azer / prova, License: GPL, npm: prova), a Node and browser test runner based on tape and browserify.

It has a built-in web application that uses watchify to automatically run tests when files change. That means you can edit code and see live results in a browser.

Slush: Streaming Scaffolds

25 Mar 2014 | By Alex Young | Comments | Tags gulp scaffolding

Slush

If you’ve been won over by Gulp, the streaming build system, then you’ve probably been wondering about scaffolding.

Slush (GitHub: klei / slush, npm: slush) takes Gulp’s streaming build concept and applies it to scaffolds, making it a Gulp-friendly Yeoman competitor.

The author, Joakim Bengtson, notes that it may take a while for the registry of available generators to update, so until then you can use the slushgenerator tag on npm.

Slush itself is fairly simple – it’s basically an interface for finding and installing generators. Like Gulp, it’s based on liftoff, the command-line tool authoring module.

Isometric Graphics with Obelisk.js

24 Mar 2014 | By Alex Young | Comments | Tags graphics libraries
Obelisk.js.

Obelisk.js (GitHub: nosir / obelisk.js, License: MIT) by Max Huang is a library for building isometric graphics.

With the simple and flexible API provided by this engine, you can easily add isometric pixel element like brick, cube, pyramid onto HTML5 canvas. Obelisk.js strictly follows the pixel neat pattern: lines with 1:2 pixel dot arrangement, leading to an angle of 22.6 degrees.

Max has written some cool examples, like a GIF to isometric convertor and text rendering.

Isometric text example.

There are also tutorials that introduce the basics, like building cubes and other primitives.

To draw a cube, you need to get an instance of a dimension, colour, and the Cube primitive. The library uses its own wrappers for colours and dimensions to set up the right co-ordinates and shading for the isometric effect:

var point = new obelisk.Point(270, 120);
var pixelView = new obelisk.PixelView(canvas, point);

var dimension = new obelisk.CubeDimension(120, 200, 60);
var color = new obelisk.CubeColor().getByHorizontalColor(obelisk.ColorPattern.GRAY);
var cube = new obelisk.Cube(dimension, color);
pixelView.renderObject(cube);

The base class is obelisk.AbstractPrimitive, which Cube inherits from. There’s a demo/ directory with more examples.

Lithium Labs Visualization Components, The Zen Approach

21 Mar 2014 | By Alex Young | Comments | Tags d3 graphics style-guides

Lithium Labs Visualization Components

Sentiment Wave

Recently I’ve been looking for D3 libraries, and coincidentally Jack Herrington sent in Lithium Labs Visualization Components (GitHub: lithiumtech / li-visualizations, License: Apache 2.0, Bower: li-visualizations). This is a set of D3 visualisations, including Sentiment Wave, Reticule, and Elegant Waves.

It includes some examples written in CoffeeScript and an example application that you can try out locally with preloaded data.

The Zen Approach

The Zen Approach by Nijikokun is a JavaScript style guide. It’s detailed and instructive, so it may work well as a beginner’s guide if you’re looking for documentation that will get existing programmers up to speed with modern JavaScript.

Multiline

Judgement call, the slowest known method is [].join. If you are building something that requires high performance do not use [].join. It’s not bad, it’s just not performant. The two fastest methods of multiline are one-line strings (no breaks), and in second using the \ character (breaks), with + (concat) trailing slightly behind. I’ll let hard data back me up on this. My personal preference is to use one line for performance (word-wrap exists people), and then [].join for non-performant things, like variables and such. Should the line require variables and need to be performant I will use + concat notation.

The author hints that it may become a book, series of talks, or an open source handout – it’s currently MIT licensed.

jspm.io

20 Mar 2014 | By Alex Young | Comments | Tags node npm modules

jspm.io

jspm.io (GitHub: jspm / jspm-cli, License: Apache 2.0, npm: jspm) by Guy Bedford is a browser package manager that supports the ES6, AMD, and CommonJS module formats. It uses SystemJS to load modules based on the ES6 module loader specification.

Loading modules in browsers looks like this:

// a specific module from the latest version of an npm package:
System.import('npm:lodash-node/modern/collections').then(function(collections) {
  collections.max([1,2,3,4]); // 4
});

// a semver-compatible version of jquery:
System.import('github:components/jquery@^2.0.3').then(function($) {
});

Notice that it allows modules to be loaded based on semantic version numbers – you can use it to load modules from a CDN this way, or on the command-line: jspm install npm:lodash-node jquery@^2.0.3.

The design of the API means you can use ES6 modules now, but still get the benefits of a Node-powered command-line tool for quickly downloading and building local bundles.

It has its own registry, so naturally the author is looking for pull requests to get new modules listed. The current registries use SPDY, which has some advantages for a module loading system. Once you’re ready to roll out projects to production, you can use the jspm command-line tool to inject package configuration and specific versions of the dependencies.

Node Roundup: cipherhub, slate, express-di

19 Mar 2014 | By Alex Young | Comments | Tags node npm modules

cipherhub

Cipherhub

Cipherhub (GitHub: substack / cipherhub, License: MIT, npm: cipherhub) by substack is a module for encrypting messages based on GitHub public keys.

The usage is simple: npm install -g cipherhub and then cipherhub USERNAME {OPTIONS} < message.txt.

If there are multiple keys for the user, then you’ll need to add a specific key with cipherhub --add USERNAME < id_rsa.pub.

slate

Slate

Slate is a new IRC client by TJ Holowaychuk that uses node-webkit. It’s a native client in a similar spirit to GitHub’s Atom.

TJ notes that the project started as a small hack that he intended to expand into a Kickstarter project, but he’s released it as an open source project instead.

Conceptually I really just wanted a clean, minimalistic IRC client, completely extensible through plugins. Ideally most of the core is written using such plugins. The entire thing should be themable, and the default theme should be programmer-friendly, aka get all the clutter out of my way, I just want to see chat logs.

I’m making a collection of Node-powered native apps, so send those in if you’re working on them.

express-di

If you’ve got addicted to dependency injection through AngularJS, then you might be interested in Express-di (GitHub: luin / express-di, License: MIT, npm: express-di) by Zihua Li:

var express = require('express');
// Require express-di
require('express-di');
var app = express();

app.factory('people1', function(req, res, next) {
  next(null, { name: "Bob" });
});

app.factory('people2', function(req, res, next) {
  next(null, { name: "Jeff" });
});

app.get('/', function(people1, people2, res) {
  res.json({
    people1: people1,
    people2: people2
  });
});

require('http').createServer(app).listen(3008);

The app.factory method is used to define a dependency, and req, res, and next are handled as default dependencies.

JScrambler

18 Mar 2014 | By Ricardo Martins | Comments | Tags sponsored-content obfuscation services
JScrambler.

Well-known libraries such as Google Closure, YUI compressor or UglifyJS minify, compress and optimize JavaScript. The techniques they use can be collectively referred to as JavaScript optimization. They are great at improving overall page load speed, but fall short if you are interested in protecting your source code from code theft and reuse, tampering, or reverse engineering.

Despite that, optimized code is often confused with obfuscated code. For example, compressed code is completely encoded, which at first glance may seem to be highly obfuscated, but a simple run is often enough to retrieve something very similar to the original code. In short, these tools do a good job optimizing your code, but they don’t protect it. (fiddle: demo, Google Compiler optimized demo and beautified optimized demo).

JScrambler goes beyond these libraries by offering advanced obfuscation (JScrambler obfuscated version) that can protect your code. It leverages obfuscation by inserting a number of different code traps to control execution and to enforce licensing. For example, you can lock the code to a list of predefined domains. If someone tries to execute the code elsewhere, the code breaks.

JScrambler just released a new version (3.5) that takes protection even more seriously. It introduces a completely new technique that provides JavaScript files with self-defending capabilities by installing a combination of anti-tampering and anti-debugging. If you try to change a single character, the code will react by breaking down intentionally (try it on JScrambler self-defending demo). It also adds new code traps to restrict execution to a certain Browser or OS, code blocks to give the developer the ability to enable/disable individual source code transformations in certain parts of the code.

There are lots of reasons why you may want to protect your code. People might try to steal your code to reuse it, perhaps to build a similar product, or to replicate a similar user experience. You may have secret algorithms or keys hidden in the code that may be easily recovered by inspecting your code. If you are selling your app, you may be worried about piracy. JavaScript can be easily copied and executed, without your authorization. And last but not least, there are all sorts of security risks, like people figuring out ways to interfere with your code, to commit fraud, or to steal data – from you, or from your users. JScrambler goes along way to combating these problems, despite the fact that there are no bulletproof solutions.

As expected, these techniques have a cost in file size and execution, but because JScrambler also has optimization features, this extra cost can be controlled – as proven by running the two protected demos. You don’t have to give up performance to get protection.

JScrambler’s web interface is simple and easy to use. In five minutes you can get your application protected. You upload the source, click a button, then wait a few seconds and download the results. JScrambler provides templates that were designed to work out of the box in most cases. It includes all sorts of JavaScript-based applications, including HTML5 Canvas, Node.js, Windows 8 WinJS Apps, etc. If you want to automate your builds, an API is also provided, and few ready to use clients, including a Node grunt task.

For more examples, see the full set on jscrambler.com.

picturePolyfill, Interfake

17 Mar 2014 | By Alex Young | Comments | Tags json node modules html5 polyfills

picturePolyfill

picturePolyfill (GitHub: verlok / picturePolyfill, License: MIT/GPL2) by Andrea Verlicchi allows you to use picture elements with srcset support so you can include high-DPI images. Here’s an example with media queries:

<picture data-alt="A beautiful responsive image" data-default-src="img/1440x1440.gif">
  <source src="img/480x480.gif"/>
  <source src="img/768x768.gif"   media="(min-width: 481px)"/>
  <source src="img/1440x1440.gif" media="(min-width: 1025px)"/>
  <source src="img/1920x1920.gif" media="(min-width: 1441px)"/>
  <noscript>
    <img src="img/768x768.gif" alt="A beautiful responsive image"/>
  </noscript>
</picture>

It doesn’t make multiple HTTP requests, so only the required images are fetched. It also takes into account browser event handling, so it won’t run while the browser is being resized.

Interfake

Interfake (GitHub: basicallydan / interfake, License: MIT, npm: interfake) by Daniel Hough is a module designed for client-side developers that makes it easy to create JSON APIs. You can create APIs using the command-line interface by making JSON files that define endpoints:

[{
  "request": {
    "url": "/whattimeisit",
    "method": "get"
  },
  "response": {
    "code": 200,
    "body": {
      "theTime": "Adventure Time!",
      "starring": [
        "Finn",
        "Jake"
      ],
      "location": "ooo"
    }
  }
}]

It supports JSONP, and you can use it programmatically in Node. The documentation has some use-case ideas, like using it for a test API for a mobile application, automated testing, and static APIs.

Rendering Large Terrain in WebGL

14 Mar 2014 | By Alex Young | Comments | Tags webgl graphics

Rendering large terrains by Felix Palmer is a tutorial that demonstrates how to render terrain with a variable level of detail. There’s a demo and the source is on GitHub. It’s built with three.js, and is based on a paper on level-of-detail distribution.

Terrain

A simple way to do better is to split our plane into tiles of differing sizes, but of constant vertex count. So for example, each tile contains 64×64 vertices, but sometimes these vertices are stretched over an area corresponding to a large distant area, while for nearby areas, the tiles are smaller.

The code uses GLSL, but the main app/ JavaScript code is all neatly organised with RequireJS, so it’s surprisingly easy to navigate and understand. The tutorial blog post also makes some of these difficult ideas more accessible, so I thoroughly recommend checking it out.

Multiline strings in JavaScript

13 Mar 2014 | By Alex Young | Comments | Tags es6 hacks

Multiline (GitHub: sindresorhus / multiline, License: MIT, npm: multiline) by Sindre Sorhus is a clever hack that allows you to write multiline strings by using a callback to wrap around a comment:

var str = multiline(function(){/*
<!doctype html>
<html>
    <body>
        <h1>❤ unicorns</h1>
    </body>
</html>
*/});

This works by calling .toString() on the callback, then running a regular expression to extract the comment: /\/\*!?(?:\@preserve)?\s*(?:\r\n|\n)([\s\S]*?)(?:\r\n|\n)\s*\*\//.

Although this is a hack, I hadn’t thought about it before. Sindre notes that this has a performance impact, but that sometimes it might be worth writing things this way for the added clarity it brings.

EcmaScript 6 will introduce template strings, which can be used for multiline strings and interpolation with ${}:

A template string uses back ticks instead of double quotes or single quotes. The template string can contain place holders, which use the ${ } syntax. The value of the expressions in the place holders as well as the text between them gets passed to a function. This function is determined on the expression before the template string. If there is no expression before the template string the default template string is used.

Node Roundup: 0.11.12, Metalsmith, Promises and Error Handling

12 Mar 2014 | By Alex Young | Comments | Tags node modules npm promises generators
Job ad: Iridize is looking for a lead frontend developer.

Node 0.11.12

Node 0.11.12 is out. It updates uv, some of Node’s C++ code in src/, and several core modules including cluster and net.

One commit that caught my attention was buffer: allow toString to accept Infinity for end by Brian White. He said he sometimes sets INSPECT_MAX_BYTES to Infinity, allowing the buffer’s contents to be printed for debugging purposes. I think it’s interesting that this works, even though it’s not something I’d usually want to do.

Metalsmith

Ian Storm Taylor sent in Metalsmith, a really cool static site generator by Segment.io. Why is it cool? Well, they had me at the entirely plugin-based API that uses chainable calls:

Metalsmith(__dirname)
  .use(drafts())
  .use(markdown())
  .use(permalinks('posts/:title'))
  .use(templates('handlebars'))
  .build();

Promises and Error Handling

Promises and Error Handling by Jon Merrifield is all about error handling with promises in Node. It has guidelines for using promises safely, including the idea that you should reject rather than throw and how to terminate chains early and safely.

Changing the then in the above code to done means that there will be no outer promise returned from this, and the error will result in an asynchronous uncaught exception, which will bring down the Node process. In theory this makes it unlikely that any such problem would make it into production, given how loudly and clearly it would fail during development and testing.

Gremlins.js, Backbone.CustomSync

11 Mar 2014 | By Alex Young | Comments | Tags testing backbone

Gremlins.js

Gremlins.js

Gremlins.js (GitHub: marmelab / gremlins.js, License: MIT) from marmelab is a monkey testing library. According to the authors, it can be used to unleash a horde of undisciplined gremlins at a web application.

If that sounds weird, there’s a handy gif in the readme that illustrates how it works: it basically throws events at your HTML, and is able to report back when something goes wrong:

Mogwais only monitor the activity of the application and record it on the logger. For instance, the “fps” mogwai monitors the number of frame per second, every 500ms. Mogwais also report when gremlins break the application. For instance, if the number of frames per seconds drops below 10, the fps mogwai will log an error.

There are various kinds of gremlins that try to uncover issues. This includes a form filler, clicker, and scroller. You can manually create hordes using a chainable API:

gremlins.createHorde()
  .gremlin(gremlins.species.formFiller())
  .gremlin(gremlins.species.clicker().clickTypes(['click']))
  .gremlin(gremlins.species.scroller())
  .gremlin(function() {
    window.$ = function() {};
  })
  .unleash();

Backbone.CustomSync

Garrett Murphey sent in Backbone.CustomSync (GitHub: gmurphey / backbone.customsync, License: MIT, npm: backbone.customsync), a more pragmatic solution for defining Backbone.sync implementations that allows you to avoid giant switch statements:

To use the extension, all you have to do is use Backbone.CustomSync.Model or Backbone.CustomSync.Collection in place of Backbone.Model and Backbone.Collection, respectively. If you don’t define one of these sync methods - createSync, for example - and Backbone attempts to save a new model, the options.error callback will be invoked automatically. Backbone.CustomSync will only perform the operations you define.

Introducing Web Components to Control Authors

10 Mar 2014 | By Matthew Phillips | Comments | Tags components tutorials
This is a guest post by Matthew Phillips, from Bitovi. You can find him on Twitter: @matthewcp and GitHub: matthewp.

At this point unless you’ve been living under a rock you’ve likely heard at least a little about web components, a collection of emerging standards created with the intent of making web development more declarative. Among other things, web components allow for custom elements, an easier way to encapsulate your widgets. If you’ve been developing with MVC frameworks there is a learning curve to using components, but once things start to click you’ll want to use them everywhere in your application. Who hasn’t wanted an easy way to insert a random cat picture?

<randomcat width="200" height="300"></randomcat>

Creating widgets that are well encapsulated is something we do on a daily basis on as JavaScript engineers. In this article I’ll explain where traditional control-based MVC has fallen short of that goal and how web components can resurrect the ease of development from the web’s early roots.

MVC’s Brick Wall

Traditional MVC frameworks encourage you to organize your view code by creating constructor functions called Controls or Views. If you’ve developed in this way you probably recognize some of the problems you encounter with this approach.

Tracking Down Instantiation

Since Controls are implemented as constructor functions that operate on a template, any sub view must be manually instantiated within one of the control’s lifecycle methods, usually either init or render. Consider the following in Backbone:

var Widget = Backbone.View.extend({
  initialize: function() {
    this.subView = new InnerWidget({ el: $('#inner') });
  },

  render: function() {
    this.$el(this.template());
    this.subView.render();
  }
});

That’s a lot of code that will become more complex as you add additional subviews or conditions for rendering.

(Lack Of) External API

While creating a widget, its common to create an external API to interact with. With traditional MVC controls, there is no standard way to do this, so it ends up being ad-hoc at the whim of the author. For example, here’s an accordion containing Panel subviews:

var PanelView = Backbone.View.extend({
  template: _.template($('#panel-tmpl').html()),

  render: function() {
    this.$el.html(this.template(this.model.toJSON()));
    return this.$el;
  }
});

var AccordionView = Backbone.View.extend({
  template: _.template($('#acc-tmpl').html()),

  addPanel: function() {
    if (panel instanceof PanelView) {
      this.$el.find('.panels').add(panel.render());
    }
  }
});

And then to use this:

var panel = new PanelView({ model: data });
accordion.addPanel(panel);

You’ll want to abstract some of these pain points to create your own “standard API” for controls. You’ll probably create some base classes with stub functions for common tasks. Pretty soon you’ve created your own mini-framework. We’re learned to put up with these little things and they don’t bother us day-to-day, but when you discover a better way it will change the way you architect your application.

Hidden Model

Similarly, widgets commonly need to interact with external model data. Most MVC frameworks provide some way to pass data into the control so a lot of people have established a pattern of passing in a “model” property to fill this hole. Having a clearer way of setting the model for a control opens a lot of doors in terms of composability. With the traditional control pattern you usually wind up using an adhoc ViewModel created with some properties passed in to the constructor and some created as part of the control’s own logic.

Enter Web Components

Web Components are a W3C spec for an HTML and JavaScript construct that, at its heart, is a way to define custom HTML elements. The spec includes:

  • Native templates (spec)
  • A way to load them (spec)
  • A method to define custom elements and extend existing ones (spec)
  • Hooks to run functions when an element is inserted in the page.

A custom element can be as complex as the main router for single page application, a simple widget to display markdown inline:

<x-markdown table-of-contents-depth="2">
# Heading

## Sub-heading

* List item
   - Sub list item
</x-markdown>

or as simple as a way to embed a YouTube video.

<rick-roll></rick-roll>

Web Components provide an alternate way to define traditional MVC based controls, with several key advantages.

Maximal Encapsulation

The power of web components is their ability to easily encapsulate the details of a widget while still maintaining the ability to compose them together, thanks to the legacy of HTML’s document-oriented roots. Because it is layout-based, web components can be neatly organized in templates, solving the instantiation problem that a control-based workflow suffers from. Applications can become more template driven rather than instantiating controls with complex event handler logic. Let’s say you were A/B testing a feature, you could have your template (in Mustache) look something like this:

{{if randomlyPicked}}
  <rick-roll></rick-roll>
{{/if}}

Obvious API layer

The API layer for web components is easy to understand. Data is passed to components through HTML attributes.

<x-injector href="/some-page.html" />

Layout is passed through the element’s inner content, as shown in the markdown example above.

Models and Templates

The web components spec includes templates for the first time in the web’s history. This means no more script tag or hidden div hacks to include templates on a page. Templates allow you to create fragments of markup to be used by your components later. They are parsed, but not rendered until inserted into the page.

Models can be bound to the templates, and, through the power of Object.observe changes in the model, would result in relevant parts of the template being automatically rendered. If you’ve used an MVC framework with template binding and observable models you’re probably already familiar with this.

Models are passed into components the same way as all types of data, through attributes.

With CanJS’ can.Component you can pass the name of your model through the attributes and get the live-binding today, without waiting for standardization to flesh out that part of the spec. This brings me to my last point…

Using Web Components today

Despite this being early days for Web Components, there are already a few options if you are interested in getting started. Polymer and X-Tags are two projects started by Google and Mozilla engineers working on the Web Components spec. These projects are bleeding-edge and break compatibility often. Additionally they don’t attempt to extend the spec with functionality that won’t eventually end up in it. What they do offer you is the ability to start using components the way they will be used when browsers have fully implemented the specifications. can.Component, on the other hand, is an early implementation of Web Components that provides additional functionality that is beyond the scope of custom elements.

can.Component adds two-way binding between a component’s model (which is not yet part of the web component spec) and its form elements. Additionally it adds declarative binding to a component’s templates through it’s scope object. Scope is an object that is used to render the template, but with the addition of declarative binding it does much more than that. The scope can be used to update the state of a component and respond to events. Here’s a typical example of can.Component that shows off all of these features:

Usage:

<color-selection></color-selection>

Implementation:

<script id="color-section-template" type="text/mustache">
  <form>
    <span class="validations">{{validationMessage}}</span>
    <input name="name" type="text" can-value="color" can-change="validate">
  </form>
</script>

This is the component’s JavaScript:

can.Component.extend({
  tag: "color-selection",
  template: can.view("#color-selection-template"),
  scope: {
    color: "blue",
    validate: function() {
      if (!isAColor(this.attr("color"))) {
        this.attr("validationMessage", "Not a valid color");
      } else {
        this.removeAttr("validationMessage");
      }
    }
  }
});

We Still Need Libraries

I hope I’ve demonstrated the way in which web components breaks some of the boundaries we’ve hit with traditional control-based MVC. At the same time the specification is intentionally low level and leaves room for libraries to improve upon the experience, as can.Component is doing today.

As a consequence of Web Component’s inherent declarative nature your code will become more condensed, with far less boilerplate. We’re truly approaching a paradigm where separation of concerns is being achieved. But you can’t appreciate the way web components changes the way you write applications until you try it yourself. So I encourage you to choose a library and start on your first widget today.

Further (required) Reading

npm-stat

07 Mar 2014 | By Alex Young | Comments | Tags npm node

Recently npm added back download stats, which means you can see how many downloads a package has had. The announcement includes the note that Mikito Takada submitted a pull request for the D3 graphs – it’s things like this that make me glad npm’s website is open source.

npm-stat

There’s a public API for the statistics, which is written using hapi.

Paul Vorbach sent in npm-stat (GitHub: pvorb / npm-stat.com, License: MIT), which generates another set of views on npm’s stats. It displays downloads per day, week, month, and year, and there are graphs for authors as well. Pages for certain authors that I won’t link to directly naturally take a while to generate, but it’s generally fairly responsive.

I’m interested in seeing what people build with npm-www and the stats public API, but so far it seems like they’ve made a big improvement over the older versions.

Book Review: Quality Code: Software Testing Principles, Practices, and Patterns

06 Mar 2014 | By Alex Young | Comments | Tags books testing jquery

Quality Code

Quality Code: Software Testing Principles, Practices, and Patterns ($44.99, eBook: $35.99, Addison-Wesley Professional) by Stephen Vance is a book about testing. It uses examples from several languages – Java is the most prominent, but there are JavaScript examples as well. The most significant part for DailyJS readers is a long practical exercise that involves testing an open source jQuery plugin, but there is a lot of general software design advice that you will find useful.

The book introduces automated testing, but also discusses how tests can be managed in real teams. One of the main points here is how the same best practices that you use for production code should go into automated tests – if you use certain object oriented patterns, small methods, SOLID principles, and so on, then these techniques should be used for test code as well.

This leads into the practice of writing maintainable test code: the relationship between engineering and craftsmanship.

Civil engineers may supervise and inspect the building of bridges or buildings, but they spend little time driving rivets, pouring concrete, or stringing suspension cables. Probably the closest to software engineers’ total immersion might be the handful of test pilots who are also aeronautical engineers, in that they participate in design, construction, inspection, and verification of the craft they fly.

There are JavaScript examples for code coverage issues, dynamic dispatch, scope, asynchronous computation and promises, and Jasmine:

Dynamic languages like JavaScript are less tied to an explicit interface, although usage still defines a de-facto interface. Jasmine’s spyOn functionality provides the full range of test-double variations by substituting a test-instrumented recording object for the function being replaced and letting you define how it behaves when invoked.

What I learned most from, though, was the higher-level advice, like “test the error paths”:

Many people stop before testing the error handling of their software. Unfortunately, much of the perception of software quality is forged not by whether the software fails, because it eventually will, but by how it handles those failures.

And the following point reinforced the way I work, mixing “spec”-like tests with integration and unit tests:

I prefer to test each defect, at least at the unit level.

Stephen sometimes talks about how most of our programming languages and tools aren’t designed specifically to support testing. One idea that runs through the book is about how to design code to be testable, and writing decoupled tests is part of this. Balancing encapsulation with access to internal state for testing is something that I think most of us struggle with.

As we have seen, verification of those internal representations sometimes occurs through interface access. Where null safety is either guaranteed by the representation or ignored by the test, we see code like A.getB().getC().getD() Despite the blatant violation of the Principle of Least Knowledge, we frequently find code like this-of course we do not write it ourselves!— in tests and production.

Chapter 12, “Use Existing Seams”, left an impression on me: it’s about the idea of finding places in code that allows you to take control of that code so you can bring it under test. Since reading that chapter I seem to have found more convenient places to grapple Express applications and test them more thoroughly.

If you write tests, but find they become unmaintainable over time, then this book may guide you to create less entangled tests. It mixes material for dynamic languages like JavaScript with statically typed languages such as C++ and Java. I find this useful as someone who writes a lot of JavaScript but works alongside Objective-C and .NET developers.

Stephen has combined years of experience into a rare, testing-focused book, that relates principles that we use to write well-designed code to the problems inherent in automated testing.