The JavaScript blog.


essays native

JavaScript as a Protocol for Native Development

Posted on .

This is a guest post by Jonathan Gros-Dubois.

These days it's hard to find a platform that can't run JavaScript - it runs in browsers, on servers (Node.js), on mobile devices (iOS, Android), TVs, drones, robots, and IoT devices. It can also be embedded inside many other popular programming languages, thanks to the JavaScript interfaces/VMs they provide. The proliferation of JavaScript runtimes makes me wonder: could JavaScript become a universal language for cross-platform, cross-device communication? Can it free us from an over-dependence on restrictive communication protocols?

If someone wants to write a custom server, which uses its own protocol or adds a layer on top of an existing protocol, they will typically write a human-readable RFC protocol specification to accompany it. Later, people can read the specification and start implementing clients for it. Usually, the people who built the server will also implement and maintain one or more 'official' clients.

Overall, the current approach works, but it has some significant drawbacks:

  • Behaviour is often inconsistent between clients. Bugs and vulnerabilities may also diverge.
  • Due to duplication of source code (across different languages), it's difficult to maintain the repositories for all of these clients. As a result, they become quickly outdated.
  • It makes it difficult for the client-server ecosystem to evolve as a whole because you're limited by the length/complexity of your RFC specification. The more features your client has, the longer your RFC has to be and the harder it is to implement/maintain across multiple platforms/languages.

As an example, the Socket.io project has published its own RFC protocol specification and this specification has formed the the basis for at least 5 different clients (note that these clients have essentially no code reuse between them):

This is the current convention for doing things and it has become a bottleneck in client-server innovation. There may be a simpler, less conventional alternative to consider:
don't write an RFC! Or to put it differently: let JavaScript code act as your specification! Thanks to recent advances, most custom, (high-level) protocols can now simply be hardcoded in JavaScript on top of WebSockets. You no longer have to write-up a lengthy specification before implementing multiple versions of (what should be) the same client. You can just write it once in JavaScript and embed it inside various JavaScript VMs (for whatever language/platform you want to support) - then you just expose the JavaScript functionality through a lightweight native interface. It's very easy to keep all your clients up to date - It’s typically just a matter of importing your updated client (.js) file and recompiling your project.

This idea occurred to us while working on SocketCluster. It started when my colleague (Lihan Li) and I decided to put together a native iOS client for SC. Our community was quite small and our existing JavaScript client was quite feature-rich, so it was clear that implementing a native Objective-C client from scratch was not an option (or at least it would have to be a severely trimmed-down version of the JS client).

We wanted to keep a lot of the 'nice to have' behaviours which the JS client provided (like auto-reconnect, pub/sub, auto-resubscribe, JWT auth token management, etc...). These behaviours would be difficult to describe as a protocol (would require a lengthy RFC specification), but we really wanted them to be available natively on iOS.

We looked at using JavaScriptCore as our JavaScript VM first because it seemed lightweight: it's built into iOS. Due to various issues and restrictions (some of which may be our own fault), we decided to use an invisible WebView as our VM instead.

The WebView approach worked out surprisingly well. We just had to declare a simple Objective-C interface with a matching JavaScript 'bridge' interface to allow calling various internal JavaScript methods from outside the VM (using native Objective-C code). The result is that we were able to create a full-featured, highly maintainable native iOS client after less than 2 weekend's worth of work! (see Resources section at the bottom of this article for the source code and the iOS-to-JavaScript bridge we used). Note that because we are not using any rendering features of the WebView, performance is not an issue.

The 'wrapper' code for the client is less than 250 lines of Objective-C and 150 lines of JavaScript in total (which is just simple binding logic) but the best part of this approach is that keeping our official client up to date is really easy.

Keep an eye out for our android client - it's coming soon.

Special thanks to OpenLearning for their feedback and support in this highly adventurous project.


Research and development was by Lihan Li with assistance from Jonathan Gros-Dubois.


essays embedding apple

How Apple Could Fix iWork: JavaScript

Posted on .

Apple and JavaScript

Apple recently updated iWork and removed a whole bunch of features. The company where I work makes a popular Mac application, and the removal of AppleScript from iWork caused a backlash from customers who rely on this feature for integration between our program, and Pages and Numbers.

AppleScript wasn't ideal, but it did the job. Now we're left in the dark, and as Apple are being typically opaque about adding it back we're not sure what to do. The way it worked before was with scripting bridge, and although SBApplication is still around, Pages and Numbers no longer have it.

Scripting Bridge

Scripting Bridge provides an Objective-C API for sending and receiving Apple events, so you can take control of applications in Objective-C. It can be used to bridge another scripting language. JSTalk is one such example:

JSTalk is a scripting language for Mac OS X built on top of JavaScript, with a bridge to Apple's Cocoa libraries. You can use it to communicate with other applications just like AppleScript does

JSTalk is built on top of Apple's JavaScriptCore, the same JavaScript engine that powers Safari. So when you write in JSTalk, you are really writing JavaScript.

There's also JavaScript Bindings for C and Objective-C. This uses SpiderMonkey to interpret your JavaScript, and bindings for Objective-C. You can even subclass native objects with JavaScript.

From my perspective as a JavaScript and Node specialist who does some Objective-C on the side, these projects have their appeal. It would be better if JavaScript was the default, though.


Apple makes JavaScriptCore available to Mac and iOS developers with an Objective-C API. This quote is from Owen Mathews at Big Nerd Ranch:

JavaScriptCore gives developers deep access to the full JavaScript runtime from Objective-C. You can syntax-check and execute scripts, access variables, receive callbacks, and share Objective-C objects, making possible a wide range of interactions. (One caveat: on iOS, there is currently no way to access the UIWebView’s runtime, so unfortunately it’s not possible to tie into web apps at this level.)

Post Scripting Bridge

Now Apple have removed Scripting Bridge from iWork, it made me want to rethink application scripting. What would it be like to run JavaScript in an application? It would be much like browser DOM scripting, except the document would be a Keynote or Pages file.

When we wrote AppleScript in the past we wanted to modify a document without having to understand how to parse its format. There's an element of driving the application, but also handling an abstracted version of the document rather than the underlying proprietary format. It's probably easier to say "make words that match this regular expression bold" than it is to write a parser for Pages documents.

I recently wrote about the importance of embedded JavaScript, and I think Apple could gain a lot by adding JavaScript APIs to their applications. It would be more agnostic than AppleScript or VBScript, which can be used with Microsoft Office. If I could somehow insert myself into the nexus of all Apple development and advocate JavaScript I would, but I don't have that kind of power, so perhaps someone reading this will!


JSON essays

JSON Extensions

Posted on .

JSON is typically used as a data interchange format. We use it for communicating with services over HTTP, and for configuration files. People are starting to take it beyond Douglas Crockford's original definition to add more features. Let's take a look at some of these extensions to JSON.

JSON Schema

JSON Schema is a JSON-based format for defining the structure of JSON data. Although the JSON Schema specification has expired, it crops up now and then in the open source community in the form of data validation modules. The idea of writing a portable JSON schema once, and then a module or framework automatically providing validation in a user interface is definitely attractive, although such projects aren't wildly popular so far.


JSON5 (GitHub: aseemk / json5, License: MIT, npm: json5) by Aseem Kishore brings ECMAScript 5-inspired enhancements to JSON. The main changes are as follows:

  • Keys don't need to be quoted and can be any valid identifier
  • Strings can be single-quoted
  • Multi-line strings are supported
  • Objects and arrays can have trailing commas
  • Inline and block comments are allowed
  • Numbers can be hex or start with a decimal point

The author's implementation is available for Node, but it'll run in most browsers (including IE6), and it has a test suite as well.


Another tweaked version of JSON is TJ Holowaychuk's eson. The main change here is TJ's parser allows functions to extend any JSON file's behaviour. It's a little bit like middleware for JSON: a function can be run over each key and value, allowing data values to be transformed.

The example in the documentation is supporting { "interval": "15 minutes" } instead of writing the millisecond value.


Looking at these deviations in JSON illustrates what authors are trying to get out of the JSON standard. Some want to relax the schema rules, others want more extensible parsers. What surprised me most about JSON in the last few years is the failure of JSON Schema to catch on. Perhaps it's lacking a killer app?


language essays

Five Common JavaScript Misunderstandings

Posted on .

Over the last few years more people have been drawn to JavaScript thanks
to libraries like jQuery and innovative server-side projects like Node.
However, JavaScript is deceptively easy to learn for existing
developers, and there are a few frustratingly awkward pitfalls for
newcomers. I've quietly been keeping notes on some of these pitfalls,
and have selected a few of my favourites in this post.

Too Many Callbacks!

New Node developers like to complain about callbacks. Deeply nested
callbacks don't read particularly well, but this isn't necessarily
JavaScript's fault. One way to mitigate this is through flow control
techniques, and chainable APIs are probably the most popular technique
in this area. For example, consider jQuery where the API feels very


If you're knee-deep in callbacks, check to see if the library you're
using has an alternative chainable API. A lot of popular libraries do,
like Underscore.js.

Chainable APIs work by returning this from their methods.
It can be quite hard to make some APIs work this way, but masking the
underlying complexity is often worth it.

There are plenty of flow control libraries available through
npm that solve this (and similar) style problems.


Experienced classical object-oriented developers often get frustrated
with JavaScript's prototypes and recreate traditional class and
inheritance patterns. I think the lack of OO-related keywords is the
main reason for this. However, there's no need to shy away from
prototypes, and prototypal inheritance is surprisingly easy to learn:

var user;

function Person(name) {
  this.name = name;

Person.prototype.greet = function() {
  console.log('Hello', this.name);

// Admin inherits from Person

function Admin(name) {
  this.name = name;

Admin.prototype = new Person;

user = new Admin('alex');

// This method was defined on Person

The key line is Admin.prototype = new Person;. I only
really understood this when I started to think about prototype chains:
Inheritance and the prototype chain.

Scope and Callbacks in Loops

I've fallen foul of this a few times over the years. When looping over a
set of values, people often mistakenly think an anonymous function will
capture the current state of the variables in the loop:

var assert = require('assert')
  , names = ['alex', 'molly', 'yuka']
  , name
  , i;

for (i = 0; i < names.length; i++) {
  name = names[i];

  if (name === 'alex') {
    setTimeout(function() {
      assert.equal(name, 'alex');
    }, 10);

The callback will execute in the future, at which point the name will
have changed because it hasn't been bound the way it seems like it
should. This will work as expected:

var assert = require('assert')
  , names = ['alex', 'molly', 'yuka'];

names.forEach(function(name) {
  if (name === 'alex') {
    setTimeout(function() {
      assert.equal(name, 'alex');
    }, 10);

People often use for instead of an iterator when
performance is desired, but in certain cases callback-based iteration
might be more readable.

this in Nested Functions

What's the value of this in a nested function?

var user;

function Person(name) {
  this.name = name;

Person.prototype.nested = function() {

  (function() {

user = new Person('alex');

The first console.log will show { name: 'alex'
, but the second will show the global object (window in client-side code). This is why a lot of code sets var self = this at the start of a method.

Future Reserved Words

The ECMA standards define Future Reserved Words that include commonly
used words like class, extends, and
super. It's probably a good idea to avoid using these
words; check with the specifications if you're unsure of a particular



language essays

Changing Perceptions of JavaScript

Posted on .

Alex Kessinger is a programmer who lives in the Bay Area. He strives to make websites, cook, and write a little bit better each day. You can find more from Alex at his blog, on Twitter, and Google+.

JavaScript is weird. There's the obvious weirdness of prototypal
inheritance, </code> vs. <code>=; but those are
just surface issues. One feature that boggles non-JavaScript programmers
is space optimizations. JavaScript is unique in this fact. Much of our
insane syntax flows from the fact that the code goes over the wire.
Space saving optimizations are not just acceptable -- they are

Think about that for a moment. How long has it been since programmers
have had to worry about the size of their code? Front-end developers do
this all the time, but to anyone else it's crazy.

At Yahoo I got really good at space optimizations. There were a bundle
of internal documents you could turn to that would offer tricks, and
rules for making your code smaller. The importance placed on these
syntax rules turned code reviews into a checking off process. Every time
we passed the list we patted ourselves on the back. We thought it was
the best use of our time, and we plodded on, writing the same code over
and over. It was prettier, smaller, and well documented, but there
wasn't a ton of original thought that went into it. At least on my part
there wasn't. I am embarrassed to say this now, but I felt like I was at
my JavaScript zenith during this time. Oh man, was I wrong.

I don't think like that anymore. My rubric for good code is how well
others can readily understand it. I no longer consider something useful
until someone else has successfully modified it. And still, the finish
line keeps speeding away from me. There is always a way to be better.

When I moved from a large place like Yahoo, where I rarely had other
kinds of engineers looking at my code, my code habits had to change.
Other kinds of engineers -- especially more polyglot engineers -- were
looking at my code. I found I had to explain myself constantly.

Their acceptance of my crazily optimised code was tepid. Over time, the
craziness started to wear on me. Every time another programmer had to
interact with my code I had to explain everything to them. I felt like a
baboon pounding my chest then leaning over and saying "see, it's simple,
right?" -- it wasn't! And I had to admit, the blame did not lie with the
language or the environment. I am sure that I had become lax in my
syntax, and my code wasn't as clean as it used to be. When there is no
one to call you on your habits, you slide a little.

I was the only front-end developer for a while, but even that wasn't the
worst part. The biggest failure was forcing other kinds of engineers to
accept the craziness, and not taking their code habits into
consideration. I thought that was how it has always been done, and will
forever be done. My coworkers enlightened me. In a small but diverse
team, shared code habits might be the most important key to
understandable code.

With shared code habits in mind, we set about mixing and matching a
front-end system that was more understandable to everyone. I had to
learn how others would see the front-end system. We went about
encapsulating things in a way that would make more sense to others. Some
engineers think of JavaScript as something that needs to disappear, like
in GWT, but our reformulation was more like a blending of technology
that kept the good parts of JavaScript.

During the transition I began to realize the reasons other programmers
reel away from front-end code. Engineers who are fully capable of
writing code for the web will be turned off by the syntax. It might even
stop them from writing web code. Then, even if they do write some code,
they switch off their analytical brains when using something they don't
like. They'll just do what they need to do to get the job done. You
won't get them to help you make the code better. In the end you took
someone who could have been a net positive on code quality, and created
a net negative on code quality. And for what; smaller code?

One of the best things we did was to switch to intermediaries.
Uglify.js is a great example of an intermediary code processor. We have begun to use a number of these
tools. Basically, you can use something like Uglify.js or
CoffeScript to bridge the gap of understanding. You can turn something that people found ugly,
into something beautiful. And then you start to switch the editorial
part of their brain on.

None of this means that I don't care about optimizing code, or keeping
things small. This is important for everyone, and especially so for
high-traffic websites like Yahoo. It is not a means to an end. It's that
90% of my checklist has been replaced by code compressors like
Uglify.js. I don't know what I would have done at Yahoo code reviews if
we started using Uglify.js. It would have been a lot of latte sipping,
and crying as the company crumbled around me.

We should have used better tools, and spent more of our time learning
better ways to code.