NoFlo, Peppy

16 Aug 2013 | By Alex Young | Comments | Tags node

NoFlo

NoFlo

NoFlo (GitHub: noflo) is a project to bring a visual-based approach to programming. It currently focuses on Node, but the authors want to branch out to other platforms if you give them enough money.

The project and the site seem interesting enough, but the Kickstarter video comes off a little bit too enthusiastic. Here is a spoonfull of delicious hyperbole for you to digest: “Nothing new has really been invented in programming in the last 50 years”, “code starts as whiteboards”, “flow-based programming can eliminate spaghetti code”.

Whether or not something is truly new is irrelevant: monads have been around less than 50 years, but even so we’re still exploring their uses after a good 20 something years (random citation: Extensibe Effects: An Alternative to Monad Transformers). Just because strongly typed object oriented programming is the dominant paradigm doesn’t stop thousands of papers a year being published about programming and related fields. Even if you spend all day writing JavaScript, Java, C#, or whatever, the field is still forging ahead: these languages themselves are evolving.

The assumption that we use whiteboards perplexes me. I don’t think I’ve used a whiteboard once in my career, and I’ve been programming commercially for 12 years now. How can making tangled masses of diagrams be a solution to spaghetti code? Won’t we just end up with spaghetti diagrams?

NoFlo has been written in CoffeeScript for simplicity.

Oh, OK… Before I give up and post that Grandpa Simpson hat gif, I’ll say this: I’m not sold on the idea, but that doesn’t mean I won’t try it. Let’s set aside the hilariously hyperbolic marketing video and see what it can do.

Peppy

Peppy (GitHub: jdonaghue / Peppy, License: FreeBSD) by James Donaghue is a CSS selector engine. He originally wrote it over four years ago, and it was well known at the time for being fast. He’s recently rewritten it with a different focus:

Peppy no longer uses large regular expressions for parsing selector strings. It now uses a selector parser that I wrote called LLSelectorParser. This is a top down parser that returns an abstract syntax tree. Peppy internally now works off of this tree.

Back when I regularly wrote the Let’s Make a Framework posts I was fascinated by how CSS frameworks work. Things have changed since then, but parsing selectors is still an interesting topic. Check out LLSelectorParser if you want to see how James implemented his.

A Look at Facebook's React

15 Aug 2013 | By Alex Young | Comments | Tags frameworks facebook ui reactive

React

React (GitHub: facebook / react, License: Apache 2.0, npm: react-tools, bower: react) from Facebook and Instagram is a UI framework. It’s maintained by Paul O’Shannessy, and Paul and Christopher Chedeau both post to the React blog regularly about new features and community news.

React is meant to be a declarative framework, so it focuses on transforming data into interface components. Data can come from servers and user input – there’s no specific transport layer support, instead other libraries like jQuery are relied on.

The first thing you’ll notice about React is it mixes HTML and JavaScript:

var HelloMessage = React.createClass({
  render: function() {
    return <div>{'Hello ' + this.props.name}</div>;
  }
});

React.renderComponent(<HelloMessage name="John" />, mountNode);

What’s going on? Well, this snippet is intended to be placed inside a script tag, with the type attribute set to text/jsx. JSX is React’s XML-based template language. It doesn’t hit the DOM until it needs to: instead a mock DOM, or faked browser is used to maintain an in-memory representation of your templates. Be aware that this comes with caveats: everything is escaped by default, so HTML entities can easily be double escaped.

You don’t need to use JSX, however: React.DOM.div({}, 'Hello ' + this.props.name) is equivalent to the previous example.

Applications made with React are based on components. Components are created with React.createClass method (not React.createComponent). Components have a “state” and “props”: the state is private, and mutable. If the state is updated, then the component will be re-rendered: this is known as reactive state, and is how React models reactive data flows. React doesn’t re-render the entire component, it tries to only render what changed:

render() methods return a description of the DOM, and React can diff this description with the in-memory representation to compute the fastest way to update the browser.

React plays well with Node-based build systems: there’s a Grunt task for building JSX, in case you don’t want to render everything in the browser. The documentation has a section on React’s tooling integration.

The documentation for React is straightforward and easy to follow: there’s a tutorial that introduces the main aspects of the framework. It’s difficult to say how it compares to other frameworks like AngularJS, but I like the distinction between mutable and immutable state and properties for modeling reactive data flows.

Node Roundup: The Future of Node, takeapeek, subtitle

14 Aug 2013 | By Alex Young | Comments | Tags node modules subtitles video
You can send in your Node projects for review through our contact form.

The Future of Node

Yakulu posted The Future of Programming in Node.js to Hacker News, which is a long message by Isaac Schlueter on the nodejs Google Group:

Callbacks will remain the de facto way to implement asynchrony. Generators and Promises are interesting and will remain a userland option.

This is not a democracy. However, there’s plenty of room for everyone’s opinion. If you want to make exciting dramatic breaking changes to node-core, and you can’t find satisfaction by writing userland npm modules, then please fork joyent/node, give it a new name and logo, and go as completely crazy as you want.

It sounds like the Node core developers are focusing on keeping Node maintainable, rather than jumping on whatever new language or API style is currently fashionable. I like this approach, because after working with faddish web frameworks that resulted in unmaintainable projects after a surprisingly short amount of time, the idea of a stable and calm core library is definitely welcome.

There were cries of a lack of innovation in the comments, setting off an interesting discussion about the future of Node. Eventually the comparison with Meteor was brought up, and Isaac talks about how it differs to Node, npm, and even how the Meteor community is managed.

It’s required reading for those of us investing in Node for the next few years.

takeapeek

takeapeek (GitHub: giodamelio / takeapeek, License: MIT, npm: takeapeek) by Gio d’Amelio is a command-line web server in the spirit of python -m SimpleHTTPServer. It supports directory indexes, and allows settings to be saved in .takeapeekrc. You can also use it as a module in Node.

It’s built with connect, and was inspired by glance by Jesse Keane.

subtitler

subtitler (GitHub: aetheon / node-opensubtitles-client, License: BSD, npm: opensubtitles-client) by Oscar Brito is an open subtitles API client.

It has a command-line tool for downloading subtitle files, and also has an event-based, asynchronous JavaScript API.

jQuery Roundup: jQuery.emphasis.js, Tipue Search

Note: You can send your plugins and articles in for review through our contact form.

jQuery.emphasis.js

jQuery.emphasis.js (GitHub: zmmbreeze / jquery.emphasis, License: MIT, jQuery: emphasis) by “mzhou” is a CSS3 text-emphasis fallback.

The author says it’s useful for East Asian languages because it adds symbols above or below characters to improve the clarity of emphasised sections. The equivalent CSS currently only works in WebKit with a vendor prefix, so this plugin will be useful to those working with these languages.

The source is quite interesting – it’s based on the CSS3 specification, which involves determining if a character is suitable for emphasis, then the insertion of fairly complex styles to simulate the emphasis marks.

Tipue image search

Tipue Search (License: MIT) is a site search plugin. It can search static JSON content, or pages on the same origin using Ajax. It also supports an image search mode, which looks for matches in JSON content that points to image URLs.

Given an index, like this:

var tipuesearch = {"pages": [
  {"title": "Tipue Search, a site search engine jQuery plugin", "text": "Tipue Search is a site search engine jQuery plugin. Tipue Search is open source and released under the MIT License, which means it's free for both commercial and non-commercial use. Tipue Search is responsive and works on all reasonably modern browsers.", "tags": "JavaScript", "loc": "http://www.tipue.com/search"},
  {"title": "Tipue drop, a search suggestion box jQuery plugin", "text": "Tipue drop is a search suggestion box jQuery plugin. Tipue drop is open source and released under the MIT License, which means it's free for both commercial and non-commercial use. Tipue drop is responsive and works on all reasonably modern browsers.", "tags": "JavaScript", "loc": "http://www.tipue.com/drop"},
  {"title": "About Tipue", "text": "Tipue is a small web development studio based in North London. We've been around for over a decade. We like minimalism with the occasional hint of retro.", "tags": "", "loc": "http://www.tipue.com/about"}
]};

Tipue can be enabled using $('#tipue_search_input').tipuesearch(); on a suitable input element.

Auto Iframe Height Plugin Update

Ilker Guller updated his Auto Iframe Height Plugin (GitHub: Sly777 / Iframe-Height-Jquery-Plugin, License: MIT, GPL) to automatically check the iframe height when the contents change.

To enable automatic resizing, you can use the watcher option passed to $.iframeHeight. It also supports several other options, including: defaultHeight, minimumHeight, and watcherTime.

Ground, Generators and Async Patterns in Node, Time Grunt

12 Aug 2013 | By Alex Young | Comments | Tags node modules grunt typescript frameworks async es6

Ground

Ground (GitHub: OptimalBits / ground, License: MIT) from OptimalBits is a web framework for Node that is written with TypeScript:

In ground, most of the application logic is moved from the server to the client, whereas the server acts mostly as an scalable, efficient distributed storage and synchronization controller.

It includes also some rather useful features such as a hierarchical routing system, an undo/redo manager, property and declarative bindings, reference counting and automatic synchronization between clients and servers. It is design to always deliver high performance and low memory consumption.

Grunt is used for building the client-side code and deploying it. It uses models in the style of the Mongoose ODM, and most parts of the framework including models have event APIs. Views are implemented using view models.

The documentation at gnd.io includes more examples and demos.

Generators and Async Patterns in Node

Gorgi Kosev sent in his essay, Analysis of generators and other async patterns in node. He looks at how generators work, how the code compares to other implementations (for example, promises), using a complexity measurement, and benchmarks of memory and execution time.

The generator solution suspend is really fast. It incurred minimal overhead of about 50-80% running time. It’s also roughly comparable with streamlinejs (when in raw callbacks mode).

Time Grunt

time-grunt (GitHub: sindresorhus / time-grunt, License: MIT, npm: time-grunt) by Sindre Sorhus displays the elapsed time of Grunt tasks:

// Gruntfile.js
module.exports = function(grunt) {
  // require it at the top and pass in the grunt instance
  require('time-grunt')(grunt);

  grunt.initConfig();
  grunt.registerTask('default', []);
}

git-html5.js, TodoMVC 1.2

09 Aug 2013 | By Alex Young | Comments | Tags node modules polymer git apps chrome

git-html5.js

Tailor

git-html5.js (GitHub: ryanackley / git-html5.js, License: MIT) by Ryan Ackley is a Git implementation for JavaScript:

git-html5.js is a pure JavaScript git client library. It implements a complete Git workflow in nothing but JavaScript. It’s meant to run in a browser environment and depends on so-called “html5” APIs. Some example use cases:

Ryan took Adobe’s Brackets app, ported to run as a Chrome packaged app, and then added git-html5.js to create an editor that can read and write to Git repositories. I’ve tried out the basics and it seems to work pretty well.

The project includes tests and build instructions, but you can just run the Tailor app in Chrome if you want to try it out.

TodoMVC 1.2

Sindre Sorhus sent in TodoMVC 1.2, an update to the monster MV asterisk collection:

We released TodoMVC 1.2 yesterday with some new app examples of frameworks from Google (Polymer), Twitter (Flight) and Facebook (React), and other improvements.

If you’re interested in seeing what a small but functional Polymer project looks like, then check it out!

Reactive JavaScript

08 Aug 2013 | By Alex Young | Comments | Tags node modules browser coffee-script reactive

As JavaScript developers, whether server-side, browser-based, or both, two things we love are composition and asynchronous APIs. Once you’ve got the hang of callbacks, promises aren’t too difficult to master, which is when libraries like q and async come into their own.

Reactive programming offers a different approach. It gives us tools that attack composition and asynchronicity head-on by focusing on data flows. This is a declarative paradigm – logic expressed through data flows. We’re entirely familiar with this: you can’t go far with Node or jQuery without thinking in terms of events. The difference, however, is a layer of formality that provides a richer language for expressing logic through change.

If you’ve ever worked with .NET, then you might have seen Reactive Extensions. There’s also RxJS (GitHub: Reactive-Extensions / RxJS, CodePlex: Rx, License: Apache 2.0, npm: rxjs), a Microsoft Open Technologies project designed to help us compose asynchronous and event-based programs using observable collections, drawing on inspiration from LINQ:

One question you may ask yourself, is why RxJS? What about Promises? Promises are good for solving asynchronous operations such as querying a service with an XMLHttpRequest, where the expected behavior is one value and then completion. The Reactive Extensions for JavaScript unifies both the world of Promises, callbacks as well as evented data such as DOM Input, Web Workers, Web Sockets. Once we have unified these concepts, this enables rich composition.

The first example in the RxJS readme demonstrates how composition can be used:

/* Only get the value from each key up */
var keyups = Rx.Observable.fromEvent(input, 'keyup')
    .select(function(e) {
        return e.target.value;
    })
    .where(function(text) {
        return text.length > 2;
    });

/* Throttle/debounce the input for 500ms */
var throttled = keyups.throttle(500 /* ms */);

/* Get only distinct values, so we eliminate the arrows */
var distinct = keyups.distinctUntilChanged();

function searchWikipedia(term) {
    var url = 'http://en.wikipedia.org/w/api.php?action=opensearch'
        + '&format=json' 
        + '&search=' + encodeURI(term);
    return Rx.Observable.getJSONPRequest(url);
}

Recently, Yang Zhang sent me Introducing reactive.coffee: reactive programming and declarative UIs in CoffeeScript, a blog post about reactive.coffee (GitHub: yang / reactive-coffee, License: MIT, bower: reactive-coffee). This is a library of reactive programming primitives – it’s not AngularJS or Knockout, but something different and more generic. It provides reactive data structures that can be used for many things, including declarative DOM construction.

It’s also designed to work with jQuery, so normal jQuery objects can be manipulated using reactive programming.

There are dozens of reactive programming modules on npm, so check those out if you’re interested in learning more. Also, if like me you’re a Unix-focused web developer, don’t underestimate Microsoft’s resources on the topic because they’ve been doing this for a long time now.

Node Roundup: 0.11.5, nvi, signobj

07 Aug 2013 | By Alex Young | Comments | Tags node modules vim security crypto
You can send in your Node projects for review through our contact form.

Node 0.11.5

Node 0.11.5 was released this week. This unstable version updates v8 and uv. It also includes fixes for buffer, child_process, dgram, fs, https, openssl, os, tls, and util.

The patch for fs by Trevor Norris is interesting (pull request) – rather than writing strings to a buffer and then the disk, it changes fs.js and src/node_file.cc to write directly to the disk instead.

nvi

nvi (GitHub: mikesmullin / nvi, License: GPLv3, npm: nvi) by Mike Smullin is a ‘very opinionated Vi clone’. I tried installing it with npm install -g nvi, but it wouldn’t run; I had to check out the repository manually. It doesn’t clone Vi or Vim in a way that I think it’s fair to call ‘clone’ – I can’t seem to get hjkl to move the cursor, and the modes have been changed to include ‘COMBO’ mode instead of Normal mode which makes using it extremely confusing for a seasoned Vim veteran.

Despite all that, and the fact that the name nvi is a bad choice, I find the project interesting because making complex text user interfaces isn’t an easy task. Also, Mike’s nvi is focused on collaborative features, which potentially makes Node a great fit.

signobj

Django has a cryptographic API for setting and reading signed cookies, and presumably you can also use this to sign API responses for RESTful JSON APIs. Inspired by this, signobj (GitHub: Submersible / node-signobj, License: MIT, npm: signobj by Ryan Munro allows you to sign JSON data with a SHA-1 HMAC:

signobj() - Signs data with secret, you can also pass in some extra hidden data that is used when hashing. This can be useful if you’re creating an access token, and you want it to become invalid when they change their password, and also don’t want the password with the public data.

Ryan said he’s been using it to sign cookies and localStorage sessions.

jQuery Roundup: spy-js, yadcf

06 Aug 2013 | By Alex Young | Comments | Tags jquery plugins sponsored-content services tables ui
Note: You can send your plugins and articles in for review through our contact form.

spy-js

spy-js.

spy-js, created by Artem Govorov, is a commercial tool that aims to make JavaScript instrumentation better. You can use it to trace, debug, and profile code that would otherwise be difficult to work with.

Although tools like Chrome DevTools are excellent, the advantage of spy-js is it can work with any browser. It also allows performance between browsers to be compared more easily.

spy-js in action.

There’s a free beta which you can download from spy-js.com, and issues are being tracked on GitHub. The documentation is also on GitHub, at spy-js/spy-js. The project isn’t open source, but the author intends to keep it free while he collects beta feedback.

For more information, take a look at spy-js.com and @SpyDashJs on Twitter.

yadcf

Yet Another DataTables Column Filter (GitHub: vedmack / yadcf, License: GPL2/BSD, jQuery: yadcf) by Daniel Reznick is a plugin for DataTables that adds filtering components. It can filter based on a select, or the jQuery UI Autocomplete widget. It also parses different data types, like delimited plain text and HTML elements.

$(document).ready(function(){
  $('#example').dataTable().yadcf([
    { column_number: 0 },
    { column_number: 1, filter_container_id: 'external_filter_container' },
    { column_number: 2, data:['Yes', 'No'], filter_default_label: 'Select Yes/No' },
    { column_number: 3, text_data_delimiter: ',', enable_auto_complete: true },
    { column_number: 4, column_data_type: 'html', html_data_type: 'text', filter_default_label: 'Select tag' }]);
});

It also works perfectly well with multiple tables on a page. For more examples, check out the yadcf showcase and the project’s readme.

Signature Pad, Windows 8.1 Apps with Open Source JavaScript

05 Aug 2013 | By Alex Young | Comments | Tags ui windows mobile canvas

Signature Pad

Signature Pad (GitHub: szimek / signature_pad, License: MIT) by Szymon Nowak is a client-side library for drawing smooth signatures. It works with touchscreens and desktop browsers, and was inspired by the Smoother Signatures post on Square’s (excellent) engineering blog.

The problem is that the signer’s finger did not move in straight lines from point to point when tracing out this shape. Rather, our touch points are sampled from a full curve that the signer’s finger traced on the touchscreen. While we can’t know the original shape between the sampled points Android gave us, straight lines are not the best guess.

Szymon’s implementation doesn’t depend on any external libraries, and can draw signatures from DATA URIs.

var canvas = document.querySelector('canvas');
var signaturePad = new SignaturePad(canvas); 
signaturePad.clear();     // Clears the canvas
signaturePad.toDataURL(); // Returns signature image as data URL
signaturePad.fromDataURL('data:image/png;base64,iVBORw0K...') // Draws signature image from data URL

Building a Windows 8.1 App using Typescript and Open Source Libraries

Ingredimeals!

This screencast by Ala Shiban demonstrates how to use some of Microsoft’s technologies with open source projects like AngularJS to create a Windows Store application.

I put together a 17 minute video that gets you started with writing a native Win8.1 app using HTML, CSS, JavaScript, TypeScript, AngularJS, Bootstrap, underscore, BankersBox and jQuery. The goal of the tutorial is to go through the end-to-end experience of developing a win8.1 app as quickly as possible while not developing a random ‘hello world’ app.

It’s a fast screencast that packs a lot in, so if you can’t keep up check out the source at AlaShiban / Ingredimeals. It uses TypeScript, Blend for Visual Studio, NuGet, jQuery, AngularJS, Underscore, and Bootstrap. Custom fonts are used to give the application a Windows 8 feel.

I haven’t used Windows 8 or Visual Studio much before, so I thought it was interesting seeing how it integrated with third-party open source JavaScript libraries through NuGet, and how debugging JavaScript worked.

Something I hadn’t seen before was MSApp.execUnsafeLocalFunction. I noticed Ala had to wrap some calls inside AngularJS (mainly things that wrote to innerHTML) to satisfy Microsoft’s security requirements for Windows Store apps.

Bootstrap 3, Node Web Development, Groc

02 Aug 2013 | By Alex Young | Comments | Tags node books bootstrap documentation

Bootstrap 3

Bootstrap 3's homepage.

Bootstrap 3 RC1 is out, and you’ll see Bootstrap 3 on the front page of getbootstrap.com:

With over ~1,600 commits, ~72,000 additions/deletions, and ~300 files changed, everything has changed. We’ve added features, removed features, and cleaned up a lot more. The v3 pull request on GitHub has all the details you’ll need for a complete list of changes and some helpful migration tips.

Before you get too excited you should know that it’ll take some work to migrate to Bootstrap 3. There’s no magic migration button as far as I know. Grids still scale up to 12 columns, but now use the class prefix .col-. Forms have different classes and tweaked markup, but the newer markup seems better to me. I don’t think we need <div class="controls"> anymore, for example.

Node Web Development - Second Edition

Node Web Development - Second Edition has been released, written by David Herron.

He’s written a post about it on his blog:

The final version of the sample application includes user authentication support, the ability to work with multiple database engines, be deployed on cloud services, and it dynamically sends data back and forth between server and browser to dynamically update the screen when other people edit things, and to support sending little messages between users of the application.

Groc

In Rock Your Doc With Groc, Our Favorite Automated Frontend Documentation Tool, a fork of Groc is discussed which adds support for tags:

We just couldn’t find a tool that gave us everything we needed. In the end, we created our own automated documentation tool by forking Docco and adding (hacking) support for Tags.

Groc, by Ian MacLeod, is a fork of docco by Jeremy Ashkenas. The post on Gilt’s blog does a good job of explaining why they need tags and how they use them.

NeDB: SQLite for Node

01 Aug 2013 | By Alex Young | Comments | Tags node modules databases code-review

NeDB (GitHub: louischatriot / nedb, License: MIT, npm: nedb) by Louis Chatriot is datastore for Node that implements a subset of MongoDB’s API. It has very modest dependencies (async, underscore, binary-search-tree, mkdirp), and can even be used in browsers:

As of v0.8.0, you can use NeDB in the browser! You can find it and its minified version in the repository, in the browser-version/out directory. You only need to require nedb.js or nedb.min.js in your HTML file and the global object NeDB can be used right away, with the same API as the server version.

The author has been enthusiastically benchmarking the project and is rather confident about its performance. And it really does look like MongoDB:

db.find({ satellites: { $lt: 'Amos' } }, function(err, docs) {
  // docs is empty since Phobos and Deimos are after Amos in lexicographical order
});

db.update({ system: 'solar' }, { $set: { system: 'solar system' } }, { multi: true }, function(err, numReplaced) {
  // numReplaced = 3
  // Field 'system' on Mars, Earth, Jupiter now has value 'solar system'
});

// Using a unique constraint with the index
db.ensureIndex({ fieldName: 'somefield', unique: true }, function(err) {
});

The persistence layer isn’t required: databases can be in-memory if desired. To understand how it all works, first take a look at how each database operation is implemented. For example, Datastore.prototype._insert on lines 265 to 268 in datastore.js calls persistence.persistNewState. The persistNewState method is called for anything that changes data (insert, update, remove). The persistNewState method itself returns early if inMemoryOnly has been set, otherwise it appends UTF8-encoded serialized models to the file that backs the database.

Models are serialized using serialize in model.js. This uses JSON.stringify with a callback that maps undefined values to null, and checks if keys are valid using similar rules to MongoDB (you can’t have keys that start with a dollar or contain a full-stop).

When data is retrieved from disk, async.waterfall is used, from Caolan McMahon’s popular module:

Runs an array of functions in series, each passing their results to the next in the array. However, if any of the functions pass an error to the callback, the next function is not executed and the main callback is immediately called with the error.

Elsewhere the async module is used to queue commands, which ensures they’re executed in the desired order. All commands are passed through an instance of the Executor class, including the loading of data from disk when the persistence layer is initialised.

By combining async and the native JSON encoder and parser, Louis has made a convincing yet lightweight MongoDB implementation. It ties together a lot of the skills required to work as a Node developer – asynchronous I/O, and event-based programming. NeDB currently has 886 stars on GitHub, so it’s clearly a popular project: I suggest taking a look at the source if you’re interested in how people use modules like async.

Node Roundup: 0.10.15, IntervalStream, StreamToMongo, fileswap-stream

31 Jul 2013 | By Alex Young | Comments | Tags node modules fs streams
You can send in your Node projects for review through our contact form.

0.10.15

Node 0.10.15 was released last week, which quickly followed 0.10.14. The newer release adds a fix for process.getuid, to address an issue on Mac OS X:

This commit should unbreak npm on OS X - it’s hitting the new ‘uid must be an unsigned int’ check when installing as e.g. user ‘nobody’ (which has an UID of -2 in /etc/passwd or 4294967294 when cast to an uid_t.)

Version 0.10.14 fixed bugs in os, url, and upgraded npm and uv.

IntervalStream

node-interval-stream (GitHub: czzarr / node-interval-stream, License: MIT, npm: interval-stream) by Stanislas Marion is a simple module that provides a Transform stream that triggers events based on an interval. The author’s example combines IntervalStream with request to display the results of a large download every 2 seconds:

var request = require('request');
var IntervalStream  = require('interval-stream');
var is = new IntervalStream(2000); // emit every 2 seconds

request('http://example.com/large_data_set.json')
  .pipe(is)
  .pipe(process.stdout);

StreamToMongo

StreamToMongo (GitHub: czzarr / node-stream-to-mongo, License: MIT, npm: stream-to-mongo), also by Stanislas Marion, allows data to be streamed to MongoDB. This could be used to stream JSON data directly into a database. The example in the readme uses the npm registry, effectively allowing you to create a structured local cache in Mongo of all the module metadata on npm.

fileswap-stream

Finally, fileswap-stream (GitHub: bpostlethwaite / fileswap-stream, License: MIT, npm: fileswap-stream) by Ben Postlethwaite allows underlying file resources to be swapped. This might be useful if you’re streaming data to log files, and want to split the files:

Write to a writable file-stream that swaps out its underlying file resources according to swapper and naming functions. This can be used for a persistent log or data stream - just stream to it 24/7 and let it swap out to new files whenever you trigger it to.

jQuery Roundup: Orangevolt Ampere, Succinct, jQuery Age

30 Jul 2013 | By Alex Young | Comments | Tags jquery plugins state-machine text time date
Note: You can send your plugins and articles in for review through our contact form.

Orangevolt Ampere

Orangevolt Ampere

If you like state machines, then you might find Orangevolt Ampere (GitHub: lgersman / jquery.orangevolt-ampere, License: MIT/GPL2) interesting. Created by Lars Gersmann, it uses AngularJS, jQuery, Boostrap, and Font Awesome and attempts to help you model single page applications by encapsulating data operations and route transitions in state machines. That means your applications get some features for “free”, like infinite undo/redo.

The author has made a presentation about Orangevolt Ampere that gently introduces the main concepts, which I recommend looking at before diving into the code.

The wizard example is a good example of how state machines can model something that would ordinarily be awkward to work with.

Succinct

Succinct (GitHub: micjamking / succinct, License: MIT) by Mike King is a text truncation plugin. It truncates based on the simplest case where you know the number of characters you want to display: $('.truncate').succinct({ size: 80 }) will cut text down to 80 characters and add an ellipsis.

jQuery Age

jQuery Age (GitHub: ksylvest / jquery-age) by Kevin Sylvestre formats and tracks dates and times as human readable text. It allows the text suffixes and grammatical formatting to be overridden, so it could be internationalised as needed.

<time datetime="2010-01-01T12:00:00Z" class="age">January 1, 2010 12:00</time>
<time datetime="2020-01-01T12:00:00Z" class="age">January 1, 2020 12:00</time>

<script type="text/javascript">
  $('.age').age();
</script>

Disc, scaleApp, Cal-HeatMap

29 Jul 2013 | By Alex Young | Comments | Tags browserify ui graphs frameworks libraries modules node

Disc

Disc

Disc (GitHub: hughsk / disc, License: MIT, npm: disc) by Hugh Kennedy is a tool for generating interactive views of browserify project bundles:

It’s especially handy for catching large and/or duplicate modules which might be either bloating up your bundle or slowing down the build process. It works with node projects too!

Running disc index.js > stats.html will generate a file that uses D3.js to display two visualisations: file count and file size. It’s a little bit like the file system usage applications for desktop systems.

scaleApp

Markus Kohlhase has been quietly working on a project called scaleApp (GitHub: flosse / scaleApp, License: MIT, npm: scaleapp) – a framework for creating single page applications. It’s designed to help you write decoupled, event-driven apps, and is influenced by Nicholas C. Zakas’s talk Scalable JavaScript Application Architecture.

The project has no dependencies and supports Node and browsers. AMD and CommonJS modules are both supported, and it can be extended with plugins. There are already plugins for internationalisation, DOM manipulation, MVC, and more.

Applications are based around modules, which are registered and “started”. Multiple instances of a module can be started, and they talk to each other using a pub/sub API. Flow control for asynchronous methods is also possible, with the runSeries and runWaterfalladdressed methods.

Projects are deployed to browsers by using “browser bundles” – Grunt is used as the build system. There is a scaleApp demo application (source), so you can see what a real application built with the framework looks like.

Cal-HeatMap

Cal-HeatMap

Cal-HeatMap (GitHub: kamisama / cal-heatmap, License: MIT, Bower: cal-heatmap) is a GitHub-style calendar heatmap created by Wan Qi Chen. It uses D3.js, and comes with a build script and tests.

var cal = new CalHeatMap();
cal.init({
  start: new Date(2000, 0),
  range: 12,
  domain: 'year',
  subDomain: 'month',
  data: 'http://example.com/api.json'
});

Data can be loaded remotely, and CSV, JSON, TSV, and plain text formats are supported. The API allows values to be highlighted, and the legend, cell size, and position are all configurable.

Ractive.js, Japanese DailyJS Translations

Ractive

Ryan McDonough sent in Ractive.js (GitHub: Rich-Harris/Ractive, License: MIT, Bower: ractive), a new alternative to libraries like AngularJS, created by developers at The Guardian. Are we still calling these libraries MVC, MV*, or something else?

Ractive.js is different. It solves some of the biggest headaches in web development - data binding, efficient DOM updates, event handling - and does so with almost no learning curve.

There’s a nice and short example in the documentation. Given the following HTML fragment:

<p>Hello ! You have
    <strong> new messages</strong>.
</p>

Then the Ractive constructor can be used to populate data:

var ractive = new Ractive({
  el: result,
  template: html,
  data: {
    user: 'Dave',
    messages: { total: 11, unread: 4 }
  }
});

ractive.set('messages.unread', 5);

However…:

Ractive.js constructs a parallel DOM representation which is aware of its dependencies on the values of user and messages.unread. When those values change, it knows exactly which parts of the real DOM need to be updated.

This could be the data-binding library with an idiomatic JavaScript API that we’ve been looking for. By focusing on the “data model” problem the project seems immediately easier to understand than some larger libraries, and the early adoption of Bower and Grunt means it should be straightforward to slot into your projects.

Japanese DailyJS Translations

Hideharu Sakai sent in his translation of my recent “Getting started with Nodebots” article: Nodebot を始めよう(著者:アレックス・ヤング). He said there’s a lot of interest in Node and DailyJS from the Japanese developer community, so thanks for that!

Getting Started with Nodebots

25 Jul 2013 | By Alex Young | Comments | Tags node modules robots nodecopter arduino
Hey, laser lips, your mama was a snow blower The johnny-five module, with art by Mike Sgier.

I’ve been collecting lots of resources relating to Arduino, Node, and robots, and there seems to be a lot of interest in the Node community around Arduino and quadcopters. The jewel in the crown of electronics-related Node modules is johnny-five (GitHub: rwldrn / johnny-five, License: MIT, npm: johnny-five. This Arduino programming framework from Bocoup provides an event-based interface into Arduino boards. The readme includes links to a wealth of examples written using JavaScript, embedded in Markdown with documentation and images. For example, Nodebot is a simple robot with motors and wheels that starts a REPL that allows you to move the robot around with commands like n.left.

Arduino works by accepting code over USB – it comes with an IDE derived from Processing which does all of that for you. In johnny-five projects, the serialport module by Chris Williams (voodootikigod) makes getting code onto your electronic creations possible. Chris has been involved with the world of Node and Arduino since 2010. The serialport module requires native compilation – the readme has instructions for Windows, OS X, and Linux.

Imagine a world where you can write JavaScript to control blenders, lights, security systems, or even robots. Yes, I said robots. That world is here and now with node-serialport. It provides a very simple interface to the low level serial port code necessary to program Arduino chipsets, X10 wireless communications, or even the rising Z-Wave and Zigbee standards. The physical world is your oyster with this goodie.

The API is event-based, so if you’re familiar with Node’s asynchronous core modules then you should be able to learn it pretty quickly. There are alternatives to johnny-five as well: duino by Cam Pedersen seems fairly mature.

NodeCopter

The nodecopter-remote module can be used to script flying drones. It’s built with johnny-five, and is popular with attendees of the NodeCopter.js events:

NodeCopter.js is a full day event where 15 - 60 developers team up in groups of 3.

Each team receives one Parrot AR Drone 2.0 and spends the day programming and playing with it. At the end of the day, each team gets to present their work to the other attendees.

I was introduced to NodeCopter.js by Andrew Nesbitt, who has spoken about Node and quadcopters at events in the UK, and organised Nodecopter London back in March.

There are a lot of other related modules on npm categorised under nodecopter – one particularly fascinating one is voxel-drone which is an AR Drone simulator in voxeljs.com.

Getting Hardware

There are a lot of cheap Arduino kits around now. The johnny-five documentation mentions the SparkFun Inventor’s Kit which is no longer available, but can be found in certain stores (I got one from Amazon a few months ago). The page for the kit has a list of the included parts, so you could order the ones you want separately.

All you really need is an Arduino board, but the important thing is the microcontroller – you can use other boards with the same or compatible microcontrollers. Even better: Arduino is open, so you can build your own boards.

If you’re not sure about Arduino but have access to a Raspberry Pi or BeagleBone, then you could try bonescript:

Bonescript is a node.js library for physical computing on embedded Linux, starting with support for BeagleBone.

The Raspberry Pi is more like a full-blown PC, but it has a simple hardware interface that does digital and analogue I/O – meaning you can connect it to sensors and motors. I managed to wire mine up to an IR sensor from a defunct laptop to make Raspbmc work with my TV remote.

If you’re hooking up hardware with Node, let me know in the comments and I’ll check out your projects!

Node Roundup: Magnolia, VMUX, joinr

24 Jul 2013 | By Alex Young | Comments | Tags node modules mongodb video webrtc
You can send in your Node projects for review through our contact form.

Magnolia

Ryan Munro sent in some of his projects, including Magnolia (GitHub: Submersible / node-magnolia, License: MIT, npm: magnolia) and Figs (GitHub: Submersible / node-figs, License: MIT). Magnolia is a MongoDB client library that has a chainable API and support for promises, which means you can compose expressions using a natural JavaScript-friendly syntax.

magnolia('user')
  .filter({_id: ObjectID('4e4e1638c85e808431000003')})
  .one()
  .then(function(user) {
      console.log('hello', user.name);
  });

Figs is a module for working with settings stored in JSON files. It supports local overrides, “parent directory clobbering”, and overriding when environmental variables are set. It also includes a command-line tool for viewing configurations.

VMUX

VMUX

VMUX (GitHub: malditogeek / vmux, License: BSD) by Mauro Pompilio is an open source video call application that works in browsers, and uses Node. Nodejitsu wrote about it as part of their open source project of the month programme. The readme includes details on how to set it up locally, but you can also try out the service running on Nodejitsu by signing in with Twitter.

Nodejitsu’s blog has had a flurry of activity recently, including a cool post about GUI console applications.

joinr

There is another MongoDB module this week that I wanted to write about: joinr (GitHub: punkave / joinr, License: MIT, npm: joinr) by Tom Boutell. This one fetches related documents by performing join-related operations:

joinr allows joins to be performed via IDs stored in a regular property (byOne) or in an array property (byArray). Joins can be performed when the ID of the related document is in the document you already have (byOne or byArray) and also when the related documents contain the IDs of documents you already have (byOneReverse and byArrayReverse).

For example, a one-to-many join through an array property looks like this:

joinr.byArray(users, 'groupIds', '_groups', function(ids, callback) {
  return groupsCollection.find({ groupIds: { $in: ids } }, callback);
}, callback);

There are more examples in the readme.

jQuery Roundup: Audio Sort, Improving fn.on with Generic Function Overloading

23 Jul 2013 | By Alex Young | Comments | Tags jquery plugins ideas audio sort
Note: You can send your plugins and articles in for review through our contact form.

Audio Sort

Audio Sort

I’ve had a difficult week of dealing with Windows 8, IE, and Visual Studio, so forgive me if I don’t indulge myself a little bit by writing about Audio Sort (GitHub: skratchdot / audio-sort, License: MIT). It’s written by “skratchdot” and is a visualisation of sort algorithms using jQuery, D3.js, subcollider.js, and Bootstrap.

When it comes to sort visualisations, I don’t think anything will beat Quick-sort with Hungarian folk dance (thanks Matias) for sheer eccentricity, but it’s a solid piece of work, and the inclusion of sound is somewhat idiosyncratic.

In a previous life I was also interested in audio programming, and subcollider.js is new to me. If you’ve ever seen SuperCollider but prefer JavaScript’s syntax then you may enjoy it.

Improving fn.on with Generic Function Overloading

Sergii Kliuchnyk sent in js-fn-overloading:

I looked at the code for the Jquery.fn.on method and realized how many methods they have which take different argument types. They do many checks to change argument values to the right types. Here’s my proposition for how to make the code clearer: js-fn-overloading and here’s an example of what Jquery.fn.on looks like with it: jqueyr-on.js

The example in the readme is quite straightforward:

var obj = {
  func: Overload('{Object|String}event, {String}[selector], {*}[data], {Function}fn', function(event, selector, data, fn){
    ok(this === obj);
    ok(typeof event === 'object' || typeof event === 'string');
    ok(typeof selector === 'undefined');
    ok(typeof data !== 'undefined');
    ok(typeof fn === 'function');
  });
};

obj.func('', 1, function(){});

The Overload function takes a configuration string which determines what types the overloaded methods should support, and how they should be mapped to arguments.

RgbQuant, Scoping AngularJS Directives

22 Jul 2013 | By | Comments | Tags images libraries browser AngularJS

RgbQuant

RgbQuant

RgbQuant (GitHub: leeoniya / RgbQuant.js, License: MIT) by Leon Sorokin is an image quantization library that runs in browsers:

Color quantization is the process of reducing an image with thousands or millions of colors to one with fewer (usually 256). The trick is to balance speed, cpu and memory requirements while minimizing the perceptual loss in output quality. More info can be found on wikipedia. Various algorithms can be found on rosettacode.org.

Internally it uses typed arrays, some ES5 array methods, and a Canvas element. The API allows the number of colours to be defined, and some more low-level control over the algorithm.

Scoping AngularJS Directives

James Donaghue sent in his articles about scoping AngularJS directives: Scoping AngularJS Directives Part 1 and Scoping AngularJS Directives Part 2.

… when both are applied to the same element, regardless of order, they have the same scope which is the scope type 3 (isolated scope). This is the most important combination to understand as you other directives may be expecting either direct or prototypical access to the nearest controller scope, but when paired with an isolated scope directive they no longer have this access. This can lead to many unexpected errors. In my opinion isolated scopes should be used only very judiciously and with exact intention understanding this behavior.