DailyJS

DailyJS

The JavaScript blog.


TagLispyScript
Featured

tutorials lisp LispyScript

Tutorial: Writing LispyScript Macros

Posted on .

This tutorial is by Santosh Rajan (@santoshrajan), the creator of LispyScript (GitHub: santoshrajan / lispyscript).

Writing LispyScript Macros

Macros are a powerful feature of LispyScript. They are much more powerful than C #define macros. While C #define macros do string substitution, LispyScript macros are code generators.

Functions take values as arguments and return a value. Macros take code as arguments, and then return code. Understanding this difference and its ramifications is the key to writing proper macros.

Functions get evaluated at runtime. Macros get evaluated at compile time, or pre-compile time to be more precise.

So, when should macros be used? When you cannot use a function! There is more to this answer than what is apparent. Consider this piece of code:

(* 2 2)

And elsewhere in the program we find this.

(* 4 4)

There is a pattern emerging here. In both cases we have the same code *, and a variable -- a number that changes in each instance of the pattern. So we reuse this pattern by writing a function:

(var square
  (function (x)
    (* x x)))

Therefore, to reuse a repeating code pattern as a function, the code pattern must meet two conditions:

  1. The code must remain the same across every instance of the code pattern.
  2. It is only the data that can change across every instance of the code pattern.

Using functions to reuse repeated code patterns has its limitations. You cannot use a function if it is the code part that changes in a repeated code pattern.

Consider the two functions below (str is an expression that adds up given strings):

(var greet
  (function (username)
    (str "Welcome " username)))

(var link
  (function (href text)
    (str "<a href=\"" href "\">" text "</a>")))

There is a repeating code pattern here. Given below is the pattern with the parts that change in capitals:

(var NAME
  (function ARGUMENTS
    (str TEMPLATE_STRINGS)))

We cannot use a function to reuse this code pattern, because the parts that change are parts of the code.

Functions are about reusing code patterns, where it is only the data that changes.

Macros are about reusing code patterns, where the code can also change.

In LispyScript, we can write a macro to reuse this code pattern. The macro needs a name, let's call it template as it happens to be a template compiler:

(macro template (name arguments rest...)
  (var ~name
    (function ~arguments
      (str ~rest...))))

Now compare this with the meta code pattern in the previous example. The arguments to this macro are the parts of the code that change -- NAME, ARGUMENTS, TEMPLATE_STRINGS -- these correspond to name, arguments rest.. in the macro definition.

Arguments can be dereferenced in the generated code by adding a ~ to the argument name. rest... is a special argument that represents the rest of the arguments to the macro after the named arguments.

This macro can be used by making a call to template:

(template link (href text) "<a href=\"" href "\">" text "</a>")

This code will expand as follows:

(var link
  (function (href text)
    (str "<a href=\"" href "\">" text "</a>")))

This expansion happens just before the expanded code is compiled. This is known as the macro expansion phase of the compiler.

Now let's try another example. We will write a benchmark macro, which benchmarks a line of code. But first we'll write a benchmark function to get a couple of related issues out of the way.

(var benchmark
  (function ()
    (var start (new Date))
    (+ 1 1)
    (var end (new Date))
    (console.log (- end start))))

This is not an example that always works. Because JavaScript can only resolve time up to milliseconds, but to benchmark an integer + operation we need a resolution down to nanoseconds.

Furthermore, the function does not scale. We need to benchmark various operations and expressions, and since this involves changes to the above code we need to write a macro. In the macro we print the result of the operation along with the elapsed time:

(macro benchmark (code)
  (do
    (var start (new Date))
    (var result ~code)
    (var end (new Date))
    (console.log "Result: %d, Elapsed: %d" result (- end start)))

It can be used like this:

(var a 1)
(var b 2)
(benchmark (+ a b))

The result printed to the console should look like the following:

Result: 3, Elapsed: 0  

Elapsed is 0 due to the millisecond resolution, but the example seems to run correctly... until one day someone attempts to do this:

(var start 1)
(var b 2)
(benchmark (+ start b))

Running this gives confusing results:

Result: NaN, Elapsed: 1  

The result is NaN, so something has gone wrong since 3 was expected. To figure out what's going on, let's look at the macro expansion:

(var start 1)
(var b 2)
(do
  (var start (new Date))
  (var result (+ start b))
  (var end (new Date))
  (console.log "Result: %d, Elapsed: %d" result (- end start))

The user has created a variable start. It so happens that the macro also creates a variable called start. The macro argument code gets dereferenced in the new scope. When (+ start b) got executed the start variable used was the start Date variable created in the macro code. This problem is known as variable capture.

When writing macros, you have to be very careful when creating a variable inside a macro. In our template macro example we were not concerned about this problem, because the template macro does not create its own variables.

In LispyScript we get around this problem by following two rules which are specified in the "guidelines" section of the document:

  1. When writing a LispyScript program, creating a variable name that starts with three underscores is NOT allowed, for example: ___varname.
  2. When writing a macro you MUST start a variable name with three underscores if you want to avoid variable capture. There are cases where you want variable capture to happen, in which case you do not need to use the three underscores. For example, when you want the passed code to use a variable defined in the macro.

The benchmark macro should be refactored using three underscores:

(macro benchmark (code)
  (do
    (var ___start (new Date))
    (var ___result ~code)
    (var ___end (new Date))
    (console.log "Result: %d, Elapsed: %d" ___result (- ___end ___start)))

Conclusion

Macros are a very powerful feature of LispyScript. It allows you to do some nifty programming, which is otherwise not possible with functions. At the same time we have to be very careful when using macros. Following the LispyScript macro guidelines will ensure your macros behave as expected.

Featured

tutorials lisp LispyScript

Tutorial: LispyScript Introduction

Posted on .

This tutorial is by Santosh Rajan (@santoshrajan), the creator of LispyScript (GitHub: santoshrajan / lispyscript).

Introduction

LispyScript is a tree structured programming language that compiles to JavaScript. A LispyScript program is made up of one or more LispyScript expressions.

(<function> arg1 arg2 arg3 ...)

A LispyScript expression is made up of an opening parenthesis, a set of elements separated by space characters, and a closing parenthesis. A LispyScript expression is a function call (this is not exactly accurate, but we will see the exceptions to this later).

The first element evaluates to a function. It may be a function reference or an anonymous function. The rest of the elements are the arguments to the function. The expression evaluates to the return value of the function.

(console.log "Hello World!")

You will notice that we called console.log, which happens to be a JavaScript function, directly from LispyScript. You can call all JavaScript functions from LispyScript, even loaded library functions and object methods. console.log also works like printf().

(console.log "2 + 2 = %d" (+ 2 2))

You can have expressions within expressions in LispyScript. Here the expression (+ 2 2) is evaluated first, and replaced with its return value 4. Then the function console.log is called with arguments string 2 + 2 = %d and value 4.

And this is all there is to the basic structure of a LispyScript program. LispyScript has a tree structure: expressions within expressions. Let's look at a tree structure almost everyone is familiar with: HTML.

<html lang="en">  
  <head>
    <title>My Home Page</title>
  </head>
  <body>
    <h1>Welcome to LispyScript</h1>
  </body>
</html>  

This is a LispyScript HTML template that generates the exact same HTML as above:

(html {lang: "en"}
  (head
    (title "My Home Page"))
  (body
    (h1 "Welcome to LispyScript")))

We will learn how LispyScript HTML templates work later. For now, you can see that LispyScript has a tree structure. This same code structure can also be a data structure in LispyScript. This is known as homoiconicity -- a fancy term for code having a data structure supported in that language.

When a compiler compiles a language like JavaScript, or Java or C, it first parses the source code into a tree structure known as the syntax tree, and then generates the machine/byte code from the syntax tree. If you could manipulate the syntax tree (which you can't in these languages) while compiling, you could actually modify the language itself in ways not possible in the above mentioned languages and most other languages.

In LispyScript your code -- for all practical purposes -- is the syntax tree itself. And since the language is homoiconic, you can treat your code as data, and even modify the syntax tree! This is what macros do.

Macros

LispyScript does not have a print expression like printf() or console.log in JavaScript. We know that we can call the JavaScript console.log from LispyScript. Now if we could somehow transform a print expression into a console.log expression, we are done!

(macro print (str rest...)
  (console.log ~str ~rest...))

(print "Hello print macro!")
(print "2 + 2 = %d" (+ 2 2))

This example extends LispyScript by adding a print expression to it. The macro expression takes the macro name print as its first argument, and a list of argument names representing the arguments that will be passed to print as the second argument. The last argument to macro is a code template that will be expanded at macro expansion time.

Here the print expression takes two arguments, the same as console.log, and we name the first argument str. The second argument rest... is a special argument which represents the rest of the arguments to print. rest... can represent zero or more arguments.

Inside the template expression we dereference the two arguments by adding a ~ to them. So what happens when we use the print expression below?

(print "Hello print macro!")

The LispyScript compiler works in two steps. First it expands any macro expressions in the code. The previous expression will be expanded to:

(console.log "Hello print macro!")

And then the compiler compiles the above expression to JavaScript

console.log("Hello print macro!");  

The expression:

(print "2 + 2 = %d" (+ 2 2))

Is expanded to:

(console.log "2 + 2 = %d" (+ 2 2))

...and compiled to:

console.log("2 + 2 = %d", (2 + 2));  

This is a very simple example and you don't gain much, but it's good enough to illustrate a simple macro. You might wonder why a print function can't be used instead of a print macro. Let's see what happens when we write a print function instead.

(var print
  (function (data value)
    (console.log data value)))

(print "2 + 2 = %d" (+ 2 2))

This will work too, but have a look at the compiled output:

var print = function(data,value) {  
    return console.log(data,value);
};
print("2 + 2 = %d",(2 + 2));  

The print function must be included alongside the call to it in the compiled output! When we used a macro the compiled code was only one line.

console.log("2 + 2 = %d", (2 + 2));  

This is only an incidental benefit of using macros. We still need to look at the real benefits. Rather than learn when to use macros, it is better to learn when not to use macros. Macros can be dangerous. Consider this function:

(var square
  (function (x)
    (* x x)))

(console.log (square 2))

The above code will print the answer 4.

This can be rewritten as a macro:

(macro square (x)
  (* ~x ~x))

(console.log (square 2))

And the expanded code is:

(console.log (* 2 2))

That's it! The answer is correct. But now try the following:

(var i 2)
(console.log (square i++))

Ouch! You get 6! An embarrassing square of an integer. If you had used the function you would have got the correct answer. This is what happened:

var i = 2;  
console.log((i++ * i++));  

When you use functions the arguments to the function get evaluated and then passed into the function. So in this case when you pass i++, the value 2 is passed as x into the function and i gets incremented. So (x * x) yields 4. In the case of the macro the argument is not evaluated when the macro expansion happens. Whatever you pass as arguments to the macro just get substituted as is in the template. So if you pass i++, it gets substituted as is in your macro template yielding (i++ * i++) which is (2 * 3)!

Conclusion

By taking advantage of tree structures and homoiconicity, LispyScript offers an alternative way of developing JavaScript programs. Although macros are powerful, they must be handled with care.