Making Macros in CoffeeScript


Introduction

JavaScript dependency management is a hot topic at the moment (see RequireJS, Dojo and StealJS). This got me thinking, why do we just treat JavaScript as dumb files to be served up to the client? Now that we have web servers that literally speak the same language, aren’t there greatly possibilities yet to be discovered? Can we write code that seamlessly merges the divide between client and server?

Well honestly? I’ve got no idea. I got a little stuck on the first problem that came to mind – how do we get the server to understand what the code is intending to do on the client? Sure, using Node we could happily execute our JavaScript. But if we wanted to have some smarts about how we deal with it, say analyze a piece of JavaScript to determine what stuff it’s dependent on, we’d actually have to parse the code. Now I’m sure this is possible; clearly Web Browsers and Node parse JavaScript quite happily. But the thought of trying to deal with that myself didn’t quite make me giddy with excitement. If only there was a language like JavaScript but had easily useable parsers on hand that let us mess with the language…

CoffeeScript

Enter CoffeeScript. In it’s own words…

CoffeeScript is a little language that compiles into JavaScript. Underneath all of those embarrassing braces and semicolons, JavaScript has always had a gorgeous object model at its heart. CoffeeScript is an attempt to expose the good parts of JavaScript in a simple way.

A nice language that will compile into JavaScript, which also exposes the good parts of JavaScript’s (gorgeous) object model in a simple way? Well doesn’t that just sound perfect. Sure enough, it’s pretty easy to compile a CoffeeScript program from JavaScript.

var coffee = require('coffee-script');

var nodes = coffee.nodes(coffee.tokens("a = 2 + 2"));

console.log( nodes.compile() ); // var a = 2 + 2;

The important bit for this post, is that we can get both the token stream and the AST nodes themselves before they’re finally compiled into JavaScript. These nodes are just simple JavaScript objects thrown together in a graph. To make messing with them even easier, the Node structures are extremely well documented on the CoffeeScript site (No really, go look at it – it’s the most attractive documentation I’ve seen in a while). For the statement “a = 2 + 2″, the node graph looks much like below:

Expressions
  Assign
    Value "a"
    Op +
      Value "2"
      Value "2"

As a first experiment, I wrote a Visitor object which would visit each node in the graph and if certain conditions were met, replace the node with a new one. In this case, I’m looking for a method call like “ADD x, y”, then I’d replace the node with another in the form “x + y”.

var addReplacementVisitor = {
	onCall: function(n, replaceCallback) {
		if (n.variable.base.value === "ADD") {
			var addOp = new nodes.Op('+', n.args[0], n.args[1]);
			replaceCallback(addOp);
		}
	}
};

So I can imagine what you’re thinking, “well done Dave, you’ve managed to replace an ADD call with an add operation. Yeah, super useful…”. My original idea was to build on this and have more advanced Visitors which would transform the node graph in grander ways. However, these things are kind of difficult to write, and doing anything even slightly more than trivial took an awful lot of code. Fortunately, I just happened to show this to our product designer Eric Wright, he took one look at and remarked – “ah, like Macro’s in Lisp” (yeah, a talented designer who’s also familiar with Lisp, way to make me feel inferior). Lisp allows you to define “macros”, essentially things that look a lot like functions, but actually act more as a find and replace on the language/AST itself. (There are a lot of places online which will give you a proper rundown).

(defmacro swap (a b)
    `(let ((temp ,a))
       (setf ,a ,b)
       (setf ,b temp)))

CoffeeScript Macros

This got me thinking – by far the easiest way of representing a graph of CoffeeScript nodes is CoffeeScript code itself. So in my first prototype, I have a CoffeeScript file designated to define macros, and then a CoffeeScript source file which the defined Macro’s are applied to. So to do something like the above we’d define a swap Macro like…

SWAP = (x,y) ->
	$tmp = x
	x = y
	y = $tmp

Quite straight forward stuff – we’re just creating a function named SWAP which takes two variables (x and y) and swaps them around. Hopefully you’re wondering about the significance of $tmp and why I’ve named it a bit funny, we’ll get to that in a moment. Imagine it working on the following CoffeeScript source:

a = 1
b = 2
c = 3
d = 4
SWAP a, b
SWAP c, d

The Macro really is just a find and replace, so when it’s found the SWAP method in the above, it’ll replace it with the body of the Macro. If we were to do this twice in the same scope, like the above. We’d expect to see two $tmp variables declared which wouldn’t be good – to prevent this, any variable in the Macro scope begining with $ will be renamed to something unique. So in my quick prototype, compiling the above would result in the following JavaScript:

var a, b, c, d, __tmp0, __tmp1;
a = 1;
b = 2;
c = 3;
d = 4;
__tmp0 = a;
a = b;
b = __tmp0;
__tmp1 = c;
c = d;
d = __tmp1;

Using your body

Usually the expressions passed to a Macro are just copied directly into the Macro’s body. But what if we wanted to wrap a Macro around a whole block of code? Well, there’s a special $body argument convention that will help us cope with that. Where most arguments are just directly copied, the $body variable is first “unwrapped” and just the expressions in it’s body will be copied. This allows us to pass a function (typically as ia Macro’s last argument) and treat it as the body of the Macro’s code. The example below hopefully demonstrates this better. Imagine we want to wrap a try…catch around all of our code so that all exceptions are swallowed.

ERRZLESS = ($body) ->
	try
		$body
	catch e
		console.log "IGNORED ERROR: #{e}"

Note how we use this macro by supplying a function as the body.

ERRZLESS ->
	throw "ARGGGGSSS"

This will produce the following JavaScript.

try {
  throw "ARGGGGSSS";
} catch (e) {
  console.log("IGNORED ERROR: " + e);
}

So what’s the point?

Hurrah, Macro’s in CoffeeScript! Well not really. This was only a little experiment of mine to see how easy it is to mess with CoffeeScript before it’s complied. Fortunately as it turns out, it’s pretty easy.

So where is this useful? These kind of techniques I’ve discussed are super useful for building internal DSL’s and meta-programming. It also looks like there’s some serious work going on to provide static metaprogramming to CoffeeScript which would give us proper Macros and an awful lot more.

Show Me The Code

All of the code for this little experiment can be found up on GitHub. If you pull it down it can be executed using the command:

coffee stirred-coffee.coffee macros.coffee main.coffee
About these ads

15 thoughts on “Making Macros in CoffeeScript

  1. One thing that might be help would be a way for the macro definition to directly manipulate the AST. This works in Lisp, I believe, because the form is so regular that it’s possible to manipulate whatever you pass in.

    For example, imagine a macro where I pass in a function that does five different things, and I want to perform them in the reverse order. I need access to the coffeescript AST to do this manipulation.

  2. Pingback: Coffeescript: stirred-coffee macro in coffee « Il Metaprogrammatore

  3. Good stuff. I’ve been slowly working my way through Lisp in the past 9 months or so. One thing I noticed is that the Lisp example you gave is not equivalent to the CoffeeScript macro example you gave. You have to create a $tmp because you weren’t scoping the variables like the way it is in the Lisp example (you know, the functions that JavaScript inherited from Scheme …) If the expanded JavaScript looked like this instead:

    var a, b, c, d;
    a = 1;
    b = 2;
    c = 3;
    d = 4;
    function(a, b) {
    tmp = a;
    a = b;
    b = tmp;
    }(a, b);
    function(a,b) {
    tmp = a;
    a = b;
    b = tmp;
    }(c, d);

    … though I don’t know how to express that in your CoffeeScript / macro system. I’d love to see a “let” special form for CoffeeScript. I will say that I had been on the fence about using CoffeeScript moving forward — seeing as how the big thing in tech right now is mobile Javascript — but the fact that someone is implementing macros has sold me.

    Thanks.

    • Hey Ho-Sheng. Yeah – you’re completely right. The code produced isn’t all that similar to it’s equivalent LISP. I’m actually a little uncomfortable with how all of this completely messes with JavaScript’s function scope. Hopefully the real take away from this is how powerful CoffeeScript can be when you’re able to mess with it’s compilation. It’s certainly worth playing with.

  4. About scopes:

    - coffeescript 1.0 introduced a “do…” structure which allows to control scopes independently from functions.

    - even if it hadn’t, that would have been the very first addition to “meta-coffee-script” (provisional name), precisely because you need it to avoid messing everything up.

    About the verbosity of meta-code:

    - indeed, it’s harder to write than regular code. A bit harder to draft, way harder to make robust.

    - This painfulness is not necessarily a bad thing: this way if there’s a way to achieve the same result through regular programming, you’ll have an incentive to choose that way, which is that right thing to do.

    - There are ways to mitigate this pain, such as quasi-quotes (“+{foo(bar)}” means “the AST which compiles into “foo(bar)”” in metalua). As for visitors, if your AST happens to be representable in XML, I think there are great ways to leverage jQuery to write them!

    • “As for visitors, if your AST happens to be representable in XML, I think there are great ways to leverage jQuery to write them!”

      Yes! That’s so obvious now that you’ve mention it, but that’s brilliant!

      • +1 Being able to identify which nodes to act on was by far the messed part of the visitor. I wonder how easy you could swap out the DOMElement part of Sizzle to work against your own tree data structures.

  5. Pingback: Coffeescript: Macro in coffeescript 2 « Il Metaprogrammatore

    • You’re a sharp guy Dom – that was certainly the plan :)

      I’ve actually come back off of that a bit since after using Stitch for a sizable JS app prototype. It really doesn’t seem too bad just to package everything up into a single js file and cache it forever. The only exception I can see is heavily modular apps, like a widget dashboard for instance.

      Still a fun project. CoffeeScript’s pretty neat as it is, being able to mess with the AST and parser makes it all the more interesting.

  6. Pingback: Freshly Brewed CoffeeScript Issue #1

  7. I can see another really useful reason for doing these kinds of replaces, especially when wrapping existing functionality. It’s the decorator pattern, where you can wrap a simple object that follows a particular convention and then wrap it with complex behaviors transparently(ish). For instance adding database persistence or transactions to a plain Javascript object, or changing the base model from a client model to a server model with automatic persistence based on where it’s used. Probably what you are getting at, so I am looking forward to where this goes :)

  8. I think what you posted was actually very logical. However,
    what about this? what if you typed a catchier post title?
    I ain’t saying your information isn’t good, but suppose you added a post title to maybe
    grab people’s attention? I mean Making Macros in CoffeeScript | David Padbury is kinda boring. You might look at Yahoo’s home page and watch how they write news titles
    to grab people interested. You might add a video or a pic or two to get people
    excited about what you’ve written. Just my opinion, it would make your blog a little bit more interesting.

  9. They now help inform the entire world, not just keep it entertained.
    There tend to be choices to prevent members of the household from viewing adult
    or questionable content. On the planet of video gaming, anything is possible.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s