Introducing StitchIt – The CommonJS Module packager for ASP.NET MVC

Introduction

One of the biggest challenges writing large client-side JavaScript single page applications is how you actually manage a large amount of JavaScript. How do you structure the content of the files? Where do you include all the script tags? What order do the script tags have to appear in? It’s all a bit of a headache.

The CommonJS Modules specification proposes a method of structuring JavaScript into self contained modules which specify what they require to run and what they expose externally. At it’s most basic – a global require function loads a module by an identifier (which typically looks like a file path), and there’s a global exports object that the module can attach it’s API on which is returned by the require call. If you’ve ever tried out Node.js it’ll be familiar to you.
calc.js

exports.add = function(n1, n2) {
	return n1 + n2;
};

app.js

var calculator = require('calc');

var result = calculator.add(2, 3); // 5

StitchIt

StitchIt is based on a great library for Node called Stitch which provides a CommonJS Module API in the browser and will automatically package your JavaScript into modules.

Disclaimer: StitchIt is not yet ready for use. It’s the result of only a few hours work on a Sunday afternoon to serve as a prototype for how CommonJS modules could really be the way to go for structuring large JavaScript applications. The code is probably awful, there’s absolutely no caching so it will rebuild everything on every request and there’s no minification. I hope to make it production ready in the coming weeks but for the time being it’s just something to look at.

That said, let’s dig into a demonstration of how it works. You start by placing all the JavaScript you want packaged into a directory – in this case I’ve used ~/Scripts/app. We initialize StitchIt in the application’s InitializeRoutes method and expose its packaged content on a path.

public static void RegisterRoutes(RouteCollection routes)
{
    ...
    routes.StitchIt()
        .RootedAt("~/Scripts/app")
        .PublishAt("app.js");
    ...
}

Inside the Scripts/app directory we’ll make a couple of our JavaScript files which will form a basic application. For this example I’m using the sample code provided in the CommonJS Module specification itself.

math.js

exports.add = function() {
    var sum = 0, i = 0, args = arguments, l = args.length;
    while (i < l) {
        sum += args[i++];
    }
    return sum;
};

increment.js

var add = require('math').add;

exports.increment = function(val) {
    return add(val, 1);
};

program.js

var inc = require('increment').increment;
var a = 1;
var b = inc(a);

console.log(b); // 2

With these in place we can pull down the file generated by StitchIt in a script tag and then use a require function attached to the global stitchIt object to execute our program.

Index.cshtml

...
<script src="/app.js"></script>
<script>
    stitchIt.require('program');
</script>
...

Wrapping third party JavaScript libraries into Modules

Very few client-side JavaScript libraries are built as CommonJS Modules, so how do we use them from our Modularized code? Let’s take jQuery as it’s probably the first JavaScript library people will want to use.

We’ll ensure that jQuery is loaded separately via a normal script tag before loading the StitchIt content. Then we’ll create a wrapper which instead of attaching an API to exports, will completely replace the module exports with the jQuery object itself grabbed from the window.

jquery.js

module.exports = window.jQuery;

Using this wrapper module we can just require it like a normal library.

app.js

var $ = require('jquery');

$(function () {
    $('body').text('Hi from jQuery');
});

Beyond JavaScript – Managing related client-side templates

If you’ve done much with KnockOut or jQuery templating you’ve probably found yourself dumping templates into script blocks in the main page body. Although this works for simple scenarios, I quickly found this practice horribly difficult to manage for large single page apps. Following RequireJS’s and Stitch’s example, I added support for adding *.html files to your JavaScript application directory. StitchIt will wrap these into a CommonJS JavaScript module so they can be required like any other JS dependency.

views/personViewTmpl.html

<div>
    <span>Hi, I'm ${name}</span>
</div>

views/personView.js

var $ = require('jquery'),
    template = require('./personViewTmpl'); // Just another module

function PersonView(el, name) {
    $.tmpl(template, { name: name }).appendTo(el);
}

exports.PersonView = PersonView;

app.js

var $ = require('jquery'),
    PersonView = require('views/personView').PersonView;

$(function () {
    var davidView = new PersonView($('body'), 'David');
});

Also notice the use of global and relative identifiers for the require call – this allows us to nicely organize our JavaScript into sub-directories that can be as deep as we need. Relative module identifiers are evaluated relative to where that module resides.

CoffeeScript

I know what you’re thinking – JavaScript is so 2010. Well thanks to Jurassic, a wonderful JavaScript runtime for .NET, simply drop in .coffee files to your app directory and they’ll automatically get compiled to JavaScript when the StitchIt package is built.

person.coffee

class Person
	constructor: (@name) ->

	sayHi: () ->
		"Hi, I'm #{@name}"

exports.Person = Person

app.js

var Person = require('person').Person;

var david = new Person('David');

console.log( david.sayHi() ); // Hi, I'm David

What needs to be done before it’s ready?

So there’s enough here to show off what I believe are very important concepts, but it’s still a fair bit off being usable. Currently all of the modules get packaged into a single .js file. Although some may disagree, I prefer wrapping all the code up into a single file which can be downloaded once and then cached forever in the client. The biggest issue now is that this .js file is getting completely regenerated on every request – this may work okay for development but ideally I’ll probably want to compare time stamps or something to only regenerate the package when it’s source files have changed.

I’ll also want to integrate some form of js minification into StitchIt. I’d ideally find a way of using SquishIt (a great tool which inspired this name). If that’s not possible I’ll probably want to integrate Google’s Closure compiler directly into StitchIt.

There’s also currently only a fraction of the CommonJS Module specification implemented. The major requirement will be support for require.paths so we can control where modules are loaded from – although I’m not sure how much sense this makes in a browser at this point.

Grand Ideas

Looking further into the future I’ve got some fairly grand plans. If I support exposing modules from resources in .NET assemblies I can imagine providing very cool NuGet integration support.

Install-Package knockout
...
var knockout = require('knockout');

I also wonder how difficult it would be to evaluate these CommonJS modules in Visual Studio itself to go towards providing some kind of AutoComplete/Intellisense support.

Probably very difficult.

Show me the code

All the code is up at GitHub – StitchIt. Please go browse and fork. I’d love to hear your thoughts.

Advertisements

CSS degrading gracefully

CSS3 has some really nice features – animation, gradients and rounded corners just to name a few. But as with most of this web stuff, you’re going to need an up to date browser like Chrome to really appreciate it. Fortunately older browsers will typically just ignore the newer stuff they don’t understand which makes it possible to deliver a degraded experience.

This afternoon I had a quick bash at creating the visuals for a progress type control. I wanted to take advantage of those nice CSS3 features in Chrome, but still have the control work in IE. Turns out, it’s wasn’t too hard.

Progress bar control in Chrome
Progress bar in IE

All the code and a running example showing the animation can be found on jsfiddle.net. The  CSS does look a little messy as browser vendors put newer CSS features behind a prefix while they’re being standardized. Using a CSS compilation tool like Less can make this a bit more bearable.

First look at Project Silk – Microsoft’s HTML5 development guidance

Project Silk is an example project designed to provide guidance on implementing HTML5 applications from the Microsoft Patterns & Practices bods. I’ve yet to have a good dig through it but at an initial glance it looks promising.

They’ve heavily utilized pubsub in for the js architecture – something I’m particularly bullish on at the moment. Instead of reaching out to something like Backbone.js for their View/Controller interaction, they’ve opted to use jQueryUI widgets. I think there’s more that can be done here but it’s pragmatic start and I’m glad to see them bring attention to the UI widget factory.

; $(function (mstats, $) {
    $.widget('mstats.fillupsPane', {
        options: {
            active: false,
            id: null
        },
        _init: function () {
            mstats.fillupsPane = this;
            mstats.pubsub.subscribe(mstats.events.vehicle.fillups.data, this._newFillupsDataHandler, this);
            this._fetchData();
        },
        fetchData: function (vid) {
            this.options.id = vid;
            this._fetchData();
        },

        // Large bulk of code excluded...

        destroy: function () {
            $.Widget.prototype.destroy.apply(this, arguments); // default destroy
            mstats.pubsub.unsubscribe(mstats.events.vehicle.fillups.data, this._newFillupsDataHandler, this);
        }
    });

} (this.mstats = this.mstats || {}, jQuery));

There’s also a suite of js tests to go with the code in QUnit – I’ve not been able to see if this is integrated into VS yet but I’m glad to see they’re there. They’ve also included a number of automated acceptance tests that can be run with VS which I’m also fond of them promoting.

So there’s a couple of directions I’d like to the project go. (disclaimer: I’ve not had a chance to actually run through the app properly so if I may well be off base on this). Currently the sample Millage application that they’re throwing together seems more of a “traditional web app with lots of javascript”, what I’d like to see is advice on creating a large “single page application”(SPA). SPA’s is currently where having a robust js architecture goes from just a good idea to essential. Of course, most developers are probably still not that far along so maybe they’re on the right side of the fence as far as guiding .NET developers go.

One of the tools that I think is really needed for decent js development in the .NET space is something that will provide dependency management for the client. I’m currently quite keen on a node.js tool called stitch which will bundle all your client js into CommonJS modules so we can use require and exports ala server side node. I’d love something like this for ASP.NET and to be quite honest if I get some spare time I’ll probably take a stab at it myself. I think if this was done well it could be a great direction for how to provide static analysis features like the Intellisense that VS devs love.

One of the first things I’d like to see the Silk team do is actually post their code on CodePlex as opposed to just providing an exe. Having to extract it every time I want to read it gets very annoying…

I’ll write some more once I’ve had a better chance to go through it. But I’d encourage ASP.NET developers to take a look. I think the team can do a lot more, but it’s a great start and something I feel ASP.NET developers will already likely learn a lot from today.

I’ll be speaking on integrating ASP.NET and jQuery at this month’s NYC .NET Meetup on the 15th. I’m sure I’ll find the time to mention Silk as well. Please drop in if you’re interested.


Originally posted on Thoughts from David Padbury.

With ChromeFrame can we consider HTML5 just another browser plugin?

It’s generally agreed that HTML5 today has a roughly similar feature set to plugin based RIA technologies like Silverlight and Flex. This feature set is available today in cutting edge browsers like Chrome, Firefox and Opera.

 

However we rarely get to make that comparison. The biggest barrier to HTML5 adoption is the significant portion of users, particularly in the Enterprise, that are still using older browsers such as IE6-8. At this point we either have the option to attempt to enhance these browsers using shims and polyfills (they can actually do a surprising amount but it’s messy none the less), or just wait until everyone upgrades.

 

Not a great situation if you know you’ll have to be supporting IE users.

 

Another option that I don’t hear talked about much is ChromeFrame. A Google produced IE plugin which seamlessly embeds the Chrome browser (V8 js engine and all…) so that when a site intentionally opts into it, it’ll take over and render the page. This is great as the biggest reason for Enterprise’s not upgrading to newer browsers is the wealth of older web based applications requiring IE. With ChromeFrame, they can carry on using their standard browser and then it’s only applications that specifically ask for being rendered by Chrome that get upgraded.

 

This could make HTML5 a very similar proposition to Silverlight or Flex for Enterprise development. But there’s a big difference – with Silverlight or Flex if you try to run them on a machine or device that doesn’t support the runtime, you simply don’t have a chance in hell. With HTML5, you’ve got a fighting chance of supporting older browsers and other devices like iPhones, Blackberries, and Androids. Isn’t that kind of compelling?

 

Of course even with the best available HTML5 application runtimes, today there’s still significant challenges delivering large HTML5 applications. Tooling and frameworks are the first things to come to mind but that’s probably fit for another post.

 

So I wonder why ChromeFrame isn’t talked about more. Could it be that Enterprises prefer more established browser plugins like Silverlight or Flash to the rather niche ChromeFrame?


Originally posted on Thoughts from David Padbury.

WCF WebSockets: First Glance

I finally got around to playing with the first drop of WebSockets support for WCF. I’m pretty familiar with WebSockets as I’ve been using Node.js to play around with them for quite a while now. The server API in Node.js is wonderful as it couldn’t be simpler – to demonstrate this, take a look at how we’d create a basic echo server.

var ws = require("websocket-server");

var server = ws.createServer();

server.addListener("connection", function(connection){
  connection.addListener("message", function(msg){
    server.send(msg);
  });
});

server.listen(8080);

When I heard that Microsoft was planning on integrating WebSockets into WCF my first thoughts weren’t all that positive. Now I know it’s not quite the case since .NET 4, but WCF has had a reputation for rather large complex API’s and heaps of XML. I was dreading what a WCF take on the above would look like. Turns out, perhaps I should be a little more optimistic.

class Program
{
    static void Main(string[] args)
    {
        var host = new WebSocketsHost(new Uri("ws://localhost:4502/echo"));
        host.AddWebSocketsEndpoint();
        host.Open();
        Console.ReadKey(true);
        host.Close();
    }
}

public class EchoService : WebSocketsService
{
    public override void OnMessage(JsonValue jsonValue)
    {
        Send(jsonValue);
    }
}

That’s really the kind of simplicity I was really hoping to see. Now the interesting question is how it’s going to look when dealing with multiple clients. Node kind of has an advantage here as it’s entirely single threaded (yeah, I said advantage), but with .NET’s recent concurrent collections and a rather nice API for dealing with events (Rx) I’m feeling pretty hopeful.

HTML5 Live Conference

I recently attended HTML5 Live, a single day conference here in New York. The conference was pitched as an event focusing on HTML5 for enterprise developers instead of designers. I was surprised to see a number of familiar faces from both the industry and the .NET/Java community. It seems that although HTML5 applications are not yet being widely developed, they’re very much on people’s minds. That said, despite the enterprise focus, once we sat down nearly everyone pulled out a MacBook or iPad, just like any other web event…

The Keynote was given by Jeffery Hammond, an expert on application development from Forrester. In my opinion, his talk was pitched perfectly. Mainly, that HTML5 has recently been extremely hyped and as people actually get into it they should be aware that they’re going to be disappointed. Although that seems like quite a negative way to start a conference on the subject, he followed it with a very pragmatic look at where HTML5 currently is, and where it’s going to be in the coming years.

HTML5 on the Gartner Hype Cycle

I found how he described us as being at the top of the “inflated expectations” curve particularly amusing as many of these I’ve seen first hand-

  1. Marketing is keen to “add HTML5”
  2. Many technologies and API are keen to be grouped with HTML5, despite being quite unrelated.
  3. “Flash/Silverlight are dead”
  4. HTML5 is cited as the solution for just about everything.
  5. Mainstream media have already picked up on the trend. (I think he referenced the New York Times)

The bottom line of his talk was that HTML5 today is an attractive option for applications that want a broad reach across mobile devices. Whereas there are still large gaps in desktop browsers, and the tooling is still very much evolving. Everyone will find HTML5 “ready” at a different time – know your users.

Adobe and Microsoft were also both represented at the event. Representatives from both companies were keen to show their love for the new HTML5 platform; I think both had “We ♥ HTML5” in their slides. On Adobe’s front, they started by demonstrating some of contributions they’ve made to webkit as a result of it’s integration with AIR – clearly making the point that HTML5 and their RIA platform can have a mutually beneficial relationship.

As for Microsoft, I was wondering quite what they would have to talk about apart from IE9. Although I expect that we’ll see some HTML5 tooling from Microsoft in the near future, currently there’s not much to speak of. I was pleasantly surprised to hear a great overview of HTML5 and a look at the tests that Microsoft are committing to various specifications. Although not much of it was particularly new to me, I’m glad that this is the kind of talk Microsoft will be giving to introduce developers to the platform. Of course there was the mandatory IE9 fishes demo.

Like most events, I found the most interesting part meeting other attendees after the talks were over. Most of the people I spoke to were there simply to gain some understanding of what HTML5 actually is. It’s clear that despite the recent buzz (or cynically maybe due to), HTML5 is a confusing term for a lot of developers, especially in the enterprise.

With CSS, LESS is more

I’ve got a bit of a secret – I’ve never really liked CSS.

Don’t get me wrong, it’s a nice idea. I’m certainly all for separating content and presentation. And yes, most importantly, it really does work in practice. Just take a look at any of the many CSS galleries that demonstrate how powerful it can be in hands much more talented than mine. However, just like there are some awesome php applications, it doesn’t mean I actually like php.

The problem I have with CSS is that I find I repeat myself a lot. For example, if you have a standard colour or margin width that is reused throughout a site, you have to repeat it every time it’s used. Now one of the first things I learned about programming as a kid is that if you’re repeating the same value, then put it in a variable. CSS, however, doesn’t have variables. Another example is when you have to frequently reuse a series of CSS declarations, like setting colour, font-family and font-weight. There’s no easy way to do that in CSS either. DRY is one of the most basic software engineering principles, and to be honest, CSS completely fails.

LESS is a Ruby tool that extends CSS to make developing it a more pleasant experience. It provides a language that is a superset of CSS – it’s the same language we know but adds variables, mixins (like methods) and basic arithmetic, among other things. Once you’ve written these LESS files, they will be compiled to plain CSS that will work just the same in any browser. Although LESS is a Ruby tool, it doesn’t mean you have to start developing everything in Ruby. In fact, the first project I used LESS on was a Java project, and the idea of requiring a Ruby install on every developer’s workstation and the build server was less than attractive. Fortunately, JRuby is an excellent Ruby runtime and installing a gem (such as LESS) couldn’t be easier. We even put JRuby into our project’s tools directory and checked it into source control. That enabled every developer and the build server to happily run LESS as part of the build process with absolutely no setup or installation.

To demonstrate what LESS can do, let’s start with a great example of CSS. Recently Christian Heilmann put together an excellent Sticky Note Effect demo to show how powerful CSS3 can be. If you’ve got a reasonably recent browser, check it out here.

CSS Sticky Note Demo

It’s a wonderful demo, but if you were to take a look at the CSS, you’d see a lot of repetition. For example, due to the way browsers have evolved support for some of the more recent CSS standards to define rotation, we have a large number of rules for a simple declaration.

-webkit-transform: rotate(-6deg);
-o-transform: rotate(-6deg);
-moz-transform:rotate(-6deg);

With LESS we can make this much more attractive. By simply defining a rotation mixin once with these rules, it becomes only a single statement to reuse them.

.rotate(@rotation) {
    -webkit-transform: rotate(@rotation);
    -o-transform: rotate(@rotation);
    -moz-transform:rotate(@rotation);
}

/* Then to use... */
ul li {
    .rotate(-6deg);
}

Another part of the language that I really like is the ability to nest rules. Take this example from the sticky note css:

ul{
  overflow:hidden;
  padding:3em;
}
ul li{
  margin:1em;
  float:left;
}
ul li a{
  text-decoration:none;
  color:#000;
  background:#ffc;
  /* lots more... */
}

The above is setting rules for ul elements, any li child elements of that ul and then any a elements of those li’s. In this example it’s not too bad, but imagine that the parent element was a specific id. If you wanted to change that id, you’d then have to change every rule related to it. LESS makes this scenario blissfully simple with nested rules. To write the same as the above, the following would be used:

ul{
  overflow:hidden;
  padding:3em;

  li {
    margin:1em;
    float:left;

    a {
      text-decoration:none;
      color:#000;
      background:#ffc;
      /* lots more... */
    }
  }
}

To finish these examples with where I started, LESS adds variables, which means we can define a named value once and reuse it throughout our styles. Need to change it? Just change it in one place.

@background: #666;

body {
   background-color:@background;
}

While LESS is already a super useful tool for developing CSS, the next version is shaping up to be even more exciting. LESS version 2 is written entirely in JavaScript. On the server, this will be run on node.js, but it can run directly inside the client’s browser. To demonstrate this I’ve rewritten the Sticky Note CSS in LESS, and instead of compiling it into CSS first, I’ve just included the new version of less.js and a link to the LESS file. Take a look at it here. When you hit the page, it’ll immediately download the LESS source, compile it and apply it to the page. Is this a good idea for a production deployment? Probably not. That said, it’s a very quick process so it’s an option. It’s certainly great for a demo like this and very useful during development. Dmitry Fadeyev has a great writeup on less.js here.

Find more information on LESS here. If you’re an die-hard .NET developer, you may want to check out the .NET port of LESS – dotLess.