Threaded JavaScript: it works!

In the context of mainstream JavaScript, multithreading often gets to play the role of the bogeyman: When somebody starts complaining about the pains of callback-based asynchronous programming, chances are someone else comes along to paint a dark picture of deadlocks and race conditions that come with threads. The conclusion then is usually that callbacks are the lesser evil, or maybe that coroutines or actors are the way to go, but unfortunately aren’t available in JavaScript today. (Actually, coroutines are part of JS 1.7 and scheduled for inclusion ES Harmony in the form of generators - yay!)

Not that this picture is completely wrong. Threads are tricky when combined with shared mutable data. If you think you can master them then you probably haven’t done a lot of it. I’ve done some over the years, and it makes Erlang with its actor based message passing, lightweight processes, and immutable data look quite attractive. Clojure and Scala have both been very successful in bringing different aspects of Erlang to the JVM. For JavaScript, the mainstream programming model has been event-based single threaded operation, which makes perfect sense for clients, and is fine for servers with low or medium workload.

The problems with single threaded event loops (and similarly, purely cooperative multithreading) on the server start to show when concurrency increases. Only scheduling on I/O may not be enough when you have hundreds or thousands of requests going on at the same time, and this will show in increased latency and reduced throughput. If you want to write a server that performs well under these conditions, you really want to use both preemptive multithreading and nonblocking I/O.

So preemtive threading does provide great performance characteristics that are hard to get by any other means. But then JavaScript spoils it with its mutable data model, right? Well it’s actually not that bad. Having worked on and with multithreaded server-side JavaScript for 10+ years, here are two points I’d like to make:

1. Threads are not evil, shared memory locking is.

2. Depending on coding style and environment, multithreaded JavaScript can range from piece of cake to arcane science.

The following is based on the actual implementation of and experience with threads in Rhino and Ringo. As a historical note, Ringo started out with shared-nothing threads where each thread got its own version of each module scope and modules were able to opt-in to shared state. Since shared modules proved to perform much better and be easy to write, that has been reversed and modules now default to shared.

The good: functional == lockless

The great news is that if you write your programs in a functional style, you may not have to worry about concurrency problems at all. Functional means that your functions should not rely on mutable state outside themselves and thus be free of side effects. Let’s repeat this, because it’s really cool: If you program functionally you get all the benefits of multithreading, such as preemtive concurrency and scaling across processor cores, without any of the downsides, such as race conditions or locking.

Fortunately JavaScript supports functional, side-effect free programming quite well compared to, say, Java, where objects typically encapsulate mutable state. Unfortunately, it also supports the exact opposite of functional programming with its global object and pseudo-classes. Ringo improves this a bit by getting the global object out of the scope chain and thus out of the way of the programmer. The only way in Ringo to set a global variable is by directly assigning to the global property.

Also, web facing JSGI modules typically consist of a flat list of function. Let’s look at a simple JSGI module that will run multi-threaded on Ringo:

var response = require("ringo/jsgi/response");
var mustache = require("ringo/mustache");

exports.index = function(req) {
    return responseHelper("./templates/welcome.html",
        {title: "Hello World!"});
}

function responseHelper(template, context) {
    var temp = getResource(template).getContent();
    return response.html(mustache.to_html(temp, context));
}

As you can see, the only thing declared outside the index and helper functions (and thus the only thing that is shared between threads calling these functions) are the module import statements. If these modules are thread-safe (ideally by using the same functional style of programming), your whole application will be. And you could easily move read-only data such as the cached template content out into module scope without too many worries.

Now JavaScript is not a purely functional language. Eventually, you’ll want or need to have shared mutable data somewhere. We’ll look at ways to do that in the last section. But before that, I’d like to talk about one otherwise great feature of JavaScript that can have inconvenient side effects when combined with threads.

The bad: closures == shared state galore

Closures are kind of an “impurely” functional concept where functions are able to access their containing scopes, even when called from somewhere else. This is a highly useful concept that allows you to attach data to functions when passing them around. Unfortunately, when running a nested function in a separate thread, it’s also recipe for race conditions.

What makes this from a theoretical to an actual problem in Ringo is the existence of the ringo/scheduler and ringo/promise modules, both of which take a function as arguement and execute it in a new thread. Of course, the most convenient way to use these modules is to define the callback functions in place as closures.

For example, here’s a (highly contrived) example that uses the setTimeout function from ringo/scheduler to create a race condition:

var {setTimeout} = require("ringo/scheduler");

exports.doSomething = function() {
    var message = "original";
    setTimeout(function() {
        message = "modified";
    }, 0);
    return message;
}

In contrast to event-loop based JavaScript, setTimeout calls the function in a separate thread. Since it is called with 0 as second argument, the function may be started immediately, running concurrently with the rest of the containing function. Thus, it’s unpredictable whether the inner function set message to “modified” before the outer function returns it or not.

Dealing with shared state

Having seen how multithreading affects JavaScript, here are some ways Rhino and Ringo let you cope with it.

Visibility

Visibility in the context of multithreading means that writes to a variable by one thread are visible by other threads. This is a challenge with modern hardware with its multiple cores, multiple levels of caches.

Rhino solves the visibility problem on a fundamental level by declaring specific fields of its JavaScript object implementation as volitile. Thus, when working with JavaScript objects and scopes in Rhino you should not normally experience problems of visibiltiy. Changes done by one thread should be immediately visible to others, and you can generally expect JavaScript objects and arrays to behave well under concurrent access.

Locking

As we saw above, concurrent access to mutable data usually requires locking. Rhino provides a sync function that corresponds to the synchronized keyword in Java. sync takes a function and an optional lock object as arguments, and it returns a function.

sync(function() {...});
sync(function() {...}, lock);

In the first form, the function returned by sync synchronizes on the this-object it is invoked with. In the second form the returned function synchronizes on the lock argument. Synchronizing means that at each point in time at most one thread may run any function synced on the lock object. When another thread tries to invoke a synced function on the same object, it will be blocked until the first thread exits its synced function.

Let’s look at an example of the closure above that fixes the race condition using synced functions:

var {setTimeout} = require("ringo/scheduler");

exports.doSomething = function() {
    const lock = {};
    var message = "original";
    var fn = sync(function() {
        setTimeout(sync(function() {
            message = "modified";
        }, lock), 0);
        return message;
    }, lock);
    return fn();
}

We’re adding a lock object at the same level as the data we want to protect. (Note that we wouldn’t need an explicit lock object if the functions we want to synchronize were invoked with the same this-object.) Then we pack both the remaining code of the outer function and the inner function into a synced function. Since the outer synced function is alway invoked before the inner one, the timeout callback has to wait until the outer synced function is done. Thus, this version of doSomething consistently returns “original”.

Locking is a very complicated topic that is impossible to cover in any depth here. I’ll just say that coarse grained locking is generally easier and less risky but also less performant than fine grained locking. When dealing with multiple locks it’s a good idea to always acquire them in the same order to avoid deadlocks.

However, a good alternative to doing your own locking is to use data structures that are already written for concurrent use.

Concurrent data structures

The java.util.concurrent package provides classes that don’t just wrap collections with synchronized methods but provide special implementations that get by with minimal or no locking. These classes are available on any Java virtual machine and Rhino’s Java integration makes using these classes very easy.

Even more advanced are Clojure’s persistent data structures. These ingenuously designed collections are immutable and thus thread-safe, but provide mutation methods returning new versions of themselves that shared as much as possible of its structure with the old version. Since these data structures are immutable they get away without any locking and are surprisingly fast.

Summary and outlook

As we have seen, multithreading in JavaScript has its highs and lows. In my opinion the pros clearly prevail. Still, I’m thinking about how the few problematic spots could be improved, especially concerning the ringo/scheduler and ringo/promise modules.

The fact that JavaScript functions are closures over mutable shared state is maybe the biggest stumbling block when trying to implement safer threading constructs such as Erlang/Scala like Actors. The way it is, the smallest unit of code we can deal with in a side-effect free way is the module, which is not very satisfactory.

One solution I’m considering is an isolate function that takes a function as argument and detatches it from its parent scope, making it behave like it was defined in global scope (which is practically read-only in Ringo). It’s also conceivable that an actor framework might do this automatically before invoking a function.

One potential source of inspiration is Clojure. Clojure’s agents might be a better fit for JavaScript than actors. Instead of sending data to a piece of code, you send code to a piece of data to replace it asynchronously. Also, Clojure’s persistent data structures described above look like a nice fit for Ringo.

The biggest problem with threads as I see it is that they are not visible when looking at the code. You just have to know about them. Once you do you’re much better prepared to sail around the dangerous cliffs. I hope this article helps getting closer to that goal.

If you found this interesting please consider giving Ringo a test drive. You may like it!

Comments