I was wondering if we could consider a concurrency module to help with
some basic concurrency constructs, in particular for working in a shared
memory threaded environments. While writing middleware for Jack, I
realized it can be almost impossible to write certain functionality in
thread safe manner without some concurrency support, like at
locks/mutexes, that are lacking from raw JavaScript. I realize that
there are certainly alternate concurrency models that do not require
locks, but I didn't think that CommonJS was attempting to force a
particular concurrency model (and I am not sure that it should either).
With the possibility that modules could be run in a threaded
environment, it seems important to have something to safeguard against
race conditions. For example, Jack runs on Jetty and Simple using
multiple threads with shared memory. While such a topic often sparks
responses suggesting that something like erlang's messaging would be
nice, but to really seriously consider that I think we would need a real
proposal and even then I am afraid it would be quite onerous to require
that something like Jack and Persevere be rewritten to follow a message
passing model, especially while preserving performance and scalability.
I think for where we are right now, we have to consider that the
concurrency model is an unknown, and provide a way for modules to
safeguard themselves from race conditions.

Anyway, here is a real simple, rough start to get going, based on some
of Java's stuff. I don't really care if it changes, I just want
something to be able to write cross engine middleware that is
thread-safe (or have someone explain how I can do it with what we have
right now).

require("concurrency").Lock -> constructor for a reentrant mutual
exclusion lock.
require("concurrency").SharedLock -> constructor for a reentrant shared
lock.
require("concurrency").ThreadLocal -> constructor for a thread-local
variable.

var lock = new (require("concurrency").Lock)();
lock.lock() -> aquires a lock
lock.unlock() -> attempts to releases a lock
lock.tryLock() -> try to aquire a lock

var threadLocal  = new (require("concurrency").ThreadLocal)();
threadLocal.get() -> get the value for the current thread
threadLocal.get(value) -> set the value for the current thread

And we could consider something like Rhino's "sync" function.
Thanks,
Kris

Hi, Kris;

Writing programs using multiple threads of JavaScript, what I call "MT JS"
and JS in a multi-threaded environment (thread-safe JS) are two quite
different things.  I know you know this, I'm just setting the stage for
further discussions.

What you're talking about is "MT JS", where two separate threads of
JavaScript share access to the same JavaScript objects.

This is possible in SpiderMonkey (caveat reador, bugs in 1.8+), I guess it
must also be possible in Rhino?  I'm not sure it is in JSCore or V8, would
love to hear from the peanut gallery.

Do you have a syntax for creating Threads from JS, or are you creating new
threads of JS from Java and then allowing access to the objects "from the
back"?  If you're planning to create Threads from JS, I think you should
consider the API that GPSEE uses:
http://www.page.ca/~wes/opensource/gpsee/docs/modules/symbols/Thread....<http://www.page.ca/%7Ewes/opensource/gpsee/docs/modules/symbols/Threa...>

I don't yet have a locking API -- haven't need one for my simple cases,
jsapi serializes property accesses which is enough.  But your mutex API
looks basic, and good. lock, unlock, tryLock. Perfecto.

I think the litmus test for "is the level-level lock API good enough" is --
can we write code like this out of it?

var myLock = new require("concurrency").Lock;
withLock(myLock, function() { doWork(); });

I believe the answer is yes:

function withLock(lock, fn)
{
   try
   {
     lock.lock();
     fn();
   }
   catch(e)
   {
     throw(e);
   }
   finally
   {
     lock.unlock();
   }

require("concurrency").Lock -> constructor for a reentrant mutual

Why do we need shared locks, and how do you see those working?  These are
just member read locks?  If so, how are updates handled?  Also, I believe
SharedLocks could be implemented in pure JS from exclusive locks, meaning it
may not be necessary to spec these now.

Instanciating real thread local variables requires modification of the
JavaScript interpreter.   I do not see that happening.  I think what you're
going after should be called Thread Local Storage. Which is is totally
doable.

var threadLocal  = new (require("concurrency").ThreadLocal)();

I take it you meant "get" in that last line.

I was just about to propose a CAS additional to this spec, but I just
realized that it's not possible with generic JS types.  The syntax I
considered was

v = require("concurrency").compareAndSwap(v, old, new);

The problem is that there is no lockless mechanism for assigning to v, as JS
is pass-by-value. Unless..

if (!require("concurrency").compareAndSwap(scope, "v", old, new))
  print("failure");

I still believe that my CAS idea a bad one: just throwing it out there to
keep others from going down that same path.

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

The immediate situation where I was thinking about this was in Jack and
Persevere where the threads are created by the servlet engine. The JS
isn't actually creating the threads, but must deal with the fact that
may be executed in multiple threads. Of course being able to actually
create the threads in JS would be good as well. Your API looks good to
me. I know Rhino's shell also has a "spawn" function (simple executes
the callback parameter in a separate thread).

There are times when shared locks (that can acquire shared/read or
exclusive/write) are helpful, but you are right, they can be implemented
with simple locks. Just thought I would throw it out there.

yeah.
Kris

I'd like to also support implicit unlocking, something like:

require("concurrency").Lock(function() {
//executes within lock/unlock boundaries

ugh, that would be more like:

var lock = new (require("concurrency").Lock)();

lock.do(function(){

On Sep 22, 7:39 am, Neville Burnell <neville.burn...@gmail.com> wrote:

which is what Wes described also  .... I need coffee 8-)

On Sep 22, 7:56 am, Neville Burnell <neville.burn...@gmail.com> wrote:

Hi folks,

To the extent that individual modules need to defend their consistency
of their state, they must assume *some* concurrency model. If a given
CommonJS module can be used in both threaded and non-threaded systems,
it must have a "worst case" to fall back onto. This worst case might
on the face of it be to build an implementation, and expose an API,
that includes explicit locking and thread-aware usage patterns. Hence
leaving the question open has forced everyone into coding for threads.
Now, how will the threaded primitives be supported on a non-threaded
system?

Does this mean we end up with 3 "flavors" of modules: (a)
non-threaded; (b) threaded; and (c) purely functional?

Perhaps I should ask this from another angle: JS programming has thus
far proceeded via event loop concurrency, and it seems like a
direction that is far less error-prone than _ad hoc_ threads and
locking. What is wrong with maintaining that?

Ihab

--
Ihab A.B. Awad, Palo Alto, CA

By way of a concrete proposal here --

What I *do* think CommonJS should standardize is an API for building
and living within "workers", each of which is single threaded, and
which pass data to one another asynchronously. This should be similar
to Gears and HTML5 workers, and should be informed by those efforts.

Ihab

Ihab:

I, too, think that JS workers are a good idea. There is another use-case to
consider, however.

If an existing MT application wants to run multiple threads of JavaScript:
how do we handle this?

Individual property access is not difficult; SpiderMonkey (and I assume
Rhino) offer serialized property access.

The tricky part is maintaining atomic updates across multiple properties of
the same object. This really is tricky, and we'd need to "stamp" modules
MT-safe if we wanted to support this in CommonJS. This means that we need a
theory of thread-safety for modules which purport to be MT safe, and it
should be consistent everywhere.

I think the following rules may be sufficient for this:
 - Underlying JavaScript engine provides serialized property access
 - require() is guarded by a mutual exclusion lock across the entire JS
runtime (all threads)
 - modules are responsible for their own theory of thread safety for
non-exported functions and variables, and can assume the presence of a
level-0 concurrency module
 - exported functions which accept non-Atom arguments must not modify those
arguments
 - exported functions which return non-Atoms must return new non-Atoms
 - Boolean, Number and String are considered Atoms.

Note also that I really believe very very strongly that absolute
minification in the theory of thread safety is absolutely paramount.

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

how do we handle this?

Forgot to clarify this -- recall that CommonJS isn't necessarily about
writing programs in JS, it's also about augment existing applications with
JavaScript.  One such example is the Apache 2 web server running the
threaded MPM. What if we .... wanted to write a JavaScript database API
which pools connections?

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

I think this is indeed a reasonable alternative (and I appreciate the
concrete proposal, I most wanted to avoid indefinitely deferring a
specification because some other model might be better without any real
solid way to move forward). A few questions/thoughts (not necessarily in
any coherent argument):

First, this is a real uphill battle. CommonJS has barely started and we
have already been herded into shared memory threading with Jack and
others due to the underlying concurrency models of the JVM and C, and
the least resistance path of building on Jetty, Simple, etc. Keeping
shared nothing concurrency (with event loops) wouldn't be a bad path,
but it'll be non-trivial to keep on that path.

Second, it seems like CommonJS has had a fairly low-level focus so far.
Naturally the low-level aspect of concurrency is shared memory threading
(that which is exposed by the hardware and what the engines must deal
with in concurrent situations). Even if we want to everyone to be doing
shared nothing architecture, should we still define concurrency
constructs like thread constructors, thread-locals, and locks in order
to be able to compose shared nothing event loops in JavaScript by using
the lower level constructs? I suppose if we expose these things, we
can't guarantee that everyone will always build on the higher level
event loops.

Also, I wanted to actually think through what event looping would look
like in one of our primary target applications, that is the web server.
Of course a web server needs to be able concurrently process requests.
If we are using event loops with nothing shared between threads, we need
separate environments for each thread (and a unique JSGI app created for
each one). This introduces some funky non-determinism. With just a few
requests coming into something like jetty, you could very well have
every request fulfilled by the same JSGI app, since every request might
finish before the next came in. A programmer may (perhaps inadvertently)
be storing some state in a per-app variable (in a closure inside the
middleware, outside the app), and expect it to be able available to all
requests. It is not until requests come in at a fast enough pace to
force multiple threads to handle requests, that another JSGI app becomes
utilized, and that variable is no longer shared across all requests.

Also, if the inter-worker communication is asynchronous (which I presume
is preferable to avoid deadlocks), it means that any type of information
sharing between workers will force us into the added complexity of
callbacks or promises (preferably promises). Something as simple as a
server-side session management would seem to entangle us into fairly
complex asynchronous code.

That being said, this is appealing to me if others think we can really
make it go. The fact that there is an existing API for this in HTML5
could perhaps allow us to sidestep a lot of bike shedding. If we think
it is desirable, maybe we could make an event-loop based fork of Jack to
try it out. I might do it sometime, unless someone else jumps on it.
Thanks,
Kris

I agree with Ihab - it would be nice if CommonJS defined APIs for a
single processes/worker and not enter the realm of threads. Javascript
does not have threads - there is not any proposal to have threads.
While some/most implementation can handle threads - it seems rather
unnecessary to leave behind ES5 and HTML5 just for lack of
imagination. Concurrency should either be done with an event loop
(which requires much more devotion to asynchronous APIs) or with
workers.

Node.js achieves high concurrency on a single event loop. Here is a
"hello world" web server benchmark I ran yesterday for a talk:
http://s3.amazonaws.com/four.livejournal/20090921/bench.png
There is no need to define extensions to javascript (threads) to write
servers - indeed, evidence suggests that doing so damns the server to
suck.

2009/9/22 Ryan Dahl <coldredle...@gmail.com>:

To me the threads model feels natural, but I know that's just because
I've banged my head on it for long enough. So I'm all for considering
fresh (or, as is the case for the event loop, refurbished) concepts.

Out of curiosity, is that Helma 1 or NG? Is there a blog post or
presentation for that talk?

Hannes

I'm interested in the detail also.

That's helma-1.6.3 using this http://helma.pastebin.com/f1d786c9 to
serve the request. (v8cgi 0.6, narwhal d147c160, node 0.1.11.) I know
this isn't fair to present benchmarks this way, without scripts,
methods, or data. I'm just being lazy. I did a more detailed benchmark
a few weeks ago (without helma)
http://four.livejournal.com/1019177.html and the setup is similar.

The talk was just a short lightening presentation:
http://s3.amazonaws.com/four.livejournal/20090922/webmontag.pdf

Do you plan on making Node be JSGI compliant? It does not look like
there is any conflict between the JSGI API and your fundamental
processing model. Or would one need to run something like Jack on top of
Node to utilize cross-server applications? I was curious what you meant
by "Narwhal". Narwhal's http module is a HTTP client, not a server
(maybe thats why it scored low!). Or did you mean Jack? And if so, which
engine was it running on? Having such low numbers with Simple would seem
suspect since Simple has such a high performance characteristics with
fairly little overhead added by Jack (or maybe there is something
expensive in there that I missed).
Kris

JSGI compliant - probably not. At least partially CommonJS compliant -
probably. At the moment I'm letting node evolve on its own.

I don't like that JSGI forces one to determine the response code and
response headers in a single function. One cannot for example, access
a database before returning the status code. Maybe I want to 404 if a
record does not exist. Well, one can do this if your database call
blocks execution inside the jack handler, but blocking implies
allocating a new execution stack with, if not threads, then
coroutines. Allocating a 2mb execution stack for each request is
rather an unacceptable sacrifice to be had at the API level, even
before an implementation is around to further slow down the server.

So, perhaps Node will have proper coroutines at some point (the
current implementation is rather bad). And perhaps someone will write
a JSGI interface on top of Node's more general HTTP interface, and
then people can access databases and wait() for the promise to
complete, and return 404 from the same function. This is not ruled
out.

The exact code being used was the same as described in
http://four.livejournal.com/1018026.html (which is a different link
than the one I gave above).

Oh, you aren't the one responsible for node?

This is perfectly doable with the JSGI model plus promises that we have
discussed, as long as the JSGI server does not grab the status from the
response object until it needs to. Here is an example JSGI App with
async database interaction, completely inline with your event loop
model, no threading or coroutines needed (although I wouldn't mind some
coroutine sugar ala JS 1.7 generators, but its not necessary):

app = function databaseApp(env){
   var resultPromise = doAsyncDatabase("select * from table where id = 3");
   var response = {
     status: 200, // the JSGI server should not write the status until
the first body write
     headers:{},
     body: {forEach: function(write){
         return resultPromise.then(function(resultSet){
            if(resultSet.length == 0){
               response.status = 404; // change the status before the
forEach promise is fulfilled
            }
            else{
                write(JSON.stringify(resultSet[0]));
            }
         });
     }}
   };
   return response;

Yeah, I guess we could do that, although hopefully it is apparent that
JSGI + promises API would not actually hinder Node (and it would be cool
to be able to run JSGI apps directly on it).
Kris

It also seems about 1.8MB too heavy, but I suppose that's neither here nor
there. Are Rhino contexts REALLY that big, thought?

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

Okay - I stand corrected. That's an important feature!

Does seem rather convoluted, doesn't it? I would still rather see
something like:

  server.addListener("request", function (request, response) {
      var resultPromise = doAsyncDatabase("select * from table where id = 3");
      resultPromise.addCallback(function (resultSet) {
           if (resultSet.length == 0) {
               response.sendHeader(404, {});
           } else {
               response.sendHeader(200, {});
               response.sendBody(JSON.stringify(resultSet));
           }
           response.finish();
      });
  });

I agree that the JSGI/promise approach is a more awkward than your
example, but I think think the philosophy is that JSGI allows the simple
case (of a basic sync function) to be very simple, and you can deal with
extra complexity as needed. In that sense, I think JSGI strikes a great
balance, making the common use cases very easy to code while still
keeping it possible to do more advance async processing.
Kris

Another thing we could consider is allowing promises to be returned
directly from the JSGI app, rather than just from the forEach call as I
demonstrated. Returning a promise from a JSGI app would look like:

app = function databaseApp(env){
   return doAsyncDatabase("select * from table where id = 3").
        then(function(resultSet){
            if(resultSet.length == 0){
               return {status: 404, headers:{}, body:""};
            }
            else{
               return {
                status: 200,
                headers:{},
                body:JSON.stringify(resultSet[0])
               };
            }
         });

That reads a lot nicer, but it would put extra burden on servers and
middleware to have to look for promises at two different places (you
still need promises from forEach in order to facilitate streamed responses).
Kris

It's worth noting that most app developers will not be coding to "raw JSGI".
You could make an adapter to JSGI+Promises that looks like that, right?

Kevin

--
Kevin Dangoor

work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com

Yeah, that is a heavy burden. The Python folks apparently had a lot of
discussion about async WSGI and they were talking about "async-aware"
middleware. A solution that doesn't require every piece of the stack to have
to inspect the return value is a big win.

Kevin

--
Kevin Dangoor

work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com

On Sep 22, 2009, at 7:06 AM, Kris Zyp wrote:

I suspect doing this would break a lot of middleware that looks at the  
response. But then again, the whole idea of JSGI middleware won't work  
at all with a purely async interface like Node's, AFAIK.

The only thing I'd worry about is tricky unexpected behaviors, but I  
suppose that can mostly be solved through documentation (listing which  
middleware is partially or fully async compliant)