Apologies in advance, but I added coroutine support to v8 + node

[Marcel Laverdet]

First off, this email is not in response to:

http://groups.google.com/group/nodejs/browse_thread/thread/c334947643c80968

And is in spite of:

http://groups.google.com/group/nodejs/browse_thread/thread/183aa44339a7c073

I don't keep up with the nodejs list too much so I didn't know I was walking into an existing flame war, but I feel that I'm bringing something pretty valuable to the table. If this is unwelcome and I should take it to the barren "javascript-coro" list please let me know.

Last weekend I got started on adding coroutine support to node, and this weekend I'm finishing up on something reasonable. v8cgi has had something similar for a while but under the hood it uses pthreads which leads to non-determinism, and lacks yielding return values. This is good for certain cases, but didn't fit my needs. So I figured out a way to get /true coroutines/ working in v8.

We've all heard the pros and cons of fibers, so I'll keep this email short. I really think this extension will make "future" implementations much more elegant, among other things. It gives you the ease of synchronous development while still working within an event-driven server.

Here's a quick example of a fiber:

    $ cat sleep.js 

    require('./node-fibers');

    var util = require('util');

    

    function sleep(ms) {

      var fiber = Fiber.current;

      setTimeout(function() {

        fiber.run();

      }, ms);

      yield();

    }

    

    Fiber(function() {

      util.print('wait... ' + new Date + '\n');

      sleep(1000);

      util.print('ok... ' + new Date + '\n');

    }).run();

    util.print('back in main\n');

    

    $ ./fiber-shim node sleep.js 

    wait... Sat Jan 22 2011 14:50:39 GMT+0900 (JST)

    back in main

    ok... Sat Jan 22 2011 14:50:40 GMT+0900 (JST)

You can get the code (and full documentation) here:

https://github.com/laverdet/node-fibers

Linux & OS X only; sorry Windows kids.

While I wouldn't consider this extension "production ready" I wouldn't say it's buggy either. At this point I'm looking for people seriously testing this for issues.

Thanks!

[Liam]

Very cool!

Examples of using this for a sequence of async fs functions?

On Jan 21, 9:59 pm, Marcel Laverdet <mar...@laverdet.com> wrote:
> First off, this email is not in response to:http://groups.google.com/group/nodejs/browse_thread/thread/c334947643...
>
> And is in spite of:http://groups.google.com/group/nodejs/browse_thread/thread/183aa44339...

[Łukasz Mielicki]

Impressive!

I was wondering if it would be doable to bring such extension as native module, you have proven that right : )

Just of curiosity, why are pthreads hooks necessary? Is there a way to run separate v8 instance in other thread from node?

I love the Fiber object although maybe maybe we could have API compatible with Rhino continuations for portability?

npm package would be great as well : )

Also please fix the shebang in fiber-shim to #!/bin/bash (it require bash).

Thanks,

Łukasz

[Marcel Laverdet]

@Liam

> Very cool!
> 
> Examples of using this for a sequence of async fs functions?

Well this just lays the groundwork that library authors need to create something more useful. What I'm envisioning is a future library which would just have a global wait() function. So you could grab a handle to 3 futures and do something like `wait(future1, future2, future3)` and it would wait until all 3 futures have been resolved before returning control to your fiber. You can write a dirty implementation of this pretty easily:

    function resumer(times) {

      var fiber = Fiber.current;

      return function() {

        if (!--times) {

          fiber.run();

        }

      }

    }

    

    var fn = resumer(3);

    getUserData(fn);

    getPageData(fn);

    writeLogs(fn);

    yield();

The `resumer` function would call run() on your fiber once you call the callback N times. But that's not the kind of code I would expect to see in a real project.

@Łukasz Mielicki

> Just of curiosity, why are pthreads hooks necessary? Is there a

> way to run separate v8 instance in other thread from node?

Uh.. the reasons for this are pretty complicated. I could have done this without hooks but it would have required patches to v8, specifically I would have had to make v8::internal::Thread or v8::internal::ThreadManager aware of my coroutines. I decided that it was best to leave v8 unpatched in order to ease early adoption.

Basically v8 has a habit of walking up the stack frame by frame doing whatever (collecting garbage, recompiling code, etc). And if you jumped to a new stack without telling v8 about it everything explodes.

However, v8 is actually thread-safe and can handle multiple stacks as long as it knows about them. So my hooks to pthread (and the LD_PRELOAD shim) are there to make pthread_self, and TLS become fiber-specific instead of thread-specific. The rest of the pthread code there is to handle the subtleties of hooking low-level library functions like that. In particular these hooks start getting called before the program is even fully loaded into memory. Some of the code is running before the heap is even ready (malloc, new, etc fail).

But the end result is that v8 sees each fiber as a new thread, however the context switches are controlled by the user instead of the scheduler. Additionally, I implemented coroutine recycling without telling v8 that the stacks are dead. So if you're creating lots of short-lived fibers there's very little setup cost for each one. To v8, it's as if you have a thread that's sitting there calling a bunch of random functions, and there's no problem with that.

> I love the Fiber object although maybe maybe we could have API

> compatible with Rhino continuations for portability?

I think Rhino continuations are harder to implement. Certainly possible, but not worth it in my opinion. The difference is that with Fiber() you have a new stack initialized before starting, however with Continuation() you have to clone the current stack in real-time. I think it'd be really tricky to pull this off right in v8. And if someone gave me Continuation() the first thing I'd do is implement Fiber() anyway.

> npm package would be great as well : )

I'm generally pretty terrible at build and deployment. See my Makefile for more information. I'd welcome someone to contribute a more node-friendly wscript or npm or whatever the cool kids are using nowadays.

> Also please fix the shebang in fiber-shim to #!/bin/bash (it require bash).

Thanks!

[James Carr]

I think webworkers have worked great for concurrency in regards to
multiple processes, I'm thinking it'd be a great model for multiple
threads as well? I'll check out the repo and play with it after I wrap
up a few things and give you some feedback.

Thanks for creating an implementation for fibers. :)

Thanks,
James

> --
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en.
>

[Marcel Laverdet]

Yes that makes perfect sense, but unfortunately you can't do that within v8's virtual machine. v8 is thread safe, but still needs mutual exclusivity. So while you could implement web workers via threads it'd be pointless because you would get 0 parallelization. I believe multiple processes is the correct solution until v8 can support running parallel threads in different Javascript contexts.

There was some talk about this on the v8-users mailing list quite some time ago [http://www.mail-archive.com/v8-u...@googlegroups.com/msg01557.html] and while everyone agreed it was a good idea, it turns out getting that support into v8 is non-trivial.

[Bradley Meck]

I for one thank you Marcel. Personally I will not use this, but it will solve many people's desires and give advocates on both sides a competition rather than theory based flaming. I have heard trouble due to some v8 API stuff with handling Errors, and would love to know how you handle them.

[Marcel Laverdet]

I think there's a time and a place for fibers. For example I won't use fibers for my long-polling XHR's, but I will use them for page loads since I know for certain it will make my code cleaner. But yes the great thing is that now people have a choice.

What kind of trouble are you talking about regarding Errors? My solution was really simple. If you throw an exception from a fiber, I catch it in C++ and then throw it again from Fiber.prototype.run(). You lose the stack trace on your exception I think, but you still have the exception and origin.

See:

    $ cat error.js

    var util = require('util');

    require('./node-fibers');

    

    var fn = Fiber(function() {

      util.print('async work here...\n');

      yield();

      util.print('still working...\n');

      yield();

      util.print('just a little bit more...\n');

      yield();

      throw new Error('oh crap!');

    });

    

    try {

      while (true) {

        fn.run();

      }

    } catch(e) {

      util.print('safely caught that error!\n');

      util.print(e.stack + '\n');

    }

    util.print('done!\n');

    

    $ ./fiber-shim node error.js 

    async work here...

    still working...

    just a little bit more...

    safely caught that error!

    Error: oh crap!

        at /Users/marcel/code/node-fibers/error.js:11:9

    done!

The exception-awareness of fibers is another great feature that will help developers write good, safe code.

On Sat, Jan 22, 2011 at 10:26 PM, Bradley Meck <bradle...@gmail.com> wrote:

I for one thank you Marcel. Personally I will not use this, but it will solve many people's desires and give advocates on both sides a competition rather than theory based flaming. I have heard trouble due to some v8 API stuff with handling Errors, and would love to know how you handle them.

--

[Łukasz Mielicki]

Hi Marcel,

I have crated experimental node API translator which leverages node-fibers, how about:

var fiberize = require('./fiberize');

var fs = fiberize(require('fs'));

fiberize.start(function() {

  var file = __filename; 

  var fd = fs.open(file, "r", 0666);

  var info = fs.fstat(fd);

  var buf = new Buffer(info.size);

  fs.read(fd, buf, 0, info.size, null);

  fs.close(fd);

  console.log(buf.toString('utf8'));  

});

All original API functions which explicitly take callback as the last argument are automatically wrapped so the function is suspended and returns second parameter passed to callback at resume (usually returns some data by convention). If first callback argument (usually err) is given, it's thrown in calling context.

Unfortunately not all node functions have callback argument given explicitly thus not all functions are translated properly, for instance fs.readFile. Although these function can be still used with explicit callback. Many of these could be translated with little patch to node.

Would you please add fiberize as contrib in node-fibers? I made the licence compatible ;)

If any of you like it, or you have some ideas what's missing or doesn't work, please let me know!

Regards,

Łukasz

[Marcel Laverdet]

Wow very cool Łukasz! You actually read my mind :). I was creating this at the very same time:

https://gist.github.com/791157

I ended up using [fn.length - 1] instead of push() to maintain optional arguments. I didn't think of wrapping entire modules though :).

I wonder what the best way for people playing with this to collaborate will be. I could make a wiki page on github to collect gists together?

2011/1/22 Łukasz Mielicki <bo...@wp.pl>

--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.

[Łukasz Mielicki]

On Saturday, January 22, 2011 3:46:41 PM UTC+1, Marcel wrote:

Wow very cool Łukasz! You actually read my mind :). I was creating this at the very same time:

https://gist.github.com/791157

I ended up using [fn.length - 1] instead of push() to maintain optional arguments. I didn't think of wrapping entire modules though :).

Right, actually it tried that initially, but got confused because logging functions skip undefined elements of array... 

I fixed that, also added async object representing original module for convenience (attached).

 

I wonder what the best way for people playing with this to collaborate will be. I could make a wiki page on github to collect gists together?

It was just an experiment, but looks automatic API translation can work. 

I think i will give it a chance on the project I'm working on : ) Hoping it can be maintained further along node.

To make it happen i think it would be good to provide whole bundle, thus:

- experiment a bit and wait a bit for some feedback : )

- bundle fiberize with node-fibers

- "npm install fibers" (see nodeprofile it's also installs separate executable, although I'm not sure if it supports so)

- add it here: https://github.com/ry/node/wiki/modules

- patch node so the callback convention is strictly followed so fiberize can cope with it : ), e.g. readFile

- I would suggest renaming fiber-shim to nodef for convenience (and add node so one can say just "nodef test.js")

- probably some additional docs

Let me know if you add fiberize to your repo. If so I'll add wiki entry.

One more word: even the API translation is hacky, it shouldn't hit performance as it adds only a little code, but I'm a somewhat concerned about translating a module each time. Preferably it should work like plain require and cache translations internally... maybe add fiberize.require('fs') ?

Regards,

Łukasz

[Łukasz Mielicki]

On Saturday, January 22, 2011 4:31:25 PM UTC+1, Łukasz Mielicki wrote:

One more word: even the API translation is hacky, it shouldn't hit performance as it adds only a little code, but I'm a somewhat concerned about translating a module each time. Preferably it should work like plain require and cache translations internally... maybe add fiberize.require('fs') ?

It's easier than i thought, just add:

module.exports.require = function(path)

{

  var mod = require(path);

  if (mod.__fiberized === undefined) {

    mod.__fiberized = fiberize(mod);

  }

  return mod.__fiberized;

}

[Marcel Laverdet]

The problem with bundling is that there's many different ways you could use fibers. While making every async call become synchronous is convenient, you lose the ability run multiple calls at once. For instance I'd like to be able to do something like this:

    var file1 = fs.readFile('/tmp/hello1');

    var file2 = fs.readFile('/tmp/hello2');

    var file3 = fs.readFile('/tmp/hello3');

    wait(file1, file2, file3);

    print(file1.get() + file2.get() + file3.get());

There's got to be loads and loads of futures libraries that could be adapted to this model.

But on the other hand being able to code sequentially is mighty convenient. It will just be a matter of choosing the right library for the job. But I think node-fibers can be a tool to make existing libraries much more powerful, rather than attempting to solve everyone's problems by itself.

2011/1/23 Łukasz Mielicki <bo...@wp.pl>

Regards,

Łukasz

[Bruno Jouhier]

Ite missa est!

If I had known this was coming, I would have slept peacefully instead
of getting up at night to write streamline.js.

I nevertheless see some value in my hack. As it relies on code
generation rather than on new runtime features, it can be used for
modules that are shared between browser and server (logic that
operates on data synchronized to offline storage for ex). And the nice
thing is that the day fibers make their way to the browser, I'll just
have to shut the transformation down and turn on fiberized modules. I
won't need to change the code because it will be fiber-ready (even
without a cleanup pass to remove the _ parameters). So streamline.js
can be a stopgap solution for people who have this problem (modules
that should also work browser side) but it seems fairly clear that
fiberized modules are the way to go in the long run (unless I missed
something important).

Great job guys.

Bruno

>  fiberize.js
> 2KViewDownload
>
>  ftest.js
> < 1KViewDownload

[Łukasz Mielicki]

On Saturday, January 22, 2011 5:09:48 PM UTC+1, Marcel wrote:

The problem with bundling is that there's many different ways you could use fibers. While making every async call become synchronous is 

You are perfectly right about this. Node-fibers is extraordinary extension, and for sure lots can be done basing on it.

I was just thinking of providing convenient start for newcomers. I'm also little overwhelmed with number of node packages available, and I'm somewhat afraid of proliferation of different approaches and APIs basing on fibers, mainly because they are all to be the basis for further libraries... wish we have some consistent approach.

Although I agree, probably the best way is to provide separate library with fibers dependency in its package.js.

Please package fibers if possible providing some simple command for bootstrapping, so I can package API wrapper separately.

  

There's got to be loads and loads of futures libraries that could be adapted to this model.

Working on it : ) I revised the approach quite a lot, but feel free to share any ideas regarding syntax.

Regards,

Łukasz

[Marcel Laverdet]

I threw fibers on npm in case anyone else wants to play around with it.

npm install fibers

I also added a `fiber-shim` wrapper, `node-fibers`. If you decide you want to always enable fiber support you can just drop `alias node=node-fibers` in your bashrc :).

2011/1/23 Łukasz Mielicki <bo...@wp.pl>

Regards,

Łukasz

[Łukasz Mielicki]

I have just published largely improved fiberize

npm install fiberize

No promises added yet, just simple sequential API, but mostly covered with tests (there are some inconsistencies to deal with).

Docs probably do not impress but it's not easy to show the value of this approach in with trivial examples. 

Feedback is welcome!

Regards,

Łukasz

[Łukasz Mielicki]

On Saturday, January 22, 2011 4:31:25 PM UTC+1, Łukasz Mielicki wrote:

On Saturday, January 22, 2011 3:46:41 PM UTC+1, Marcel wrote:

Wow very cool Łukasz! You actually read my mind :). I was creating this at the very same time:

https://gist.github.com/791157

I ended up using [fn.length - 1] instead of push() to maintain optional arguments. I didn't think of wrapping entire modules though :).

Right, actually it tried that initially, but got confused because logging functions skip undefined elements 

I found out my original approach was good, node expects fs.open(path, 'r', callback) rather than fs.open(path, 'r', undefined, callback), even if that works sometimes.

Łukasz