V8 and pthreads

[mschwartz]

I was toying with pthreads and V8 and it all pretty much works as I expected.

The repo is here:

https://github.com/mschwartz/v8t

It's a pretty small amount of code, so all the v8 API calls used are not so obfuscated.

The test program, test.js is:

var NUM_THREADS = 1000;

var nnn = 0;

var pthread = builtin.pthread;

log(pthread);

function Thread(fn) {

var args = [];

for (var i=1; i<arguments.length; i++) {

args.push(arguments[i]);

}

this.fn = fn;

this.args = args;

this.t = pthread.spawn(this.run, this);

}

Thread.prototype.run = function(me) {

me.fn.apply(me, me.args);

};

function thread(n) {

var nnnn = 0;

log('thread ' + n);

while (true) {

nnn++;

nnnn++;

log('thread ' + this.t + ' here ' + nnn + ' ' + nnnn);

}

}

function tester(i, n) {

log(i + ' ' + n);

sleep(n);

log('TESTER ' + i + ' EXITING');

}

function main() {

var threads = [];

if (true ) {

log('spawning');

new Thread(tester, 1, 5);

new Thread(tester, 2, 5);

new Thread(tester, 3, 6);

for (var i=0; i<NUM_THREADS; i++) {

threads.push(new Thread(thread, i));

}

while (true) {

var tid = pthread.wait();

log('parent: ' + tid + ' exited');

}

}

else {

log('here');

}

}

A couple of things to note:

First, there is no actual "wait for any thread to exit" kind of functionality in pthreads.  I implemented my own using doubly linked lists and pthread_cond_signal().  pthread_t may be radically different on various operating systems, so I implemented my own threadId scheme, similar to PIDs.

When I spawn 1000 threads, they don't seem to get equal treatment by the scheduler.  Sample output of the test.js program:

81073 thread 514 here 85641 116

81073 thread 713 here 85642 37

81073 thread 712 here 85643 38

81073 thread 711 here 85644 39

81073 thread 710 here 85645 40

81073 thread 709 here 85646 41

81073 thread 515 here 85647 116

81073 thread 516 here 85648 116

81073 thread 517 here 85649 116

81073 thread 518 here 85650 116

81073 thread 519 here 85651 116

81073 thread 520 here 85652 116

81073 thread 521 here 85653 116

81073 thread 522 here 85654 116

81073 thread 523 here 85655 116

81073 thread 524 here 85656 116

81073 thread 525 here 85657 116

81073 thread 526 here 85658 116

81073 thread 527 here 85659 116

81073 thread 528 here 85660 116

81073 thread 529 here 85661 116

81073 thread 635 here 85662 104

81073 thread 634 here 85663 105

81073 thread 530 here 85664 116

81073 thread 531 here 85665 116

81073 thread 680 here 85666 67

81073 thread 679 here 85667 68

81073 thread 678 here 85668 69

81073 thread 532 here 85669 116

81073 thread 533 here 85670 116

81073 thread 741 here 85671 11

81073 thread 740 here 85672 12

81073 thread 739 here 85673 13

As you can see, the first 600 or so threads seem to get equal processor time, while the remaining ones seem really starved for processor time.  I'm not sure if this is due to the OSX pthread scheduler algorithm (I tried all 3 variants, not much difference) or something about how v8::Locker works.

Anyhow, I hope the code is of interest to someone.

[Stephan Beal]

On Mon, Jul 2, 2012 at 1:46 PM, mschwartz <myk...@gmail.com> wrote:

Anyhow, I hope the code is of interest to someone.

Absolutely. i tried to do the same in SpiderMonkey several years ago and failed miserably. It actually worked on single-core systems (where no real MT was going on), but as soon as i moved to 2-core it all exploded very horribly.

Thank you for the code and link :).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/

http://gplus.to/sgbeal

[Michael Schwartz]

I just compiled the program on Ubuntu and the execution of threads is quite uniform.  

For all the great work done in OSX for the user interface (the apps are outstanding), it really lacks in the core OS features and the command line as well.

--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users

[Stephan Beal]

On Mon, Jul 2, 2012 at 3:00 PM, Michael Schwartz <myk...@gmail.com> wrote:

I just compiled the program on Ubuntu and the execution of threads is quite uniform.  

For all the great work done in OSX for the user interface (the apps are outstanding), it really lacks in the core OS features and the command line as well.

LOL! i'm completely anti-Apple, but i'm going to say this anyway: to be fair, OSX is a desktop OS and Ubuntu is (at its heart) a server OS, and both are optimized for their purposes. It would be "highly unusual" for most desktop apps to spawn 600 threads, i think, and i don't know anyone would installs 3000 Euro laptops in their server racks.

[Michael Schwartz]

I'm a recent Apple convert.  The quality of the software is just amazing.  I use Ubuntu at work for my workstation, and I love it, too.  But the difference in the quality of apps and the overall desktop user experience doesn't compare with Lion.

The test program actually spawns 1000 threads :)

In a WWW serving environment, I could see a single request using 3 threads to overlap operations in parallel.  One thread could be querying a database, another could be doing some RESTful type API call to a remote server, etc.  It might be useful for dealing with pipelined requests, and to implement something like SPDY or WebSockets as well.

The drawback to threads and v8 is that due to Locker, you can only have one thread actually executing JS code inside V8.  This means a thread that does a LOT of work in JS would hog V8 so others wouldn't run (blocked in Locker).

However, a long running thread could call sleep(0) or pthread_yield() periodically to allow the other threads to run.

[Stephan Beal]

On Mon, Jul 2, 2012 at 3:11 PM, Michael Schwartz <myk...@gmail.com> wrote:

However, a long running thread could call sleep(0) or pthread_yield() periodically to allow the other threads to run.

From what i understand, every time the JS-side code calls back into v8 (e.g. via a function call), v8 gets a chance to interrupt it. This doesn't protect against infinite loops which don't call back into v8, of course. while(true){++i} would "probably" never yield.

[Michael Schwartz]

Yeah, that is exactly what I'm talking about (while loop like that).  Not that it needs to be infinite to be harmful, but long running would starve the other threads.

A filter/map/reduce with an array.sort() could be harmful enough, depending on the size of the data set.  I'll know soon enough :)

[Pablo Sole]

You don't seem to be using any context preemption (and you're over a
single Context) so, what would happen if a thread does a long running
operation (or may be just a long script) over one Thread? Or may be are
you thinking on implementing Unlockers for all those kind of operations?

I'm implementing something around this lines too (for a more specific
purpose though) and decided to use a diff Isolate per-thread to have
full parallel execution. Of course in my case I don't need to share much
information between threads, otherwise it would be impossible to do so.

Cheers,
pablo.

[Michael Schwartz]

Wow, that's a great tip. I implemented it and long running threads work fine and get preempted.

I guess the disadvantage is thread safety and synchronization of data accesses. I'm not clear on when a thread running in v8 might be preempted. Likely on entering or exiting functions (when doing stack checks, I think I remember reading about).

[Stephan Beal]

On Mon, Jul 2, 2012 at 6:00 PM, Michael Schwartz <myk...@gmail.com> wrote:

I'm not clear on when a thread running in v8 might be preempted.  Likely on entering or exiting functions (when doing stack checks, I think I remember reading about).

FWIW, that's also my recollection.

[Pablo Sole]

On 07/02/2012 01:00 PM, Michael Schwartz wrote:
> Wow, that's a great tip. I implemented it and long running threads work fine and get preempted.

Glad to help!

> I guess the disadvantage is thread safety and synchronization of data accesses. I'm not clear on when a thread running in v8 might be preempted. Likely on entering or exiting functions (when doing stack checks, I think I remember reading about).

AFAIK, any StackGuard instance is preemptible, regex matching also check
for preemption, some loops and there must be some other places too. This
mechanism is used by chromium, I suppose, to switch between Contexts
(frames/iframes) of a same tab (Isolate?). I'm completely making this up
and it's probably not accurate, but must be something like that :).

The rule of thumb for thread safety should be to not trust on a Locker
instance (which can be preempted) for any resource sync managed or
potentially modified from outside of v8. A C++ function wrapped inside a
FunctionTemplate though, cannot be preempted of course, so that's safe.

pablo.

[Michael Schwartz]

Well, I was talking about data access from two threads running JS with preemption enabled.

Consider a doubly linked list, like this:

https://github.com/mschwartz/SilkJS/blob/dev/modules/List.js

If one thread calls each() and another is calling remove(), each() will fail if the remove() is preempted before both assignments are done.

[Michael Schwartz]

You might find this interesting.

I tried setting preemption time to 10ms and output of my script was sporadic (in bursts). I set it to 1ms and it got way better. I set it to 0ms and it seems perfect.

I'm not sure what kind of performance impact this setting has.

On Jul 2, 2012, at 10:52 AM, Pablo Sole wrote:

>

[Pablo Sole]

The ms you set is used here:

void ContextSwitcher::Run() {
while (keep_going_) {
OS::Sleep(sleep_ms_);
isolate()->stack_guard()->Preempt();
}
}

So what you're doing basically is to preempt on every preemptible point.
There might be a better way of doing this without the need of a thread
constantly setting the PREEMPT flag on the current StackGuard instance
(several times actually). May be to set the flag ON by default on the
StackGuard constructor.

Anyway, it's interesting indeed as that would be the fairest
distribution possible from an engine perspective, may be somebody from
the engine team can tell us more accurately what's the overhead involved
in a preemption task switch.

pablo.

[Michael Schwartz]

Yeah, I figure there's a bit of overhead, even if it's in the constant sleep()/wakeup context switches alone.

I initially chose 10ms because I think it's close to the OS scheduler's quantum.

I also figure the time should be a bit longer than what I expect typical "long time running threads" to actually run. "Long time" would be longer than what you'd do in an event-loop style scheme. And from my experience with SilkJS, I typically see sub millisecond entire HTTP request CPU times, and requests that do quite a bit of CPU work take < 10ms, and requests that read in 25 files and parse them for /** … */ comments and run the content through github-flavored-wiki can take a second. Those really long running ones would be scheduled by Unlockers in the C++ code (reading files) anyhow, so it probably would work out at 10ms. So I figured.

Yet the main thread was clearly starved for CPU cycles….

[Jorge]

I've been playing with threads and V8 too: <https://github.com/xk/node-threads-a-gogo> :-P

Cheers,
--
Jorge.

[mschwartz]

It's not working as I hoped.

I'm seeing random crashes in v8 Locker class when I implement a JS thread that does some work, then spawns itself just before calling pthread_exit().

On OSX, I see the thread get respawned 15 times and it works like it should.  The 16th time, segfault.  I ran it under gdb and see this:

14020 test3 exiting 16

14020 test3 respawn 16

runner

a

b

c

Program received signal EXC_BAD_ACCESS, Could not access memory.

Reason: KERN_INVALID_ADDRESS at address: 0xfffffffffffffff8

[Switching to process 14020 thread 0x1923]

v8::internal::ExitFrame::GetStateForFramePointer (

    fp=<value temporarily unavailable, due to optimizations>, 

    state=0x105080400) at ../src/frames.cc:556

556 ../src/frames.cc: No such file or directory.

in ../src/frames.cc

However, spawning 1000 threads in the main program still works.

The runner, a, b, c lines are printf() I put in the code to see where it's dying.

The runner() function is a wrapper around entering v8 from a newly started thread. 

static void *runner(void *p) {

    printf("runner\n");

    ThreadInfo *t = (ThreadInfo *)p;

printf("a\n");    

    pthread_setspecific(tls_key, t);

printf("b\n");    

    pthread_detach(pthread_self());

    {

printf("c\n");    

        Locker loc;

printf("d\n");    

        HandleScope scope;

printf("e\n");    

        Handle<Value>v = t->o->Get(String::New("runHandler"));

printf("f\n");    

        if (v.IsEmpty()) {

printf("g\n");    

            printf("No runHandler\n");

printf("h\n");    

            pthread_exit(NULL);

        }

        else if (!v->IsFunction()) {

printf("i\n");    

            printf("No runHandler (not a function)\n");

printf("j\n");    

            pthread_exit(NULL);

        }

printf("k\n");    

        Handle<Function>func = Handle<Function>::Cast(v);

printf("l\n");    

        Handle<Value>av[1];

printf("m\n");    

        av[0] = t->o;

printf("n\n");    

        // TryCatch tryCatch;

        printf("CALL\n");

        v = func->Call(context->Global(), 1, av);

        // if (v.IsEmpty()) {

            // printf("Exception!");

            // ReportException(&tryCatch);

        // }

    }

    printf("exitx\n");

    pthread_exit(NULL);

}

You can see it's getting a Locker, and dying in the HandleScope creation.

The JS code looks like this:

function test3a() {

    log('test3a alive ' + this.threadId);

    this.on('exit', function() {

        log('test3a respawn ' + this.threadId);

        new Thread(test3);

        // while(1);

    });

    log('test3a sleeping ' + this.threadId);

    // sleep(1);

    log('test3a exiting ' + this.threadId);

}

function test3() {

    log('test3 alive ' + this.threadId);

    this.on('exit', function() {

        log('test3 respawn ' + this.threadId);

        new Thread(test3a);

        // while (1);

    });

    log('test3 sleeping ' + this.threadId);

    // sleep(1);

    log('test3 exiting ' + this.threadId);

}

function main() {

    // for (var i=0; i<1000; i++) {

        // sleep(2);

        // new Thread(test4);

    // }

    new Thread(test3);

    while (1) sleep(1);

}

The pthread create function looks like this:

static JSVAL create(JSARGS args) {

    ThreadInfo *t;

    t = new ThreadInfo;

    t->threadId = nextThreadId++;

    t->o = Persistent<Object>::New(args[0]->ToObject());

    {

        Unlocker ul;

        pthread_create(&t->t, NULL, runner, t);

    }

    return Integer::New(t->threadId);

}

(The source to runner posted above).

Also of note is if I throw an Error() from JS in a thread, it's caught, but the TryCatch doesn't seem to have valid data.

[Stephan Beal]

On Thu, Jul 5, 2012 at 4:21 PM, mschwartz <myk...@gmail.com> wrote:

        // TryCatch tryCatch;

        printf("CALL\n");

        v = func->Call(context->Global(), 1, av);

        // if (v.IsEmpty()) {

            // printf("Exception!");

            // ReportException(&tryCatch);

        // }

Out of curiosity: did you comment that TryCatch out because it is no longer relevant for you or because it doesn't work? i remember having "unspecified headaches" with local-scoped TryCatches, very possibly due to my inadvertent misuse of them. If the above is how they are supposed to be used, i'll take another look at the code i had such problems with.

Also of note is if I throw an Error() from JS in a thread, it's caught, but the TryCatch doesn't seem to have valid data.

Ah, that's what i get for not reading to the end before asking questions :/.

[Michael Schwartz]

I use that TryCatch logic in the main() function to catch compile errors and initial script loading/running errors.  It seemed to me that spawning a new thread and entering V8 from the new thread was a similar situation.