Multiple instances and static data

[SteveC]

Hi,

I know theres been some discussion lately about users wanting to have
multiple independent V8 instances that can be accessed from multiple
threads without global locking.

I'd like the same; V8 is such a clean piece of code (aside from this
one issue ;-) that I'd love to use it, but I really do need to call
multiple instances from multiple threads without one global per-
process lock (of course, a lock per V8 instance would be fine).

Now, I expect this is naive of me, but I notice there is lots of
static data in the codebase. if I were to refactor this into some kind
of 'GlobalContext' class, could I not simply instantiate multiple of
those GlobalContexts without them interfering with each other? Or is
there funky stuff to do with thread local storage or something else
that would make this fail?

Best regards,
Steve.

[Stephan Beal]

On Wed, Sep 16, 2009 at 5:23 PM, SteveC <sdcli...@googlemail.com> wrote:

Now, I expect this is naive of me, but I notice there is lots of
static data in the codebase. if I were to refactor this into some kind
of 'GlobalContext' class, could I not simply instantiate multiple of
those GlobalContexts without them interfering with each other?

One of the major diffs between v8 and, e.g., SpiderMonkey is that none of the v8 API takes a Context/Engine argument. In SpiderMonkey, nearly every function takes a Context argument (as it should, IMO). v8's usage of static data instead of an explicit argument inherently limits it in regards to threading/concurrent access. While v8 could, in theory, be a widely-applicable engine, it was designed with the sole purpose of supporting Chrome. The API limitations which that decision have led to will always limit v8 severely in terms of concurrency and re-use. e.g., the main reason which v8 does not call GC at engine shutdown is because doing so slows down the shutdown of Chrome, as one v8 developer admited on the list some months ago. That Chrome-induced limitation bleeds over into my apps, however, and number of the types i wrap (e.g. DB handles) require that their dtors get called at some point, and v8 cannot guaranty that for me.

OTOH: there has never been, AFAIK, a working SWIG wrapper generator for JavaScript. As far as i've been able to determine (by exploring SWIG), this is because the Context/Engine argument required by the JS API makes it nearly impossible to bind in to the SWIG system (which doesn't want to know about that Context/Engine). Since v8 doesn't have this argument, i expect that someone will be able to create SWIG wrapper generator for v8.

> Or is there funky stuff to do with thread local storage or something else
> that would make this fail?

Thread-local storage could probably be used to "solve" (at least partially) this "problem" (assuming TLS is available on all platforms Chrome wants to run on), but that would also make it more difficult (i believe) for more than one thread to interact with the same VM (because each thread would see its thread-local Context, rather than the global one). IMO, the API should be completely rewritten, inserting a Context argument as the first argument to nearly every function (that is, anywhere a Context is internally needed and cannot be extracted from another argument). But the project is now too big for such a change.

The "global everything" aspects of v8 will, over time, probably turn out to be the biggest hindrance to its adoption.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/

[Stephen Clibbery]

Hi, thanks for the response.

I guess to do this I am talking about a fork of the V8 code, which is
certainly unfortunate. I'd just like to get an idea of the work
involved; I need to embed a JS engine, and none of them seem like an
exact fit for my needs; TraceMonkey and JavaScriptCore look like they
may be more heavyweight than V8, and may have more runtime
dependencies as well.

I need something as lightweight as possible (that wont cause a bloated
executable or have tons of dll-type dependencies) because the app is
installed through downloading. For reference, I have used Lua for
scripting up till now, which adds a grand total of about 300k of
executable code to the app. I don't expect to get near that value from
a JS engine, but still the smaller the better; more than a Mb or two
of code from the JS engine will probably be unacceptable.

Anyway, so, say I was to create an 'Engine' class to wrap up V8, and
stuff all the static data in there. And then either use TLS to store
an Engine reference, or modify the functions to take an 'Engine&'
parameter. Would that work? Obviously forking is bad, and modifying
lots of function signatures would be a fairly major job, but could I
expect that to be enough to deal with the locking issue?

Best Regards,
Steve

[Erik Corry]

2009/9/16 SteveC <sdcli...@googlemail.com>

Now, I expect this is naive of me, but I notice there is lots of
static data in the codebase. if I were to refactor this into some kind
of 'GlobalContext' class, could I not simply instantiate multiple of
those GlobalContexts without them interfering with each other? Or is
there funky stuff to do with thread local storage or something else
that would make this fail?

Basically I think that would work and would probably be a good idea.  It's hard to say what the performance hit would be but we think it might be minimal.  The generated code would be basically unchanged.  I don't think you should underestimate the work involved:  There are

a lot of static variables scattered around.

One idea that has been bandied around in the V8 team is to change the API minimally by

adding a GlobalContext parameter to the V8::Locker constructor.  So when you lock V8 you

tell it which GlobalContext you want to use.  The locker writes the GlobalContext into the

operating system provided thread local storage (not to be confused with V8s ThreadLocal

stuff which is completely different and arguably a misuse of the term - mea culpa!).  Thread

local storage tends to be very fast so it's not unrealistic (we think) to get the GlobalContext

where we need it (pretty much everywhere!).

The model provided by this is that you can have several GlobalContexts or Isolates and they share nothing with each other.  The memory use is basically multiplied by the number of GlobalContexts you want to have.  For each GlobalContext you can have 1 or more threads, just like you can now with the one implicit GlobalContext.  (Probably the normal way to use it would be to have one GlobalContext per physical CPU.)  A V8 thread can't move between GlobalContexts.  Communication between JS programs running in different GlobalContexts would have to be done in C++.  There can be more than one Context in a GlobalContext, but a Context is bound to one GlobalContext.

On the plus side, the VM doesn't get much more complicated, there are still no difficult-to-debug race conditions between threads and it shouldn't be noticably slower for the case we really care about: Chromium.

The first step http://images.google.com/images?q=first+step+is+doozie would be to make a single GlobalContext, put its address in thread local storage and measure how much that slowed things down.

Patches welcome, as we like to say. :-)

--
Erik Corry, Software Engineer
Google Denmark ApS.  CVR nr. 28 86 69 84
c/o Philip & Partners, 7 Vognmagergade, P.O. Box 2227, DK-1018 Copenhagen K, Denmark.

[Stephen Clibbery]

Hi,

> One idea that has been bandied around in the V8 team is to change the API
> minimally by
> adding a GlobalContext parameter to the V8::Locker constructor. So when you

That sounds nice and simple from the API point of view, and would also
minimise the amount of function-signature refactoring required. I've
actually done something a bit similar to the whole 'TLS for storing
GlobalContexts' idea in the rest of my app, so at least I should be
familiar with the general pattern.

> The first step http://images.google.com/images?q=first+step+is+doozie would
> be to make a single GlobalContext, put its address in thread local storage
> and measure how much that slowed things down.

Hehe; well I think my very first step is to do a quick evaluation of
the amount of static variables I would have to mess with :-)
But yes, some profiling would be a good start, since of course you
guys won't want to introduce any performance issues just to support
functionality not needed by Chrome. Even if the TLS has a noticeable
cost, we might still be able to have a #define switch to change
between the full TLS thing and a simpler version, where there is one
static GlobalContext, which would surely cost nothing to access :-)

> Patches welcome, as we like to say. :-)

Excellent! That would save any worrying about forking/maintainance etc
for me :-)
The choice for me is either mess with V8, or try to get TraceMonkey
going. I will go and look see how much work there might be in adding
the GlobalContext idea to V8. If its practical, I will very seriously
look at doing this work.

Best Regards,
Steve.

[Stephen Clibbery]

Hi,

Well, I've had a look at the codebase, and I reckon there are just
over three hundred static variables that would need to go into a
Global Context. I'm not saying I'm definitely going to do this yet,
but if I do have a go, I have a few practical questions:

* What's the best way to measure any possible performance regressions?
* I can only develop for Windows, but a TLS-based solution will likely
need a little porting to other platforms; is this ok?
* Are there any published code-style guidelines for V8?
* Should I try to land one big patch with all the changes, or find a
way to do it more incrementally?
* Anything else I should know? :-)

Best Regards,
Steve

[Luis Furquim]

On Thu, Sep 17, 2009 at 3:17 PM, Stephen Clibbery
<sdcli...@googlemail.com> wrote:
> * I can only develop for Windows, but a TLS-based solution will likely
> need a little porting to other platforms; is this ok?

It would be much appreciated if it runs in Linux.

Best Regards,

--
Luis Otavio de Colla Furquim
Não alimente os pingos
Don't feed the tribbles - http://www.thinkgeek.com/geektoys/plush/ac6e/
By the nice edges of dataly graphs I shall walk
http://www.furquim.org/chironfs/

[Ivan Posva]

Stephen,

Please do not underestimate the effort involved here. In heap.h alone
there are about 150 statics which you would need to move to an isolate
along with the other widely distributed statics in the other C++
classes. Many of the static definitions are hidden in massive macros,
so a simple count of "static" is going to give you inaccurate results.

As far as code-style and other things to consider please take a look
at http://code.google.com/p/v8/wiki/Contributing.

In general bigger patches are harder to review. So a set of smaller
patches is a.) easier to review, b.) faster to assess if you are on
the right track and c.) gives you early indication about the overhead
for accessing all current static data through thread local storage.

Also we try to not have too many different #ifdef in the code because
that reduces maintainability, increases testing complexity and
complicates matters when using the same pre-built library in different
settings. As an example, a Linux distribution might only have one
libv8.so, should this be the version using isolate support or not?
Having said that it means that the TLS based statics should have a
negligible effect on V8 performance if they are supposed to be
successful.

Cheers,
-Ivan

[Erik Corry]

2009/9/17 Stephen Clibbery <sdcli...@googlemail.com>

Hi,

Well, I've had a look at the codebase, and I reckon there are just
over three hundred static variables that would need to go into a
Global Context. I'm not saying I'm definitely going to do this yet,
but if I do have a go, I have a few practical questions:

* What's the best way to measure any possible performance regressions?

The V8 benchmark suite is checked in as part of the source.  There's a standalone version that runs with the standalone V8 shell.  I think you just write "shell run.js" in the benchmark directory.
 

* I can only develop for Windows, but a TLS-based solution will likely
need a little porting to other platforms; is this ok?

All commits have to work on all platforms.  Here's one way to achieve that:  Anything that is platform dependent needs to go in each of the *platform* files.  If you put something in the windows version of the file you can add an empty implementation to the other files.  Then you have to persuade someone to add an implementation that works for each of the other platforms.  Once they are there you can commit something that makes use of the new functionality.
 

* Are there any published code-style guidelines for V8?

http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml

There's a python script, tools/presubmit.py which checks for some of the style guide, but not all.  Here is a small selection of the things it doesn't check:

1) * and & in type declarations go next to the type name, not next to the object.  So it's "Object* foo" not "Object *foo"
2) static and instance member variables are written with lower_case_and_a_trailing_underscore_
3) Method names have CamelCase unless they are trivially simple accessors in which case they use the name of the variable they access without the trailing underscore
4) Comments should be sentences with correct punctuation.
 

* Should I try to land one big patch with all the changes, or find a
way to do it more incrementally?

Incrementally.  Apart from anything else, a single patch means your work could all be wasted if for some reason we can't accept the patch.
 

* Anything else I should know? :-)

You need to agree to the contributor's agreement and your employer may need to sign the corporate agreement.

[Stephen Clibbery]

Hi,

Thanks for the reply. I'm beginning to see just how much work it would
be. A simple global search for static variables just doesn't get close
to the amount of work really involved. There's a lot of data not
explicitly marked static (including, as you say, variables defined in
macros). There's probably weeks if not months of work here.

I also think it will be hard to accurately estimate the performance
impact of TLS without actually doing all the work, because until all
the modules are converted, we probably won't know exactly which call
sites would need to access the TLS data. A starting point might be a
throwaway prototype that just compares direct static data access to
TLS access. It would be hard to simulate appropriate cache-usage
patterns in such a test though. I'm also not sure exactly how to read
the results: without knowing the potential call-sites in the V8 code,
it would be difficult to say exactly how fast one TLS access would
need to be compared to the existing static access. Then there's the
question of whether all platforms would have similarly fast TLS
access.

Anyway, this (V8 with this patch) would still be my preferred solution
overall, but it's probably not practical for me to put this amount of
work in at this time :-(

Best Regards,
Steve

[nicity]

Hi,
For my rough v8mt prototype, one thread and concatenated v8 benchmarks
it looks like following:
12 seconds for all:
Richards: 1178
DeltaBlue: 1212
Crypto: 789
RayTrace: 1732
EarleyBoyer: 1994
RegExp: 213
Splay: 2073
Score: 1081

for original version:
11 seconds for all:
Richards: 1176
DeltaBlue: 1251
Crypto: 788
RayTrace: 1646
EarleyBoyer: 2374
RegExp: 371
Splay: 1498
Score: 1141

Back to investigating such a difference for last benchmarks :)
Best, Maxim Mossienko

[nicity]

http://code.google.com/p/v8/issues/detail?id=510