Thinking about JS_THREADSAFE yesterday, reminded me of this issue, which is something Bill brought up recently. The question is how to organize the API in the future: what file(s) to put it in, what namespaces, and how to note APIs as deprecated, experimental, quasi-internal, or whatever.
The current system seems to be a sort of natural outgrowth from the previous system of having everything in jsapi.h with a JS_ token prefix. We got C++ stuff, so we used a JS:: namespace for that. Gecko used things that we didn't want to consider public, so they went into jsfriendapi.h. It's all reasonable enough, but I think it's time for a fresh look. Bill's observations were that it might help compile times to have separate files for separate topics (like GC), and that jsfriendapi.h isn't necessarily buying us a lot.
A whole bunch of questions:
- Is there any reason not to split out the API into multiple API files by topic?
- Is it necessary (or even possible) to have header files that are either all public API or contain no public API? We could just use namespaces to express what is public API.
- Given that we don't particularly support a stable API at the moment, is there really a point to clearly delineating stable from unstable APIs? The main reason I can see is just to provide for the future. But it seems like we'd need a good idea of what the future API will be. If it's a new C++ API, then the existing stable API is the empty set, and we can just slowly grow it.
- The distinction that seems more meaningful to me is public vs. private: it is possible for the outside to call this method vs. it's not possible at all. We can then expose a public API + the minimal number of other functions to support Gecko and debugging. Do we have a really good way of enforcing public vs private APIs in C++? If not, how do we want to express it?
- What about the GC friend API functions? AFAIK, most VMs don't provide a GC API, but they do provide various unsupported settings and hooks. I assume GC controls are more tightly coupled to the implementation, and they don't want to try to support them over time. Possibly the implementers also consider GC APIs hard to use correctly and want to discourage general usage. It seems sensible enough to follow tradition and make them public but not API, but that's not the only way to go.
- Debugging functions? e.g. js_DumpString. I see no reason not to make them API--debugging and profiling are so fundamental to our use of the system we might as well officially support them.
- DOM/XPConnect functions? e.g. JS_SplicePrototype, JS_ShrinkGCBuffers. I don't know much about these, so I don't have too much to say. I gather that no one other than DOM should really be using them? It seems to me they should go in a special bucket for DOM, then. I also wonder if future versions of the web engine shouldn't try to be more decoupled from the JS engine...
> - DOM/XPConnect functions? e.g. JS_SplicePrototype
This one should just go away. It's only used in 3 places:
1) GSP invalidation. Will go away once the GSP is a proxy.
2) GSP installation. Likewise, modulo item 3 below.
3) Global object setup. This one will need _something_: the problem is that the global's proto chain needs to have various things on it, but those things can't be easily created before the global, afaict, both because they need to be parented to the global and because creation for some of them wants to have a global to cache them on. Object.prototype has this problem already, so presumably the JS engine already has a solution for it. We "just" need to generalize that a bit. ;)
> JS_ShrinkGCBuffers.
This seems to be a hackaround for the JS engine leaving empty gcchunks and arenas lying around? See bug 713916. I'd be happy to nuke it if possible!
> I gather that no one other than DOM should really be using them?
It seems to me like the "give the global an interesting proto chain" is possibly somewhat DOM-specific. The empty gcchunks thing, on the other hand, would probably matter to any embedding that cares....
> I also wonder if future versions of the web engine shouldn't try to be more decoupled from the JS engine...
Where by "web engine" you mean servo or Gecko or whatnot?
Right now servo is instead coupling tighter (e.g. allocating DOM nodes out of the JS heap).
On Tuesday, October 30, 2012 8:34:02 AM UTC-7, Boris Zbarsky wrote:
> On 10/30/12 11:14 AM, Dave Mandelin wrote:
> > I also wonder if future versions of the web engine shouldn't try to be more decoupled from the JS engine...
> Where by "web engine" you mean servo or Gecko or whatnot?
> Right now servo is instead coupling tighter (e.g. allocating DOM nodes > out of the JS heap).
That example doesn't sound so bad to me. Exposing a memory allocator isn't a core feature of a JS engine but at least it can be done in a sane way. I would think it would still make it very hard to switch to a different JS engine, if that's an option you'd want to have.
The coupling that I don't like is things like depending on an implementation detail of SpiderMonkey, or adding arbitrary pieces of functionality that are highly specific to some Gecko purpose and can't be understood as a general-purpose service.
> The coupling that I don't like is things like depending on an implementation detail of SpiderMonkey
Agreed. The largest current sources of such dependencies are GC+rooting and the use of explicit jsapi for things like security checks, handling of typed arrays, etc.
For that last one, it doesn't help that typed arrays are part of SpiderMonkey proper... I don't believe that's the case in V8 or JSC, last I checked, but my info may be out of date.
With the WebIDL bindings we've been sort of trying to end up with C++ objects that represent things that are passed from JS, so we're not dependent on bits of JSAPI. The things we haven't done that with so far are "any" (JS::Value), "object" (JS::Object*), and typed arrays (because we can't really create those without using JSAPI). Plus the fact that security checks rely on compartment so need a JSContext.
Well, plus workers, where life is much much worse. Those are very tightly coupled to SpiderMonkey GC for lifetime management.
It would be interesting to me to see what we could do to decouple more of these sorts of things, for sure. Decoupling from GC is pretty hard, though.
> On 10/30/12 2:24 PM, Dave Mandelin wrote:
>> The coupling that I don't like is things like depending on an
>> implementation detail of SpiderMonkey
> Agreed. The largest current sources of such dependencies are GC+rooting
One other thing on that. The reason servo is putting the DOM on the JS heap is not just so it can use the convenient memory area. The idea is to basically use the binary data APIs to implement the DOM, to a large extent... And the reason for that is because of a desire to avoid cycle-collection and just have JS GC manage all the cyclic stuff the DOM does.
Now how reasonable this desire is is a good question. But certainly really decoupling from the JS GC setup means handling cyclic references somehow... This has been a perennial problem in WebKit, where they _are_ decoupled, and it's really easy to cause leaks... and they have no way at all to implement some parts of WebIDL without leaking currently.
On Wednesday, October 31, 2012 10:38:38 AM UTC-7, Boris Zbarsky wrote:
> On 10/30/12 2:46 PM, Boris Zbarsky wrote:
> > On 10/30/12 2:24 PM, Dave Mandelin wrote:
> >> The coupling that I don't like is things like depending on an
> >> implementation detail of SpiderMonkey
> > Agreed. The largest current sources of such dependencies are GC+rooting
> One other thing on that. The reason servo is putting the DOM on the JS
> heap is not just so it can use the convenient memory area. The idea is
> to basically use the binary data APIs to implement the DOM, to a large
> extent... And the reason for that is because of a desire to avoid
> cycle-collection and just have JS GC manage all the cyclic stuff the DOM
> does.
> Now how reasonable this desire is is a good question. But certainly
> really decoupling from the JS GC setup means handling cyclic references
> somehow... This has been a perennial problem in WebKit, where they
> _are_ decoupled, and it's really easy to cause leaks... and they have no
> way at all to implement some parts of WebIDL without leaking currently.
At a high level, using the GC to collect DOM cycles is very appealing. I don't know the details, though. Maybe we should get people together more often to talk about things like GCing DOM and how to handle typed arrays.
> 3) Global object setup. This one will need _something_: the problem is that the global's proto chain needs to have various things on it, but those things can't be easily created before the global, afaict, both because they need to be parented to the global and because creation for some of them wants to have a global to cache them on. Object.prototype has this problem already, so presumably the JS engine already has a solution for it. We "just" need to generalize that a bit. ;)
The "solution" for Object.prototype is not generalizable. In fact the whole lazy standard class setup (which absolutely shouldn't be used for Function or Object, except they are now, and XPConnect makes it hard to make them not-lazy) is really not something to emulate or build upon.
What's needed here is something that lets you create a global object and specify any intermediates along its prototype chain before you reach Object.prototype (and null after it). That thing would then be the very first function you call after you've created a JSRuntime/JSContext. I have no idea at all what this might look like, especially if it's to be something nice and readable and understandable as JS_NewGlobalObject is (well, modulo that requiring you to specify a JSClass with special flags set, and various other semi-implicit requirements). Right now I believe lazy-standard-classes would get in the way of implementing this. Until XPConnect gets its house in order, and we can at the very least scale back lazy standard classes to not cover Function and Object, I don't expect to give it much or any thought.
> What's needed here is something that lets you create a global object and specify any intermediates along its prototype chain before you reach Object.prototype (and null after it).
Specify how? Those are generally speaking various somewhat complicated (e.g. use custom classes) objects, which need to be parented to the global object...
> That thing would then be the very first function you call after you've created a JSRuntime/JSContext.
Hrm. Did you mean compartment, or actually JSContext? Because we have a 1-to-many relationship between JSContext and globals!
> I have no idea at all what this might look like, especially if it's
to be something nice and readable and understandable as JS_NewGlobalObject is (well, modulo that requiring you to specify a JSClass with special flags set, and various other semi-implicit requirements).
We're going to need custom classes for the stuff on the proto chain of the global. And some of it will have to be proxies, for that matter. Is that what you meant?
> Right now I believe lazy-standard-classes would get in the way of implementing this. Until XPConnect gets its house in order
Why is XPConnect relevant here? I'm happy to try to fix things in XPConnect if I know what needs fixing!
We're getting off into js-engine.internals weeds here, but we *are* still somewhat discussing JSAPI considerations here, at least... :-)
On 11/14/2012 05:26 PM, Boris Zbarsky wrote:
> Specify how? Those are generally speaking various somewhat complicated (e.g. use custom classes) objects, which need to be parented to the global object...
Some sort of method that you call that you provide a list of { jsclass, functionspecs, propertyspecs } (or a list of roughly morally equivalent things), then the engine does magic to give you a global object whose prototype chain matches that list, then has a standardly-constructed Object.prototype, then ends in null. Basically from the point of view of the embedder the whole process of creating a global object and its prototype chain would be atomic.
Of course even this might not be enough, if those objects needed custom privates, or wanted to have reserved slots set in special ways from the start. Or if they needed to be proxies, as you note as a requirement. Which is all what makes designing an interface to support this quite complex.
>> That thing would then be the very first function you call after you've created a JSRuntime/JSContext.
> Hrm. Did you mean compartment, or actually JSContext? Because we have a 1-to-many relationship between JSContext and globals!
But you have a one-to-one relationship between compartments and globals. A compartment is an internal implementation detail (leaky sometimes, but I hope we can change this). From the JSAPI point of view, we should offer ways to construct globals; the compartment would get constructed at the same time, but the JSAPI user wouldn't even know it.
I did mean JSContext, because I was speaking in a semi-futurish world where we've unified JSRuntime and JSContext, after we've eliminated that 1-to-many relationship. I believe bholley is working on this, among other xpconnect/browser/js-engine intersection point issues.
> We're going to need custom classes for the stuff on the proto chain of the global. And some of it will have to be proxies, for that matter. Is that what you meant?
Yes, fun details like those, exactly.
>> Right now I believe lazy-standard-classes would get in the way of implementing this. Until XPConnect gets its house in order
> Why is XPConnect relevant here? I'm happy to try to fix things in XPConnect if I know what needs fixing!
I tried making standard class init non-lazy a year or so ago. I ran into the issue that XPCWrappedNative::WrapNewGlobal offers the explicit option of choosing whether standard classes are lazy or eager. Just making it all non-lazy caused the browser to not start up, throwing up some sort of alert about something going wrong. I wish I remembered exactly what it was.
I *think* it may have had something to do with the newly-created global not having its private data set yet, as something about the "// Set the private to the XPCWrappedNative." comment in XPCWrappedNative::WrapNewGlobal looks familiar to me. It may (I'm increasingly confident in my memory the more I think about this) be that initializing standard classes was defining properties on the global, which was triggering global class hooks, which were complaining mightily because at that early phase of execution the global didn't have its private yet. As I said, a mess, kind of. Although, if that recollection is accurate, maybe it's as much the JSAPI's fault for not supporting this as it is XPConnect's fault for depending on lazy init so intricately. :-)
> Some sort of method that you call that you provide a list of { jsclass, functionspecs, propertyspecs } (or a list of roughly morally equivalent things), then the engine does magic to give you a global object whose prototype chain matches that list, then has a standardly-constructed Object.prototype, then ends in null. Basically from the point of view of the embedder the whole process of creating a global object and its prototype chain would be atomic.
OK. We could do something like that; for my purposes we wouldn't even need functionspecs/propertyspecs, since I can just set those later, but _would_ need a proxyhandler for the proxy that's on the proto chain of the window. But yeah, something like that would be just fine for my purposes.
> Of course even this might not be enough, if those objects needed custom privates, or wanted to have reserved slots set in special ways from the start.
Apart from the global itself (which needs a reserved slot pointing to the actual DOM window) and the proxy handler and proxy private for the gsp, I don't think we need anything like that for the window proto chain. I guess that's a pretty big "apart"... ;)
We'd have to go through later and define some .constructor properties on those prototypes, and some of those might need custom bits, but that can be done post-facto without too much trouble I'd think.
> I did mean JSContext, because I was speaking in a semi-futurish world where we've unified JSRuntime and JSContext, after we've eliminated that 1-to-many relationship.
Um. At that point we'd have one JSContext for all windows, period. So I certainly hope we would not be limited to creating globals to right after we created the JSContext... I feel like I'm just misunderstanding something.
>> Why is XPConnect relevant here? I'm happy to try to fix things in XPConnect if I know what needs fixing!
> I tried making standard class init non-lazy a year or so ago. I ran into the issue that XPCWrappedNative::WrapNewGlobal offers the explicit option of choosing whether standard classes are lazy or eager. Just making it all non-lazy caused the browser to not start up, throwing up some sort of alert about something going wrong. I wish I remembered exactly what it was.
I see. This is definitely worth revisiting, but yes, trying to resolve stuff on the global (or more precisely, invoke its resolve hook) before it has its private slot set would not have worked well....
This API is starting to sound very much like a "do all the DOM Window magic" kind of API. :(
> Um. At that point we'd have one JSContext for all windows, period. So I certainly hope we would not be limited to creating globals to right after we created the JSContext... I feel like I'm just misunderstanding something.
Overreading, perhaps. :-) You wouldn't at all be limited to then, just that the ordinary course of using the JSAPI would have you create your runtime, then create the global, then go to town with it, no other fuss necessary. Embeddings that wanted more global objects could of course create them basically any time they wanted after the runtime was created.
