JSDebugv2: Goals
johnjbarton
* Support for debugging all Javascript code, including inline code in
HTML documents, code via element.appendChild(), document.write, and
evalInSandBox,
* Passing the current callstack on every callback from the debugger,
* Support for onCaught when a catch block succeeds, and onUnCaught when
none do so,
* Support for default JS operation with user code unwired, on a page.
Specifically Firebug has to block scrolling on the web page when on a
breakpoint, which makes looking the page hard.
* Explicit information on the security principals, in way we can use to
clarify error messages,
* Support for compilation begin/end (I think it is in the current design),
* Code re-writing and stack rewriting access (for metacompilers),
* Callback on too-much-CPU time (before the popup dialog), and on GC points,
* Information on JIT fail-to-optimize,
* Better focus on when the debugger is active (not in the goals but I
saw stuff about this in the text),
* Built-in support for walking the object graph, with an API consistent
with the rest of the debugger API (to replace the current memory addon),
* Support for "onObjectCreated",
* Ability to unroll the stack by one frame and re-call the function
(after we have replaced the function),
* High performance interrupt hooks. For example, the ability to push JS
code into the debugger and have it JITed into user code for logging
support,
(Sorry if this repeats something I've said before)
jjb
Jim Blandy
> Among goals for JSDebugv2 I'd like to see:
>
> * Support for debugging all Javascript code, including inline code in
> HTML documents, code via element.appendChild(), document.write, and
> evalInSandBox,
Definitely. If these don't work, they're bugs.
> * Passing the current callstack on every callback from the debugger,
>
> * Support for onCaught when a catch block succeeds, and onUnCaught when
> none do so,
My js::dbg2 draft does both of these.
> * Support for default JS operation with user code unwired, on a page.
> Specifically Firebug has to block scrolling on the web page when on a
> breakpoint, which makes looking the page hard.
I agree, but there are experienced folks who are telling me I'm wrong,
and that scrolling should just be prohibited. That seems like a pain in
the neck. Much better to just "unwire" the page, as you say.
> * Explicit information on the security principals, in way we can use to
> clarify error messages,
This is a great idea.
> * Support for compilation begin/end (I think it is in the current design),
It is.
> * Code re-writing and stack rewriting access (for metacompilers),
Oy. :) Well, let me say this: assuming I'm understanding what you mean,
this would require deep changes in the interpreter itself, since aspects
of the JS stack are implemented in terms of the C++ stack. But if the
SpiderMonkey core ever supported this, then the debugging interface
should, too.
> * Callback on too-much-CPU time (before the popup dialog), and on GC
> points,
GC, probably not, unless the new GC compartments work differently than I
understand they will. But too much CPU seems definitely useful.
> * Information on JIT fail-to-optimize,
That's out of scope for this work, but I agree it's needed.
> * Better focus on when the debugger is active (not in the goals but I
> saw stuff about this in the text),
I don't understand this.
> * Built-in support for walking the object graph, with an API consistent
> with the rest of the debugger API (to replace the current memory addon),
I don't understand this either.
> * Support for "onObjectCreated",
You'd like a hook called each time JavaScript allocates an object?
> * Ability to unroll the stack by one frame and re-call the function
> (after we have replaced the function),
Huh. I think I can see how to define what this should mean: the
continuation of the current function call is available; we just want to
create a new activation that shares that continuation. How we'd rig
that at the interpreter level, I'm not sure. JaegerMonkey might be even
harder. This would take some guts.
> * High performance interrupt hooks. For example, the ability to push JS
> code into the debugger and have it JITed into user code for logging
> support,
That's interesting. Then you could do profiling, call graph collection,
and stuff like that. It might be best to put this in the compiler,
rather than having to think about an implementable interface for
patching running code.
These are more specific than the goals I've put on the wiki page. I'll
be turning my work items into bugs soon, owned by some umbrella bug, and
I'll file the items above that aren't (to my mind) already enclosed some
other task as blockers as well.
johnjbarton
> On 05/17/2010 10:11 PM, johnjbarton wrote:
>> Among goals for JSDebugv2 I'd like to see:
>> * Support for default JS operation with user code unwired, on a page.
>> Specifically Firebug has to block scrolling on the web page when on a
>> breakpoint, which makes looking the page hard.
>
> I agree, but there are experienced folks who are telling me I'm wrong,
> and that scrolling should just be prohibited. That seems like a pain in
> the neck. Much better to just "unwire" the page, as you say.
There are two layers of debugging here. One is chrome layer and one is
content layer. From the chrome layer point of view, scrolling is just
another operation, we can't allow it when the application is halted.
From the content layer point of view, scrolling is a meta-operation
which can be allowed. The content layer is an application on the
platform; the chrome layer is an application on the OS. So the goal here
would be to support separation of the content layer to we can allow
chrome layer meta-operations to give a better debugger at the content layer.
>...
>> * Code re-writing and stack rewriting access (for metacompilers),
>
> Oy. :) Well, let me say this: assuming I'm understanding what you mean,
> this would require deep changes in the interpreter itself, since aspects
> of the JS stack are implemented in terms of the C++ stack. But if the
> SpiderMonkey core ever supported this, then the debugging interface
> should, too.
I think there could be middle ground here. JS code loaders (caja, Google
closure, ...) already do code rewriting and Firebug already does stack
rewriting. So the question is what can we do to provide better support
for this existing activity?
>
>> * Better focus on when the debugger is active (not in the goals but I
>> saw stuff about this in the text),
>
> I don't understand this.
Sorry, I meant: allow me to debug just one sphere. That got dropped from
the goals, but I think it's in your design.
>
>> * Built-in support for walking the object graph, with an API consistent
>> with the rest of the debugger API (to replace the current memory addon),
>
> I don't understand this either.
Given |window.foo| I want to answer the question: what are all the other
references to the object referenced by |window.foo|?
That reference graph should be in terms that can be easily expressed as
property paths (|window.bar.x|)
>
>> * Support for "onObjectCreated",
>
> You'd like a hook called each time JavaScript allocates an object?
Given |window.foo| the developer wants to know "on what source line was
the value set into |window.foo|" (watchpoints) and "on what source line
was the value created" (onObjectCreated). There may be other better ways
to answer the question than a hook, it's just one I know.
>
>> * Ability to unroll the stack by one frame and re-call the function
>> (after we have replaced the function),
>
> Huh. I think I can see how to define what this should mean: the
> continuation of the current function call is available; we just want to
> create a new activation that shares that continuation. How we'd rig that
> at the interpreter level, I'm not sure. JaegerMonkey might be even
> harder. This would take some guts.
Something like this is essential. Java has had hot code replacement for
years.
> These are more specific than the goals I've put on the wiki page.
Yes, but the one wiki page devotes a paragraph to webworkers which makes
no sense to me, it's a minor feature compared with other items here.
jjb
Colin Fraser
I've been following this with interest and I'm glad to see some concerted
effort in improving the debug capabilities.
One thing I'd like to see as an embedder of the spidermonkey engine (i.e.
non-browser use) is a way to get these debugging capabilities without
needing to pull in a whole lot of the other mozilla stack.
I don't know how feasible this is, as a lot of the capabilities sound like
they would be implemented at a high-ish level, but if it was possible to
implement the debug data transport layer close to the C it would seem to be
ideal for me.
Regards,
Colin
Jim Blandy
It sounds like what you are trying to do is avoid bringing XPConnect
into the picture --- is that right?
My current thinking is that js::dbg2 will be part of the SpiderMonkey
source tree, and will simply be another C++ API supported by the engine.
jsd2IDebuggerService will be in a sibling directory like jsd, and will
be a much thinner layer over js::dbg2 than jsd is over jsdbgapi.
So if you are working with SpiderMonkey alone, and not the rest of the
mozilla tree, you will get js::dbg2.
Unless we completely fail to get the kind of isolation we think we can,
the remote protocol server will be in JavaScript, and will thus depend
on the jsd2IDebuggerService, which means XPConnect will be part of that
picture. You will need to implement your own JS interface on top of
js::dbg2 if you want to avoid XPConnect.
John J Barton
It seems that the trend nowadays is to build the JS bridge via ctypes.
It moves the ugly parts into JS (and we're good with that ;-). Maybe we
don't need IDebuggerService?
jjb
Wes Garland
> Unless we completely fail to get the kind of isolation we think we can, the
> remote protocol server will be in JavaScript, and will thus depend on the
> jsd2IDebuggerService, which means XPConnect will be part of that picture.
> You will need to implement your own JS interface on top of js::dbg2 if you
> want to avoid XPConnect.
FWIW -- IIUC, JSD v1 and related stack looks like this:
- jsdbgapi is parallel to jsapi
- JSD sits on top of jsdbgapi
- Corba component sits on top of JSD (jsd_iiop) - ~IDL reflection
- Java component sits on top of JSD (netscape.jsdebugger) - ~ higher level
than a plain reflection
- jsdIDebuggerService sits on top of JSD via XPConnect - ~IDL reflection
- Firebug and Venkman sit on top of jsdIDebuggerService
- jsdb sits on top of JSD and reflects JSD into JavaScript (jsd_refl.cpp)
- debugger.js (shell debugger) sits on top of jsdb
I'm not sure if Colin is aware of debugger.js - it's certainly not trivially
usable in late-model spidermonkey (ping me in email or IRC if you want my
fork).
JJB's idea is particularly intriguing, because the most likely candidates
for actually using this -- embedder JS, Venkman, and Firebug -- all "speak"
JSCTypes, which would reduce the number of competing JSDv2 reflections down
to one. Additionally, there may be benefits to taking IDL + XPConnect out of
the loop (I am not qualified to comment there), and it may be easier to
write automatic test cases that can run in the js shell.
Now, of course, there is one big draw back to JS CTypes - they don't play
nice with a C++ API, but if JSDv2 were extern 'C'...
Some things to consider :)
Wes
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
Colin Fraser
"Jim Blandy" wrote:
> It sounds like what you are trying to do is avoid bringing XPConnect
> into the picture --- is that right?
Yes, that is what I would like to avoid if possible. I guess what I'd
really like would be the compiled-in out-of-process debug capabilities that
V8 appears to have (although I've never actually used V8 in anger so can't
comment how these work in practice).
"Wes Garland" wrote:
> I'm not sure if Colin is aware of debugger.js - it's certainly not
> trivially
> usable in late-model spidermonkey (ping me in email or IRC if you want my
> fork).
I don't think I have looked at that. I did download the Firebug sources
while looking for ideas - is it the same thing or a different
implementation?
For the record I have got basic breakpoint/step/trace/watch working in my
system using the lowest level jsdbgapi, but it has been slow going :)
Colin
Wes Garland
> Yes, that is what I would like to avoid if possible. I guess what I'd
> really like would be the compiled-in out-of-process debug capabilities that
> V8 appears to have (although I've never actually used V8 in anger so can't
> comment how these work in practice).
>
That's kind of what I'd like to see, too -- but I'll take what I can get and
write the rest. :)
I don't think I have looked at that. I did download the Firebug sources
> while looking for ideas - is it the same thing or a different
> implementation?
No, debugger.js is a debugger for the standalone js shell.
johnjbarton
> Hi,
>
> "Jim Blandy" wrote:
>> It sounds like what you are trying to do is avoid bringing XPConnect
>> into the picture --- is that right?
>
> Yes, that is what I would like to avoid if possible. I guess what I'd
> really like would be the compiled-in out-of-process debug capabilities that
> V8 appears to have (although I've never actually used V8 in anger so can't
> comment how these work in practice).
The V8 approach is, in my opinion, counter productive for both the
internal debugger developer and the clients of the interface. V8 can be
competitive as a developer platform only by committing much more
resource to tools.
>
>
> "Wes Garland" wrote:
>> I'm not sure if Colin is aware of debugger.js - it's certainly not
>> trivially
>> usable in late-model spidermonkey (ping me in email or IRC if you want my
>> fork).
>
> I don't think I have looked at that. I did download the Firebug sources
> while looking for ideas - is it the same thing or a different
> implementation?
Firebug 1.7 will be closer to what you could use: the layer on top of
jsd will be a server you can connect to remotely.
>
> For the record I have got basic breakpoint/step/trace/watch working in my
> system using the lowest level jsdbgapi, but it has been slow going :)
Exactly my point. That is why the V8 approach is not appropriate for
this problem.
jjb
>
> Colin
>
>
Wes Garland
will be a
> server you can connect to remotely.
Will it be possible to connect Firebug 1.7 to things other than something
like a brower? i.e. Can the layer on top of jsd live alone, or be
replicated somehow for use without XPConnect, nsIJSDebuggerService etc? (A
protocol like v8's is enough for this for JS embeddings - but you rightly
point out that it is pretty lacking from a browser-debugging POV).
johnjbarton
>> Firebug 1.7 will be closer to what you could use: the layer on top of jsd
> will be a
>> server you can connect to remotely.
>
> Will it be possible to connect Firebug 1.7 to things other than something
> like a brower? i.e.
Yes, but mainly because we don't consider XUL apps be browsers. If the
question is "will it be possible to use Firebug 1.7 in a non-XUL-app
world", I say it would take some work.
> Can the layer on top of jsd live alone, or be
Could be, but not a goal for us. That is, we're not interested in JS
only feature.
> replicated somehow for use without XPConnect, nsIJSDebuggerService etc? (A
No, since 1.7 will still be on jsd which uses jsdIDebuggerService.
We would be interested in collaborating on a server supporting JSDebugv2
if the JS-only features did not conflict with the full debug feature
set. One important question would be: is there a JS-callable socket
solution that does not use XPConnect?
jjb
Jim Blandy
> There are two layers of debugging here. One is chrome layer and one is
> content layer. From the chrome layer point of view, scrolling is just
> another operation, we can't allow it when the application is halted.
> From the content layer point of view, scrolling is a meta-operation
> which can be allowed. The content layer is an application on the
> platform; the chrome layer is an application on the OS. So the goal here
> would be to support separation of the content layer to we can allow
> chrome layer meta-operations to give a better debugger at the content
> layer.
I buy that. There's just a lot I need to learn before I understand what
it'll take to deliver on that idea.
>>> * Code re-writing and stack rewriting access (for metacompilers),
>>
>> Oy. :) Well, let me say this: assuming I'm understanding what you mean,
>> this would require deep changes in the interpreter itself, since aspects
>> of the JS stack are implemented in terms of the C++ stack. But if the
>> SpiderMonkey core ever supported this, then the debugging interface
>> should, too.
>
> I think there could be middle ground here. JS code loaders (caja, Google
> closure, ...) already do code rewriting and Firebug already does stack
> rewriting. So the question is what can we do to provide better support
> for this existing activity?
By "code rewriting", you just mean "loading new code"? Okay. I guess
what always catches me up thinking about "code rewriting" is that I
expect it to affect the code being executed by the currently active
stack frames, and the code to be run by function values currently
floating around in the data graph. Except in limited cases, that's just
completely ill-defined, and would be miserable to support going forward.
But if you mean that global functions get new definitions, but active
frames continue to execute the old code, and extant closure objects
continue to refer to the old code, then that's a case we have to support
anyway, simply because we support eval. So I'm not sure what's being
discussed here.
By stack rewriting... again, I'm trying to imagine what specifically
this means, and I come up with stuff that's either trivial (popping
frames; of course we'll support that) or off the charts (deleting and
inserting calls from the middle of the stack? What?!?).
>>> * Better focus on when the debugger is active (not in the goals but I
>>> saw stuff about this in the text),
>>
>> I don't understand this.
>
> Sorry, I meant: allow me to debug just one sphere. That got dropped from
> the goals, but I think it's in your design.
Yeah. Modulo details, I think we're on the same page here.
>>> * Built-in support for walking the object graph, with an API consistent
>>> with the rest of the debugger API (to replace the current memory addon),
>>
>> I don't understand this either.
>
> Given |window.foo| I want to answer the question: what are all the other
> references to the object referenced by |window.foo|?
>
> That reference graph should be in terms that can be easily expressed as
> property paths (|window.bar.x|)
Oooh. That would be super-useful; if SpiderMonkey had the machinery,
we'd certainly want to expose it.
>>> * Support for "onObjectCreated",
>>
>> You'd like a hook called each time JavaScript allocates an object?
>
> Given |window.foo| the developer wants to know "on what source line was
> the value set into |window.foo|" (watchpoints) and "on what source line
> was the value created" (onObjectCreated). There may be other better ways
> to answer the question than a hook, it's just one I know.
Ah --- given the address of a malloc block, some valgrind tools can give
you a backtrace of when that block was allocated. That's certainly
valuable. And Robert O'Callahan's chronomancer is an awesome tool built
on valgrind that can directly answer "where did this value get set"
questions without requiring you to re-run the program from the
beginning, as watchpoints do:
http://weblogs.mozillazine.org/roc/archives/2007/08/announcing_chro.html
These would be killer tools, although I don't know how to implement them
efficiently.
>>> * Ability to unroll the stack by one frame and re-call the function
>>> (after we have replaced the function),
>>
>> Huh. I think I can see how to define what this should mean: the
>> continuation of the current function call is available; we just want to
>> create a new activation that shares that continuation. How we'd rig that
>> at the interpreter level, I'm not sure. JaegerMonkey might be even
>> harder. This would take some guts.
>
> Something like this is essential. Java has had hot code replacement for
> years.
I guess the reason I hesitate is that I haven't used these tools, so I
don't really know what the semantics of these operations are (as I've
hinted at above). Where we can give the operation reasonable semantics
in the terms used by the language specification, then I'm game, but I
need to feel like I really know what's going on first.
johnjbarton
> On 05/18/2010 08:34 AM, johnjbarton wrote:
>> Given |window.foo| I want to answer the question: what are all the other
>> references to the object referenced by |window.foo|?
>>
>> That reference graph should be in terms that can be easily expressed as
>> property paths (|window.bar.x|)
>
> Oooh. That would be super-useful; if SpiderMonkey had the machinery,
> we'd certainly want to expose it.
This page summarizes how far we have gotten, using the code of Dion
Almer and Atul Varma:
http://getfirebug.com/wiki/index.php/Firebug_Memory_Profiler
We can probably continue to hack away, but performance issues limit us
to toy programs now.
>>>> * Support for "onObjectCreated",
>>>
>>> You'd like a hook called each time JavaScript allocates an object?
>>
>> Given |window.foo| the developer wants to know "on what source line was
>> the value set into |window.foo|" (watchpoints) and "on what source line
>> was the value created" (onObjectCreated). There may be other better ways
>> to answer the question than a hook, it's just one I know.
>
> Ah --- given the address of a malloc block, some valgrind tools can give
> you a backtrace of when that block was allocated. That's certainly
> valuable. And Robert O'Callahan's chronomancer is an awesome tool built
> on valgrind that can directly answer "where did this value get set"
> questions without requiring you to re-run the program from the
> beginning, as watchpoints do:
>
> http://weblogs.mozillazine.org/roc/archives/2007/08/announcing_chro.html
>
> These would be killer tools, although I don't know how to implement them
> efficiently.
I'll send you paper on a new approach to this by Salman Mirghsemi.
>
>>>> * Ability to unroll the stack by one frame and re-call the function
>>>> (after we have replaced the function),
>>>
>>> Huh. I think I can see how to define what this should mean: the
>>> continuation of the current function call is available; we just want to
>>> create a new activation that shares that continuation. How we'd rig that
>>> at the interpreter level, I'm not sure. JaegerMonkey might be even
>>> harder. This would take some guts.
>>
>> Something like this is essential. Java has had hot code replacement for
>> years.
>
> I guess the reason I hesitate is that I haven't used these tools, so I
> don't really know what the semantics of these operations are (as I've
> hinted at above). Where we can give the operation reasonable semantics
> in the terms used by the language specification, then I'm game, but I
> need to feel like I really know what's going on first.
Yes I think we are talking about the same thing, development time
support that allows experimentation, even if some are duds and some
require do-overs. Javascript is so much more malleable than Java that
we should be able to provide fast interactive development support.
jjb
Colin Fraser
"johnjbarton" wrote
> We would be interested in collaborating on a server supporting JSDebugv2
> if the JS-only features did not conflict with the full debug feature set.
> One important question would be: is there a JS-callable socket solution
> that does not use XPConnect?
>
There is a project at www.jsdb.org (javascript for databases) that extends
the basic Spidermonkey shell and includes socket support and a socket based
debugger that seems to be fairly well featured (written to the jsdbgapi). I
don't know if it is an extension of the debugger.js that Wes mentioned.
I'll try and have another look to see if there is any crossover with the new
JSDebug2 proposals.
Colin
johnjbarton
> On 5/19/2010 4:49 PM, Jim Blandy wrote:
>> On 05/18/2010 08:34 AM, johnjbarton wrote:
...
>>>>> (after we have replaced the function),
>>>>
>>>> Huh. I think I can see how to define what this should mean: the
>>>> continuation of the current function call is available; we just want to
>>>> create a new activation that shares that continuation. How we'd rig
>>>> that
>>>> at the interpreter level, I'm not sure. JaegerMonkey might be even
>>>> harder. This would take some guts.
>>>
>>> Something like this is essential. Java has had hot code replacement for
>>> years.
>>
>> I guess the reason I hesitate is that I haven't used these tools, so I
>> don't really know what the semantics of these operations are (as I've
>> hinted at above). Where we can give the operation reasonable semantics
>> in the terms used by the language specification, then I'm game, but I
>> need to feel like I really know what's going on first.
>
> Yes I think we are talking about the same thing, development time
> support that allows experimentation, even if some are duds and some
> require do-overs. Javascript is so much more malleable than Java that we
> should be able to provide fast interactive development support.
If you look at the video on this page:
http://code.google.com/events/io/2010/sessions/chrome-developer-tools.html
around minute 26 you can see a demo of the Google Chrome version of this
feature.
jjb
Rob Campbell
> around minute 26 you can see a demo of the Google Chrome version of this
> feature.
> jjb
I just popped in to mention this and see that you've already seen it,
John. The way Pavel described it in the video, it seems they pop the
top layer off the call-stack, make the changes to the function/script
where the breakpoint lives, then let the user continue execution to
the breakpoint, new code in place.
As Jim suggests, this would probably require some JS engine work to
make it happen, but is a very neat feature.
~ rob