Debugger API feedback

[David Bruant]

Hi,

I've been playing with the new Debugger API and I'd like to provide some
feedback and ask questions.

The first piece of feedback is that overall, the API is excellent. I
particularly love the idea of the Debugger.DebuggeeWouldRun Exception.
Having a mirror-like API with 1:1 correspondance between a debuggee
object and a Debugger.Object is also a brilliant idea.


* No way to pause
Looking at the Debugger API, I see a way to set breakpoint or trigger
some behavior onDebuggerStatement, but I don't see a way to pause the
program to enable interactive debugging. To quote the wiki [1]:
"Your event handling methods run in the same thread as the debuggee, on
the same JavaScript stack: when the event occurs, the debuggee pauses
while your handler methods run, and resumes (unless you say otherwise)
when your methods return."
but it seems that "unless you say otherwise" is limited to
returning/yielding/throwing early or canceling the program (by returning
null) or DOSing (which is a stupid idea) [2].

It would be awesome if there was a debugger.pause() (or a Debugger.Pause
resumption value) and debugger.resume() method (and a debugger.isPaused
property I guess). I have no strong opinion on the "how", but being able
to pause a program to enable interactive debugging seems like an
important use case.

I wondered how the Firefox built-in debugger was doing to pause the
program (since they actually do). Apparently, they use a ThreadState
global [3]. I haven't been able to find what it is and how it works.


* setBreakpoint and the 'hit' property
>From my own code:

var bpHandler = {
hit: function(frame){
console.log('just hit breakpoint');
}
};

// later
script.setBreakpoint(offset, bpHandler);

why can't setBreakpoint just take a function as an argument instead of
an object with a 'hit' property?


* "findScripts" instead of "getAllScripts".
I've fixed MDN [4] which was talking about a "getAllScripts" method
which has been replaced by findScripts. Is the semantics unchanged?
Are there other API or semantics differences?
Is MDN expected to be up-to-date?
Which documentation is up-to-date related to the implementation?


* block-scoped variables
ES.next will standardize 'let' and 'const' (which already exist in
Firefox anyway) which will be block-scoped variables. How will these be
accessible in a frame? Will it be possible to differenciate the fact
that they are block-scoped within a frame?


* Web Workers
If a program spawns a Web Worker, how can I debug what happens in the
worker?


* WeakMap, Map and Set
Are there introspection methods for them? Specifically for WeakMaps
since in a regular program, it's not possible to enumerate the keys.

Thank you very much,

David

[1] https://wiki.mozilla.org/Debugger
[2] https://wiki.mozilla.org/Debugger#Resumption_Values
[3] https://hg.mozilla.org/mozilla-central/rev/6e1983c0efbb
[4]
https://developer.mozilla.org/index.php?title=en/SpiderMonkey/JS_Debugger_API_Reference/Debugger&action=diff&revision=14&diff=16

[David Bruant]

Le 21/05/2012 20:01, Jason Orendorff a écrit :
> David,
>
> Thanks very much for the detailed comments. We'll make sure the bugs
> get on file.
Thanks :-)

> On 5/19/12 7:35 AM, David Bruant wrote:
>> * No way to pause

>> (...)
>
> I'm sorry for the long response that follows.
Don't be, that's an interesting (and I understand complicated) topic.

> In an event-driven GUI program, it's awkward to pause in the middle of
> event handler code. For the program to remain responsive, you have to
> be in the event loop processing events. But to debug the code, it has
> to remain paused on the stack. Dilemma! You have two options:
>
> 1. Capture the current continuation (in the call/cc sense) of the
> paused code you're debugging.
I'm sorry, but I didn't understand this sentense at all :-s
What is a continuation?
What is the call/cc sense? (and what other senses are they?)

> Set it aside in the debugger for later. Return to the event loop.
>
> Unfortunately, we don't really have this option when debugging in
> Gecko. Our C++ codebase doesn't support call/cc, nor does our
> implementation of JS; we would need both.
>
> 2. While the debuggee code is on the stack, run a nested event loop.
> This is what the built-in debugger and Firebug do. They do some ad hoc
> UI event filtering, delivering events to the browser chrome and other
> tabs but not to the debuggee.
>
> Now, there is a high-level, asynchronous, remote-able protocol,
> implemented atop the Debugger API, which has messages like "interrupt"
> and "resume":
> https://wiki.mozilla.org/Remote_Debugging_Protocol
> It's implemented, and the built-in debugger uses it; see
> toolkit/devtools/debugger/server and particularly the
> dbg-script-actors.js file in that directory. This protocol lets you
> debug from out-of-process and pause everything (the entire browser, if
> you want).
Having this capability with a remote protocol, but not locally is an
intriguing idea. But thanks for the pointer, I'll take a look.

>> * setBreakpoint and the 'hit' property
>> From my own code:
>>
>> var bpHandler = {
>> hit: function(frame){
>> console.log('just hit breakpoint');
>> }
>> };
>> // later
>> script.setBreakpoint(offset, bpHandler);
>>
>> why can't setBreakpoint just take a function as an argument instead of
>> an object with a 'hit' property?

> Yeah, I know, right? There is a reason for this, not necessarily a
> good reason. It's because everything else in the API that has hooks is
> an object.
> debugger.onEnterFrame // 'this' is the Debugger
> frame.onPop // 'this' is the Debugger.Frame
> etc.
>
> The user always has a choice between using a closure and using 'this'
> to store extra data for the hook to use. Both OO style and lisp style
> work. Kinda neat?
I tend to think that the OO style for callbacks is too verbose. Having
to name a callback seems to me like an overhead. As a proof, i'll take
the fact that I have never seen the {handler: fct} DOM event handlers
form in production websites.

> I dunno, If I had this one to design over again, I'd probably do it a
> little differently.
Interesting.
For the short term, I'll probably wrap my function with "function
wrap(f){f.hit = f; return f;}".

>> * Web Workers
>> If a program spawns a Web Worker, how can I debug what happens in the
>> worker?

> Workers aren't debuggable yet. I filed bug 757133 for this. It is
> actually not a huge amount of work, and probably none inside the JS
> engine. So if you're interested in taking on some or all of it, let us
> know. Or if it's just a priority for you, again, let us know.
No priority on that as far as I'm concerned.
As we discussed back in December, I'm interested in working on a
debugger that would allow to go back in time [1]. For the moment, I'm
just playing around with the API and seeing what can and cannot be done
with it.
Regarding priorities, having the whole object watchpoint [2] would the
priority for me. Something else that would be interesting would be
knowing when a variable value has changed [3]. As a workaround, I've
written a piece of code that can detect that [4]. It's obviously a very
expensive process :-)

Thanks for your answers,

David

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=738965
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=638053
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=740548
[4]
https://github.com/DavidBruant/Herodotus/blob/e4586bde77e690eafcbdaf7f766c994334c1acc2/herodotus.js#L87-129

[Boris Zbarsky]

On 5/21/12 2:59 PM, David Bruant wrote:
> What is a continuation?
> What is the call/cc sense?

http://en.wikipedia.org/wiki/Call-with-current-continuation has a sort
of explanation.

-Boris

[Jim Blandy]

On 05/19/2012 05:35 AM, David Bruant wrote:
> * No way to pause

> Looking at the Debugger API, I see a way to set breakpoint or trigger
> some behavior onDebuggerStatement, but I don't see a way to pause the
> program to enable interactive debugging. To quote the wiki [1]:
> "Your event handling methods run in the same thread as the debuggee, on
> the same JavaScript stack: when the event occurs, the debuggee pauses
> while your handler methods run, and resumes (unless you say otherwise)
> when your methods return."
> but it seems that "unless you say otherwise" is limited to
> returning/yielding/throwing early or canceling the program (by returning
> null) or DOSing (which is a stupid idea) [2].
>
> It would be awesome if there was a debugger.pause() (or a Debugger.Pause
> resumption value) and debugger.resume() method (and a debugger.isPaused
> property I guess). I have no strong opinion on the "how", but being able
> to pause a program to enable interactive debugging seems like an
> important use case.
>
> I wondered how the Firefox built-in debugger was doing to pause the
> program (since they actually do). Apparently, they use a ThreadState
> global [3]. I haven't been able to find what it is and how it works.

The way to get the effect of a 'pause' is to start up a nested event
loop. Since the event loop is an aspect of the browser, and not of the
JavaScript engine per se, it's not covered in Debugger.

The code for this is in ThreadActor.prototype._nest, in
toolkit/devtools/debugger/server/dbg-script-actors.js. That uses
'preNest' and 'postNest' hooks defined in
toolkit/devtools/debugger/server/dbg-browser-actors.js.

[Jason Orendorff]

On 5/21/12 1:59 PM, David Bruant wrote:
>> In an event-driven GUI program, it's awkward to pause in the middle of
>> event handler code. For the program to remain responsive, you have to
>> be in the event loop processing events. But to debug the code, it has
>> to remain paused on the stack. Dilemma! You have two options:
>>
>> 1. Capture the current continuation (in the call/cc sense) of the
>> paused code you're debugging.
> I'm sorry, but I didn't understand this sentense at all :-s

> What is a continuation?

> What is the call/cc sense? (and what other senses are they?)

A continuation is "what happens next" after a certain point in a
program. I don't think it'll help this discussion to get into it, but
I'd be happy to chat with you about it. It's very hard to explain.

Let me try this question over again. I'll try two different approaches.

1. Basically when you're in a debugging hook, the stack looks like this:

my_debugging_hook
debuggee code
debuggee code
debuggee code
the main event loop
main()

Suppose you call dbg.pause(), and then you return. What's supposed to
happen?

2. You don't need a dbg.pause() method because the Debugger API
automatically pauses and resumes the debuggee as needed! If a debugging
hook is running, that means the debuggee is paused. If you want the
debuggee to stay paused, just don't return!

>> Now, there is a high-level, asynchronous, remote-able protocol,
>> implemented atop the Debugger API, which has messages like "interrupt"
>> and "resume":
>> https://wiki.mozilla.org/Remote_Debugging_Protocol
>> It's implemented, and the built-in debugger uses it; see
>> toolkit/devtools/debugger/server and particularly the
>> dbg-script-actors.js file in that directory. This protocol lets you
>> debug from out-of-process and pause everything (the entire browser, if
>> you want).
> Having this capability with a remote protocol, but not locally is an
> intriguing idea. But thanks for the pointer, I'll take a look.

The remote debugging protocol works fine in-process, as well as
remotely. The built-in debugger uses it. You can use it.

> I tend to think that the OO style for callbacks is too verbose.

Yeah, I'm not going to reply to this in detail because we basically agree.

-j

[Jason Orendorff]

David,

Thanks very much for the detailed comments. We'll make sure the bugs get
on file.

On 5/19/12 7:35 AM, David Bruant wrote:
> * No way to pause
> Looking at the Debugger API, I see a way to set breakpoint or trigger
> some behavior onDebuggerStatement, but I don't see a way to pause the
> program to enable interactive debugging. To quote the wiki [1]:
> "Your event handling methods run in the same thread as the debuggee, on
> the same JavaScript stack: when the event occurs, the debuggee pauses
> while your handler methods run, and resumes (unless you say otherwise)
> when your methods return."
> but it seems that "unless you say otherwise" is limited to
> returning/yielding/throwing early or canceling the program (by returning
> null) or DOSing (which is a stupid idea) [2].
>
> It would be awesome if there was a debugger.pause() (or a Debugger.Pause
> resumption value) and debugger.resume() method (and a debugger.isPaused
> property I guess). I have no strong opinion on the "how", but being able
> to pause a program to enable interactive debugging seems like an
> important use case.
>
> I wondered how the Firefox built-in debugger was doing to pause the
> program (since they actually do). Apparently, they use a ThreadState
> global [3]. I haven't been able to find what it is and how it works.

I'm sorry for the long response that follows.

In an event-driven GUI program, it's awkward to pause in the middle of
event handler code. For the program to remain responsive, you have to be
in the event loop processing events. But to debug the code, it has to
remain paused on the stack. Dilemma! You have two options:

1. Capture the current continuation (in the call/cc sense) of the paused

code you're debugging. Set it aside in the debugger for later. Return to

the event loop.

Unfortunately, we don't really have this option when debugging in Gecko.
Our C++ codebase doesn't support call/cc, nor does our implementation of
JS; we would need both.

2. While the debuggee code is on the stack, run a nested event loop.
This is what the built-in debugger and Firebug do. They do some ad hoc
UI event filtering, delivering events to the browser chrome and other
tabs but not to the debuggee.

Now, there is a high-level, asynchronous, remote-able protocol,
implemented atop the Debugger API, which has messages like "interrupt"
and "resume":
https://wiki.mozilla.org/Remote_Debugging_Protocol
It's implemented, and the built-in debugger uses it; see
toolkit/devtools/debugger/server and particularly the
dbg-script-actors.js file in that directory. This protocol lets you
debug from out-of-process and pause everything (the entire browser, if
you want).

> * setBreakpoint and the 'hit' property
> From my own code:
>
> var bpHandler = {
> hit: function(frame){
> console.log('just hit breakpoint');
> }
> };
>
> // later
> script.setBreakpoint(offset, bpHandler);
>
> why can't setBreakpoint just take a function as an argument instead of
> an object with a 'hit' property?
Yeah, I know, right? There is a reason for this, not necessarily a good
reason. It's because everything else in the API that has hooks is an object.
debugger.onEnterFrame // 'this' is the Debugger
frame.onPop // 'this' is the Debugger.Frame
etc.

The user always has a choice between using a closure and using 'this' to
store extra data for the hook to use. Both OO style and lisp style work.
Kinda neat?

I dunno, If I had this one to design over again, I'd probably do it a
little differently.

> * "findScripts" instead of "getAllScripts".
> I've fixed MDN [4] which was talking about a "getAllScripts" method
> which has been replaced by findScripts. Is the semantics unchanged?
> Are there other API or semantics differences?
> Is MDN expected to be up-to-date?
> Which documentation is up-to-date related to the implementation?

https://wiki.mozilla.org/Debugger is the thing to trust at this point.

Or ask in #devtools or look at the source code or try it out...

> * block-scoped variables
> ES.next will standardize 'let' and 'const' (which already exist in
> Firefox anyway) which will be block-scoped variables. How will these be
> accessible in a frame? Will it be possible to differenciate the fact
> that they are block-scoped within a frame?

They're already accessible using the Debugger object. Use frame.eval to
run code in the debuggee's current innermost scope. Use
frame.environment to query the current innermost scope directly. (And of
course frame.environment.outer is the next enclosing scope.)

> * Web Workers
> If a program spawns a Web Worker, how can I debug what happens in the
> worker?
Workers aren't debuggable yet. I filed bug 757133 for this. It is
actually not a huge amount of work, and probably none inside the JS
engine. So if you're interested in taking on some or all of it, let us
know. Or if it's just a priority for you, again, let us know.

> * WeakMap, Map and Set
> Are there introspection methods for them? Specifically for WeakMaps
> since in a regular program, it's not possible to enumerate the keys.
>

Yep, good idea. We need to do this. Jim's going to file bugs.

-j

[Jim Blandy]

On 05/21/2012 11:59 AM, David Bruant wrote:
>>> * setBreakpoint and the 'hit' property
>>> From my own code:
>>>
>>> var bpHandler = {
>>> hit: function(frame){
>>> console.log('just hit breakpoint');
>>> }
>>> };
>>> // later
>>> script.setBreakpoint(offset, bpHandler);
>>>
>>> why can't setBreakpoint just take a function as an argument instead of
>>> an object with a 'hit' property?
>> Yeah, I know, right? There is a reason for this, not necessarily a
>> good reason. It's because everything else in the API that has hooks is
>> an object.
>> debugger.onEnterFrame // 'this' is the Debugger
>> frame.onPop // 'this' is the Debugger.Frame
>> etc.
>>
>> The user always has a choice between using a closure and using 'this'
>> to store extra data for the hook to use. Both OO style and lisp style
>> work. Kinda neat?

> I tend to think that the OO style for callbacks is too verbose. Having
> to name a callback seems to me like an overhead. As a proof, i'll take
> the fact that I have never seen the {handler: fct} DOM event handlers
> form in production websites.

I'm not very familiar with content programming, so I could be missing
something, but I don't think {handler:fct}-style handlers are popular
for DOM handlers because such handlers can use the DOM element itself as
their 'this' object --- if I say elt.onclick = function f() {...}, then
the 'this' that f receives is elt. That's good enough. Isn't this how
it's usually done?

In the case of a breakpoint, there's no natural 'this' value to pass.
The Debugger.Script instance is to broad: the handler wants
per-breakpoint context. The thought was that breakpoint handlers could
use the same object as both its internal representation of a breakpoint
and as a breakpoint handler.

If the handler function were the sole per-breakpoint client object, then
one would need to store per-breakpoint context in one of two places:

- As variables from enclosing environments, captured by the handler
function. This is fine, but it means that all code that wants access to
those variables must be written within their scope. It's nice to be able
to place code elsewhere that can operate on a breakpoint's contextual data.

- As properties on the function object itself. Since JavaScript
functions are just objects, this is fine, but it apparently is not
something people often do. We wanted to avoid requiring new coding patterns.

[Jim Blandy]

David, I think we've filed bugs for the issues you've brought up. Is
there anything you've brought up that is not covered by a bug?

[David Bruant]

A "Breakpoint" abstraction could be what's needed here. It could carry
all the information related to the breakpoint that can be useful. The
minimum would be {script, offset}, but having a 'line' (and 'column'
when available) property could be handy (although a getter on
Breakpoint.prototype could do the trick)

> The thought was that breakpoint handlers could use the same object as
> both its internal representation of a breakpoint and as a breakpoint
> handler.
>
> If the handler function were the sole per-breakpoint client object,
> then one would need to store per-breakpoint context in one of two places:
>
> - As variables from enclosing environments, captured by the handler
> function. This is fine, but it means that all code that wants access
> to those variables must be written within their scope. It's nice to be
> able to place code elsewhere that can operate on a breakpoint's
> contextual data.
>
> - As properties on the function object itself. Since JavaScript
> functions are just objects, this is fine, but it apparently is not
> something people often do. We wanted to avoid requiring new coding
> patterns.

Interesting. I had not really thought about per-breakpoint context. I
haven't had the need yet.
The Breakpoint instances could be used here I think.

> The way to get the effect of a 'pause' is to start up a nested event
> loop. Since the event loop is an aspect of the browser, and not of the
> JavaScript engine per se, it's not covered in Debugger.

I guess that's also the reason there is no way to inspect the event loop
in Debugger?
Is there a way to inspect it with the remote debugging protocol? I can't
find one.
The idea here would be to be able to list all pending events (user
inputs, tiemouts, intervals, postMessage-messages (from workers or
cross-frame), etc.).

> David, I think we've filed bugs for the issues you've brought up. Is
> there anything you've brought up that is not covered by a bug?

I had mentionned the event loop on github, but now, I'm not sure it
belongs to the Debugger API.


Historically, the event loop has not been part of ECMAScript/JavaScript
and rather part of the browser. But time has passed and the "cultural"
JavaScript concurrency model (event loop, run-to-completion) has
expanded to node.js. The JSConf 2012 River Trail demo [1] starts with a
description of this model and the benefits that come with it.
There is also a pending proposal for this model to be part of ECMAScript
[2].
With all that in mind, wouldn't it be worth bringing the event loop to
the JS engine? (I'm aware it's certainly a lot of work, but I'm asking
the question as more of a theorical statement).

David

[1] http://2012.jsconf.us/ (sorry for the inaccurate link)
[2] http://wiki.ecmascript.org/doku.php?id=strawman:concurrency

[Jason Orendorff]

On 5/22/12 5:44 AM, David Bruant wrote:
> Le 21/05/2012 23:09, Jim Blandy a écrit :
>> In the case of a breakpoint, there's no natural 'this' value to pass.
>> The Debugger.Script instance is to broad: the handler wants
>> per-breakpoint context.
> A "Breakpoint" abstraction could be what's needed here. It could carry
> all the information related to the breakpoint that can be useful. The
> minimum would be {script, offset}, but having a 'line' (and 'column'
> when available) property could be handy (although a getter on
> Breakpoint.prototype could do the trick)

You can build it yourself in JS. The Debugger API was designed to
support this kind of thing.
https://gist.github.com/2769076

(Note: perhaps that code should interact with your onNewScript hook, so
that if you reload the file, and it gets compiled again, your
breakpoints are automatically set. But it depends on what you want. The
API is low-level enough to let you build whichever behavior you need.)

>> The way to get the effect of a 'pause' is to start up a nested event
>> loop. Since the event loop is an aspect of the browser, and not of
>> the JavaScript engine per se, it's not covered in Debugger.
> I guess that's also the reason there is no way to inspect the event
> loop in Debugger?

Right, that's the reason. The Debugger API is an API into the JS engine,
so its design reflects a few architectural details about how we draw the
line between our JS engine and the rest of the browser.

Other components of the browser have their own APIs, naturally.

> Is there a way to inspect it with the remote debugging protocol? I
> can't find one.
> The idea here would be to be able to list all pending events (user
> inputs, tiemouts, intervals, postMessage-messages (from workers or
> cross-frame), etc.).

There should be.

> With all that in mind, wouldn't it be worth bringing the event loop to
> the JS engine? (I'm aware it's certainly a lot of work, but I'm asking
> the question as more of a theorical statement).

I think we could *expose* the event loop to the JS engine with or
without a massive code reorg. And it's certainly worth doing (i.e. we
have to do it) if the concurrency proposals are adopted.

-j

[David Bruant]

Le 22/05/2012 16:12, Jason Orendorff a écrit :
> On 5/22/12 5:44 AM, David Bruant wrote:
>> Le 21/05/2012 23:09, Jim Blandy a écrit :
>>> In the case of a breakpoint, there's no natural 'this' value to
>>> pass. The Debugger.Script instance is to broad: the handler wants
>>> per-breakpoint context.
>> A "Breakpoint" abstraction could be what's needed here. It could
>> carry all the information related to the breakpoint that can be
>> useful. The minimum would be {script, offset}, but having a 'line'
>> (and 'column' when available) property could be handy (although a
>> getter on Breakpoint.prototype could do the trick)
>
> You can build it yourself in JS. The Debugger API was designed to
> support this kind of thing.
> https://gist.github.com/2769076

True. I will do that.
By the way, the for(of) is really cool. I had forgotten that I don't
need to care about cross browser support.

Your breakAt is interesting in how expressive it is especially since it
abstracts out offsets.

>>> The way to get the effect of a 'pause' is to start up a nested event
>>> loop. Since the event loop is an aspect of the browser, and not of
>>> the JavaScript engine per se, it's not covered in Debugger.
>> I guess that's also the reason there is no way to inspect the event
>> loop in Debugger?
>
> Right, that's the reason. The Debugger API is an API into the JS
> engine, so its design reflects a few architectural details about how
> we draw the line between our JS engine and the rest of the browser.
> Other components of the browser have their own APIs, naturally.

Is there a place where I can find all these APIs?

>> Is there a way to inspect it with the remote debugging protocol? I
>> can't find one.
>> The idea here would be to be able to list all pending events (user
>> inputs, tiemouts, intervals, postMessage-messages (from workers or
>> cross-frame), etc.).
>
> There should be.

In the sense that there is one in the protocol and I haven't found it or
that it's missing? Or that there is a way which is not in the protocol?

David

[David Bruant]

Le 21/05/2012 20:54, Jim Blandy a écrit :

> On 05/19/2012 05:35 AM, David Bruant wrote:
>> * No way to pause
>> Looking at the Debugger API, I see a way to set breakpoint or trigger
>> some behavior onDebuggerStatement, but I don't see a way to pause the
>> program to enable interactive debugging. To quote the wiki [1]:
>> "Your event handling methods run in the same thread as the debuggee, on
>> the same JavaScript stack: when the event occurs, the debuggee pauses
>> while your handler methods run, and resumes (unless you say otherwise)
>> when your methods return."
>> but it seems that "unless you say otherwise" is limited to
>> returning/yielding/throwing early or canceling the program (by returning
>> null) or DOSing (which is a stupid idea) [2].
>>
>> It would be awesome if there was a debugger.pause() (or a Debugger.Pause
>> resumption value) and debugger.resume() method (and a debugger.isPaused
>> property I guess). I have no strong opinion on the "how", but being able
>> to pause a program to enable interactive debugging seems like an
>> important use case.
>>
>> I wondered how the Firefox built-in debugger was doing to pause the
>> program (since they actually do). Apparently, they use a ThreadState
>> global [3]. I haven't been able to find what it is and how it works.

> The way to get the effect of a 'pause' is to start up a nested event
> loop. Since the event loop is an aspect of the browser, and not of the
> JavaScript engine per se, it's not covered in Debugger.
>

> The code for this is in ThreadActor.prototype._nest, in
> toolkit/devtools/debugger/server/dbg-script-actors.js. That uses
> 'preNest' and 'postNest' hooks defined in
> toolkit/devtools/debugger/server/dbg-browser-actors.js.

Ok. So, I've played with enterNestedEventLoop and exitNestedEventLoop.
>From reading the code of dbg-script-actors.js [1], it was not obvious
that the code "freezes". Knowing it does, the preNest and postNest hooks
make a lot of sense.
Freezing like that is a bit hard, because there is no way to "boostrap"
the nested event loop. My understanding is that the way it is
bootstraped in the Firefox debugger is that all the "intelligence" of
the debugger does not run within the process of the debuggee but rather
in its own window (which is why the debugger does have its own window).
The nested event loop that run in the same context than the debuggee is
controlled by messages sent by the "intelligent" debugger.

In conclusion, it is not possible to do an interactive debugger unless
creating a separate window. Doing otherwise was probably to begin with.

>From an API point of view, it could be interesting if
enterNestedEventLoop took a 'bootstrap' function as argument. When
called, enterNestedEventLoop would create the nested event loop and
start a new stack by calling the boostrap function. But we're not there yet.

Thanks again for all the answers and guidance,

David

[1]
https://hg.mozilla.org/mozilla-central/file/64187d60fae7/toolkit/devtools/debugger/server/dbg-script-actors.js#l190

[Jason Orendorff]

On 5/22/12 4:59 PM, David Bruant wrote:
> Le 21/05/2012 20:54, Jim Blandy a �crit :

>> The way to get the effect of a 'pause' is to start up a nested event
>> loop. Since the event loop is an aspect of the browser, and not of the
>> JavaScript engine per se, it's not covered in Debugger.
>>
>> The code for this is in ThreadActor.prototype._nest, in
>> toolkit/devtools/debugger/server/dbg-script-actors.js. That uses
>> 'preNest' and 'postNest' hooks defined in
>> toolkit/devtools/debugger/server/dbg-browser-actors.js.
> Ok. So, I've played with enterNestedEventLoop and exitNestedEventLoop.
> From reading the code of dbg-script-actors.js [1], it was not obvious
> that the code "freezes". Knowing it does, the preNest and postNest hooks
> make a lot of sense.
> Freezing like that is a bit hard, because there is no way to "boostrap"
> the nested event loop.

What do you mean by "bootstrap" here? Keep in mind, basically all
enterNestedEventLoop does is:

while (!exitNestedEventLoopWasCalled())
NS_ProcessNextEvent();

> My understanding is that the way it is
> bootstraped in the Firefox debugger is that all the "intelligence" of
> the debugger does not run within the process of the debuggee but rather
> in its own window (which is why the debugger does have its own window).

I don't know if this is right�it might be simpler than you think�but I'm
not the right person to say either way. You might want to take this to
dev.platform or ping me on IRC and we'll go find the right person.

-j

[Jim Blandy]

On 05/22/2012 02:59 PM, David Bruant wrote:
> Freezing like that is a bit hard, because there is no way to "boostrap"

> the nested event loop. My understanding is that the way it is

> bootstraped in the Firefox debugger is that all the "intelligence" of
> the debugger does not run within the process of the debuggee but rather
> in its own window (which is why the debugger does have its own window).

> The nested event loop that run in the same context than the debuggee is
> controlled by messages sent by the "intelligent" debugger.

What do you think of the way jorendb does it? (I assume you've seen
jorendb; if not: https://github.com/jorendorff/jorendb) It's similar:
the repl isn't a single loop entered once per debugging session;
instead, the repl is entered each time the debuggee pauses, and the repl
exits when it wants to continue/step/finish.

Note that jorendb's 'print' command, based on
Debugger.Frame.prototype.eval, is a separate case: that is called from
the repl, and returns to the same iteration of the repl. (If we hit a
breakpoint while running the eval, then that spins up a second, nested
repl.)

I don't see it as the debuggee "freezing"; I see it more as the debugger
and debuggee sharing a single stack, which means that all the debugger's
frames need to be gone whenever the debuggee is to continue.

[David Bruant]

Hi Jason, Jim,

Sorry for the late answer. I was away for a bit and I took some time to
do some drawings to give proper answers.

Le 23/05/2012 23:13, Jason Orendorff a écrit :
> On 5/22/12 4:59 PM, David Bruant wrote:

>> Le 21/05/2012 20:54, Jim Blandy a écrit :
>>> The way to get the effect of a 'pause' is to start up a nested event
>>> loop. Since the event loop is an aspect of the browser, and not of the
>>> JavaScript engine per se, it's not covered in Debugger.
>>>
>>> The code for this is in ThreadActor.prototype._nest, in
>>> toolkit/devtools/debugger/server/dbg-script-actors.js. That uses
>>> 'preNest' and 'postNest' hooks defined in
>>> toolkit/devtools/debugger/server/dbg-browser-actors.js.
>> Ok. So, I've played with enterNestedEventLoop and exitNestedEventLoop.
>> From reading the code of dbg-script-actors.js [1], it was not obvious
>> that the code "freezes". Knowing it does, the preNest and postNest hooks
>> make a lot of sense.

>> Freezing like that is a bit hard, because there is no way to "boostrap"
>> the nested event loop.
>

> What do you mean by "bootstrap" here? Keep in mind, basically all
> enterNestedEventLoop does is:
> while (!exitNestedEventLoopWasCalled())
> NS_ProcessNextEvent();

I have made some images to explain what I meant for "bootstrap".
Something about a thousand words. See
http://davidbruant.github.com/JSRuntime-SVGs/ if you see less than 4
images, clone g...@github.com:DavidBruant/JSRuntime-SVGs.git and open
index.html locally (I don't know what's the difference, but the github
version does some 404 on my SVGs for no reason).
I assume my mental representation to be accurate, but please correct me
if any of what I've drawn is inaccurate.

The first image is a regular JavaScript run-time with a stack on the
left, a queue at the bottom and a heap (graphical position of elements
in the heap is completely irrelevant here).
For the queue, messages can come from different sources, like DOM
events, postMessage messages, timeouts, intervals, etc. One message is
treated at a time. On processing, a message "creates" the initial frame
then, run-to-completion, etc.

For the first image, I have drawn the top frame in a different color to
mean that it's a debugger frame (on the same stack, but not the same
"type of code").

On the second image, enterNestedEventLoop is called. A new stack and a
new queue are created. Until exitNestedEventLoop, new messages will go
in the nested frame and be treated in the nested stack.
One important thing to point out for the nested event loop is that both
the stack and
the queue are empty initially. Code will run in this stack/queue only if
messages are
sent to the queue from different sources (like another window sending a
message, etc.)

What I mean by "bootstraping the nesting event loop" is described in the
third image. On creation, instead of creating empty stack and queue, the
nested event loop would be created with an initial frame on the stack.
This frame could be created thanks to a function passed as an argument
to enterNestedEventLoop.

>> My understanding is that the way it is
>> bootstraped in the Firefox debugger is that all the "intelligence" of
>> the debugger does not run within the process of the debuggee but rather
>> in its own window (which is why the debugger does have its own window).
>

> I don't know if this is right—it might be simpler than you think—but

> I'm not the right person to say either way. You might want to take
> this to dev.platform or ping me on IRC and we'll go find the right person.

I will.

Le 24/05/2012 00:56, Jim Blandy a écrit :
> On 05/22/2012 02:59 PM, David Bruant wrote:
>> Freezing like that is a bit hard, because there is no way to "boostrap"
>> the nested event loop. My understanding is that the way it is
>> bootstraped in the Firefox debugger is that all the "intelligence" of
>> the debugger does not run within the process of the debuggee but rather
>> in its own window (which is why the debugger does have its own window).
>> The nested event loop that run in the same context than the debuggee is
>> controlled by messages sent by the "intelligent" debugger.
> What do you think of the way jorendb does it? (I assume you've seen
> jorendb; if not: https://github.com/jorendorff/jorendb) It's similar:
> the repl isn't a single loop entered once per debugging session;
> instead, the repl is entered each time the debuggee pauses, and the
> repl exits when it wants to continue/step/finish.

I wasn't aware of jorendb, thanks for sharing that.
Apparently, the trick here is the synchronous block caused by readline.
I feel a bit weird about it, but that's probably cultural. I've been
taught to conform to a JS model where you should never block.

In a way, it's fortunate that readline is defined as a synchronously
blocking primitive. If SpiderMonkey's readline was following Node.js
design [1], it would not be possible to do the same thing.
Also, I'm not sure this can be reproduced for any use case. readline
works well for a command-line tool. Even if it was, I don't think
synchronous blocking primitive should be the way to go in general.

> I don't see it as the debuggee "freezing"; I see it more as the
> debugger and debuggee sharing a single stack, which means that all the
> debugger's frames need to be gone whenever the debuggee is to continue.

I used "freezing" in the context of using enterNestedEventLoop, which,
as I drew in the second image freezes the state of the stack and queue
of the original event loop.

After the 3 initial pictures, I started to think of what I originally
meant by "pausing the debuggee" and that's the 4th image. When calling
"pause", the bottom of the stack would be "lifted up" to the top of the
debuggee frames and a new queue is created. Until calling "resume", the
messages would only accumulate in the new queue and the stack would
never go back the debuggee code.
I don't know if it's easy to do. While explaining, I realized that the
"boostraped nested event loop" is equivalent or very close to that model.

After more thoughts, I've realized that I can hack the bootstraping
myself by opening another window which role will be to wake me up after
having called enterNestedEventLoop.

David

[1] http://nodejs.org/api/readline.html#readline_rl_question_query_callback