Starting work on a Chrome Devtools addon

[Adam Crabtree]

Howdy all,

I have LinkedIn's blessing to work full-time for 2 weeks on building a node.js addon[1] for remote Chrome Devtools debugging. Since I've never built an addon, this is a rather ambitious task for my first addon, and nearly everyone in the community would benefit from this, I thought I'd solicit interest in working with me on it, either directly via pull requests or indirectly by giving advice on building node.js addons.

Details:

Per Pavel Feldman of the Chrome Devtools team, it's possible, with possibly some limited support from Chrome to decouple some APIs if necessary. For me personally, the inability to use the native Chrome debugger has always been a little frustrating, and this is my attempt to do what I can to resolve it. Any non-trivial contributions will receive Contributor permissions to accelerate collaboration.

Pavel has also graciously provided me with the following high-level overview of where to start:

// Begin Quote

There is not much info available. These are the rough hints for you (use cs.chromium.org to navigate Blink source). Then we could chat over VC / hangouts / Skype.

[Compile]

InspectorController is a container for remote debugging protocol agents. We have one for the Page (InspectorController.cpp) and we have one for the workers (WorkerInspectorController.cpp that has limited functionality). Given that Node is more like a worker (pure JS execution environment), take a look at that file for the list of agents to compile in your module (InspectorDebuggerAgent, InspectorConsoleAgent, InspectorProfileAgent, etc.). So you would need to compile all of them.

These agents would pull ScriptDebugServer.cpp, ScriptProfiler.cpp, Script* etc. These are wrappers around v8, should compile against v8.h. Some of them (ScriptProfiler) and most of the agents will reference Node and that's what we need to fix. In fact, most of them will pull Page, Document, Node for no good reason and that's what we need to fix upstream, in Blink.

You would also need to generate InspectorBackendDispatcher and InspectorFrontend from devtools/protocol.json. First one will parse incoming JSON messages and will dispatch them on controller's agents, second will generate typed API for agents to send messages back. Some tweaking would be necessary here as well since currently we generate a giant dispatch and giant front-end instead of per-domain files. Having them per-domain would let you only take what you really need.

[Wire]

Lets assume it all compiled. Now you need to instantiate this controller and call dispatchMessageFromFrontend on it with every message from the front-end. Front-end will issue them using web socket transport, so you need to have a server web socket as a part of your Node module that would accept connection and do the right thing to the messages (call dispatchMessageFromFrontend). And for the way back, your m_frontendChannel would need to send info back using that web socket.

[Debugging]

After compiling and wiring, console will start working. But you also need to make things work while on breakpoint. For that, you would need to implement runMessageLoopOnPause in your version of WorkerScriptDebugServer.

As I mentioned, it is quite some work both downstream (in the Node module) and upstream (to remove poor Blink dependencies from agents). But it might be worth it.

Regards

Pavel

// End Quote

Feel free to submit pull requests or post issue to https://github.com/crabdude/lookingglass

Cheers,

Adam Crabtree

[1] http://nodejs.org/api/addons.html

--
Better a little with righteousness
       than much gain with injustice.
Proverbs 16:8

[Jorge Chamorro]

On 10/07/2013, at 01:32, Adam Crabtree wrote:

> Howdy all,
>
> I have LinkedIn's blessing to work full-time for 2 weeks on building a node.js addon[1] for remote Chrome Devtools debugging.

Hi,

What is the idea, connecting from a node to the debugger of a remote chrome browser, or to debug a node remotely from the devtools of a chrome browser, or...

Sorry if I'm being dense :-)

Have you seen danny coates' node-inspector? Is it something like that?

Cheers,
--
( Jorge )();

[Andrey]

Same question here - what is the difference compared to node-webkit-agent? Would it require extra agent to run in addition to node.js process or it's going to use Chrome native networking and talk v8 debugger protocol directly?

[Floby]

I was gonna answer that https://github.com/dannycoates/node-inspector already existed, but it hasn't been updated in 2 years.

[Ben Noordhuis]

On Wed, Jul 10, 2013 at 10:01 AM, Floby <floren...@gmail.com> wrote:
> I was gonna answer that https://github.com/dannycoates/node-inspector
> already existed, but it hasn't been updated in 2 years.

See [1], we've taken over maintenance. We'll probably update Danny's repo soon.

[1] https://github.com/strongloop/node-inspector

[Adam Crabtree]

First, I'm definitely not interested in duplicating any work. As a software engineer, my goal has always been to write as little code as possible to achieve my goals.

Second, with regards to node-inspector...

It's my understanding that node-inspector, and it's step-brother node-webkit-agent[1] both consume V8's D8 debugging protocol, and that there are intrinsic limitations to using D8. I have a little experience with this while at Palm / HP webOS when I got node-inspector working against the Touchpad, and we suffered the same problems of D8's feature subset. Either way, should definitely talk to the guys at Strongloop (who are awesome for taking over node-inspector) and Camilo Aguilar (c4milo) who's been working with the Chrome Devtools guys and maintains the only active equivalent.

That being said, if I understand the scope of the problem correctly, my goal is fundamentally different (and from what I understand still possible). Over time, both Danny Coates and Camilo Aguilar have admitted to struggling to keep their projects up to date with the latest devtools code. Ideally, I won't even need to touch the devtools code.

Chrome already supports remote Chrome debugging[2], and my (hopefully achievable) goal is to implement this protocol in a node.js addon so that debugging a node process would be indistinguishable from debugging a remote Chrome process. Obviously there's no DOM, no Blink, no UI, which may cause complications, but I guess I'll find that out along the way.

Any additional insight or input into any of this is appreciated.

Cheers,

Adam Crabtree

[1] https://github.com/c4milo/node-webkit-agent

[2] https://developers.google.com/chrome-developer-tools/docs/debugger-protocol

--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Adam Crabtree]

Also, checkout https://github.com/CrabDude/lookingglass/blob/master/GettingStarted.md where I've copied Pavel's high-level overview of what's involved. Essentially, node will be a lot like debugging a remote webworker, so I guess that obviates the UI issues. 

[mgutz]

awesome, ty in advance

[Antonio D'Ettole]

Hi Adam,

you might want to take a look at bugger [1]. I've been using it for a few weeks with the inspector in the latest version of Chrome and it works like a charm. I believe it uses the Chrome debugging protocol which you mentioned.

I put Jan (its creator) in CC, perhaps you might have a chat.

Cheers!

Antonio

[1] https://github.com/jkrems/bugger

[Jan Krems]

Hey,

if you need a reference for implementing directly against the Chrome Devtools, I wrote a small tool inspired by node-inspector that does exactly that - you start your script with "bugger my_script.js", this opens up a websocket server that is compatible with the devtools protocol. Then you can just open chrome://devtools/devtools.html?ws=127.0.0.1:8058/websocket in Chrome. No patching of anything required. You can find the source here: https://github.com/jkrems/bugger. Especially the mapping of data types between v8 debugger and RemoteObjects is quite tricky since the documentation is barely existing, but maybe you find something useful in there. If you have any further question, feel free to reach out to me directly.

Cheers,

Jan

[phidelta]

Hi,

I am also constantly living in bugger (great work Jan!). I am currently looking into extending it to provide saving of local changes.

Also bugger does writes to disconnected web-sockets if you do a reload of the dev-tools which is sometimes necessary when more sourcefiles get add via require(). So I am also looking into finding a way to either have bugger catch those send errors if client connections go away or even better forcing dev-tools to update its source list when changes occurr. Maybe putting more effort into bugger, would be time better spent.

Regards, Philipp

On Wednesday, July 10, 2013 1:32:29 AM UTC+2, Adam Crabtree wrote:

[Jan Krems]

Hi,

there's a source compiled event in v8 which I'm currently not handling which when I had it working had some performance impacts (on startup it would trigger a ton of back and forth traffic between devtools, bugger and v8), but it's definitely on the bug/todo list of bugger, as is the annoying websocket disconnected error. I currently don't have nearly as much time to work on bugger as I'd want to, so I'd be more than happy to review & accept PRs if you have any improvements.

Thanks,

Jan

--

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---

You received this message because you are subscribed to a topic in the Google Groups "nodejs" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nodejs/KlriDbLz1ho/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nodejs+un...@googlegroups.com.

[David Beck]

Wow.. bugger looks awesome! I love how you can use Chrome's command-shift-O to get a list of the functions in the current file. Always missed that with node-inspector.

Does the support for coffeescript source maps mean that you can actually debug in coffeescript? Any tips on how to set this up? We have not been able to figure it out!

Thanks in advance, and for your work on this tool!

David Beck

[Jan Krems]

Generally: yes, you should be able to debug coffee-script directly with bugger. But... it's broken in the current beta. The biggest issue is Chrome not liking that the compiled file and the source file have the same filename. I tried different ways to work around it, but the current solution (patching the source filenames by appending ".src") breaks in some interesting ways. Will maybe have to revert to one of the previous solutions.

TL;DR: It's broken in the current beta, not your fault.

Cheers,

Jan Krems

P.S.: A colleague showed me the debugging support in WebStorm today, they support step-by-step coffee-script debugging by now. It's cumbersome because you have to start the app and the debugger separately (one of the things I always hated about node-inspector) and you are forced into big-IDE-land, but it seemed somewhat stable and working quite nicely. Just as a FYI.

P.P.S.: I hope I will find some time this weekend to get the next beta out. Fingers crossed that I also get a fix for coffee-script debugging in.

P.P.P.S.: Since apparently some people are reading these mails: My biggest wish in the whole wide world would be to have some proper integration tests for bugger but I can't get the devtools-url to open in chrome via automation (e.g. webdriver/wire). I suspect it to be some security-policy, though I feel like it shouldn't be a security issue. Does anyone have an idea how to trick chrome into opening these kind of urls? It's also blocking them when I try to open them via CLI params.

[Miroslav Bajtoš]

On Friday, July 12, 2013 7:00:33 PM UTC+2, David Beck wrote:

Wow.. bugger looks awesome! I love how you can use Chrome's command-shift-O to get a list of the functions in the current file. Always missed that with node-inspector.

Does the support for coffeescript source maps mean that you can actually debug in coffeescript? Any tips on how to set this up? We have not been able to figure it out!

The upcoming version of node-inspector [1] has all of this too, plus few features that are not yet implemented in bugger - e.g. live edit of javascript sources or modify variable values.

To enable source maps, you need to invoke the coffeescript compiler with -m option, i.e.

    coffee -c -m [files]

This way the compiler adds a magic string with a link to the original coffee-script source to the generated javasript. That's all you need to make it work.

[1] https://github.com/strongloop/node-inspector