[nodejs] [ANN] node-webkit-agent

[Camilo Aguilar]

Hey guys, 

I've been working in https://github.com/c4milo/node-webkit-agent in my spare time. The idea is to develop an agent for nodejs than can communicate with webkit inspector front-end. As many of you already know, Webkit folks announced recently the remote debugging protocol and node-webkit-agent is an attempt to implement

every domain of the protocol so that we can use webkit inspector as a development tool in nodejs as well. 

How is it different from node-inspector?

Well, I basically took a different approach. Danny Coates' approach was to have a patched version of webkit inspector front-end and it worked fine at the beginning but then, it just was really hard for him to keep it up to date due to the insanely fast v8 and webkit development process. So, in node-webkit-agent there is no front-end, at least for now. it just enables a websocket service for the webkit built-in frontend to connect with it. It means that basically you can connect to the agent with an URL like this: 

http://trac.webkit.org/export/head/trunk/Source/WebCore/inspector/front-end/inspector.html?ws=localhost:1337

You can start also a Chrome browser using the --remote-debugging-port that will serve the front-end for you and just change the parameter **ws** in the URL to the host and port where your node-webkit-agent is listening.

So far CPU and heap profiling are working fine, node-webkit-agent is using right now a patched version of Danny's v8-profiler. I was trying to communicate with him but it never came back so I haven't send him pull requests yet.

Anyways, this is pretty much a work in progress, pull requests and feedback are pretty much appreciated.

Best, 

-- 

Camilo Aguilar

[pfeldman]

On Wednesday, May 2, 2012 10:22:08 PM UTC+4, Camilo Aguilar wrote:

Hey guys, 

I've been working in https://github.com/c4milo/node-webkit-agent in my spare time. The idea is to develop an agent for nodejs than can communicate with webkit inspector front-end. As many of you already know, Webkit folks announced recently the remote debugging protocol and node-webkit-agent is an attempt to implement

every domain of the protocol so that we can use webkit inspector as a development tool in nodejs as well. 

Heh. We are the WebKit folks you are referring to.

 

How is it different from node-inspector?

Well, I basically took a different approach. Danny Coates' approach was to have a patched version of webkit inspector front-end and it worked fine at the beginning but then, it just was really hard for him to keep it up to date due to the insanely fast v8 and webkit development process. So, in node-webkit-agent there is no front-end, at least for now. it just enables a websocket service for the webkit built-in frontend to connect with it. It means that basically you can connect to the agent with an URL like this: 

http://trac.webkit.org/export/head/trunk/Source/WebCore/inspector/front-end/inspector.html?ws=localhost:1337

You can start also a Chrome browser using the --remote-debugging-port that will serve the front-end for you and just change the parameter **ws** in the URL to the host and port where your node-webkit-agent is listening.

I like your approach, but it has one problem associated with it. When we commit to the protocol backwards compatibility, we only commit to backwards compatible backend and only on the set of the public (non-hidden) methods. So "newer" front-end is not guaranteed to work with the old backend. [While adding new features to the WebInspector front-end we often use "hidden" protocol methods and rely upon their existence on the ToT backend]. Our long term goal though is to make all the methods public and allow newer versions of the front-end to be able to attach to the older backends. It is just that it is not the priority for us right now.

So far CPU and heap profiling are working fine, node-webkit-agent is using right now a patched version of Danny's v8-profiler. I was trying to communicate with him but it never came back so I haven't send him pull requests yet.

I think Debugger domain is the one that helps the most, so am looking forward to its implementation.

 

Anyways, this is pretty much a work in progress, pull requests and feedback are pretty much appreciated.

 

An alternative path for achieving the same goal would be to extract a part of the Web Inspector backend (native code) from WebKit and compile it against node.js and its v8 natively. Inspector backend (at least its debugger and profiler domains) do not depend on WebKit much, they depend on the layer that abstracts us from V8 / JSC. You could bundle the part that is based on V8 with a handful of inspector classes, hack them a bit and make a part of node. You would then be able to take the front-end from the same WebKit revision and hence guarantee the compatibility. That would require a number of small refactorings in these native classes, but we'll accept them back in WebKit, so you won't need to fork much. As a free bonus, you'll be getting the backend support for the new debugging features automatically. But that is clearly more work with less tangible result at first.

Regards

Pavel

Best, 

-- 

Camilo Aguilar

[cam...@cloudescape.com]

On Friday, May 4, 2012 1:11:59 AM UTC-4, pfeldman wrote:

On Wednesday, May 2, 2012 10:22:08 PM UTC+4, Camilo Aguilar wrote:

Hey guys, 

I've been working in https://github.com/c4milo/node-webkit-agent in my spare time. The idea is to develop an agent for nodejs than can communicate with webkit inspector front-end. As many of you already know, Webkit folks announced recently the remote debugging protocol and node-webkit-agent is an attempt to implement

every domain of the protocol so that we can use webkit inspector as a development tool in nodejs as well. 

Heh. We are the WebKit folks you are referring to.

Yeah, sorry about that. The main audience of that email was the nodejs community. I wrote it in a hurry, talked with Paul Irish in the last minute and he suggested me to let this group know about it as well.

 

 

How is it different from node-inspector?

Well, I basically took a different approach. Danny Coates' approach was to have a patched version of webkit inspector front-end and it worked fine at the beginning but then, it just was really hard for him to keep it up to date due to the insanely fast v8 and webkit development process. So, in node-webkit-agent there is no front-end, at least for now. it just enables a websocket service for the webkit built-in frontend to connect with it. It means that basically you can connect to the agent with an URL like this: 

http://trac.webkit.org/export/head/trunk/Source/WebCore/inspector/front-end/inspector.html?ws=localhost:1337

You can start also a Chrome browser using the --remote-debugging-port that will serve the front-end for you and just change the parameter **ws** in the URL to the host and port where your node-webkit-agent is listening.

I like your approach, but it has one problem associated with it. When we commit to the protocol backwards compatibility, we only commit to backwards compatible backend and only on the set of the public (non-hidden) methods. So "newer" front-end is not guaranteed to work with the old backend. [While adding new features to the WebInspector front-end we often use "hidden" protocol methods and rely upon their existence on the ToT backend]. Our long term goal though is to make all the methods public and allow newer versions of the front-end to be able to attach to the older backends. It is just that it is not the priority for us right now.

Understood. There have been a couple of users having problems with node-webkit-agent workflow too. So, I'll add a working front-end to the project then. Thanks for the heads up regarding the backwards compatibility.

 

So far CPU and heap profiling are working fine, node-webkit-agent is using right now a patched version of Danny's v8-profiler. I was trying to communicate with him but it never came back so I haven't send him pull requests yet.

I think Debugger domain is the one that helps the most, so am looking forward to its implementation.

I'm close with the Debugger but I got stuck implementing "pause" and "resume". I thought initially that v8 would stop the main thread when calling Debug.breakExecution but it doesn't. I'm using the v8 --expose_debug_as flag to expose the javascript debug object by the way. I also saw the mechanism you guys are using for pause and resume in the v8 bindings and I was wondering if I should do something similar. Although, I'm working only in JS so I'm not sure yet how that should work. Any direction here is appreciated :).  

 

Anyways, this is pretty much a work in progress, pull requests and feedback are pretty much appreciated.

 

An alternative path for achieving the same goal would be to extract a part of the Web Inspector backend (native code) from WebKit and compile it against node.js and its v8 natively. Inspector backend (at least its debugger and profiler domains) do not depend on WebKit much, they depend on the layer that abstracts us from V8 / JSC. You could bundle the part that is based on V8 with a handful of inspector classes, hack them a bit and make a part of node. You would then be able to take the front-end from the same WebKit revision and hence guarantee the compatibility. That would require a number of small refactorings in these native classes, but we'll accept them back in WebKit, so you won't need to fork much. As a free bonus, you'll be getting the backend support for the new debugging features automatically. But that is clearly more work with less tangible result at first.

I think I looked to that possibility before starting node-webkit-agent and my main fear was to keep it up to date without too much effort. I also thought it was a process difficult to automate. Anyways, I'm going to give it another try in a separated project.