weinre extension system

[Haitao Feng]

Hi Patrick Mueller,

In your OSCON 2011 talk, you have mentioned "extension system that works"  as a current issue logged. Now the extension system works for the jetty version while not for the node.js version.
Are you re-designing the interfaces and going to give us a better extension system in node.js?

Currently I am working on a prototype to get Ripple-UI (https://github.com/blackberry/Ripple-UI) as an extension of weinre, Basically I want to get the control panel as a weinre
client, and pass the simulator events (such as accelerometer info) to the weinre target through the weinre server. In the weinre target, I will get the device APIs (phonegap, wac, webworks, tizen)
injected and listen to the simulator events from the message loop in target-script.js.

From my usage, I need the weinre infrastructure for client/channel/server/channel/target connection and message loop in client and target. I do not need the inspector UI part. I have gotten
accelerometer working (snapshot attached, I could drag the accelerometer panel to play the game in the right) by hacking a SimulatorEvent.idl, inject the SimulatorEvent into WebInspector
at client.coffee and register a handle in target.coffee. This is not perfect. So I'd like to know whether you could make "client/channel/server/channel/target connection and message loop in
client and target" as a skeleton and easily support third-party APIs, maybe even make the inspector panel as an extension of weinre.

Thanks
-Haitao

[Patrick Mueller]

On Wednesday, February 22, 2012 4:01:22 AM UTC-5, Haitao Feng wrote:

In your OSCON 2011 talk, you have mentioned "extension system that works"  as a current issue logged. Now the extension system works for the jetty version while not for the node.js version.
Are you re-designing the interfaces and going to give us a better extension system in node.js?

I wasn't very happy with the extension system that existed in the jetty version, hence the issue on extension system redesign [1].

 

Currently I am working on a prototype to get Ripple-UI (https://github.com/blackberry/Ripple-UI) as an extension of weinre, Basically I want to get the control panel as a weinre
client, and pass the simulator events (such as accelerometer info) to the weinre target through the weinre server. In the weinre target, I will get the device APIs (phonegap, wac, webworks, tizen)
injected and listen to the simulator events from the message loop in target-script.js.

Sounds interesting.  And hard.  To get one existing UI to run in another UI - for UI's at this level (very high) - seems difficult to me.  I suspect the radical differences in the UIs, presented in the same browser window, may also be a bit jarring.

But it's a good use case :-)

From my usage, I need the weinre infrastructure for client/channel/server/channel/target connection and message loop in client and target. I do not need the inspector UI part. I have gotten
accelerometer working (snapshot attached, I could drag the accelerometer panel to play the game in the right) by hacking a SimulatorEvent.idl, inject the SimulatorEvent into WebInspector
at client.coffee and register a handle in target.coffee. This is not perfect. So I'd like to know whether you could make "client/channel/server/channel/target connection and message loop in
client and target" as a skeleton and easily support third-party APIs, maybe even make the inspector panel as an extension of weinre.

Sounds even more interesting.

The IDL story is probably not one that we will continue to go down; it's been phased out of Web Inspector code for quite a while, actually.  They have some kind of a JSON-based service interface definition story now. 

I think the way to do this is to define the client-target IPC system first, for extensions, then we need to figure out how to get code loaded in the client and the target.  Lastly, how do you "install" one of these extensions on either the server, so it's available to all clients, and/or just the clients, and/or (I guess) the targets.

I think we'll likely need to add some new IDL interfaces to make this work, but I don't want folks doing the kind of work you're doing to be having to define interfaces in IDL - instead, you should be programming at the level of the IPC system.

Would you like to take a stab at a design? I've created a wiki page [2] to capture thoughts.  

[1] https://issues.apache.org/jira/browse/CB-86 

[2] http://wiki.apache.org/cordova/WeinreExtensions

[Haitao Feng]

On Wednesday, February 22, 2012 10:36:53 PM UTC+8, Patrick Mueller wrote:

On Wednesday, February 22, 2012 4:01:22 AM UTC-5, Haitao Feng wrote:

In your OSCON 2011 talk, you have mentioned "extension system that works"  as a current issue logged. Now the extension system works for the jetty version while not for the node.js version.
Are you re-designing the interfaces and going to give us a better extension system in node.js?

I wasn't very happy with the extension system that existed in the jetty version, hence the issue on extension system redesign [1].

Thanks for this information. The requirements are more comprehensive than my specific simulator implementation.
 

 

Currently I am working on a prototype to get Ripple-UI (https://github.com/blackberry/Ripple-UI) as an extension of weinre, Basically I want to get the control panel as a weinre
client, and pass the simulator events (such as accelerometer info) to the weinre target through the weinre server. In the weinre target, I will get the device APIs (phonegap, wac, webworks, tizen)
injected and listen to the simulator events from the message loop in target-script.js.

Sounds interesting.  And hard.  To get one existing UI to run in another UI - for UI's at this level (very high) - seems difficult to me.  I suspect the radical differences in the UIs, presented in the same browser window, may also be a bit jarring.

But it's a good use case :-)

Yes, the UI part is hard. If the web inspector could be an extension and optional, it might not be a big issue to show the remote panel (for management) and simulator panel only.
 

From my usage, I need the weinre infrastructure for client/channel/server/channel/target connection and message loop in client and target. I do not need the inspector UI part. I have gotten
accelerometer working (snapshot attached, I could drag the accelerometer panel to play the game in the right) by hacking a SimulatorEvent.idl, inject the SimulatorEvent into WebInspector
at client.coffee and register a handle in target.coffee. This is not perfect. So I'd like to know whether you could make "client/channel/server/channel/target connection and message loop in
client and target" as a skeleton and easily support third-party APIs, maybe even make the inspector panel as an extension of weinre.

Sounds even more interesting.

The IDL story is probably not one that we will continue to go down; it's been phased out of Web Inspector code for quite a while, actually.  They have some kind of a JSON-based service interface definition story now. 

You have a python script to generate JSON file from IDL and have IDLTools.coffee, they are very convenient for me to do the above prototype. Is the generated JSON has the same format with Web Inspector JSON-based service interface? Actually I have found limited information on web inspector extension system by Googling, could you give me more information on how your extension system works in your jetty version? :)
 

I think the way to do this is to define the client-target IPC system first, for extensions, then we need to figure out how to get code loaded in the client and the target.  Lastly, how do you "install" one of these extensions on either the server, so it's available to all clients, and/or just the clients, and/or (I guess) the targets.

I think we'll likely need to add some new IDL interfaces to make this work, but I don't want folks doing the kind of work you're doing to be having to define interfaces in IDL - instead, you should be programming at the level of the IPC system.

Would you like to take a stab at a design? I've created a wiki page [2] to capture thoughts.  

Thanks for the invitation. I will learn how your existing system works, how web inspector extension works, how Google dev tools and dev tools extension (chrome.experimental.devtools) before writing something valuable on this area.

[Patrick Mueller]

On Thursday, February 23, 2012 4:46:41 AM UTC-5, Haitao Feng wrote:

Yes, the UI part is hard. If the web inspector could be an extension and optional, it might not be a big issue to show the remote panel (for management) and simulator panel only.

But it actually works the other way.  The remote panel is just a plain old Web Inspector panel.  Switching the "top-most" framework from Web Inspector to something else seems like a lot of work.  Interesting work, but not worth the effort.

You have a python script to generate JSON file from IDL and have IDLTools.coffee, they are very convenient for me to do the above prototype. Is the generated JSON has the same format with Web Inspector JSON-based service interface? Actually I have found limited information on web inspector extension system by Googling, could you give me more information on how your extension system works in your jetty version? :)

The IDL tooling and runtime was only designed to be used statically. I never really had a thought that you might want to use IDL to define your interface - seems like too much work.  I think for extensions, the idea would be that there's an IPC  mechanism between client-server-targer, and you send messages between those agents.  Probably you would identify messages with an id of some kind - "org.muellerware.classBrowser" - and then register a callback to handle messages when they arrive from another agents.

In terms of weinre extension API == Web Inspector API, no.  I think the only real API for the extensions was "synchronous eval()".

 

 Thanks for the invitation. I will learn how your existing system works, how web inspector extension works, how Google dev tools and dev tools extension (chrome.experimental.devtools) before writing something valuable on this area.

The weinre extension stuff should be easy to figure out, there's not much there - if you've gotten anything to run with it, like the DOM Monster extension I saw in your screen shot :-) then you've touched all the APIs.

It would certainly be nice to be able to match the Web Inspector extension story, not sure how close you can actually get.  

Another option for "weinre extensions" you might think about is making them available as Chrome extensions.  That is, your weinre extension is just a Chrome extension that comes to life when it realizes weinre is running.  This kind of maps into the direction I was thinking with "where is the extension installed".  In the Jetty version, the extension is "installed" on the server, which means EVERY client has the extension installed, and NO client can ever install it's own extensions.  That doesn't scale for something like debug.phonegap.com.  The story should be that you install an extension in the client.  And an extension should never inject code into the server.

[HaiTao Feng]

On Thu, Feb 23, 2012 at 10:00 PM, Patrick Mueller <pmu...@gmail.com> wrote:

On Thursday, February 23, 2012 4:46:41 AM UTC-5, Haitao Feng wrote:

Yes, the UI part is hard. If the web inspector could be an extension and optional, it might not be a big issue to show the remote panel (for management) and simulator panel only.

But it actually works the other way.  The remote panel is just a plain old Web Inspector panel.  Switching the "top-most" framework from Web Inspector to something else seems like a lot of work.  Interesting work, but not worth the effort.

I agree with you after diving into your weinre source codes. The codes are clean and clear and not that hard to read as I have imaged :)

 

You have a python script to generate JSON file from IDL and have IDLTools.coffee, they are very convenient for me to do the above prototype. Is the generated JSON has the same format with Web Inspector JSON-based service interface? Actually I have found limited information on web inspector extension system by Googling, could you give me more information on how your extension system works in your jetty version? :)

The IDL tooling and runtime was only designed to be used statically. I never really had a thought that you might want to use IDL to define your interface - seems like too much work.  I think for extensions, the idea would be that there's an IPC  mechanism between client-server-targer, and you send messages between those agents.  Probably you would identify messages with an id of some kind - "org.muellerware.classBrowser" - and then register a callback to handle messages when they arrive from another agents.

Here is the current idl for SimulatorEvent, 

module sim {

    interface SimulatorEvents {

        void setAccelerometerInfo(double x, double y, double z, string date);

    };

}

From the Ripple-UI implementation, it is possible to define all the simulator event through this idl. I am working on this part to get more events into SimulatorEvents. 

In terms of weinre extension API == Web Inspector API, no.  I think the only real API for the extensions was "synchronous eval()".

 

 Thanks for the invitation. I will learn how your existing system works, how web inspector extension works, how Google dev tools and dev tools extension (chrome.experimental.devtools) before writing something valuable on this area.

The weinre extension stuff should be easy to figure out, there's not much there - if you've gotten anything to run with it, like the DOM Monster extension I saw in your screen shot :-) then you've touched all the APIs.

It would certainly be nice to be able to match the Web Inspector extension story, not sure how close you can actually get.  

Another option for "weinre extensions" you might think about is making them available as Chrome extensions.  That is, your weinre extension is just a Chrome extension that comes to life when it realizes weinre is running.  This kind of maps into the direction I was thinking with "where is the extension installed".  In the Jetty version, the extension is "installed" on the server, which means EVERY client has the extension installed, and NO client can ever install it's own extensions.  That doesn't scale for something like debug.phonegap.com.  The story should be that you install an extension in the client.  And an extension should never inject code into the server.

Thanks for pointing out this possibility. I spent some time reading Chrome extensions. What kind of Chrome extension do you mean? Is it a devtools extension, like https://github.com/NV/chrome-devtools-autosave? Or a normal extension which injects the extension UI and JS into  a weinre client? 

BTW: It seems that my current work could be implemented on top of devtools extension as client, server and target (inspected window) are running in the same desktop machine.

--
From the weinre Google Group -- http://groups.google.com/group/weinre

[Patrick Mueller]

On Fri, Feb 24, 2012 at 02:18, HaiTao Feng <haita...@gmail.com> wrote:

On Thu, Feb 23, 2012 at 10:00 PM, Patrick Mueller <pmu...@gmail.com> wrote:

The IDL tooling and runtime was only designed to be used statically. I never really had a thought that you might want to use IDL to define your interface - seems like too much work.  I think for extensions, the idea would be that there's an IPC  mechanism between client-server-targer, and you send messages between those agents.  Probably you would identify messages with an id of some kind - "org.muellerware.classBrowser" - and then register a callback to handle messages when they arrive from another agents.

Here is the current idl for SimulatorEvent, 

module sim {

    interface SimulatorEvents {

        void setAccelerometerInfo(double x, double y, double z, string date);

    };

}

From the Ripple-UI implementation, it is possible to define all the simulator event through this idl. I am working on this part to get more events into SimulatorEvents. 

Again, not sure that defining IDL is the way for you to go; are you rebuilding weinre after you make changes to the IDL?  You don't really want to have to do that.  Instead, we want to define some new IDL for a new interface - the "Extension" interface.  That includes some kind of IPC mechanism to send messages between clients and targets.

 

Another option for "weinre extensions" you might think about is making them available as Chrome extensions.  That is, your weinre extension is just a Chrome extension that comes to life when it realizes weinre is running.  This kind of maps into the direction I was thinking with "where is the extension installed".  In the Jetty version, the extension is "installed" on the server, which means EVERY client has the extension installed, and NO client can ever install it's own extensions.  That doesn't scale for something like debug.phonegap.com.  The story should be that you install an extension in the client.  And an extension should never inject code into the server.

Thanks for pointing out this possibility. I spent some time reading Chrome extensions. What kind of Chrome extension do you mean? Is it a devtools extension, like https://github.com/NV/chrome-devtools-autosave? Or a normal extension which injects the extension UI and JS into  a weinre client? 

I was thinking a normal extension, not a devtools extension.

 

BTW: It seems that my current work could be implemented on top of devtools extension as client, server and target (inspected window) are running in the same desktop machine.

Right.  Interesting.  So the perfect story for you, would be if weinre's extension system worked EXACTLY the same as the devtools extension.  I've not looked at this enough to know if that's possible, but would be VERY curious if we could make this work.  Maybe that's where you should start - try to build it as a devtools extension for Chrome, then we'll see if we can make it so the same code (probably packaged differently) would work in weinre.

--
Patrick Mueller
http://muellerware.org