ff-devtools-libs: making a module out of devtools.html

[James Long]

Jason and I are working on moving the debugger to be HTML/React/Redux. A
project with this heavy development really makes me want to solve our HTML
development workflow. And that means: develop it totally as a normal
webpage. It's just an HTML file that you can open and debug with Cmd+Alt+I.

I extracted out the cool parts of devtools.html and made them into a
module: https://github.com/jlongster/ff-devtools-libs. I published it to
npm so you can just do `npm install ff-devtools-libs`, and then some
webpack module aliasing and it works. Read the README for more info.

(This is just an experiment, but it solved our debugger workflow)

The core thing is a WebSocket transport. I only made sure the
DebuggerClient is working, but no reason we can't get other stuff to work
too. With a few lines of code I can connect to a remote debugging instance.
You need to run a proxy server to get this to work.

There are other dependencies as well. The idea is if you code against this
lib, you can "drop in" your code into the Firefox env and it will work the
same. I think it's a neat idea: even if we don't use webpack in production,
we might be able to support that environment for this workflow.

Lots to figure out (l10n/themes/etc) but I know many of you are working on
similar things.

- James

[Patrick Brosset]

Thanks James. This looks great.
For anyone interested in testing this out a little more, I suggest looking
at James' debugger-experiment repo (
https://github.com/jlongster/debugger-experiment) since that contains the
necessary package.json and webpack.config.js files necessary to put it all
together.

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

[James Long]

Thanks Patrick! I meant to say that. I forgot to commit the latest
package.json which includes ff-devtools-libs. Let me know if anyone has any
questions. Still very much an experiment.

On Tue, Apr 5, 2016 at 7:56 AM, Patrick Brosset <pbro...@mozilla.com>
wrote:

[Patrick Brosset]

So, I do have one comment. The lib doesn't include /devtools/server/actors
files, which makes it impossible to use protocol.js actors, since Front
classes are defined in the same modules as the corresponding Actor classes.
It's still possible to send requests to the actors with
client.request(packet), but that makes it a lot harder.
Not sure of the implications of just including the actor files in the lib
though...

[James Long]

Chatted some on IRC. I'm not sure what to do about those files. Anything
ported to ff-devtools-libs requires changes: no `loader.lazyGetter`, etc.
I'd like to minifiy the amount of files that we need to keep up-to-date.
But protocol.js requires including all of the actor files, which is a large
amount of code. It also seems more likely for protocol.js files to change
the protocol.js accidentally because it auto-generates the fronts.

Basically, any files loaded by the client needs to run in non-privileged
mode, and that seems like a much larger task for the actors. Maybe I'm
missing a simple way we could make this work.

I was hoping long-term we could make any code loaded by the client run
non-privileged *by default*, so ff-devtools-libs goes away.

I don't really like that the client loads the server files, but that's a
larger discussions. I'd prefer some way to describe the protocol in a
generic file that both the client and server include.

- James




On Tue, Apr 5, 2016 at 9:35 AM, Patrick Brosset <pbro...@mozilla.com>

[Nick Fitzgerald]

Maybe this is obvious and you've already discussed it, but...

Would splitting out the protocol "types" into their own interface
definition file that both the client and server would require work? The
protocol type defitinition file would be pretty much JSON with the request
and response metadata that is part of the protocol.js `method` function
right now, and would not need to have elevated privileges.

+------------------------+
+---------->
​
|Protocol Type Definition|
| +------------------------+
| ^
| |
| |
+----+-+ +--+---+
|Actors| |Fronts|
+------+ +------+

​(X --> Y is "X requires Y")​

[James Long]

On Tue, Apr 5, 2016 at 11:21 AM, Nick Fitzgerald <nfitz...@mozilla.com>
wrote:

> Maybe this is obvious and you've already discussed it, but...
>
> Would splitting out the protocol "types" into their own interface
> definition file that both the client and server would require work? The
> protocol type defitinition file would be pretty much JSON with the request
> and response metadata that is part of the protocol.js `method` function
> right now, and would not need to have elevated privileges.
>
> +------------------------+
> +---------->
> ​
> |Protocol Type Definition|
> | +------------------------+
> | ^
> | |
> | |
> +----+-+ +--+---+
> |Actors| |Fronts|
> +------+ +------+
>
> ​(X --> Y is "X requires Y")​
>
>

That sounds perfect, yep! Don't know how much work that is, but that seems
ideal and would be a good way to quickly see the protocol defs and maybe
even auto-generate docs.

[Jim Blandy]

On Tue, Apr 5, 2016 at 8:21 AM, Nick Fitzgerald <nfitz...@mozilla.com>
wrote:

> Would splitting out the protocol "types" into their own interface
> definition file that both the client and server would require work?
>

This seems like a great idea. I would use an "executable data" approach,
but whatever.

[Jason Laster]

Here's chrome DevTools protocol.json file, which might be a useful reference

https://src.chromium.org/viewvc/blink/trunk/Source/devtools/protocol.json

[Dave Camp]

fwiw, we implement a root-actor command that dumps json of all protocol.js
definitions (I forget the name), so that might be a good starting point.

-dave

On Tue, Apr 5, 2016 at 8:27 AM, James Long <jl...@mozilla.com> wrote:

> On Tue, Apr 5, 2016 at 11:21 AM, Nick Fitzgerald <nfitz...@mozilla.com>
> wrote:
>
> > Maybe this is obvious and you've already discussed it, but...
> >

> > Would splitting out the protocol "types" into their own interface

> > definition file that both the client and server would require work? The
> > protocol type defitinition file would be pretty much JSON with the
> request
> > and response metadata that is part of the protocol.js `method` function
> > right now, and would not need to have elevated privileges.
> >
> > +------------------------+
> > +---------->
> > ​
> > |Protocol Type Definition|
> > | +------------------------+
> > | ^
> > | |
> > | |
> > +----+-+ +--+---+
> > |Actors| |Fronts|
> > +------+ +------+
> >
> > ​(X --> Y is "X requires Y")​
> >
> >
>
> That sounds perfect, yep! Don't know how much work that is, but that seems
> ideal and would be a good way to quickly see the protocol defs and maybe
> even auto-generate docs.

[J. Ryan Stinnett]

On Wed, Apr 6, 2016 at 9:53 AM, Dave Camp <dc...@mozilla.com> wrote:
> fwiw, we implement a root-actor command that dumps json of all protocol.js
> definitions (I forget the name), so that might be a good starting point.

Yes, dumpProtocolSpec:
https://dxr.mozilla.org/mozilla-central/rev/a235bfcc8c411169b82420c503775c1a3e7edad5/devtools/server/protocol.js#1456

I mentioned this to :jlongster on IRC. There's also the volcan SDK
client that to interact with tools based on this spec:

https://dxr.mozilla.org/mozilla-central/source/addon-sdk/source/lib/dev/volcan.js

However, I don't believe volcan is actively used currently.

- Ryan

[James Long]

On Wed, Apr 6, 2016 at 10:58 AM, J. Ryan Stinnett <jry...@gmail.com> wrote:

> On Wed, Apr 6, 2016 at 9:53 AM, Dave Camp <dc...@mozilla.com> wrote:
> > fwiw, we implement a root-actor command that dumps json of all
> protocol.js
> > definitions (I forget the name), so that might be a good starting point.
>
> Yes, dumpProtocolSpec:
>
> https://dxr.mozilla.org/mozilla-central/rev/a235bfcc8c411169b82420c503775c1a3e7edad5/devtools/server/protocol.js#1456

I looked at that thoroughly and even tried to write my own (because that
seemed rough, there's even a console.log leftover in there). But due to my
inexperience with protocol.js I wasn't sure how much info I needed to
serialize. And the major missing point is the ability to read this spec and
rebuild all of the types. I tried to do that but it was too much work, but
maybe someone with better knowledge of protocol.js could do it.

Seems like the steps for each file that uses protocol.js is:

* Dump out a serialized protocol spec for all types from that file
* Pull out any fronts and move to a different file
* Change `FrontClass` to take a spec instead of an actorType