Implementing a Chrome DevTools Protocol server in Firefox

[Michael Smith]

Hi everyone,

Mozilla DevTools is exploring implementing parts of the Chrome DevTools
Protocol ("CDP") [0] in Firefox. This is an HTTP, WebSockets, and JSON
based protocol for automating and inspecting running browser pages.

Originally built for the Chrome DevTools, it has seen wider adoption
with outside developers. In addition to Chrome/Chromium, the CDP is
supported by WebKit, Safari, Node.js, and soon Edge, and an ecosystem of
libraries and tools already exists which plug into it, for debugging,
extracting performance data, providing live-preview functionality like
the Brackets editor, and so on. We believe it would be beneficial if
these could be leveraged with Firefox as well.

The initial implementation we have in mind is an alternate target for
third-party integrations to connect to, in addition to the existing
Firefox DevTools Server. The Servo project has also expressed interest
in adding CDP support to improve its own devtools story, and a PR is in
flight to land a CDP server implementation there [1].

I've been working on this project with guidance from Jim Blandy. We've
come up with the following approach:

- A complete, typed Rust implementation of the CDP protocol messages and
(de)serialization lives in the "cdp" crate [2], automatically generated
from the protocol's JSON specification [3] using a build script (this
happens transparently as part of the normal Cargo compilation process).
This comes with Rustdoc API documentation of all messages/types in the
protocol [4] including textual descriptions bundled with the
specification JSON. The cdp crate will likely track the Chrome stable
release for which version of the protocol is supported. A maintainers'
script exists which can find and fetch the appropriate JSON [5].

- The "tokio-cdp" crate [6] builds on the types and (de)serialization
implementation in the cdp crate to provide a server implementation built
on the Tokio asynchronous I/O system. The server side provides traits
for consuming incoming CDP RPC commands, executing them concurrently and
sending back responses, and simultaneously pushing events to the client.
They are generic over the underlying transport, so the same backend
implementation could provide support for "remote" clients plugging in
over HTTP/WebSockets/JSON or, for example, a browser-local client
communicating over IPDL.

- In Servo, a new component plugs into the cdp and tokio-cdp crates and
acts on behalf of connected CDP clients in response to their commands,
communicating with the rest of the Servo constellation. This server is
disabled by default and can be started by passing a "--cdp" flag to the
Servo binary, binding a TCP listener to the loopback interface at the
standard CDP port 9222 (a different port can be specified as an option
to the flag).

- The implementation we envision in Firefox/Gecko would act similarly: a
new Rust component, disabled by default and switched on via a command
line flag, which binds to a local port and mediates between Gecko
internals and clients connected via tokio-cdp.

We chose to build this on Rust and the Tokio event loop, along with the
hyper HTTP library and rust-websocket which plug into Tokio.

Rust and Cargo provide excellent facilities for compile-time code
generation which integrate transparently into the normal build process,
avoiding the need to invoke scripts by hand to keep generated artifacts
in sync. The Rust ecosystem provides libraries such as quote [7] and
serde [8] which allow us to auto-generate an efficient, typed, and
self-contained interface for the entire protocol. This moves the
complexity of ingesting, validating, and extracting information from
client messages out of the Servo- and Gecko-specific backend
implementations, helps to ensure they conform correctly to the protocol
specification, and provides a structured way of upgrading to new
protocol versions.

As for Tokio, the event loop and Futures-based model of concurrency it
offers maps well to the Chrome DevTools Protocol. RPC commands typically
execute simultaneously, returning responses in order of completion,
while the server continuously generates events to which the client has
subscribed. Under Tokio we can spawn multiple lightweight Tasks,
dispatch messages to them, and multiplex their responses back over the
single client connection. The Tokio event loop is nicely self-contained
to the one or, optionally, more threads it is allocated, so the rest of
the application doesn't need to be aware of it.

Use of Tokio is becoming a standard in the Rust ecosystem---it's worth
mentioning that Mozilla funds Tokio development [9] and employs some of
its primary developers. Servo currently depends on an older version of
the hyper HTTP client/server library, and consequently this is already
present in the Firefox tree. The current release of hyper is built on
top of Tokio, so upgrading hyper, either as maintenance or to take
advantage of the forthcoming HTTP/2 support, would require pulling in
Tokio anyway. The current release of rust-websocket, from which Servo
derives its WebSockets implementation, also supports Tokio.

The alternative to Tokio for the networking layer would likely be to
build on top of Necko. This presents its own share of problems.

No Rust bindings to Necko currently exist, so writing those in the first
place would become a prerequisite for the rest of the work, and may
require integration with technologies like XPCOM which also lacks Rust
support. Then rewriting the CDP networking layer on top of Necko would
duplicate effort between the CDP implementations for Gecko and Servo,
the latter of which does not presently use Necko and may not be able to
take it on as a dependency.

Furthermore, it is my understanding through conversations with others
that while Necko's HTTP client implementation is mature, what HTTP
*server* implementation exists is bare-bones, and that at present there
is no support for upgrading HTTP connections to WebSockets in order to
implement a WebSocket server. Then building on top of Necko would entail
effectively developing a new library within Necko, when we could instead
adopt existing technologies (Tokio, hyper, rust-websocket) which are
already in use by other Mozilla projects and the surrounding Rust ecosystem.

Feedback, suggestions, and guidance are very much welcome!
https://bugzil.la/1391465 is the bug for the CDP server implementation
in Firefox.

-Michael Smith [:mismith]

[0] https://chromedevtools.github.io/devtools-protocol/
[1] https://github.com/servo/servo/pull/18133
[2] https://github.com/devtools-html/cdp
[3]
https://github.com/devtools-html/cdp/blob/master/json/browser_protocol.json
[4] https://www.spinda.net/files/mozilla/cdp/doc/cdp/tools/index.html
[5] https://github.com/devtools-html/cdp/blob/master/update_json.sh
[6] https://github.com/devtools-html/tokio-cdp
[7] https://github.com/dtolnay/quote
[8] https://github.com/serde-rs/serde
[9]
https://blog.mozilla.org/blog/2017/04/10/mozilla-awards-365000-to-open-source-projects-as-part-of-moss/

[Jet Villegas]

Can you summarize the desired outcomes?
e.g.,
1. people using devtools in Chrome can also debug Firefox
2. devtools for Chrome can be ported to Firefox
3. devtools for Firefox can be used with Chrome
4. ...

This is potentially a very large API surface to support, and I'm skeptical
about our ability to emulate Chromium's behavior when attached to devtools.
For example, we've been exposing new API's for Layout geometry to the
front-end and devtools, and would like to do more of that sort of tight
integration with our Layout engine. I'm not sure we could guarantee bug
compatibility when a Chrome devtool requires similar-but-not-quite
functionality. In other words, it sounds like your proposed protocol
emulation is feasible, but I'm less certain we can live up to the expected
semantics of things going over that wire.

--Jet

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[David Burns]

Do we know if the other vendors would see value in having this spec'ed
properly so that we have true interop here? Reverse engineering seems like
a "fun" project but what stops people from breaking stuff without realising?

David

[Eric Rescorla]

I assume this is going to involve TLS (generally this is a requirement for
H2). In Firefox, this is done with NSS. Does Tokio/Hyper cleanly separate
out the TLS stack so that you can do that?

-Ekr

[Karl Dubost]

Michael,

Le 31 août 2017 à 07:56, David Burns <dbu...@mozilla.com> a écrit :
> Do we know if the other vendors would see value in having this spec'ed
> properly so that we have true interop here?

Yeah I had the same train of thoughts than David when I read the initial message.

There is currently Browser Testing and Tools Working Group Charter which does only Web Driver API. David is a co-chair there.
https://www.w3.org/2016/05/browser-testing-tools-charter.html

The 2011 charter was open on more APIs
https://www.w3.org/2011/08/browser-testing-charter.html


But I had a slight memory of a long time ago attempt, which indeed didn't take off.
There was this talk at XTech 2008 by Mike Smith (W3C)
https://www.w3.org/2008/Talks/05-07-smith-xtech/slides.pdf

Also this talk at the 2008 W3C Technical Plenary
by Chaals (Opera then)
on standardizing Scope (Opera Dragonfly Inspection API)
"Standards API and Debuggers"
https://www.w3.org/2008/10/1022-dragonfly-chaals/talk.html
https://www.w3.org/2008/10/22-tp-minutes.html
http://operasoftware.github.io/scope-interface/


But I don't remember if there was a workshop or anything and it just didn't build up steam.


--
Karl Dubost, mozilla 💡 Webcompat
http://www.la-grange.net/karl/moz

[Michael Smith]

On 8/30/2017 15:25, Jet Villegas wrote:
> Can you summarize the desired outcomes? [...] This is potentially a very large API surface to support, and I'm skeptical about our ability to emulate Chromium's behavior when attached to devtools.

The Chrome DevTools Protocol is split up into separate commands
(client-to-server RPC) and events (server-initiated push). These can be
implemented individually, so we can pick and choose which parts of the
protocol to tackle and in what order.

Supporting an entire devtools suite as a client (i.e. "use Chrome
devtools with Firefox") would *not* be an initial target. Instead I've
been looking toward adding enough support so that some of the
third-party testing and automation tooling which exists for the CDP can
be used with Firefox.

Sonar [0], for example, is a web linting tool which uses the CDP. The
surface area it touches is not all that large [1], consisting of
straightforward operations like opening and closing tabs, navigating to
URLs, clearing the cache, and observing page load and network events.
We've also looked at the WebPageTest agent implementation for Chrome
[2]: its core functionality also requires navigating between URLs and
observing page load and network events.

I'm aware of other places where the CDP is used to collect performance
and profiling data as well as console output, as part of internal
automated testing. This is testing that isn't being done with Firefox,
but could be if we supported a limited subset of the protocol. There's a
common set of functionality here which covers much of the use cases
developers have in the wild, without having to touch potentially-hairier
parts such as layout inspection.

*** So, desired outcome #1: be compatible with enough of the Chrome
DevTools Protocol that we're a better target for testing and automation.
This complements the recent work in implementing a headless mode.

There are also editors and editor plugins which use the CDP to provide
HTML/CSS live preview and JS debugging support. I mentioned Brackets in
my original post, and similar plugins exist for Atom, VS Code, Sublime
(see [3]). The more developers that adopt these, the more that are
building web pages with Chrome as the primary target (or Edge, which is
also adding support). But if Firefox had a CDP implementation which
supported the subset these tools need, we would suddenly become
compatible with them. The subset that VS Code's plugin touches [4] looks
very approachable.

*** Desired outcome #2: be compatible with enough of the Chrome DevTools
Protocol that we're a better target for web development. This
complements recent additions to the Firefox DevTools (eg. the CSS Grid
Inspector) which can be used as points of differentiation once we've
arrived on the playing field.

On 8/30/2017 15:25, Jet Villegas wrote:
> For example, we've been exposing new API's for Layout geometry to the
> front-end and devtools, and would like to do more of that sort of tight
> integration with our Layout engine.

Implementing parts of the Chrome DevTools Protocol wouldn't mean ripping
out the Firefox one: we can keep doing tight, Firefox-specific
integrations with our own devtools.

Where it makes sense, we might replace parts used by the Firefox
devtools with CDP equivalents---perhaps in conjunction with the
RemoteDebug.org initiative [5], which is trying to standardize a common
remote debugging protocol from parts of the CDP. The Console and JS
Debugger seem like good targets for this, and Debugger.html can already
communicate with CDP-based servers. But this would, I think, be further
down the line from the goal of integrating with clients outside the browser.

*** Desired outcome #3: Firefox's CDP server implementation supports
external (remote) clients, but is designed in such a way that it can be
applied efficiently to internal use-cases where appropriate. This has
informed the development direction of the cdp/tokio-cdp Rust libraries.

-Michael

[0] https://sonarwhal.com/
[1] https://github.com/sonarwhal/sonar/issues/377#issuecomment-314929118
[2]
https://github.com/WPO-Foundation/wptagent/blob/master/internal/devtools.py
[3] https://remotedebug.org/integrations/
[4]
https://github.com/Microsoft/vscode-chrome-debug/blob/c77829dad40c8f2f783a03333ae23d61ef79cbbc/test/debugProtocolMocks.ts#L24-L77
[5] https://remotedebug.org/

On 8/30/2017 15:25, Jet Villegas wrote:
> Can you summarize the desired outcomes?
> e.g.,
> 1. people using devtools in Chrome can also debug Firefox
> 2. devtools for Chrome can be ported to Firefox
> 3. devtools for Firefox can be used with Chrome
> 4. ...
>
> This is potentially a very large API surface to support, and I'm skeptical
> about our ability to emulate Chromium's behavior when attached to devtools.
> For example, we've been exposing new API's for Layout geometry to the
> front-end and devtools, and would like to do more of that sort of tight
> integration with our Layout engine. I'm not sure we could guarantee bug
> compatibility when a Chrome devtool requires similar-but-not-quite
> functionality. In other words, it sounds like your proposed protocol
> emulation is feasible, but I'm less certain we can live up to the expected
> semantics of things going over that wire.
>
> --Jet
>
>
>
>
>

[Josh Matthews]

On 8/30/17 2:55 PM, Michael Smith wrote:
> Use of Tokio is becoming a standard in the Rust ecosystem---it's worth
> mentioning that Mozilla funds Tokio development [9] and employs some of
> its primary developers. Servo currently depends on an older version of
> the hyper HTTP client/server library, and consequently this is already
> present in the Firefox tree. The current release of hyper is built on
> top of Tokio, so upgrading hyper, either as maintenance or to take
> advantage of the forthcoming HTTP/2 support, would require pulling in
> Tokio anyway. The current release of rust-websocket, from which Servo
> derives its WebSockets implementation, also supports Tokio.

One clarification - while all of the Servo codebase exists in
mozilla-central/servo/, only the dependencies used by Stylo are vendored
in third-party/rust. That means that neither hyper nor tokio are
vendored in mozilla-central at the moment, or part of any
mozilla-central build.

Cheers,
Josh

[Michael Smith]

On 8/30/2017 15:56, David Burns wrote:
> Do we know if the other vendors would see value in having this

spec'ed properly so that we have true interop here? Reverse engineering 
seems like a "fun" project but what stops people from breaking stuff
without realising?

Fortunately we're not reverse engineering here (for the most part), all
protocol messages are specified in a machine-readable JSON format which
includes inline documentation [0] --- this is what the cdp Rust library
consumes. The spec is versioned and the authors do seem to follow a
proper process of introducing new features as "experimental",
stabilizing mature ones, and deprecating things before they're removed.

There's the Chrome DevTools Protocol viewer which provides a
human-readable view [1], or if you prefer, the Rustdoc generated from
the cdp crate [2] which carries over all the same information from the
specification JSON.

Exposing a typed Rust interface is intended to help us avoid "breaking
stuff without realizing". Other projects like Node.js have also adopted
the CDP and an ecosystem of applications have grown around it which
provide a counterweight against random breakages on Chrome's end.

As far as standardization goes, I'm aware of the RemoteDebug.org [3]
initiative which has been trying to standardize parts of the CDP as a
cross-browser remote debugging protocol. From what I recall from
conversations with others I believe Edge is also interested in this, as
they're introducing their own CDP implementation/compatibility layer.

On 8/30/2017 17:20, Eric Rescorla wrote:
> I assume this is going to involve TLS (generally this is a
requirement for H2). In Firefox, this is done with NSS. Does Tokio/Hyper
cleanly separate out the TLS stack so that you can do that?

The Chrome DevTools Protocol actually only runs over HTTP/1.1 and
WebSockets, without TLS (binding to a port on a local interface, so
transport security is less of a concern). My reference to HTTP/2
concerned the possibility of Servo wanting to support that protocol,
which would necessitate an upgrade to the future hyper version that will
support it, which in turn would mean pulling Tokio into Servo.

On 8/30/2017 18:04, Josh Matthews wrote:
> One clarification - while all of the Servo codebase exists in
mozilla-central/servo/, only the dependencies used by Stylo are vendored
in third-party/rust. That means that neither hyper nor tokio are
vendored in mozilla-central at the moment, or part of any
mozilla-central build.

Isn't it vendored into third_party/rust/hyper [4] on mozilla-central?

-Michael

[0] https://github.com/ChromeDevTools/devtools-protocol/tree/master/json
[1] https://chromedevtools.github.io/devtools-protocol/
[2] https://www.spinda.net/files/mozilla/cdp/doc/cdp/tools/index.html
[3] https://remotedebug.org/
[4] http://searchfox.org/mozilla-central/source/third_party/rust/hyper

[James Graham]

On 31/08/17 02:14, Michael Smith wrote:
> On 8/30/2017 15:56, David Burns wrote:
> > Do we know if the other vendors would see value in having this
> spec'ed properly so that we have true interop here? Reverse engineering
> seems like a "fun" project but what stops people from breaking stuff
> without realising?
>
> Fortunately we're not reverse engineering here (for the most part), all
> protocol messages are specified in a machine-readable JSON format which
> includes inline documentation [0] --- this is what the cdp Rust library
> consumes. The spec is versioned and the authors do seem to follow a
> proper process of introducing new features as "experimental",
> stabilizing mature ones, and deprecating things before they're removed.

I think that the reverse engineering part is not the wire protocol,
which is usually the most trivial part, but the associated semantics. It
doesn't seem that useful to support the protocol unless we behave in the
same way as Chrome in response to the messages. It's the specification
of that behaviour which is — as far as I can tell — missing, and which
seems likely to involve reverse engineering.

In general it seems unfortunate if we are deciding to implement a
proprietary protocol rather than opting to either extend something that
is already a standard (e.g. WebDriver) or perform standardisation work
alongside the implementation. What alternatives have we considered here?
Is it possible to extend existing standards with missing features? Or
are the current tools using the protocol so valuable that we don't have
any choice but to support them on their terms? If it's the latter, or we
just think the Chrome protocol is so technically superior to the other
options that we would be foolish to ignore it, can we work with Google
to get it standardised? I think some meaningful attempt at
standardisation should be a prerequisite to this kind of protocol
implementation shipping in Firefox.

[Ted Mielczarek]

On Wed, Aug 30, 2017, at 08:20 PM, Eric Rescorla wrote:
> I assume this is going to involve TLS (generally this is a requirement
> for
> H2). In Firefox, this is done with NSS. Does Tokio/Hyper cleanly separate
> out the TLS stack so that you can do that?

This was mostly answered in another reply, but just to be clear: yes,
Hyper allows plugging in alternate TLS stacks. This is very commonly
used with the `native-tls` crate[1] by way of `hyper-tls`[2], which uses
the native TLS stack on Windows/macOS, and OpenSSL on Linux.

1. https://github.com/sfackler/rust-native-tls
2. https://github.com/hyperium/hyper-tls

[Jim Blandy]

Definitely, yes. As Michael said, some core subset of the protocol is
already a de-facto standard. The process of actual specification, with
committees and processes and imprimaturs and all that, is getting started,
with interest from several major players, including Microsoft and Google.

On Wed, Aug 30, 2017 at 3:56 PM, David Burns <dbu...@mozilla.com> wrote:

> Do we know if the other vendors would see value in having this spec'ed
> properly so that we have true interop here? Reverse engineering seems like
> a "fun" project but what stops people from breaking stuff without
> realising?
>

> David

> > Use of Tokio is becoming a standard in the Rust ecosystem---it's worth
> > mentioning that Mozilla funds Tokio development [9] and employs some of
> its
> > primary developers. Servo currently depends on an older version of the
> > hyper HTTP client/server library, and consequently this is already
> present
> > in the Firefox tree. The current release of hyper is built on top of
> Tokio,
> > so upgrading hyper, either as maintenance or to take advantage of the
> > forthcoming HTTP/2 support, would require pulling in Tokio anyway. The
> > current release of rust-websocket, from which Servo derives its
> WebSockets
> > implementation, also supports Tokio.
> >

[Jim Blandy]

Some possibly missing context: Mozilla Devtools wants to see this
implemented for our own use. After much discussion last summer in London,
the Firefox Devtools team decided to adopt the Chrome Debugging Protocol
for the console and the JavaScript debugger. (The cases for converting the
other tools like the Inspector are less compelling.)

Speaking as the designer of Firefox's protocol, the CDP is a de-facto
standard. The Firefox protocol really has not seen much uptake outside
Mozilla, whereas the Chrome Debugging Protocol is implemented with varying
degrees of fidelity by several different browsers. "Proprietary" is not the
right term here, but in the sense of "used nowhere else", one could argue
that it is Mozilla that is using the proprietary protocol, not Chrome. In a
real sense, it is more consistent with Mozilla's mission for us to join the
rest of the community, implement the CDP for the tools where it makes
sense, and participate in its standardization, than to continue to push a
protocol nobody else uses.

The devtools.html JavaScript debugger already implements the Chrome
protocol. We've implemented adapters like Valence that implement the
Firefox protocol in terms of the Chrome protocol. So while it's true that
not everything is documented at the standards of a WHATWG specification
(yet), in practical terms, there hasn't been much problem getting things
going.


On Thu, Aug 31, 2017 at 2:50 AM, James Graham <ja...@hoppipolla.co.uk>
wrote:

> On 31/08/17 02:14, Michael Smith wrote:
>

>> On 8/30/2017 15:56, David Burns wrote:
>> > Do we know if the other vendors would see value in having this spec'ed
>> properly so that we have true interop here? Reverse engineering seems like
>> a "fun" project but what stops people from breaking stuff without realising?
>>

>> Fortunately we're not reverse engineering here (for the most part), all
>> protocol messages are specified in a machine-readable JSON format which
>> includes inline documentation [0] --- this is what the cdp Rust library
>> consumes. The spec is versioned and the authors do seem to follow a proper
>> process of introducing new features as "experimental", stabilizing mature
>> ones, and deprecating things before they're removed.
>>
>
> I think that the reverse engineering part is not the wire protocol, which
> is usually the most trivial part, but the associated semantics. It doesn't
> seem that useful to support the protocol unless we behave in the same way
> as Chrome in response to the messages. It's the specification of that
> behaviour which is — as far as I can tell — missing, and which seems likely
> to involve reverse engineering.
>
> In general it seems unfortunate if we are deciding to implement a
> proprietary protocol rather than opting to either extend something that is
> already a standard (e.g. WebDriver) or perform standardisation work
> alongside the implementation. What alternatives have we considered here? Is
> it possible to extend existing standards with missing features? Or are the
> current tools using the protocol so valuable that we don't have any choice
> but to support them on their terms? If it's the latter, or we just think
> the Chrome protocol is so technically superior to the other options that we
> would be foolish to ignore it, can we work with Google to get it
> standardised? I think some meaningful attempt at standardisation should be
> a prerequisite to this kind of protocol implementation shipping in Firefox.
>

[Josh Matthews]

On 8/30/17 6:14 PM, Michael Smith wrote:
> On 8/30/2017 18:04, Josh Matthews wrote:
> > One clarification - while all of the Servo codebase exists in
> mozilla-central/servo/, only the dependencies used by Stylo are vendored
> in third-party/rust. That means that neither hyper nor tokio are
> vendored in mozilla-central at the moment, or part of any
> mozilla-central build.
>
> Isn't it vendored into third_party/rust/hyper [4] on mozilla-central?

Ah, it turns out that it's brought in by the geckodriver crate rather
than Servo.

Cheers,
Josh

[Jim Blandy]

On Thu, Aug 31, 2017 at 2:50 AM, James Graham <ja...@hoppipolla.co.uk>
wrote:

> In general it seems unfortunate if we are deciding to implement a
> proprietary protocol rather than opting to either extend something that is
> already a standard (e.g. WebDriver) or perform standardisation work
> alongside the implementation.

That would be unfortunate, I agree. I hope that's not what's happening
here. By adopting the CDP console and JS debugger protocols I think we are
doing exactly what Mozilla should do: encouraging the adoption and
formalization of de-facto standards.

What alternatives have we considered here?

The Devtools team had extensive discussions a year ago in London about
whether to continue to develop Firefox's own protocols, or whether to go
with what Chrome, Safari, and now Edge are using. We considered the
question on a tool-by-tool basis. Harald Kirschner has started to assemble
a list of the third-party tools that use the CDP.

[James Graham]

On 31/08/17 19:42, Jim Blandy wrote:
> Some possibly missing context: Mozilla Devtools wants to see this
> implemented for our own use. After much discussion last summer in
> London, the Firefox Devtools team decided to adopt the Chrome Debugging
> Protocol for the console and the JavaScript debugger. (The cases for
> converting the other tools like the Inspector are less compelling.)
>
> Speaking as the designer of Firefox's protocol, the CDP is a de-facto
> standard. The Firefox protocol really has not seen much uptake outside
> Mozilla, whereas the Chrome Debugging Protocol is implemented with
> varying degrees of fidelity by several different browsers. "Proprietary"
> is not the right term here, but in the sense of "used nowhere else", one
> could argue that it is Mozilla that is using the proprietary protocol,
> not Chrome. In a real sense, it is more consistent with Mozilla's
> mission for us to join the rest of the community, implement the CDP for
> the tools where it makes sense, and participate in its standardization,
> than to continue to push a protocol nobody else uses.

I entirely agree that the current Firefox protocol is also proprietary.
However I also assumed that it's considered an internal implementation
detail rather than something we would expect people to interoperate
with. If that wasn't the case then I apologise: I should have complained
earlier :)

Going forward, if we implement a "de-facto" standard that is not
actually standardised, we are assuming a large risk, in addition to the
problems around our stated values. An obvious concern is that Google are
free to change the protocol as they like, including in ways that are
intentionally or accidentally incompatible with other implementations.
We also know from past experience of implementing "de-facto" standards
that implementation differences end up hardcoded into third party
consumers (i.e. web pages in the case of DOM APIs), making it impossible
to get interoperability without causing intolerable short-term breakage.
This has prevented standardisation and compatibility of "de-facto"
standards like innerText and contentEditable, which remain nominally
equivalent but actually very different in all browsers.

If people are starting to standardise not just the protocol but also the
semantics of CDP, that's great. But people tend to vastly underestimate
how long standardisation will take, and overestimate the resources that
they will find to work on it. So it would be good to see concrete
progress before we are actually shipping.

[Harald Kirschner]

On Thursday, August 31, 2017 at 2:51:41 AM UTC-7, James Graham wrote:
> On 31/08/17 02:14, Michael Smith wrote:
> > On 8/30/2017 15:56, David Burns wrote:
> > > Do we know if the other vendors would see value in having this
> > spec'ed properly so that we have true interop here? Reverse engineering
> > seems like a "fun" project but what stops people from breaking stuff
> > without realising?
> >

> In general it seems unfortunate if we are deciding to implement a
> proprietary protocol rather than opting to either extend something that
> is already a standard (e.g. WebDriver) or perform standardisation work
> alongside the implementation. What alternatives have we considered here?
> Is it possible to extend existing standards with missing features? Or
> are the current tools using the protocol so valuable that we don't have
> any choice but to support them on their terms? If it's the latter, or we
> just think the Chrome protocol is so technically superior to the other
> options that we would be foolish to ignore it, can we work with Google
> to get it standardised? I think some meaningful attempt at
> standardisation should be a prerequisite to this kind of protocol
> implementation shipping in Firefox.

For more history on executing on a de-facto standard; remotedebug.org was started as an effort to spec the protocol by representatives from Edge, Chrome and Firefox. Chrome committed to keeping the stable parts of the API stable, which so far worked. For comparison, WebDriver has gone a similar route, experimenting in the open while standardized the stable parts of the API.

To the idea of having one tool like Web Driver solving all remote debug needs: Having WebDriver focussed on Automation and not on matching CDP's more advanced use cases avoid reinventing the wheel.

[Jim Blandy]

Google has indicated a willingness to participate in standardizing the
protocol.

If we switch from a devtools protocol used only by us to a tooling protocol
used by the rest of the industry, that is strictly an improvement over the
status quo, even if our implementation deviates from the others' to some
degree: developers whose tooling doesn't run into the points of
incompatibility can now include Firefox in their scripting and tests, and
the maintainers of tooling that use the CDP can work around
incompatibilities.

I think it's a mistake to analogize this too closely to content-exposed
APIs. Those are under extreme backwards compatibility pressure, of a sort
that I think doesn't hold here. Unlike a web user, who simply sees a broken
page and can't do anything about it, our audience is more technical, and
can do things like upgrade tools when things don't work.


On Thu, Aug 31, 2017 at 1:09 PM, James Graham <ja...@hoppipolla.co.uk>
wrote:

[Jack Moffitt]

> I entirely agree that the current Firefox protocol is also proprietary.
> However I also assumed that it's considered an internal implementation
> detail rather than something we would expect people to interoperate with. If
> that wasn't the case then I apologise: I should have complained earlier :)

We would like a console and debugger for Servo, and those seemed
pretty common. We did an initial implementation of the Firefox
protocol, but hat protocol changed out from under us, and some of the
work we did broke and wasn't not easily fixable.

At the very least there is a desire to interoperate between Servo and
Firefox and the current protocol isn't very helpful for this. The
Chrome one has a huge advantage of being documented, even if not all
the semantics are. I was also under the impression that the design of
the Chrome protocol is better than the Firefox one because you can
ingest the JSON messages and only handle the things you want. It seems
like that would make it more robust to future changes.

Is there another alternative besides CDP you'd like to propose? I
think there's a strong case that doing something besides the current
Firefox protocol is a good idea, and the CDP seems to have some nice
qualities. Is there an alternative that is also suitable, or is the
preference that we simultaneously get involved in standardizing CDP
(or at least ensuring that all parties are amenable to doing that
work)?

jack.

[Jim Blandy]

Certain bits of the original post are getting more emphasis than I had
anticipated. Let me try to clarify why we in Devtools want this change or
something like it.

The primary goals here are not related to automation and testing. They are:

- to allow Devtools to migrate the console and the JS debugger to the
CDP;
- to start a tools server that can be shared between Gecko and Servo;
- to replace Gecko's devtools server, implemented in JS, with one
implemented in Rust, to reduce memory consumption and introduce less noise
into performance and memory measurements



and to help us share code with Servo. Our user interfaces already work with
the CDP.

[Harald Kirschner]

On Thursday, August 31, 2017 at 1:10:30 PM UTC-7, James Graham wrote:
> If people are starting to standardise not just the protocol but also the
> semantics of CDP, that's great. But people tend to vastly underestimate
> how long standardisation will take, and overestimate the resources that
> they will find to work on it. So it would be good to see concrete
> progress before we are actually shipping.

As mentioned in my last email, CDP already provides a stable core. We did review existing tools for API dependencies to understand what the minimum viable set should include to be useful for developers. We are still reaching out to developers and browser-testing services to capture their feedback, but preliminary results are captured in [1]. The list so far suggests that there is a shortlist of APIs solving most needs.

[1]: https://docs.google.com/document/d/19bimjSpwbiNOPFiEWzBsJ9PtB6cvEaBQCY1Y8UValRk/edit#

[Jim Blandy]

Sorry for the premature send. The complete message should read:

The primary goals here are not related to automation and testing.

- We want to migrate the Devtools console and the JS debugger to the CDP,
to replace an unpopular protocol with a more widely-used one.

- Servo wants its devtools server to use an industry-standard protocol, not
Firefox's custom thing.

- We'd like to have a devtools server we can share between Gecko and Servo.

- We'd like to move devtools server code out of JS and into a language that
gives us better control over memory use and performance, because the
current server, implemented in JS, introduces a lot of noise into
measurements, affecting the quality of the performance data we're able to
collect.

- We'd like to share front ends (i.e. clients) between Firefox and Servo.
devtools.html already implements both the Firefox protocol and the CDP.

Using a protocol that's more familiar to web developers doing automation
and testing is also good, and we're hoping that will have ancillary
benefits. For example, it turns out that there is lots of interest in our
new JS debugger UI, which has hundreds of contributors now. I don't know
why people want a JS debugger UI, but they do. I believe that Firefox's use
of a bespoke protocol prevents many similar opportunities for collaboration.

[James Graham]

On 31/08/17 21:22, Jack Moffitt wrote:
> Is there another alternative besides CDP you'd like to propose?
>

I don't have an alternate proposal, and I feel like I must have been
unclear at some point. I'm not saying "this is bad, period". I'm
certainly not saying "this is bad because it isn't WebDriver". Given
that people seem to be in agreement that technically the CDP is good,
I'm saying "this is bad if it remains a vendor-controlled, partially
documented, pseudo-standard". The fact that there is apparently interest
in creating a standard is reassuring, but there doesn't seem to be any
recent activity on remotedebug.org, so it's hard to tell what the real
status is, or whether people have understood the amount of work that
entails. I am slightly worried that there have been several replies
suggesting that poor interoperability above the message layer won't be a
big problem because the users will be technical and therefore happy to
absorb the cost of backwards-incompatible changes between releases.
Experience from WebDriver is that this isn't true.

[David Burns]

I don't think anyone would disagree with the reasons for doing this. I,
like James who brought it up earlier, am concerned that we from the emails
appear to think that implementing the wire protocol would be sufficient to
making sure we have the same semantics.

As mentioned by Karl earlier, it was part of the Browser Testing Tools WG,
but was dropped due to lack of interest from vendors but this thread is now
suggesting that feeling has changed. I am happy, as chair, to see about
adding it back when we recharter, which luckily at the end of September!

I am happy to create a separate thread to get this started.

David

[Jim Blandy]

On Mon, Sep 4, 2017 at 7:36 AM, David Burns <dbu...@mozilla.com> wrote:
> I don't think anyone would disagree with the reasons for doing this. I,
like James who brought it up earlier, am concerned that we from the emails
appear to think that implementing the wire protocol would be sufficient to
making sure we have the same semantics.

LOL, give us a little credit, okay? The authors of the email do not think
that. We want to have a properly written specification and conformance
tests. I think you're reading "we have no interest in established
standardization processes" when what we wrote was "the process is in very
early stages".

Do you think the Browser Testing Tools WG is the right body to work on a JS
debugging and console protocol, used by interactive developer tools? That
seems like a surprising choice to me.

Also - at least as far as I know - this is not where the current
participants in the discussion (Kenneth Auchenberg or Christian Bromann, to
name two) have been working. Is having a previously uninvolved standards
committee take up an area in which current activity is occurring elsewhere
considered friendly and cooperative behavior? It seems unfriendly to me. I
would like to avoid upsetting the people I'm hoping to work closely with.

I think the people who have actively participating in the work should be
the ones to decide which standards body to collaborate with.

[James Graham]

On 04/09/17 23:34, Jim Blandy wrote:
> On Mon, Sep 4, 2017 at 7:36 AM, David Burns <dbu...@mozilla.com> wrote:
>> I don't think anyone would disagree with the reasons for doing this. I,
> like James who brought it up earlier, am concerned that we from the emails
> appear to think that implementing the wire protocol would be sufficient to
> making sure we have the same semantics.
>
> LOL, give us a little credit, okay? The authors of the email do not think
> that. We want to have a properly written specification and conformance
> tests. I think you're reading "we have no interest in established
> standardization processes" when what we wrote was "the process is in very
> early stages".
>
> Do you think the Browser Testing Tools WG is the right body to work on a JS
> debugging and console protocol, used by interactive developer tools? That
> seems like a surprising choice to me.

It is certainly not the only possible venue, but if you want to do the
work at the W3C then it's probably the easiest way to get things going
from a Process point of view, since this kind of protocol would be in
the general remit of the group, and the rechartering could add it
specifically. Certainly the people currently in the group aren't the
right ones to do the work, but adding new participants to work
specifically on this would be trivial.

> Also - at least as far as I know - this is not where the current
> participants in the discussion (Kenneth Auchenberg or Christian Bromann, to
> name two) have been working. Is having a previously uninvolved standards
> committee take up an area in which current activity is occurring elsewhere
> considered friendly and cooperative behavior? It seems unfriendly to me. I
> would like to avoid upsetting the people I'm hoping to work closely with.

I think you have misinterpreted the intent here. I don't think anyone is
interested in doing a hostile takeover of existing work. But there is
concern that the work actually happens. Pointing at remotedebug.org,
which has been around since 2013 without producing any specification
materials, isn't helping assuage my concerns, and I guess others are
having a similar reaction. It is of course entirely possible that
there's work going on that we can't see. But my interpretation of
David's email is that he is trying to offer you options, not force you
down a certain path. The W3C is not always the right venue to work in,
but it is sometimes sought out by organisations who would likely
participate in this work because of its relatively strong IPR policy.

I should stress that irrespective of venue I would expect this
standardisation effort to take years; people always underestimate the
work and time required for standards work. It will certainly require us
to commit resources to make it happen.

[Jim Blandy]

As an offer of help, from a group whose charter covers this work, that's
very welcome. I felt that I was being shepherded into something on behalf
of others for whom I cannot speak, which was uncomfortable!

For my own sake, I am disinclined to participate in a standardization
effort outside of the usual institutions. I think we could benefit from the
experience of people who have produced web standards before. I'm not
well-connected enough yet to anticipate what folks will say, but I'll
suggest the Browser Testing Tools WG when the opportunity comes up.

[ken...@auchenberg.dk]

Hi everyone,

Kenneth, the original initiator of RemoteDebug here. I just want to provide an overview of the state of the RemoteDebug initiative and how I see CDP fit into the broader tooling ecosystem.

Status of RemoteDebug

Now several years after I initiated RemoteDebug a lot has happened in the tooling ecosystem, as we today have several 3rd party tools build on and around CDP by the big players (Debugger.html by Mozilla, Lighthouse+ Puppeteer by Google, and Sonar + VS Code debuggers by Microsoft), which has reinforced that CDP is the de-facto DevTools "standard" protocol.

At the last Chrome DevSummit (2016) we had a meeting where Firefox DevTools, Edge DevTools and Chrome DevTools where present to discuss the latest RemoteDebug proposal, the RemoteDebug DevTools Core Specification [1], which could serve as an implementation starting point for new vendors who want to embrace CDP compatibility but focus on the stable parts of the API. At the meeting it was agreed that Google would focus on stabilizing most of the CDP API in its CDP 1.2 version (which today is the most recent version) while continue to experiment with new APIs marked as experimental in the protocol.

When looking at the RemoteDebug DevTools Core and the stable APIs within CDP it's clear that most of the API surface has been stable sine the early WebKit days, which is good news for existing tools and for new implementors. The API differences can be browsed through the RemoteDebug Compatibility Tables [10]

From a process perspective the idea of the RemoteDebug DevTools Core Specification has been to embrace the same process as the ServiceWorker specification which was build around GitHub, and later promoted to a W3C spec once a foundation has been agreed upon by multiple vendors. I'm open to transition RemoteDebug and all related work to the W3C Browser Testing Tools WG, if this could help ensure commitment from Mozilla and others.

Until now most of the conversations for RemoteDebug/CDP has been ad-hoc conversations between Chrome, Firefox, Edge and VS Code as the number of persons involved in the protocol discussions has been very limited.


Overview of CDP in the tooling system

As I started out by saying then CDP has emerged as the defacto "DevTools" standard protocol as most of the major players now have tools and project based on CDP. I would look at CDP in a similar way to WebDriver as something that can live in parallel to what-ever Firefox currently exposes today.

From Microsoft (my employer) we have been pushing CDP as our preferred protocol for our VS Code debuggers in vscode-chrome-debug-core [2], which is now powering 6x VS Code debuggers and the Node debugger in Visual Studio. Microsoft backed Project Sonar adapted by the JS Foundation is also based on CDP [4] Additionally it was just announced at the Edge Summit 2017 that Microsoft Edge is gonna introduce the Edge DevTools protocol based on JSON RPC over WebSockets with the aim to be highly compatible with CDP. This means that future version of Edge DevTools most likely will be solely powered by a CDP dialect. [3].

Microsoft has also contributed to remotedebug-ios-webkit-adapter which enables iOS WebKit to be debugged from VS Code and Chrome DevTools, by working as an API adaptor mapping the WebKit API to CDP. [8]

Google contributed an CDP implementation for Node.js, which has now become the default debugging protocol for Node under the name "Inspector protocol" which has enables tools like Chrome DevTools to debug Node out of the box. [5]. We have also seen Google building tools like Lighthouse [6] and Puppeteer [7] which also are based on CDP, and used in Chrome Headless where CDP is the preferred API.

As already mentioned in this thread, when we have also seen Mozilla building debugger.html which is powered by CDP [11].

On top of these larger projects the open-source community has also been building a broad range of tools and CDP clients [9]

It can be discussed whether CDP is the most technical elegant solution, but it can't be ignored that it's a well proven protocol (JSON RPC over WebSockets) which has been used since Webkit introduced remote debugging, and today we all major vendors either exposing CDP or consuming CDP in some form (Apple, Microsoft, Mozilla, Google, Opera, Samsung) within their web runtime or tooling. CDP isn't perfect, and is know to have shortcomings for feature/API detection, but these are things that easily can be resolved by introducing this to the API surface.

From my perspective we have now reached a critical point in CDP's adaption as we have a rich tooling ecosystem using the APis while most of the API surface has been stabilized in CDP 1.2, which speaks in favor of moving forward with a formel spec and for other vendors to begin their implementations.

[1] https://github.com/RemoteDebug/devtools-core-spec
[2]
https://github.com/Microsoft/vscode-chrome-debug-core
[3] https://mobile.twitter.com/auchenberg/status/908721908782428160
[4] https://sonarwhal.com/
[5] https://github.com/nodejs/node/pull/6792
[6] https://github.com/GoogleChrome/lighthouse
[7] https://github.com/GoogleChrome/puppeteer
[8] https://github.com/RemoteDebug/remotedebug-ios-webkit-adapter
[9] https://github.com/ChromeDevTools/awesome-chrome-devtools
[10] http://compatibility.remotedebug.org/
[11] https://github.com/devtools-html/debugger.html