Shipping DevTools as an add-on?

[Alexandre poirot]

Devtoolers,

This post is about 'the how'. If you want to learn and ask about 'the
why', here's this previous thread:
"Devtools impact on Firefox package and performances":
https://groups.google.com/forum/#!topic/mozilla.dev.developer-tools/rUmAom_L3kc

Installing DevTools as an add-on has been recently made possible
thanks to the work done for the add-on contribution workflow [1].

With a set of simple patches [2], you can already experience DevTools
installed as an add-on:
* Install a Firefox with Devtools removed. See [3] for d/l links,
* Enjoy a slimmer Firefox where "Tools > Web developer" menu contains
only "View Source" (which is a platform feature, not a devtools one),
* Go to about:config and toggle "xpinstall.signatures.required" pref to false,
* Install add-on from link [4],
* Be relieved, the Devtools are back! Menu, key shortcuts, toolboxes, ...

These patches are about:
- Stripping Devtools from Firefox builds (everything: frontend,
backend, prefs, locales, ...),
- Fixing Firefox codepaths relying on Devtools. Two typical patterns
are: firefox trying to read a Devtools pref and throws because they
are not set. The second is firefox code using event-emitter module
which lives in devtools...
- Fixing small bugs in the DevTools add-on (install the prefs on addon
startup and fix a l10n url bug),
- Generating the DevTools add-on on Taskcluster.

What to do next if we want to move forward?
- Get these patches landed. It shouldn't be too hard. They need some
polish, but none is very complex.
- Reinsert some UI elements builtin into Firefox in order to ease
installing this
add-on.
If we strip devtools it looks like Firefox no longer has any. We still
need to reference them in the UI and make it super easy to install'n
use them.
We already have a good experience with that in WebIDE as we
auto-install ADB Helper and Valence. This would surely require a
fine-tuned and polished work involving UX thoughts.
- Figure out all the release engineering issues it raises!

For me, this is mostly a release engineering thing:
- How to build the add-on?
I imagine that's quite straithforward. I already have a Taskcluster
Task to build it. But we need some more work to get it signed.
- How to release it?
One unique Add-on for all Firefoxes? One per major version? One per
changeset? Per locale? Per experiment?
- How to keep running tests on CI?
I have multiple options here. It is mostly about mozilla-central
build system and Taskcluster tasks running DevTools tests.

The complexity depends entirely on how we plan to release it and it
isn't really specific to using add-ons as distribution channel!
We would have to answer to the same questions if we end up hosting
tools on https.

I think gofaster and test pilot teams have very valuable information
and process to share! But I'm convinced that's a reasonable goal if we
release add-ons for each Firefox release/mozilla-central changeset.
The key to simplify things here is to avoid multiple combinations.
You know this eternal story where the old guard prefer a unique repo
with a unique changeset where we can test everything with a very
precise environment.

Finally, I keep hearing about using Service Workers to release
DevTools. But we are far from being able to use that. We can only
distribute tools via https and Service Workers if our panels are able
to load without any privileges. We surely can for the new Debugger.
But this is still in progress for the Console and the Inspector. Even
the Debugger isn't ready for release. And what about everything else?

But suppose we are ready. The add-on has the advantage of stripping
more things, basically everything related to devtools (frontend,
backend, prefs, l10n, firefox integration).
And as everything ship altogether, it will also ease handling versioning.
Then, there are many advantages of packaged versus hosted. We
learned a lot about that in Firefox OS and that is why Packaged
WebApps won over Hosted ones:
* Signatures: better control of what we ship and what users receive and run,
* Performances: files are read from disk, always. Reading files from jar
is optimized. No need of caching or merging resources to reduce HTTP
requests...
* Offline: you do not have to think about offline at all.
Now you can surely address that using recent Web APIs like Service
Worker, but it requires some more work.

++
Alex

[1] Contributing on DevTools from Nightly
https://developer.mozilla.org/en-US/docs/Tools/Contributing/Contribute_on_nightly

[2] Patch queue for DevTools as Add-on experiment
https://hg.mozilla.org/try/log/81230c5331508cefef68e76cf3c86413dd4b3a85

[3] Firefox builds without DevTools
# Windows:
https://archive.mozilla.org/pub/firefox/try-builds/apo...@mozilla.com-81230c5331508cefef68e76cf3c86413dd4b3a85/try-win32/firefox-52.0a1.en-US.win32.zip
# MaxOS:
https://archive.mozilla.org/pub/firefox/try-builds/apo...@mozilla.com-81230c5331508cefef68e76cf3c86413dd4b3a85/try-macosx64/firefox-52.0a1.en-US.mac.dmg
# Linux64:
https://archive.mozilla.org/pub/firefox/try-builds/apo...@mozilla.com-81230c5331508cefef68e76cf3c86413dd4b3a85/try-linux64/firefox-52.0a1.en-US.linux-x86_64.tar.bz2

[4] DevTools as an Add-on (built from a Taskcluster Task)
https://queue.taskcluster.net/v1/task/UgBuF_jrSkOEzzpyT-pQ3A/runs/0/artifacts/public%2Fdevtools%40mozilla.org.xpi

[Patrick Brosset]

Thanks for the write-up Alex.

You're making a set of very good points. Here's what I'm hearing:
- With a few simple patches we can already make this work today.
- There's a lot of prior experience we can rely on.
- We would be able to remove both the frontend and backend. Not only does
this allow to save more Kb when downloading Firefox, it also would get rid
of the frontend-backend compatibility problems! (in fact we'd just be
moving the problem to between the backend and the platform, but I'm tempted
to think this would be simpler, I might be wrong though).

I'd love to hear what others think about this approach but, to me, it
sounds very tempting.
It would get us very quickly a lot closer to the following things:
- delivering features to users based on criteria we think are important
rather than only because they shipped in nightly and rode the trains from
there
- helping make the Firefox download artifact smaller

However, it wouldn't get us closer to making the rest of our front-end more
web-like and therefore compatible in the future with other environments:
inside Servo, as a standalone app to debug node, running in other browsers,
etc.
This is, however, an effort that can continue in parallel to shipping
DevTools to Firefox users as an addon.

Can you clarify the following paragraph though? I'm not sure I understand:

[Alexandre poirot]

2016-11-02 15:52 GMT+01:00 Patrick Brosset <pbro...@mozilla.com>:
> Can you clarify the following paragraph though? I'm not sure I understand:
>
>> I think gofaster and test pilot teams have very valuable information
>> and process to share!

gofaster team knows how to release addons to firefoxes. And I think
they know how to release a given version to a specific set of firefox
users.
Test pilot team knows how to work against firefox via addons.

>> But I'm convinced that's a reasonable goal if we
>> release add-ons for each Firefox release/mozilla-central changeset.

That's just two very conservative ways to handle versionning between
firefox and devtools:
1) One devtools addon per Firefox version. One for 49, another one for
50, ... Each addon updates as each firefox gets updated
2) One devtools addon per Firefox changeset. This is most likely going
to happen if devtools stays on mozilla-central and will happen behind
the scene on automation. But that doesn't mean we end up releasing
every single version of the devtools addon.

The opposite proposal would be to release only one version of devtools
somehow compatible with all firefox versions.

>> The key to simplify things here is to avoid multiple combinations.

Combinations is the worse with the "opposite proposal". You would have
to ensure that a given code works fine with many firefox versions. It
introduces challenges regarding QA and engineering.
Regarding code, it means compatibility layers.
Regarding QA, it means more work. The more combinations there is, the
more test you have to run.
Every time you update devtools you would have to retest it against
many firefoxes.

>> You know this eternal story where the old guard prefer a unique repo
>> with a unique changeset where we can test everything with a very
>> precise environment.

That's a way to refer to conservative option 2. Devtools are still on
mozilla-central, they do have a changeset on mozilla-central, which
happen to also relate to a precise Firefox binary. We do run automated
tests and QA on this unique changeset and release this.
It is easier for everyone working on software after the developer to
have a unique repo and changeset.
Everyone can refer to a given changeset and know what's inside and
stamp it for the next step.
Developer to QA, QA to release engineering, Release engineering to users.

[Nick Fitzgerald]

I just skimmed, so apologies if this is already mentioned, but:

Another benefit that would come out of this is always having a matching
server + client, and making protocol compatibility issues largely a thing
of the past. This would be huge.

Of course, this would now require feature detection of newer platform APIs
on the server, but this is a much easier problem.

On Wed, Nov 2, 2016 at 8:22 AM, Alexandre poirot <poiro...@gmail.com>
wrote:

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

[James Long]

This is great and thorough research. The only line I have a comment on is
this:

> No need of caching or merging resources to reduce HTTP requests...

Merging JS files into a single JS bundle is mostly to solve the module
problem. We don't have any way to load JS modules, but we *really* want to
write them, so webpack solves this very nicely and enables a lot of other
things too. So it's not really about reducing HTTP requests.

We should also think about other clients. I hope we don't just forget about
protocol versioning/compatibility if we do this. I hope to see more people
connecting to Firefox with other tools, which is a huge market if you ask
me (all the atom/code/etc integration), so we still need to think about
protocol compatibility.

That's the big upside to the hosted version: we just serve up the right
version of the frontend, which is a really nice way to solve it.

However, you are right to point out that it feels like we're not close to
serving all the tools that way. The debugger is ready (it should ride the
trains), and maybe memory tool, but the console and especially inspector
still have a lot of work. So I don't know what the right next step is, and
I haven't been involved in these discussions recently. Just wanted to make
bring up a few points.


On Wed, Nov 2, 2016 at 1:49 PM, Nick Fitzgerald <nfitz...@mozilla.com>
wrote:

[J. Ryan Stinnett]

On Wed, Nov 2, 2016 at 9:04 AM, Alexandre poirot <poiro...@gmail.com>
wrote:

> What to do next if we want to move forward?
> - Get these patches landed. It shouldn't be too hard. They need some
> polish, but none is very complex.
> - Reinsert some UI elements builtin into Firefox in order to ease
> installing this
> add-on.
> If we strip devtools it looks like Firefox no longer has any. We still
> need to reference them in the UI and make it super easy to install'n
> use them.
> We already have a good experience with that in WebIDE as we
> auto-install ADB Helper and Valence. This would surely require a
> fine-tuned and polished work involving UX thoughts.
>

Assuming we move forward with this add-on approach you've outlined, then
these UI issues will be a critical thing to get right. I assume Dev.
Edition and Nightly would come with the add-on preinstalled, but other
channels might not?

Like Joe and Bryan mentioned in the previous thread, we'll need to work out
the UX, such as:

* Keep all menu items and shortcuts as stubs that triggers add-on install
* Developer Mode which triggers add-on install when first enabled

We'll also want some kind of progress display on screen while download and
install is happening.

> But suppose we are ready. The add-on has the advantage of stripping
> more things, basically everything related to devtools (frontend,
> backend, prefs, l10n, firefox integration).
> And as everything ship altogether, it will also ease handling versioning.
>

I agree that a single add-on sounds a lot simpler to release and keep track
of than releasing separate sites for each panel all with their own
versions. At the very least, it seems like a good stepping stone towards
greater flexibility in how we ship without having to commit to most complex
scenario (one site per panel plus a bunch of add-ons for all the non-web
panels). We're still free to develop each panel however we like, I am just
talking about where we draw the line around the package that is versioned
and shipped.

If we proceed with this and then decide we want independent versioning per
panel, we'll definitely need to spend some time on great release /
versioning tooling for ourselves to avoid creating a giant headache.

- Ryan

[J. Ryan Stinnett]

On Wed, Nov 2, 2016 at 9:52 AM, Patrick Brosset <pbro...@mozilla.com>
wrote:

> - We would be able to remove both the frontend and backend. Not only does
> this allow to save more Kb when downloading Firefox, it also would get rid
> of the frontend-backend compatibility problems! (in fact we'd just be
> moving the problem to between the backend and the platform, but I'm tempted
> to think this would be simpler, I might be wrong though).
>

This plan by itself doesn't seem like it removes the need for
compatibility. There are several scenarios to consider:

# Debugging Local Firefox

For this case, I assume we'd download the right add-on for the exact
Firefox build you have. Firefox RDP compatibility code is unnecessary here,
since we know the client and server match.

# Debugging Remote Firefox (Firefox for Android, etc.)

The remote Firefox needs a DevTools server that matches whatever platform
version it is. Historically, we have committed to supporting remote
debugging across all active channels, i.e. desktop Firefox Nightly
debugging release Firefox for Android. Assuming we keep this flexibility,
there are many ways we could choose to proceed for this scenario:

* Continue maintaining compatibility code like today so that clients can
talk to older servers
* Somehow acquire (through add-ons, remote web server, etc.) the version of
the DevTools used in the remote target, use this as the front end so you
have matching versions
* New client could install the DevTools server into the remote target,
giving you matching RDP versions (this would likely fail though, since the
server would likely expect new platform features)

So, add-ons could help with some of this by becoming the "unit of delivery"
that would allow fetching an older UI in this scenario, but there's still
more surrounding choices to make before compatibility could be removed. We
discussed a lot of these options in 2015, before the idea of an add-on had
really been discussed. You can review the document[1] from that time to
read about pros and cons for each of these.

# Debugging non-Firefox things (Chrome, Node.js, Servo)

For remote targets like these where we don't write the server at all, all
we have is the protocol. Presumably, we'd define some set of supported
versions for each of these things, and we'd need to maintain enough
compatibility code to connect to all of them from one client, which sounds
similar to what the new debugger and new console have been doing to support
both Firefox and Chrome protocols.

# Other Clients Debugging Local Firefox (Atom, VS Code, random CLI tools,
etc.)

Like James already mentioned, editors and other things that want to connect
to local Firefox will expect a certain protocol level from the server that
we'd need to maintain over time.

# Summary

Delivering DevTools as an add-on could be leveraged into a way to have a
matching client when connecting to Firefox for Android, but it requires
additional work to fetch the right version, load a different add-on than
what you'd use locally, etc., so the add-on idea alone isn't enough to
remove compatibility. Furthermore, we're interested in supporting other
scenarios that seem unaffected by this, so we'll likely still need
compatibility of one form or another.

[1]:
https://docs.google.com/document/d/1qg7EkMGNmvCho7YrenbR1LQrw6Zf6WsmBWh-SLUmy6M/edit

- Ryan

[mratc...@mozilla.com]

Great to see that we have made such rapid progress on this. As long as it is really easy or even transparent to install devtools I am completely on board.

My one concern is that every Firefox change has to be tested against the very latest revision of DevTools and vice versa but I am sure our automation people would be happy to do that.

/me idly wonders if there are other parts of Firefox that could be separate modules.

[Alexandre poirot]

2016-11-03 13:30 GMT+01:00 <mratc...@mozilla.com>:
> Great to see that we have made such rapid progress on this. As long as it is really easy or even transparent to install devtools I am completely on board.

Note that I'm just exposing opportunities here. I'm not actually doing
it right away.

> My one concern is that every Firefox change has to be tested against the very latest revision of DevTools and vice versa but I am sure our automation people would be happy to do that.

Doing that is what we currently do. Everytime a commit is made to
mozilla central in /devtools/ we run all tests
(firefox/platform/devtools), and vice versa. Everytime a commit is
made in /browser/ or /toolkit, /dom/, ... we run all tests again.

> /me idly wonders if there are other parts of Firefox that could be separate modules.

Session restore?
http://blog.techno-barje.fr/post/2016/03/14/session-restore-web-extension/
But I'm not sure anyone wants a firefox *without* session restore.

[Alexandre poirot]

Yes, this add-on strategy doesn't remove the necessity of compatiblity
between frontend and backend.
It only removes it for Firefox debugging itself, but not when using
remote debugging against Fennec, node.js, chrome and ios targets via
valence add-on.

[francesc...@gmail.com]

> But suppose we are ready. The add-on has the advantage of stripping
> more things, basically everything related to devtools (frontend,
> backend, prefs, l10n, firefox integration).

Hi,
Was there any concrete thought about l10n in this new environment?

For example:
* Would the strings live where the code lives (outside of m-c)?
* Would the add-on follow the localization of the browser, i.e. would it ship with all the languages? It might be expensive in terms of size.
* Would I be able to download/use the add-on in a language different from the one I'm using for Firefox?

DevTools have been a painpoint for some locales: language is obviously technical and most terms might not be available in some smaller languages, and developers might be more comfortable with English in this context. But we also know that there are plenty of DevTools users in languages other than English, so it's important that we keep DevTools a project with good localizability.

Francesco