disabling the desktop/Android Web Runtimes

[Myk Melez]

Folks,

I think we should disable the desktop and Android Web Runtimes in
Firefox and Fennec.

The Web Runtime projects were initiated by Mozilla Labs in 2011 and
developed by a Runtime Engineering team at MoCo that shipped the desktop
runtime in 2012 and the Android runtime in 2013. They're designed to run
"Open Web Apps," which in practice has meant "Firefox OS apps" hosted by
the Firefox Marketplace.

Both runtimes were deprioritized by MoCo around the times they initially
shipped, and they've seen little activity since. Bugs have gone unfixed,
enhancements have gone unimplemented, and unit tests have broken and
been disabled. Although their core functionality continues to work,
their developer and user experiences are poor, and their technical debt
is substantial and growing.

The Runtime Engineering team has also been disbanded, and its engineers
have been reassigned to other projects. MoCo no longer invests in the
runtimes, and it hasn't for years.

Nor has a significant community of other contributors evolved. App users
and developers file the occasional bug, but code contributions are few
and far between. And they're usually bustage fixes for
Firefox/Fennec/Gecko changes that inadvertently break the runtimes,
which makes the runtimes a burden on the developers of those projects.

I'm one of the early contributors to the runtimes, and I'm the module
owner for the Desktop Runtime module (and the relevant peer for Fennec),
which makes me ultimately responsible for them. I still think the
runtimes could provide a good experience for developers and users who
want to build and run web apps with deep native integration.

But it would take significant, ongoing investment to provide that
experience, and there's no evidence of that happening. Absent such
investment, and a commitment to maintain it, the benefits that the
runtimes provide don't justify their risks and costs to developers,
users, and other projects.

Disabling them isn't as simple as flipping a switch. It entrains other
work, like updating the docs on MDN to stop describing desktop/Android
as platforms for Open Web Apps and updating Marketplace to stop
soliciting apps from developers and distributing apps to users on those
platforms.

There's also the question of how to let users know that their apps will
stop working and give them time to migrate to other apps (or to versions
of those apps that run within the browser). And there's the question of
what to do with the mozApps API on those platforms. I don't have all the
answers yet, but I'm making this proposal now anyway in order to get
input on it before making a final decision.

If I decide to disable the runtimes, I'll work with Marketplace,
Firefox/Fennec, and Gecko/mozApps peers to identify and address the
various dependencies, after which I'll complete the work to disable them
and remove their code from the mozilla-central repository.

Let me know your questions, concerns, and other thoughts.

Regards,
-myk

_______________________________________________
firefox-dev mailing list
firef...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev

[Mark Hammond]

+1 - great or dead, right?

(Personally I'd prefer to see this as "great", but I need to face
reality here...)

Mark

[Ehsan Akhgari]

This sounds great to me!  I hope that this means that we can unship the non-standard mozApps API on desktop and Android as AFAICT the web app runtime is the only use case for it on those platforms.

Cheers,

Ehsan

--

Ehsan

[Mike Hommey]

Isn't the web app support (presumably through the runtime) exactly what was
announced would be improved in Fennec during Mozlando? (with hires
icons, full screen display, etc.)

Mike

[Nicholas Alexander]

It's similar but the technical implementation is independent.  The "Web App Runtime" on Fennec that Myk refers to is the system that produces APK files wrapping a web app, and then arranges to run that web app in its own process.  (Independent of the Fennec process!)  That set of features is essentially untested and slows development down.  The future of Web Apps in Fennec is not 100% clear, but the features discussed in Mozlando are intended to be used and integrate more with Fennec-the-browser-process than stand beside it.  Parsing manifests to extract hires icons is possible in both worlds, but many things require lots of work on the Fennec side -- opening homescreen bookmarks fullscreen or with custom chrome UI, for example.

margaret and I can provide more technical detail as required.

Best,

Nick

[Nicholas Alexander]

+1 from me.

This is a large and mostly untested piece of the Fennec code base that I'm happy to cull.

Best,

Nick

[Margaret Leibovic]

+1 from me as well.

We've already taken steps in Fennec to stop promoting the Marketplace, and we've removed about:apps from our UI.

Margaret

[David Rajchenbach-Teller]

I'll be sad to lose the promise of this feature, but yeah, great or
dead. Does anyone know if we going to have/keep the ability to bookmark
to the desktop/start menu/dock on desktop and mobile, as a replacement?

Cheers,
David

[Nicholas Alexander]

Hi David,

On Tue, Dec 15, 2015 at 11:37 AM, David Rajchenbach-Teller <dte...@mozilla.com> wrote:

I'll be sad to lose the promise of this feature, but yeah, great or
dead. Does anyone know if we going to have/keep the ability to bookmark
to the desktop/start menu/dock on desktop and mobile, as a replacement?

The ability to bookmark to the home screen will remain in Fennec.  Such bookmarks are the intended entry point for whatever Progressive Web App features Fennec implements, although I don't know all the details.

Best,

Nick

[Myk Melez]

Nicholas Alexander

2015 December 14 at 17:23

It's similar but the technical implementation is independent.  The "Web App Runtime" on Fennec that Myk refers to is the system that produces APK files wrapping a web app, and then arranges to run that web app in its own process.  (Independent of the Fennec process!)  That set of features is essentially untested and slows development down.  The future of Web Apps in Fennec is not 100% clear, but the features discussed in Mozlando are intended to be used and integrate more with Fennec-the-browser-process than stand beside it.  Parsing manifests to extract hires icons is possible in both worlds, but many things require lots of work on the Fennec side -- opening homescreen bookmarks fullscreen or with custom chrome UI, for example.

What Nick said!

I would only add that the manifest in question is different between these two approaches to supporting web apps. The Web Runtime uses the Open Web App (i.e. FxOS) manifest [1], while the demonstration at Mozlando used the W3C manifest [2].

-myk

[1] https://developer.mozilla.org/en-US/Apps/Build/Manifest
[2] https://developer.mozilla.org/en-US/docs/Web/Manifest

[Myk Melez]

Mark Hammond

2015 December 14 at 16:10

+1 - great or dead, right?

Right, although this proposal isn't part of the "Great or Dead" program. I'm proposing it per my responsibility as module owner/peer to seek the best possible outcome for the code under my control.

FWIW, I'm not sure the runtimes could be made "great," given the significant (and growing) constraints imposed by the various platforms that it supports. However, I'm pretty sure it could be made "good," at least for a subset of users who are willing to accommodate or work around those constraints.

Regardless, it won't be either without investment, and there's been no prospect of such investment for years. So it's already been implicitly killed and exists in a kind of undead state. Presumably it would eventually be explicitly killed by the "Great or Dead" program, but it's my Lennie, and I feel responsible for doing so myself.

-myk

[Myk Melez]

Ehsan Akhgari

2015 December 14 at 16:17

I hope that this means that we can unship the non-standard mozApps API on desktop and Android as AFAICT the web app runtime is the only use case for it on those platforms.

I think it does, yes. You and Fabrice co-own the "mozApps API & UI" module, so I would just confirm that with Fabrice (cc:ed).

-myk

[Myk Melez]

David Rajchenbach-Teller

2015 December 15 at 08:37

I'll be sad to lose the promise of this feature, but yeah, great or
dead. Does anyone know if we going to have/keep the ability to bookmark
to the desktop/start menu/dock on desktop and mobile, as a replacement?

I've had discussions with various folks about ways that Firefox could use the W3C manifest to better support web apps on desktop, but I'm not aware of any plans to do something like you describe.

-myk

[Fabrice Desré]

On 12/15/2015 09:35 AM, Myk Melez wrote:
>> Ehsan Akhgari <mailto:ehsan....@gmail.com>

No objections on my side.

Fabrice
--
Fabrice Desré
b2g team
Mozilla Corporation

[Andrew Sutherland]

On Mon, Dec 14, 2015, at 06:45 PM, Myk Melez wrote:
> Let me know your questions, concerns, and other thoughts.

Retiring the web runtime (WebRT) makes a lot of sense. I think it would
also be great to use this as a (non-blocking) opportunity to figure out
the suggested road-map for apps leveraging the WebRT/packaged apps
mechanism for getting powerful/dangerous permissions granted.

Previous statements by :sicking have indicated a desire to recognize
that some APIs are sufficiently dangerous and hard to explain in a way
that users can truly make an informed decision (ex: TCP) and that the
least-bad solution is to expose these dangerous APIs only to add-ons via
a mechanism that requires a trusted intermediary/adviser to assist in
the decision[1]. This also makes a lot of sense to me.

I had erroneously concluded that the WebExtensions work for Firefox
would address this need. It turns out I had missed the difference
between Google Chrome's *Extensions* APIs at
https://developer.chrome.com/extensions/api_index and the Google Chrome
*Apps* APIs at https://developer.chrome.com/apps/api_index. In an
informal conversation with :bwinton at the Mozlando gathering last week
he indicated that the WebExtensions work is currently not planning to
implement the Google Chrome Apps platform/API.

:bwinton also indicated that likely the best solution for my desktop
email client "app" needs at this time is to use Firefox's legacy-prone
Jetpack/add-on SDK or more traditional legacy chrome.manifest/XULy
manifest. This makes sense since it provides both a) access to the
permissions as well as b) a client-side packaging mechanism that does
not depend on me ensuring that my web-server is never compromised[2].
Since these are both legacy, they are obviously not long-term solutions.

It'd be great if we could pick out a direction for solutions like this.
Here are some mix-and-match options using the specific example of the
TCP scenario of an email app but I believe are applicable to other
scenarios too:

- "protocol driver" (iframe?): Along the lines of the WebUSB discussions
on dev-platform (specifically
https://groups.google.com/d/msg/mozilla.dev.platform/keBpwdYtuGU/g7VyZFwqCwAJ),
do expose our TCPSocket API (and others) to add-ons that must be
reviewed, but require that the add-on be a minimal-ish well-reviewed JS
IMAP/SMTP/etc. protocol implementation that is something that we can
explain to users or that is by its very nature not something they need
to even be told about. If the IMAP driver cannot be tricked into
establishing connection to devices behind your firewall that are not
IMAP servers, maybe there is no risk. Or maybe there is still a risk of
accessing private IMAP servers, in which case the risk can be mitigated
by the driver add-on using a prompt to confirm the user wants to "Allow
'mailapp.example.com' to talk to the email server at
'secret-corporate-email.example.com'. The primary risk to the user in
such a case becomes them revealing their credentials to a bad actor, but
that's entirely separate from the TCP issue.

- "subresource integrity rooted by a signed payload in the web app
manifest and additional CSP constraints if service workers are
involved": The (implemented!) subresource integrity spec at
https://w3c.github.io/webappsec-subresource-integrity/ lets a page say
"use this script that should have this hash". If the root page load
could be required by a signed hash in the site's web app manifest (spec
which allows for extensions at https://w3c.github.io/manifest/ but which
says nothing about signing stuff), this could allow a (fully static) app
to effectively be signed. Once dynamism and/or serviceworkers come into
play, however, CSP needs to be bound into the signature that requires
the app only load script resources from a signed manifest of resource
paths and their hashes.

- "Implement/try and standardized the Chrome Apps stuff/some subset of
it".

More TCP specific:

- "Sockets as a service": Side-step the TCP security issues by running
the TLS in the browser but having an external proxy do the actual TCP
(which could be TOR). Whiteout.io supported something like this for
testing/demo purposes I think. This approach is not without its many
issues.

- "Get a server that doesn't need TCP". Fastmail is trying to get an
IMAP-sucessor JMAP (https://jmap.io/) standardized which could obsolete
the need for TCP. Email clients using standard protocols need not
require raw TCP (w/TLS) access, but today it does. Have email apps
encourage and enable users to pick and migrate to providers supporting
web-friendly JMAP and all the goodness that modernity brings like valid
TLS certificates and push notifications. While in some ways this is not
realistic, in many ways email apps really should be helping users
escape/avoid bad providers and this could potentially help improve the
status quo.

Andrew

1: Under current ecosystems this is accomplished via
per-browser/platform single-vendor add-on/app marketplaces with users
potentially having secondary mechanisms for expressing trust in
apps/add-ons. In general these secondary mechanisms are less than
optimal with there being no easy way for the user to express "I trust
the EFF, I want to use the EFF's trusted marketplace/trust-graph to help
me decide what's safe to use in my browser and help keep me safe in the
event of compromise of the app's signing infrastructure" or "I trust the
developers of this one app and trust them to make a
compromise-attestation via these declared secondary channels if they get
compromised."

2: Specifically, :myk's cool https://github.com/mykmelez/tcpsocketpup
could let me host my https://github.com/asutherland/glodastrophe efforts
that look a lot like
https://clicky.visophyte.org/files/screenshots/20151201-061945.png on
something like https://glodastrophe.visophyte.org/ and have users grant
the required permissions to the origin to be able to use (moz)TCPSocket
to access their email. This works great until the server is
compromised. I can of course pick a more competent webhost than myself
that depends on me authenticating myself/the content using multi-factor
auth (possibly even using usually-offline crypto keys), and maybe that
is also an answer. Arguably using addons.mozilla.org does already
depend on me keeping my AMO credentials safe unless I locally sign my
packages... but I'm not sure how that mixes with the new AMO-signing
stuff...

[Geoff Lankow]

I assume we're planning to implement/already have implemented the W3C version? Could you post bug number(s)?

Also, is there a way for those of us not at Mozlando to see the demo? I'm curious and/or nosy.

GL

_______________________________________________
firefox-dev mailing list
firefox-dev-4eJtQ...@public.gmane.org
https://mail.mozilla.org/listinfo/firefox-dev

[Myk Melez]

Geoff Lankow

2015 December 16 at 00:34

I assume we're planning to implement/already have implemented the W3C version? Could you post bug number(s)?

The platform work to support W3C manifests is meta-bug 997779:

    https://bugzilla.mozilla.org/show_bug.cgi?id=997779

Much of that work is already done. But its utility depends on integration into our products (Firefox, Fennec, FxOS) to give users a better experience of discovering and using web apps.

Some product-specific bugs are dependencies of that meta-bug, like Firefox bug 1186134 and FxOS bug 1003876:

    https://bugzilla.mozilla.org/show_bug.cgi?id=1186134
    https://bugzilla.mozilla.org/show_bug.cgi?id=1003876

Others are dependencies of Fennec meta-bug 1212648:

    https://bugzilla.mozilla.org/show_bug.cgi?id=1212648

Also, is there a way for those of us not at Mozlando to see the demo? I'm curious and/or nosy.

I think the demo in question is Bill Walker's presentation at the closing plenary, which isn't publicly available. cc: Bill to confirm.

-myk

[Bill Walker]

Hello all,

The first thing I'd personally like to see in our for W3C App Manifest is to use the manifest as a source for high quality icons when pinning to the home screen or other similar features. I'd like us to explore the notion of a heuristic that notices when a user frequently visits a site offering a manifest and suggests pinning to the home screen; there are some thorny questions to be answered there.

https://dev.opera.com/blog/installable-web-apps/
https://developers.google.com/web/updates/2015/03/increasing-engagement-with-app-install-banners-in-chrome-for-android?hl=en

-Bill

--

-Bill

[Marco]

The Web Runtime project has been my first serious contribution to Mozilla,
so I'm kind of sad to see it die, but I strongly agree with Myk.

We haven't spent much time in the last few years to improve it, we have never
really finished up the work to integrate it fully in Firefox.

I do hope and I'm confident that we've learned from our mistakes and we will
make the experience with web-web apps (or, progressive web apps :D) better
and well integrated in Firefox (e.g. by making them more discoverable in
the homepage or in the new tab page, by making them pinnable in some way,
by making it easy to find them again in the activity stream, etc.).

- Marco.

[David Bialer]

I'd like to understand the implications for existing apps that are installed.   We are getting 400K apps installed per month on desktop and 25K per month installed on Android.  There are about 3.3M total apps installed on Desktop and 900K on Android.

Will these apps stop working?   Or will they only stop working if they use specific APIs (and exactly which ones)?  Can apps be removed if they are already on a user's machine?

Marketplace has plans to remove submission and updates of apps for Desktop and Android in Q1, and disable the ability to install/open apps from Marketplace.

I put together some options last August for deprecating WebRT here (though perhaps dated): https://docs.google.com/presentation/d/1LG6HFCf0TY0fiweaoM_UDr74W24PhpJQSbwFT6X_9BY/edit#slide=id.p

We decided to 'do nothing' at that point, but now we will need to put some plans in place so that we don't strand uses with apps they can't use and developers with supporting apps they don't wish to.

[Myk Melez]

David Bialer

2015 December 17 at 10:21

I'd like to understand the implications for existing apps that are installed.   We are getting 400K apps installed per month on desktop and 25K per month installed on Android.  There are about 3.3M total apps installed on Desktop and 900K on Android.

Do you have data on the number of those installs that are actively being used (or the number of users that are using any apps)? It'd also be interesting to know the kinds of apps that have been installed, specifically their use of privileged APIs.

The data won't necessarily affect my decision about whether or not to disable the runtime, since no number of users will make the runtime viable without ongoing investment. But it would still be useful to know how much usage we'd break.

Will these apps stop working?   Or will they only stop working if they use specific APIs (and exactly which ones)?

The apps would stop working once the user upgrades Firefox to a version that no longer includes the runtime. They'd start working again if the user then downgrades Firefox, and they'd remain working if the user avoids upgrading Firefox, so there would be a way to work around the change, although it wouldn't be recommended (and would grow increasingly problematic over time).

Can apps be removed if they are already on a user's machine?

We could make Firefox automatically uninstall the apps, but I wouldn't recommend it, as it would be an abrupt and unexpected change. It'd also be complex to implement.

Rather, I'd change the message that apps display when they can't find a runtime (f.e. because the user installs an app and then uninstalls Firefox) to explain the situation and direct the user to more information about it.

I might also add a startup warning dialog to the apps in the version of Firefox *before* the version in which we disable the runtime, explaining that the runtime will be disabled in the next version, to give users time to prepare for it.

Marketplace has plans to remove submission and updates of apps for Desktop and Android in Q1, and disable the ability to install/open apps from Marketplace.

Woot! Do you know when Marketplace is scheduled to disable the ability to install/open apps? Is that also in Q1, or will it happen later?

I put together some options last August for deprecating WebRT here (though perhaps dated): https://docs.google.com/presentation/d/1LG6HFCf0TY0fiweaoM_UDr74W24PhpJQSbwFT6X_9BY/edit#slide=id.p

Thanks, this is very useful! And a good reminder that disabling the runtime would entrain other work, in Marketplace, MDN, etc. I'll work with you and other folks to coordinate the various dependencies.

We decided to 'do nothing' at that point, but now we will need to put some plans in place so that we don't strand uses with apps they can't use and developers with supporting apps they don't wish to.

Indeed. Thanks for engaging. If I decide to go ahead with disabling the runtime, I know it'll create work for you and the Marketplace team. I appreciate your help with figuring out the best way to make it happen!

-myk

[Robert Kaiser]

Nicholas Alexander schrieb:

> The ability to bookmark to the home screen will remain in Fennec. Such
> bookmarks are the intended entry point for whatever Progressive Web App
> features Fennec implements, although I don't know all the details.

Doesn't sound to me like it would be the same. Right now, I'm running a
("hosted") web "app" quite a lot on my Android tablet because I can use
web technologies to run something that starts decently fast and launches
directly into full screen and acts like a full application, without the
browser chrome that is pretty useless in that one. If that doesn't work
any more, I may switch to non-wen technology instead to get the same
effect. That doesn't sound like it is where Mozilla wants to go. OTOH, I
understand the arguments esp. around low usage and missing maintenance.

KaiRo

[Myk Melez]

Robert Kaiser

2015 December 28 at 12:32

Doesn't sound to me like it would be the same. Right now, I'm running a ("hosted") web "app" quite a lot on my Android tablet because I can use web technologies to run something that starts decently fast and launches directly into full screen and acts like a full application, without the browser chrome that is pretty useless in that one.

Homescreen bookmarks should launch as fast as open web apps (if not faster). If they don't, then that's a bug we should fix.

As for opening without browser chrome, that's bug 1126479:

    https://bugzilla.mozilla.org/show_bug.cgi?id=1126479

See meta-bug 1212648 for other issues regarding support for progressive apps in Fennec:

    https://bugzilla.mozilla.org/show_bug.cgi?id=1212648

-myk

[Myk Melez]

Myk Melez

2015 December 14 at 15:45

I think we should disable the desktop and Android Web Runtimes in Firefox and Fennec.

Thanks to all of the participants in this conversation for your careful consideration and thoughtful responses!

After reviewing the responses to this proposal over the last few weeks, I've concluded that we should indeed disable the desktop and Android runtimes, as they aren't viable without substantial additional investment in their development and maintenance, and there's no such investment forthcoming.

So I filed bug 1238079 to disable the desktop runtime:

    https://bugzilla.mozilla.org/show_bug.cgi?id=1238079

And Margaret filed bug 1235869 to disable the Android runtime:

    https://bugzilla.mozilla.org/show_bug.cgi?id=1235869

As I noted earlier, this work will entrain a variety of dependencies, especially on Marketplace and MDN, which need to stop offering/documenting desktop and Android as a deployment target for Open Web Apps. So there'll be additional bugs to track other work. But they'll be dependencies of the two aforementioned ones, so those are the ones to follow if you want to track all the work to implement this proposal.

-myk

[Robert Kaiser]

Myk Melez schrieb:
>> Robert Kaiser <mailto:ka...@kairo.at>

Thanks, I hope we can make "progressive web apps" (or actually websites
that work just like mobile apps or even desktop applications) run with a
good experience on all form factors that Firefox is available.

If we can do that in a more standardized way and more in line with the
web and Firefox development than previously, that's a good thing for sure.