Proposal: Cross-origin Service Workers

[Benjamin Francis]

What if a Service Worker could allow a web site/web app to expose a web
service to other apps, even while offline?

Currently we have several not very web-like proprietary JavaScript APIs in
Gecko for sharing data between apps in Firefox OS, like the Contacts API,
DataStore API, Settings API and Device Storage API.

It seems that a more webby alternative for these use cases could be for a
hosted web service (e.g. tied to your Firefox Account) to handle the
storage of data like contacts and photos, accessed by web apps via a REST
API. Currently the problem with that is that REST APIs are not available
while offline.

The misleadingly named "navigator.connect" API [1] being proposed by Google
for offline cross-origin communication via Service Workers actually
includes not only cross-origin postMessage, but cross-origin handling of
onfetch events. The example use cases given include a font service which
could provide fonts to multiple origins while offline using a Service
Worker. I wonder whether this could be extended to allow JSON to be
fetched from another origin over XHR/fetch while offline?

Imagine a gallery app hosted at gallery.firefoxos.com which stores your
photos online, but caches them locally as blobs in IndexedDB. A camera app
at camera.firefoxos.com could use a REST API at gallery.firefoxos.com to
save photos to your gallery, and to get thumbnails of recently taken
photos. If gallery.firefoxos.com had a Service Worker registered to handle
onfetch events originating in a fetch request from camera.firefoxos.com
then this could also work offline.

A web page at dialer.firefoxos.com could send an HTTP request:

GET http://contacts.firefoxos.com/{contactId}
Accept: application/json

which is intercepted by a Service Worker which provides a response even
when offline.

Being "cacheable" and a "layered system" where "the client can not tell
whether it is connected directly to the end server or an intermediary along
the way" are key principles of a RESTful architecture [2]. Service Workers
just make this far more powerful by making the cache programmable!

Could this even allow us to expose APIs using common standards on the web
like CalDAV, CardDAV and OpenSearch while offline? contacts.firefoxos.com
could provide an OpenSearch API for the Rocketbar to search the user's
contacts for example, as could any other app.

CORS allows web sites to provide cross-origin web services over HTTP, how
could this work with Service Workers?

I've filed an issue on GitHub [3] for the navigation.connect specification
to propose this as a potential use case, and I'm interested to know what
you think of this idea?

Ben

1. https://github.com/mkruisselbrink/navigator-connect
2. http://en.wikipedia.org/wiki/Representational_state_transfer
3. https://github.com/mkruisselbrink/navigator-connect/issues/23
<http://en.wikipedia.org/wiki/Representational_state_transfer>

[Dale Harvey]

Initial reaction is this would be a great idea, I have seen people do a
similiar thing by using emscriptem to compiles nodes http library and
patching it to use chromes tcp server API, this let them expose PouchDB as
a HTTP server to web content running in the browser (that works offline)

This was obviously a very hacky solution that wasnt widely usable, however
it seems like this would be a way to make it viable

> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi
>

[Anders Rundgren]

You mean that a service worker would listen to localhost? According to Google this concept is dangerous and they are trying to constrain it

[Benjamin Francis]

Taking inspiration from the fonts example at
https://github.com/mkruisselbrink/navigator-connect/blob/gh-pages/explainer.md

First, you need to overcome the bootstrapping problem of registering the
cross-origin Service Worker (this could probably all be done artificially
for pre-installed apps).

A request from dialer.firefox.com to contacts.firefox.com...

GET https://contacts.firefox.com/contacts/123
Accept: application/json
[...]

Could return the response...

GET https://contacts.firefox.com/contacts/123
Status:200 OK
Content-length:385
Content-type: application/json
Service-Worker-Scope: /
Service-Worker-Script: /js/service_worker.js
[...]

Which would trigger the fetching of /js/service_worker.js which would
return the response...

GET https://contacts.firefox.com/js/service_worker.js
Status:200 OK
Content-Encoding:gzip
Content-Length:971
Content-Type:text/javascript; charset=utf-8
Service-Worker-Allowed: /
[...]

Which registers a Service Worker "/js/service_worker.js" with the scope "/"
as if contacts.firefox.com itself registered it.

>From that point on, the contacts service worker could intercept and respond
to HTTP requests made from dialer.firefox.com to contacts.firefox.com, only
once dialer.firefox.com's service worker has let the request go to the
network.

service_worker.js from contacts.firefox.com might handle these "fall
through requests" as follows...

self.addEventListener('install', function(e) {
e.handleFallThroughRequests(['/']);
});

self.addEventListener('fetch', function(e) {
if (e.isFallThrough) {
}
});

What I'm not sure about is how these requests could be authenticated on the
client side for a given user? Could any existing authorisation mechanisms
on the web be re-used locally somehow in the context of Service Workers?

On 1 April 2015 at 17:00, Dale Harvey <da...@arandomurl.com> wrote:

> Initial reaction is this would be a great idea, I have seen people do a
> similiar thing by using emscriptem to compiles nodes http library and
> patching it to use chromes tcp server API, this let them expose PouchDB as
> a HTTP server to web content running in the browser (that works offline)
>
> This was obviously a very hacky solution that wasnt widely usable, however
> it seems like this would be a way to make it viable
>
>
>

[Dave Huseby]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 04/01/2015 08:02 AM, Benjamin Francis wrote:
> What if a Service Worker could allow a web site/web app to expose a
> web service to other apps, even while offline?

I really like this idea. It would solve the biggest remaining issue
for the crypto-ish hardware framework. Right now, one of the main
requirements of the crypto-ish hardware framework is that Mozilla will
*NOT* be authoring the "driver" code for each piece of crypto-ish
hardware.

With this cross-origin service workers, I think it would be possible
for Yubico developers to author a Yubikey app that would implement a
cross-origin service worker that other apps can use to access the
functions of a Yubikey. This puts the support burden on the
manufacturers.

TL;DR

What is crypto-ish hardware? Two-factor auth hardware, hardware
bitcoin wallets, smart cards, secure elements, entropy sources, etc.
The hope for the framework is to make all of these bits of hardware
available to web apps. Until now, there was no idea how to avoid
having Mozilla engineers writing driver code into gecko.

- --dave
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCAAGBQJVIwllAAoJEJ7v31qiCP4gZ/4P/2pTzLZe/4czfTSLf7Q9xqNV
zCU4wZQVSkZ/sj/yaEPFLpFh6PJGLAW7Ej/7+t7TZxWMWLl4RydX0rkKi+qhnpY1
+mXE1VpGebSAQ35Ar3linMyn5acsEzUv2bxwfk45xlD3+c/bjYEGSCBmalapFR2q
OCO8SKGvLqeIBKAOCfj5B5Uzhybzsrsqs9DpodVpBnQHqwJtfhP1BUn0eb8BkCjC
dC92i9Me/MXcCuyrR+Iim1tPkCETKSzOCMtVx9c+pnhiPCY772TsEWnU6/EvG3mh
2v7vyn7UliZA7uWtWwPKt18qXJPtPYol2WWeAl4CYYVNEr3VxK8frXLWsHRT4JEA
jXpAfbtpW5R7GFcnFz8BEx8CmJAsiuQvZyNlPXRk2Xm4nH1QmTTjNBtEbxLSrkTY
YfZGTePNeXTup2jLUkP5tormuoZ65wjBB58osZS/pvkFLSkvQdZUSletgz+u/+t2
vH2Ij0V3yuxc1Bgts943GKtUCc32AANxs+AEKD/jALqxsQ+RGefh/Wii2NS68Z7e
OaSrB5K9bCOeKKVRE9KiOBCQIfqVqy/C2FV1noBc28tebKSQ4Pupc/urbt9A38xa
RcmrnpE6CFDjKDSLgGlXUOxV+4KRMq5OJP+aSE4DXllrUn2wWtt14dZQmnqQ9YVa
0KPOCkLktEzXVx6S2zx5
=4wPu
-----END PGP SIGNATURE-----

[Ehsan Akhgari]

On 2015-04-02 5:54 AM, Benjamin Francis wrote:
> What I'm not sure about is how these requests could be authenticated on the
> client side for a given user? Could any existing authorisation mechanisms
> on the web be re-used locally somehow in the context of Service Workers?

If the authentication is handled by the service worker itself (which
could mean it would ask some remote server) and is tied to something
like a client-side cookie, then the SW can deal with the authentication
similarly to how a server would, but auth models that rely on the user
interaction such as OAuth obviously doesn't work here...

[Martin Thomson]

On Wed, Apr 8, 2015 at 6:38 AM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> If the authentication is handled by the service worker itself (which could
> mean it would ask some remote server) and is tied to something like a
> client-side cookie, then the SW can deal with the authentication similarly
> to how a server would, but auth models that rely on the user interaction
> such as OAuth obviously doesn't work here...

I don't see why not. As long as the SW had sufficient information to
generate the tokens. That said, if there are any secrets involved in
generating the tokens (as there probably should be), you don't want to
be shipping them out to browsers. I can imagine some narrow cases
where the SW could authorize access to other resources with some
restrictions.

[Ehsan Akhgari]

Hmm, actually yes, if the SW can talk to an external server to get the
token from then this would work, but of course that doesn't work offline.

[Martin Thomson]

On Wed, Apr 1, 2015 at 8:02 AM, Benjamin Francis <bfra...@mozilla.com> wrote:
> What if a Service Worker could allow a web site/web app to expose a web
> service to other apps, even while offline?

I've been thinking about this on and off. And this relates back to
Ehsan's earlier suggestion [1] regarding navigator.connect.

There are several separate concerns to address:

1. Opting in to having requests intercepted by a SW. It seems like
that is part of the SW design. Personally, I think I'd be OK with the
SW were always present once installed.

2. Installing the SW. Obviously, this is straightforward for
same-origin code, but it doesn't work well cross-origin. The target
origin needs to be able to run code or provide some information about
what JS to run. navigator.connect seems to propose HTTP header fields
for this purpose. It also mentions having some way to frame in the
target origin so that it could run code, but notes how clumsy that is
(and I agree). I could see this working well with /.well-known/ too.

3. The concern Alex Russell raised regarding multiple layers of
routing: if the origin making the request gets a chance to intercept
with its SW as well as the origin that serves the request, then there
are two SWs involved, potentially two lots of cache. He was concerned
about memory and a few other things too. I don't share that concern,
FWIW.

There was a call about this topic apparently, but I can't find
anywhere that the results of the discussion were posted.


[1] https://groups.google.com/d/msg/mozilla.dev.platform/-cNLULs08As/XXbINlBiEYUJ

[Ehsan Akhgari]

I believe there was a face to face meeting in San Francisco about this,
but I don't know where the discussion results was posted if anywhere (I
wasn't attending.)

FWIW I still think that we should explore cross origin service workers
before looking to implement navigator.connect, since I still think that
the use cases for navigator.connect are a (less important) subset of the
use cases for cross origin service workers.

[Martin Thomson]

On Thu, Apr 9, 2015 at 3:32 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> FWIW I still think that we should explore cross origin service workers
> before looking to implement navigator.connect, since I still think that the
> use cases for navigator.connect are a (less important) subset of the use
> cases for cross origin service workers.

I agree. I definitely prefer the model you've described.

The question of how a site opts in (or whether there is an option)
seems secondary to that question.

[Benjamin Francis]

On 9 April 2015 at 23:47, Martin Thomson <m...@mozilla.com> wrote:

> On Thu, Apr 9, 2015 at 3:32 PM, Ehsan Akhgari <ehsan....@gmail.com>
> wrote:
> > FWIW I still think that we should explore cross origin service workers
> > before looking to implement navigator.connect, since I still think that
> the
> > use cases for navigator.connect are a (less important) subset of the use
> > cases for cross origin service workers.
>
> I agree.
>

+1

What's the next step?

[Ehsan Akhgari]

On 2015-04-09 6:47 PM, Martin Thomson wrote:
> On Thu, Apr 9, 2015 at 3:32 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
>> FWIW I still think that we should explore cross origin service workers
>> before looking to implement navigator.connect, since I still think that the
>> use cases for navigator.connect are a (less important) subset of the use
>> cases for cross origin service workers.
>

> I agree. I definitely prefer the model you've described.
>
> The question of how a site opts in (or whether there is an option)
> seems secondary to that question.

I think the model described in the current navigator.connect explainer
based on HTTP headers is actually not terrible. It doesn't solve the
issue of the first time request, but perhaps that's OK for now. I think
we can look into ways of opting in in advance of the first request for
the site once this feature has shipped and has proven popular among Web
developers.

Jonas, Anne, do you have any opinions on this?

[Martin Thomson]

On Thu, Apr 9, 2015 at 9:14 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> I think the model described in the current navigator.connect explainer based
> on HTTP headers is actually not terrible.

I agree. From a protocol perspective, I'd prefer to see this manifest
as a link relation, but that's window dressing. I don't see any easy
way to ensure that a SW is installed in a timely fashion without
actually talking to the site first.

I think that as a practical matter, we will see service workers in
this context provide a signal to the origin server that they are in
place, so that the server can provide a different response. For
instance, the server might choose to send an error if the SW isn't in
place. (I can't see servers using Vary properly there, but one can
live in hope...Cache-Control: max-age=0 is probably OK.)

[Ehsan Akhgari]

On 2015-04-10 12:16 PM, Martin Thomson wrote:
> On Thu, Apr 9, 2015 at 9:14 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
>> I think the model described in the current navigator.connect explainer based
>> on HTTP headers is actually not terrible.
>
>
> I agree. From a protocol perspective, I'd prefer to see this manifest
> as a link relation, but that's window dressing. I don't see any easy
> way to ensure that a SW is installed in a timely fashion without
> actually talking to the site first.

Right.

> I think that as a practical matter, we will see service workers in
> this context provide a signal to the origin server that they are in
> place, so that the server can provide a different response. For
> instance, the server might choose to send an error if the SW isn't in
> place. (I can't see servers using Vary properly there, but one can
> live in hope...Cache-Control: max-age=0 is probably OK.)

Hmm, I'm not sure if I understand this part. What do you mean by the
"origin server"?

The model I had in mind was something like this: mysite.com wants to
invoke an API from coolrestapi.com (for example, using XHR). If
coolrestapi.com has a service worker installed, then the SW can
intercept the XHR and respond to it however it chooses to (either by
actually attempting to reach out to coolrestapi.com, or synthesizing the
response offline without ever letting the request to get to the
network.) If coolrestapi.com doesn't have a SW installed, the request
goes to the network as normal, and through a response header,
coolrestapi.com can instruct the browser to install its SW at the right
scope to make the interception work the next time the given Rest API is
called into.

Is this what you have in mind as well?

[Martin Thomson]

On Fri, Apr 10, 2015 at 10:46 AM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> The model I had in mind was something like this: mysite.com wants to invoke
> an API from coolrestapi.com (for example, using XHR). If coolrestapi.com
> has a service worker installed, then the SW can intercept the XHR and
> respond to it however it chooses to (either by actually attempting to reach
> out to coolrestapi.com, or synthesizing the response offline without ever
> letting the request to get to the network.) If coolrestapi.com doesn't have
> a SW installed, the request goes to the network as normal, and through a
> response header, coolrestapi.com can instruct the browser to install its SW
> at the right scope to make the interception work the next time the given
> Rest API is called into.
>
> Is this what you have in mind as well?

Yep. Exactly.

This is less important, but do you see any reason to allow mysite.com
to opt out of intercept by the coolapi.com SW? Or for the coolapi.com
SW to opt out of interception for other origins? I tend to think that
we should just route the request to the SW in all cases, just because
that is simpler.

I ask because it seems that the SW spec seems to imply that an origin
is required to opt in to SW use. This would be a divergence from that
policy.

BTW, "origin server" is a term defined in the HTTP RFC as "the server
that is authoritative for a resource" or something like that.

[Ehsan Akhgari]

On 2015-04-10 1:56 PM, Martin Thomson wrote:
> On Fri, Apr 10, 2015 at 10:46 AM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
>> The model I had in mind was something like this: mysite.com wants to invoke
>> an API from coolrestapi.com (for example, using XHR). If coolrestapi.com
>> has a service worker installed, then the SW can intercept the XHR and
>> respond to it however it chooses to (either by actually attempting to reach
>> out to coolrestapi.com, or synthesizing the response offline without ever
>> letting the request to get to the network.) If coolrestapi.com doesn't have
>> a SW installed, the request goes to the network as normal, and through a
>> response header, coolrestapi.com can instruct the browser to install its SW
>> at the right scope to make the interception work the next time the given
>> Rest API is called into.
>>
>> Is this what you have in mind as well?
>
> Yep. Exactly.
>
> This is less important, but do you see any reason to allow mysite.com
> to opt out of intercept by the coolapi.com SW? Or for the coolapi.com
> SW to opt out of interception for other origins? I tend to think that
> we should just route the request to the SW in all cases, just because
> that is simpler.

I agree, I can't think of any use case which would need either of the
above opt-outs.

> I ask because it seems that the SW spec seems to imply that an origin
> is required to opt in to SW use. This would be a divergence from that
> policy.

Well, right now in the spec, the fetch to be intercepted needs to be
associated with a controlled document. We do need to lift that
restriction for allowing SWs to intercept cross origin.

> BTW, "origin server" is a term defined in the HTTP RFC as "the server
> that is authoritative for a resource" or something like that.

I see, thanks. :-)

[Benjamin Francis]

On 9 April 2015 at 23:32, Ehsan Akhgari <ehsan....@gmail.com> wrote:

> I believe there was a face to face meeting in San Francisco about this,
> but I don't know where the discussion results was posted if anywhere (I
> wasn't attending.)
>

> FWIW I still think that we should explore cross origin service workers
> before looking to implement navigator.connect, since I still think that the
> use cases for navigator.connect are a (less important) subset of the use
> cases for cross origin service workers.
>

It looks like a cross-origin Service Workers proposal might get split out
of navigator.connect into the Service Workers spec itself
https://github.com/mkruisselbrink/navigator-connect/issues/28

[Jonas Sicking]

On Thu, Apr 9, 2015 at 9:14 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:

> On 2015-04-09 6:47 PM, Martin Thomson wrote:

>>
>> On Thu, Apr 9, 2015 at 3:32 PM, Ehsan Akhgari <ehsan....@gmail.com>
>> wrote:
>>>
>>> FWIW I still think that we should explore cross origin service workers
>>> before looking to implement navigator.connect, since I still think that
>>> the
>>> use cases for navigator.connect are a (less important) subset of the use
>>> cases for cross origin service workers.
>>
>>

>> I agree. I definitely prefer the model you've described.
>>
>> The question of how a site opts in (or whether there is an option)
>> seems secondary to that question.
>
>

> I think the model described in the current navigator.connect explainer based

> on HTTP headers is actually not terrible. It doesn't solve the issue of the
> first time request, but perhaps that's OK for now. I think we can look into
> ways of opting in in advance of the first request for the site once this
> feature has shipped and has proven popular among Web developers.
>
> Jonas, Anne, do you have any opinions on this?

First off, I agree that using a cross-origin service worker handling
network requests is a great way to enable offlining cross-site
communication. So definitely something that we should do.

A nice property of that solution is that it falls back nicely to
letting the server handle the request if the SW is not installed.
Though of course that then requires the user to be online.

Another nice property is that it doesn't require any changes to
initiating website.

Regarding navigator.connect(), I don't think that it's proposal works
at all right now. I.e. I don't think we can hook up
navigator.connect() to SWs, as long as SWs require installation before
they can be used. Having an cross-origin messaging system which stalls
for tens of seconds the first time it's used is not a good idea.

I think the best solution here is to either make navigator.connect()
not use SWs, or to enable SWs to run without being installed first.

/ Jonas

[Anders Rundgren]

On Tuesday, April 7, 2015 at 12:32:18 AM UTC+2, Dave Huseby wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 04/01/2015 08:02 AM, Benjamin Francis wrote:
> > What if a Service Worker could allow a web site/web app to expose a
> > web service to other apps, even while offline?
>
> I really like this idea. It would solve the biggest remaining issue
> for the crypto-ish hardware framework. Right now, one of the main
> requirements of the crypto-ish hardware framework is that Mozilla will
> *NOT* be authoring the "driver" code for each piece of crypto-ish
> hardware.

It does not solve the crypto-ish hardware issue that is the reason why Secure/Convenient/Decentralized web-payments is in the same state as 20 years ago.

> With this cross-origin service workers, I think it would be possible
> for Yubico developers to author a Yubikey app that would implement a
> cross-origin service worker that other apps can use to access the
> functions of a Yubikey. This puts the support burden on the
> manufacturers.

One strong point with U2F is that a bit-level standard makes a built-in driver feasible which is not the case for other security hardware stuff out there.

[Benjamin Francis]

I've filed a bug for this.

Cross-origin Service Workers
https://bugzilla.mozilla.org/show_bug.cgi?id=1169621