Intent to Implement: Background Fetch API

[Jennifer]

Contact emails

hark...@chromium.org, aw...@chromium.org, jakear...@chromium.org, pe...@chromium.org

Spec

https://github.com/jakearchibald/background-fetch

Design doc

Summary

Background Fetch API provides a service worker based download and upload mechanism which is persistent across service worker and browser restarts.

The system should provide a developer-controllable interface to schedule, query, pause, and cancel background downloads and uploads. After a scheduled fetch completes, the system will notify the developer of the outcome of the operation by sending an event to the service worker.

In progress downloads and uploads will be handled by the download manager and will be visible to users in a notification. The user will be able to pause and cancel actions from that interface.

Downloads and uploads should continue even if the developer's website is not loaded in a tab, and if Chrome is closed, the download should resume when Chrome opens again.

Motivation

If a developer wants to download a file, they can initiate the download or upload via fetch or XHR, but those fetches will be canceled if the user navigates away from the tab or the service worker is shut down.  Many developers have larger fetches that are needed for applications such as audio, video, photos, game levels, map packs, etc. which can’t be reliably fetched in the time available to a tab or service worker.

This API gives those developers the ability to queue up a fetch and then be evicted from memory without losing progress on the action. The fetch will continue while the app is not resident, and when the fetch is complete the service worker will be woken up and the app will be able to process the result.

Interoperability and Compatibility Risk

Standards discussion has already begun with web developers in the spec linked above. Additionally, we plan to release the feature using Origin Trials to get additional feedback before launching.

Edge: No signals

Firefox: No signals

Safari: No signals

Web developers: Positive

Ongoing technical constraints

None

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes, except WebView. Investigations are pending as to whether it is applicable in WebView. We expect that mobile platforms will be the primary users.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=684578

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5712608971718656

Requesting approval to ship?

No.

[rsl...@chromium.org]

On Tuesday, January 24, 2017 at 10:12:47 AM UTC-8, Jennifer wrote:

Contact emails

hark...@chromium.org, aw...@chromium.org, jakear...@chromium.org, pe...@chromium.org

Spec

https://github.com/jakearchibald/background-fetch

Design doc

I realize this is an I2I, and so the barriers are very low (especially without shipping), but I am concerned at least with some of the goals of the spec outlined, because they do seem somewhat incompatible or concerning from a Web Platform/interop side.

In particular, all of the goals of "Allow the OS to..." seem problematic in terms of being able to explain the behaviour in terms of Fetch, which allows the loading behaviour to be predictable for both script authors and server operators. I'm not aware of any OS that adheres, either implicitly through current behaviour or explicitly through specified API contract, to the behaviours of Fetch, and that seems to open a potential can of interoperability issues.

Consider a hypothetical scenario of a large download being initiated, handed off to the OS (as captured in the explainer), and the user pauses the download. Later, they attempt to resume (by issuing a range request to the server). What happens if the server doesn't support range requests? What happens if the server issues a redirect - does the OS handle it, or does the SW? If the OS handles it, does it observe any CORS behaviours with respect to that redirect?

I highlight this issue now because it seems to go to the heart of the integration/implementation - if we don't expect the OS to adhere to Fetch behaviours, then it seems unlikely we'd ever be able to reasonably ship this without causing/introduce compatability issues. And if we don't expect to be able to leverage the OS integration, does that affect our excitement (as implementors) about the value proposition of this, either for implementations or for authors?

I'm not trying to suggest this isn't a valuable primitive for the platform, nor do I disagree that if we can hand to the OS, and allow the browser to terminate, we might be able to greatly improve the battery scenario on mobile. However, it seems a dangerous coupling for the Web Platform to be extending it's "network implementation" to the OS, because OS's don't have the same platform predictability or interoperability concerns that the Web Platform does, and so it seems like we need to anticipate and design for this up front in the implementation, otherwise, we'll never be in a position to reliably ship.

[Jake Archibald]

On Tue, 24 Jan 2017 at 18:46 <rsl...@chromium.org> wrote:

On Tuesday, January 24, 2017 at 10:12:47 AM UTC-8, Jennifer wrote:

Contact emails

hark...@chromium.org, aw...@chromium.org, jakear...@chromium.org, pe...@chromium.org

Spec

https://github.com/jakearchibald/background-fetch

Design doc

Consider a hypothetical scenario of a large download being initiated, handed off to the OS (as captured in the explainer), and the user pauses the download. Later, they attempt to resume (by issuing a range request to the server). What happens if the server doesn't support range requests? What happens if the server issues a redirect - does the OS handle it, or does the SW? If the OS handles it, does it observe any CORS behaviours with respect to that redirect?

In the initial implementation Chrome will handle the download. In future it could be handed off to a smaller browser process, or OS, as long as it's spec compliant. I want the spec to allow for optimisations like this.

Range requests are a bigger issue, as they aren't covered by HTML specs at all. I've done a little work around this (https://github.com/w3c/ServiceWorker/issues/703 & https://github.com/whatwg/fetch/issues/144) but there's a much bigger task around defining the kind of ranged-responses that are acceptable to particular ranged-requests, and when ranged-requests can be made.

[Ben Kelly]

On Tue, Jan 24, 2017 at 1:12 PM, Jennifer <hark...@chromium.org> wrote:

Firefox: No signals

FWIW, I think we are positive on the overall effort here.  Jake has already incorporated some feedback from us into his proposal.  For logistical reasons we probably will not implement for a while.

Note, it would be great to have some WPT tests as part of this implementation to clarify any ambiguity in the spec.

Thanks.

Ben

[Ryan Sleevi]

On Tue, Jan 24, 2017 at 11:00 AM, Jake Archibald <jakear...@google.com> wrote:

In the initial implementation Chrome will handle the download.

I don't know how well the Chrome DownloadManager tracks Fetch integration. I don't think it has paid as close attention for things like the resume case (which, AIUI, was a more recent addition). The frame-attached path (ResourceDispatcherHostImpl-enabled path) should be, for the most part (nb: haven't checked handling of things like redirects), but the frame-unattached path may represent a problem.

Since this makes that behaviour 'more' web-observable (to authors, not just site operators as it is today), that's probably something that should be included in the implementation plan to avoid issues with I2S, and since it's an implementation issue, doesn't block I2I.

 

In future it could be handed off to a smaller browser process, or OS, as long as it's spec compliant. I want the spec to allow for optimisations like this.

Right. I think it's unlikely we'd ever see an OS spec compliant, least of all in part because Fetch is a Living Standard and that's anathema to all of the OSes in which we currently ship and to our shipping model of the Web Platform. So even if the spec allows for such optimizations, I'm not optimistic we can make it.

That said, I'm encouraged to hear you acknowledge the spec-compliance dependency as a blocker to the OS integration (which didn't seem to be called out in either the explainer or the design doc), and just want to encourage that, without tests, this will be tremendously easy to regress/diverge.

 

Range requests are a bigger issue, as they aren't covered by HTML specs at all. I've done a little work around this (https://github.com/w3c/ServiceWorker/issues/703 & https://github.com/whatwg/fetch/issues/144) but there's a much bigger task around defining the kind of ranged-responses that are acceptable to particular ranged-requests, and when ranged-requests can be made.

Yup. And that's part of the compat-concerns with using the ChromeDownloadManager / downloads subsystem - because they perform Range requests, it means we've got an unspec'd behaviour that would (by this) be exposed to authors.

Again, not a blocker to I2I, but I wanted to make sure to call it out in the event if affects the implementation design and integration, because it seems like something that would affect the Intent to Ship if we left unresolved. I'm glad to hear it's on your radar, though.

[d...@google.com]

I took a look at this as part of last week's Intent to... triage rotation.  My notes / feedback:

Assuming XSS on a domain, an attacker could queue up downloading that happens on behalf of the victim domain, in the background, even after the user has closed the browser.  Though if UI is always shown then this is probably not all that interesting.

There are a handful of things that likely just need a bit of clarification in the spec:

• What happens with downloads initiated from incognito mode?  Should they terminate immediately once the incognito session is over?

• The spec doesn’t mention credentials.  It appears that backgroundFetch.fetch() needs to be called on a request object that has been set up beforehand.  I can imagine that an implementation would likely punt the request to an external downloading process that is longer-lived than the browser itself.

• On the desktop, does the download stop if the user closes the notification?  As spec’d, it shouldn’t be possible to close the notification and have the download persist in the background.

• What happens if you clear browsing data while a download is in progress?

• How is the user aware of ongoing downloads on a mobile browser?

• What’s the scope of a “tag”?  (Per-origin?  What about protocol scheme?)  I would expect the tag mechanism to obey the SOP.

• Is CORS respected?

• Is this mechanism regulated appropriately by CSP?  (E.g.: connect-src)

[Jake Archibald]

On Mon, 30 Jan 2017 at 15:49 <d...@google.com> wrote:

Assuming XSS on a domain, an attacker could queue up downloading that happens on behalf of the victim domain, in the background, even after the user has closed the browser.  Though if UI is always shown then this is probably not all that interesting.

Yeah, this is already possible without background fetch. http://output.jsbin.com/fimukur/quiet triggers a 1gb download which continues after the tab has closed.

• What happens with downloads initiated from incognito mode?  Should they terminate immediately once the incognito session is over?

Filed https://github.com/jakearchibald/background-fetch/issues/17 - we can cover this by saying what happens if the related service worker registration is unregistered.

• The spec doesn’t mention credentials.  It appears that backgroundFetch.fetch() needs to be called on a request object that has been set up beforehand.  I can imagine that an implementation would likely punt the request to an external downloading process that is longer-lived than the browser itself.

Credentials are opt-in with fetch, and background fetch should behave the same way. I've filed https://github.com/jakearchibald/background-fetch/issues/18 to think about what should happen if the download is performed in mutliples ranges and cookies change.

• On the desktop, does the download stop if the user closes the notification?  As spec’d, it shouldn’t be possible to close the notification and have the download persist in the background.

At the very least it mustn't be possible to lose the ability to abort the fetch.

• What happens if you clear browsing data while a download is in progress?

This will also be covered by https://github.com/jakearchibald/background-fetch/issues/17 

• How is the user aware of ongoing downloads on a mobile browser?

I think the plan is for the notification to be sticky. 

• What’s the scope of a “tag”?  (Per-origin?  What about protocol scheme?)  I would expect the tag mechanism to obey the SOP.

Per service worker registration. There can be many of those per origin.

• Is CORS respected?

Yes. You can make a no-cors request which will get you a response that you cannot inspect, but can later use in APIs that accept no-cors responses, such as <img>.

Basically, no different to how fetch() works today. 

• Is this mechanism regulated appropriately by CSP?  (E.g.: connect-src)

Filed https://github.com/jakearchibald/background-fetch/issues/19 - background fetch should be bound by the same rules as fetch().

Thanks for the review, many good points!

[David Ross]

Ok, that all looks good.  Thank you for filing those issues.

Dave

[Rick Byers]

Given the support from Mozilla, please get this moved to a WICG repo ASAP to ensure everyone who may want to provide feedback is able.

I'm excited to see us fleshing out the details here and working towards design consensus :-)

[Jake Archibald]

Yes, I've been a bit lazy there. I'm aiming to do this and a second-pass on the explainer (incorporating feedback) next week.