Intent to Ship: readable streams in Fetch API

[Kenji Baheux]

Contact emails

Engineering: yhi...@chromium.org, tyos...@chromium.org, ho...@chromium.org

PM: kenji...@chromium.org

Spec

https://streams.spec.whatwg.org/ (editor: dom...@chromium.org)

https://fetch.spec.whatwg.org/#fetch-api (note: preparation for merging of this patched version of the spec)

Summary

This is a follow-up to shipping the fetch API on the global scope (see this LGTMed intent to ship; Chrome 42). This intent to ship is about a first cut at integrating the Streams API with the Fetch API. Concretely, we are making readable streams available to the Fetch API through Response.body. A readable stream allows for a saner way to process data in a progressive manner.

Comparison between XHR and Fetch x readable streams:

XHR	Fetch x ReadableStream
Can “kind-of stream” and then only text content	Can stream binary data and text content
The whole response needs to be buffered which dramatically increases memory usage. This makes some use cases impossible to achieve on mobile devices where memory constraints are more severe.	Only the unread bits are buffered which dramatically reduces memory usage. This makes memory constraints on mobile devices a distant concern.
Since streaming was not an officially recognized use case, browsers have different strategies for buffering received data: some expose data to the consumer immediately others buffer small responses and wait for a larger chunk to complete before exposing to the consumer Also, historically browsers have had different restrictions on which content-types can be be read incrementally: some allow "text/html, others only allow "application/x-javascript."	The Streams API is aptly named (well defined behavior and relevant APIs for streaming use cases).
Can’t stream “infinite” source of data as one would quickly hit the memory wall.	Can stream infinite source of content (e.g. web radio).

Use cases

• Streaming video from a single video file without the need for plugins, or server-side endpoints to retrieve small chunks; can just keep the single HTTP connection open and read as desired.

• Parsing of a web radio stream for metadata (e.g. title of the song).

• Stream a complete set of game levels, show a progress bar and let the user start playing as soon as the first level is decoded.

• A server could forward the binary data created by one user to an open response stream for another user, e.g. video game streaming or music broadcast

• Allows manual progressive decoding/presentation of any binary format (e.g. progressive textures in a game)

    API level detail (includes known caveats)

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/cCPJTZbnjCw/9vFx8DHjjAEJ

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Demo

Pre-requisite: Chrome Canary with chrome://flags/#enable-experimental-web-platform-features enabled.

Description: find the position of a particular sequence within the first billion digits of PI without consuming a billion bytes of memory!

Link: https://domenic.github.io/streams-demo/

Debuggability

On par with XMLHttpRequest.

Compatibility Risk: pluses and minuses

+ There is a WHATWG spec for the Fetch API and a WHATWG spec for the Streams API

+ Fetch API has already shipped in M40 (in Service Worker) and in M42 (global scope)

+ The spec is backed by a large test suite and a well maintained polyfill allowing us to test our implementation for compatibility issues.

- ReadableStream saw a late and significant spec change to address issues discovered via 253 (see the summary in the merged pull request for more details).

+ the change ensures that the Streams API is able to better encompass different implementation strategies, including ones necessary for file streams as an example.

+ it also results in better developer ergonomics.

+ This intent covers a small subset of the Streams API. There might be future spec changes affecting developer created readable streams, but this intent to ship would not be affected (only covers user agent created streams).

+ Microsoft has expressed interest in integration with MSE (see TPAC minutes).

- Microsoft has already shipped a previous incarnation of the Streams API.

   + However, there is no worry to have about APIs clashing with each other.

+ Igalia folks and Youen Fablet are contributing an implementation to webkit (metabug; first target being ReadableStream) and code reviews seem to be going fine.

+ Mozilla has expressed interest in streams (for reference: implementation bug).

+ Jonas Sicking generally agrees that *a* streams API would be valuable...

- ... but has expressed concerns over the design of the current Streams API (IRC log on Feb 5th, w3c ml on March 9th, ).

+ Ben Kelly, who has been very active in the spec discussion, provided an argumentation in favor of the current design (IRC log)

- Jonas has also expressed concerns about the fact that you can’t transfer a ReadableStream between thread (issue/276)

+ Domenic, Ben and Blink folks have been working on pre-requisite work (tentative pull request). We believe that the compatibility risk is significantly low given the scope of this intent to ship.

- Another area of concerns was about the design of the constructors

+ This intent to ship doesn’t include constructors

+ Regarding the Fetch API, Mozilla is implementing it (bugzilla) and expects that it can ship it in “Firefox Developer Edition version 39 later this month” (source).

Tracking bug

crbug.com/240603

Link to entry on the feature dashboard

https://www.chromestatus.com/features/6730533392351232

---

[end of the intent to ship]

---

Bonus track!

Upcoming

This is a tentatively prioritized list of items that are on the team’s radar.

We are eager to hear your feedback, and preference among the 2 P1s with use cases via #StreamsAPI #Blink!

P1+. Ensuring backpressure signals are propagated down to the network stack, so that if you are slow consuming the stream, the browser can signal to the server to slow down through TCP window mechanisms. This is not available in XHR nor web sockets [1] [2].

P1. Add writable streams and integrate with fetch (e.g. allowing progressive upload with request body, or progressive creation of response body in a service worker...)

P2. Developer constructible streams: instead of only getting readable or writable streams from the fetch API, allow developers to create their own, wrapping arbitrary sources or sinks or transformations.

[Jochen Eisinger]

I'm concerned that when we add developer constructible and writeable streams, we'll end up in a similar position as with Promises where cutting over from the C++ implementation to the C++/JS implementation was, let's say, non trivial (even not taking the spec change into account).

What are the chances that we run into the same situation here?

Given that there's work underway to add the JS part, what's the downside of just waiting for this to be done before we ship?

[Adam Klein]

On Tue, Mar 24, 2015 at 9:52 AM, Jochen Eisinger <joc...@chromium.org> wrote:

I'm concerned that when we add developer constructible and writeable streams, we'll end up in a similar position as with Promises where cutting over from the C++ implementation to the C++/JS implementation was, let's say, non trivial (even not taking the spec change into account).

What are the chances that we run into the same situation here?

Can you explain what you found particularly problematic in the Promises cutover? I think that would help evaluate the possible risk of similar issues appearing for Streams.

FWIW, as someone who was quite involved in the Promise transition, a good number of the problems had to do with timing issues that wouldn't pop up for Streams.

- Adam

[Jochen Eisinger]

On Tue, Mar 24, 2015 at 10:10 AM Adam Klein <ad...@chromium.org> wrote:

On Tue, Mar 24, 2015 at 9:52 AM, Jochen Eisinger <joc...@chromium.org> wrote:

I'm concerned that when we add developer constructible and writeable streams, we'll end up in a similar position as with Promises where cutting over from the C++ implementation to the C++/JS implementation was, let's say, non trivial (even not taking the spec change into account).

What are the chances that we run into the same situation here?

Can you explain what you found particularly problematic in the Promises cutover? I think that would help evaluate the possible risk of similar issues appearing for Streams.

IIRC there were two problems: the semantics of microtasks changed subtly (esp. for promises that resolved due to C++ events), and (somewhat related) resolving promises from C++ was rather complicated as this suddenly required a JS context.

I think we would have had an easier time to change these if the API hadn't been already launched at that point.

[Yutaka Hirano]

My understanding is that the impact of streams in V8 is smaller than the impact of promises in V8. Promises needed microtasks which didn't work well (particularly with workers) at that time. In the range of this shipment, Readable[Byte]Stream doesn't contain "magic" and I expect smoother transition than promises case.

Another difference is that promises were used by various modules but we currently have only one customer for streams: Fetch API.

[Domenic Denicola]

From: blin...@chromium.org [mailto:blin...@chromium.org] On Behalf Of Jochen Eisinger

> I'm concerned that when we add developer constructible and writeable streams, we'll end up in a similar position as with Promises where cutting over from the C++ implementation to the C++/JS implementation was, let's say, non trivial (even not taking the spec change into account).
>
> What are the chances that we run into the same situation here?

Yutaka has already given the implementation-side answer to this, but from the spec side, there’s a pretty clear boundary that should make this transition easy. A given ReadableStream instance is given all its specific behavior via an “underlying source” constructor argument (similar to how a specific promise instance is given its behavior by an initializer function constructor argument). As we cut over, we would leave the underlying source implementation in C++, even as the ReadableStream class moves to JS.

> Given that there's work underway to add the JS part, what's the downside of just waiting for this to be done before we ship?

I think there's definite value in getting this new functionality to web developers soon. Kenji's table from the intent to ship is a pretty compelling showing of the new abilities it adds. The work to create a developer-constructible readable stream could be done soon-ish (one extra release, maybe), if everything goes right, on both the spec and implementation side. But assuming that would be a classic failure mode; if we take the planning fallacy into account it could be two or three more releases. Given that I don't see much _upside_ in waiting, shipping this subset makes a lot of sense to me. It's a big win for developers who have suffered under XHR's paltry streaming abilities for so long :)

[Jochen Eisinger]

Yesterday, the Blink API owners met and discussed this proposal.

We discussed the balance between shipping useful features as soon as possible vs polishing the implementation, and decided that in this case, the benefits from shipping early outweigh the concerns, so please go ahead and enable readable streams for fetch.

best

-jochen

[Benjamin Kelly]

I was looking at this again recently and noticed that your intent-to-ship mentions ReadableStream, but the fetch-with-streams proposal document uses ReadbleByteStream:

  https://github.com/yutakahirano/fetch-with-streams/

Can you confirm whether you are intending to ship ReadableStream or ReadableByteStream now?

I ask because, as far as I know, the spec for ReadableByteStream is not complete and is not yet in the streams API:

  https://streams.spec.whatwg.org/

Thanks!

Ben

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Domenic Denicola]

Right, fetch-with-streams doesn’t seem to be as clear about this as it should. I will file a pull request.

 

The plan on the spec side is to define ReadableByteStream as a superset of ReadableStream. (The tracking issue is #300.) It will have the exact same behavior as a ReadableStream if you use the ReadableStream subset. And the fetch body stream should be a ReadableByteStream, at least, once that thing is defined.

 

The plan on the implementation side is to ship the body stream as a subset of the whole ReadableByteStream implementation. The subset is missing for now:

·         The byte-specific stuff that ReadableByteStream will layer on top of ReadableStream

·         Other methods from ReadableStream, like pipeTo and pipeThrough (and possibly tee, depending on how discussions in #311 go)

 

The reason we’re not simply shipping the fetch body stream as a ReadableStream, and then later upgrading it to a ReadableByteStream, is just that we’d prefer to avoid the minor back-compat risk of `res.body.constructor` (and possibly `res.body instanceof ReadableStream`) changing during the upgrade.

[Benjamin Kelly]

On Sun, Apr 5, 2015 at 12:41 AM, Domenic Denicola <d...@domenic.me> wrote:

The reason we’re not simply shipping the fetch body stream as a ReadableStream, and then later upgrading it to a ReadableByteStream, is just that we’d prefer to avoid the minor back-compat risk of `res.body.constructor` (and possibly `res.body instanceof ReadableStream`) changing during the upgrade.

 

Well, this seems to avoid a minor back-compat issue at risk of larger back-compat issues if the ReadableByteStream changes in unexpected ways before being finalized.

It seems to me it would be safer to use ReadableStream and just not expose res.body.constructor.  (Also, I didn't think chrome was shipping stream constructors yet.)

Just my opinion, though.  Thanks for explaining!

Ben

[Domenic Denicola]

From: Benjamin Kelly [mailto:bke...@mozilla.com]

> Well, this seems to avoid a minor back-compat issue at risk of larger back-compat issues if the ReadableByteStream changes in unexpected ways before being finalized.

I think that would be a huge failure on our part in speccing. If ReadableByteStream is not swap-in compatible with ReadableStream, it's almost pointless to have it exist :)

> It seems to me it would be safer to use ReadableStream and just not expose res.body.constructor.  (Also, I didn't think chrome was shipping stream constructors yet.)

Right, not shipping constructors yet! But all class prototypes have a .constructor property, pointing to the constructor function whose .prototype is that prototype. Even if the constructor function always throws when called (by author code), it's still there, and can be used for identity testing. Try e.g. `window.constructor` and `window.constructor.name`.

[Benjamin Kelly]

Just out of curiosity, does this include full body streaming support from other producers of Response objects, like Cache.match()?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Takeshi Yoshino]

Yes. Response objects retrieved from a Cache using Cache.match() also have .body.

Takeshi