Intent to Ship: Streams API: transferable streams

[Adam Rice]

Contact emails

ri...@chromium.org, yhi...@chromium.org

Explainer

https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md

Specification

https://streams.spec.whatwg.org/

Design docs

https://docs.google.com/document/d/1_KuZzg5c3pncLJPFa8SuVm23AP4tft6mzPCL5at3I9M/edit

TAG review

https://github.com/w3ctag/design-reviews/issues/332

Summary

The streams APIs provide ubiquitous, interoperable primitives for creating, composing, and consuming streams of data. A natural thing to do with a stream is to pass it to a web worker. This provides a fluent primitive for offloading work to another thread. Transferable streams add this capability by allowing ReadableStream, WritableStream, and TransformStream objects to be passed as arguments to postMessage().

Link to “Intent to Prototype” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/P97xJm1TFj4/PA0FKwLhBwAJ

Risks

Interoperability and Compatibility

Interoperability risk is low as the feature adds no new API surface, instead enhancing the existing postMessage() functionality to support transfer of streams. postMessage() will throw an exception in browsers where it is not implemented, making it easy to test for.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/430)

WebKit: No signal (https://www.mail-archive.com/webki...@lists.webkit.org/msg29629.html)

Web developers: Positive (https://github.com/whatwg/streams/pull/956#issuecomment-428540655) Most users of WebRTC Insertable Streams have also adopted transferable streams. I haven't been able to find any negative opinions.

Ergonomics

This is always used with postMessage(). In the current iteration, objects read or written to the stream are always cloned. The overhead hasn't been observed to be a problem so far in practice.

Activation

The feature is impossible to polyfill. However, feature detection is easy. Early-adopters may need to maintain two code paths.

Security

Data can be passed between different render processes. This means the deserialisation code in particular needs to be written with care to account for the fact that the other render process may be compromised. Mostly we're using existing methods for this, so the incremental security risk is not too high.

Debuggability

No specific debugging facility is required. Ordinary JavaScript debugging works as well as can be expected given the cross-realm nature of the feature.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes. This feature is purely implemented in Blink and so cross-platform support is automatic.

Is this feature fully tested by web-platform-tests?

Extensive web platform tests are under review at https://github.com/web-platform-tests/wpt/pull/24546.

Tracking bug

https://crbug.com/894838

Demo links

https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5298733486964736

This intent message was generated by Chrome Platform Status.

[Yoav Weiss]

On Tue, Aug 18, 2020 at 11:32 AM Adam Rice <ri...@chromium.org> wrote:

Contact emails

ri...@chromium.org, yhi...@chromium.org

Explainer

https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md

Aside: Would be good to highlight the bits directly related to transferable streams in the explainer. As is, it's hard for someone unfamiliar with Streams to understand the examples from the explainer alone.

It's unclear what's the feature detection story here. 

Would the code examples work in browsers that haven't yet shipped transferable streams? (I'm guessing no, but want to confirm)

Can developers know that the feature is not supported?

Would they need to care in the future when transferable streams enable zero copy object transfers? (e.g. maybe transfer large objects only when no copy is involved)

 

Specification

https://streams.spec.whatwg.org/

Design docs

https://docs.google.com/document/d/1_KuZzg5c3pncLJPFa8SuVm23AP4tft6mzPCL5at3I9M/edit

TAG review

https://github.com/w3ctag/design-reviews/issues/332

This seems like an early review. Did a more thorough review happen?

Summary

The streams APIs provide ubiquitous, interoperable primitives for creating, composing, and consuming streams of data. A natural thing to do with a stream is to pass it to a web worker. This provides a fluent primitive for offloading work to another thread. Transferable streams add this capability by allowing ReadableStream, WritableStream, and TransformStream objects to be passed as arguments to postMessage().

Link to “Intent to Prototype” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/P97xJm1TFj4/PA0FKwLhBwAJ

Risks

Interoperability and Compatibility

Interoperability risk is low as the feature adds no new API surface, instead enhancing the existing postMessage() functionality to support transfer of streams. postMessage() will throw an exception in browsers where it is not implemented, making it easy to test for.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/430)

WebKit: No signal (https://www.mail-archive.com/webki...@lists.webkit.org/msg29629.html)

Those requests are fairly recent. At the same time, since this is now part of a WHATWG spec, I'm assuming that there was multi-engine interest. Is that a correct assumption?

 

Web developers: Positive (https://github.com/whatwg/streams/pull/956#issuecomment-428540655) Most users of WebRTC Insertable Streams have also adopted transferable streams. I haven't been able to find any negative opinions.

Ergonomics

This is always used with postMessage(). In the current iteration, objects read or written to the stream are always cloned. The overhead hasn't been observed to be a problem so far in practice.

Activation

The feature is impossible to polyfill. However, feature detection is easy. Early-adopters may need to maintain two code paths.

Security

Data can be passed between different render processes. This means the deserialisation code in particular needs to be written with care to account for the fact that the other render process may be compromised. Mostly we're using existing methods for this, so the incremental security risk is not too high.

Debuggability

No specific debugging facility is required. Ordinary JavaScript debugging works as well as can be expected given the cross-realm nature of the feature.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes. This feature is purely implemented in Blink and so cross-platform support is automatic.

Is this feature fully tested by web-platform-tests?

Extensive web platform tests are under review at https://github.com/web-platform-tests/wpt/pull/24546.

Tracking bug

https://crbug.com/894838

Demo links

https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5298733486964736

This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAC_ixdyr9icisrjJiP7zja7MAGOjdCaY7GrVSvnpHBB0Vc_Y2A%40mail.gmail.com.

[Domenic Denicola]

On Tue, Aug 18, 2020 at 9:07 AM Yoav Weiss <yo...@yoav.ws> wrote:

On Tue, Aug 18, 2020 at 11:32 AM Adam Rice <ri...@chromium.org> wrote:

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/430)

WebKit: No signal (https://www.mail-archive.com/webki...@lists.webkit.org/msg29629.html)

Those requests are fairly recent. At the same time, since this is now part of a WHATWG spec, I'm assuming that there was multi-engine interest. Is that a correct assumption?

We've been pinging Gecko and WebKit on GitHub for about a month, and reached out directly via private email a couple weeks ago to try to get more engagement. That resulted in unofficial support from a WebKit engineer in that private thread.

In the spec editors' judgment, this was enough to proceed with merging the pull request. At the same time, we recognize that the Blink process requires a more formal position, thus the signal requests in this Intent.

Hope this helps!

[Adam Rice]

Aside: Would be good to highlight the bits directly related to transferable streams in the explainer. As is, it's hard for someone unfamiliar with Streams to understand the examples from the explainer alone. 

Thank you for the feedback. I will update the explainer. 

It's unclear what's the feature detection story here. 

postMessage() throws an exception if transferable streams are not implemented. So a simple feature detector would be

function hasTransferableStreams() {

  const rs = new ReadableStream();

  try {

    postMessage(rs, '*', [rs]);

    return true;

  } catch {

    return false;

  }

}

Would the code examples work in browsers that haven't yet shipped transferable streams? (I'm guessing no, but want to confirm)

No, they would fail with a pretty self-explanatory exception ("ReadableStream object could not be cloned" in Chrome, "The object could not be cloned" in Firefox).

Can developers know that the feature is not supported?

Yes, using the exception from postMessage().

Would they need to care in the future when transferable streams enable zero copy object transfers? (e.g. maybe transfer large objects only when no copy is involved)

This will need a modification to the Streams API to explicitly support zero-copy. I don't know what the feature detection story will be, but existing code won't suddenly start behaving differently.

Those requests are fairly recent. At the same time, since this is now part of a WHATWG spec, I'm assuming that there was multi-engine interest. Is that a correct assumption?  

Correct. We now have public support from WebKit: https://lists.webkit.org/pipermail/webkit-dev/2020-August/031350.html. We haven't heard anything from Mozilla.

Thanks,

Adam

[Chris Harrelson]

Hi Adam, did you have a response on this question also? Thanks.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAC_ixdzwik4bGjH8fo%3DnF28dCrxvJRYL69cxSCATf1K_m9tDvw%40mail.gmail.com.

[Adam Rice]

This seems like an early review. Did a more thorough review happen?

 

Hi Adam, did you have a response on this question also? Thanks.

Sorry, I missed this one. I have requested a full review at https://github.com/w3ctag/design-reviews/issues/551.

[Chris Harrelson]

LGTM1 to ship, even though the TAG review is not done.

This LGTM is conditional on TAG not having feedback that leads to a change in API. TAG review may happen very soon according to the labels on the github issue.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAC_ixdzF2_XbPWYE1nqC6k05D1-XEQ1VVVgytKbdUaZayiZczQ%40mail.gmail.com.

[Yoav Weiss]

LGTM2 with similar conditions

[Alex Russell]

LGTM3