Intent to Ship: writable streams

[Domenic Denicola]

Contact emails

ri...@chromium.org, tyos...@chromium.org, dom...@chromium.org

Spec

https://streams.spec.whatwg.org/#ws

TAG review: https://github.com/w3ctag/spec-reviews/issues/163

Summary

WritableStream is the writable counterpart to ReadableStream (which shipped in 2015). It provides a standard abstraction for writing streaming data to a sink, with backpressure and queuing. Sinks can be either built-in or implemented in JavaScript.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/f1zzIHtRomQ/foZ6f5dKBwAJ

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Interoperability Risk

Low.

Writable streams, and streams in general, have been the result of an active multi-vendor collaboration, guided the entire way by a large set of web-platform-tests which are co-developed with the specification and with other vendors. "Code coverage" of the writable streams portion of the spec currently stands at 96.63%. Readable streams have paved the way here with excellent interoperability, being deployed in Edge, WebKit, and Blink with compatible semantics, with Gecko following close behind.

• Edge: has partially shipped readable streams and has expressed general support for writable streams

• Firefox: In development and actively collaborating on the spec repo; their main implementer has already prototyped writable streams, although I don't know if he committed that code anywhere yet

• Safari: In development. An earlier version of the spec ships in Safari Tech Preview (but not stable Safari); we are in contact with them about updating.

• Web developers: Positive

Compatibility Risk

Low.

The WritableStream standard has stabilized. There has been a lot of activity with new tests being written and small adjustments since the Intent to Implement, as we work to nail down edge cases. None of these have made breaking changes to the API, and in general the changes have only been about edge case behaviors such as what happens when you close and then immediately abort the stream. At this point we are happy even with all the edge case behaviors.

You can view the issues being tracked in the spec repository. At this time all of the remaining issues are around future features (tagged "V2").

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=658144

Entry on the feature dashboard

https://www.chromestatus.com/feature/5928498656968704

[Adam Rice]

Safari: In development. An earlier version of the spec ships in Safari Tech Preview (but not stable Safari); we are in contact with them about updating.

Update: they just shipped it in 10.1. It is interoperable for creating WritableStreams: the sink API is the same. pipeTo() from a ReadableStream to a WritableStream is also interoperable. The API for writing directly to a WritableStream has changed significantly; for the time being pipeTo() is the best choice for writing to a WritableStream to work cross-browser. I have written non-trivial code in Chrome that worked with Safari's implementation with no changes.

[Dimitri Glazkov]

LGTM1

[Rick Byers]

LGTM2

Thanks for the excellent WPT test suite!

On Fri, Mar 31, 2017 at 10:30 AM, Dimitri Glazkov <dgla...@chromium.org> wrote:

LGTM1

On Thu, Mar 30, 2017 at 11:26 PM Adam Rice <ri...@chromium.org> wrote:

Safari: In development. An earlier version of the spec ships in Safari Tech Preview (but not stable Safari); we are in contact with them about updating.

Update: they just shipped it in 10.1. It is interoperable for creating WritableStreams: the sink API is the same. pipeTo() from a ReadableStream to a WritableStream is also interoperable. The API for writing directly to a WritableStream has changed significantly; for the time being pipeTo() is the best choice for writing to a WritableStream to work cross-browser. I have written non-trivial code in Chrome that worked with Safari's implementation with no changes.

On 31 March 2017 at 14:50, Domenic Denicola <dom...@chromium.org> wrote:

Contact emails

ri...@chromium.org, tyos...@chromium.org, dom...@chromium.org

Spec

https://streams.spec.whatwg.org/#ws

TAG review: https://github.com/w3ctag/spec-reviews/issues/163

Summary

WritableStream is the writable counterpart to ReadableStream (which shipped in 2015). It provides a standard abstraction for writing streaming data to a sink, with backpressure and queuing. Sinks can be either built-in or implemented in JavaScript.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/f1zzIHtRomQ/foZ6f5dKBwAJ

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Interoperability Risk

Low.

Writable streams, and streams in general, have been the result of an active multi-vendor collaboration, guided the entire way by a large set of web-platform-tests which are co-developed with the specification and with other vendors. "Code coverage" of the writable streams portion of the spec currently stands at 96.63%.

Whoa, how did you get that!  Is the technique applicable only to V8 extras or also C++ code?

[Chris Harrelson]

LGTM3

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[PhistucK]

How does Blink fare in Web Platform Tests for this feature?

Any failing tests that other browsers pass? Percentage?

☆PhistucK

[Domenic Denicola]

On Sat, Apr 1, 2017 at 5:59 AM Rick Byers <rby...@chromium.org> wrote:

LGTM2

Thanks for the excellent WPT test suite!

Writable streams, and streams in general, have been the result of an active multi-vendor collaboration, guided the entire way by a large set of web-platform-tests which are co-developed with the specification and with other vendors. "Code coverage" of the writable streams portion of the spec currently stands at 96.63%.

Whoa, how did you get that!  Is the technique applicable only to V8 extras or also C++ code?

The trick is in the wording here: it's coverage of the spec, not Chrome's implementation :). The spec has a corresponding reference implementation, which we run the web platform tests against, and then use JavaScript code coverage tools on. We could probably do similar work for Chrome's V8 extras implementation; that would be fun. But the hope is that if we cover 100% of the spec, that will also improve coverage for implementations.

[Adam Rice]

Blink's implementation suffered a serious blow when I landed https://github.com/w3c/web-platform-tests/pull/4830, "WritableStream: Add tests for re-entrant calls inside strategySize". This tests a case where if a developer does something pathological (and useless) the internal state of the WritableStream gets messed up. No implementation is going to pass that test until they've incorporated the corresponding spec revision.

The good news is I have a CL in the works which gets Blink back to passing 100%.

[PhistucK]

What about the other implementations? Is there a page that shows passing percentages for all of the experimental implementations?

☆PhistucK

[Adam Rice]

What about the other implementations? Is there a page that shows passing percentages for all of the experimental implementations?

Not that I know of. I tried looking at the tests at http://w3c-test.org/streams/writable-streams/ manually in Safari, but the change in the API for directly writing to a WritableStream means that most of the tests fail in the call to WritableStream getWriter().

[PhistucK]

This is not a good sign... That means that the implementation will ship while already non-interoperable.

I understand that you are in contact with them, but Safari releases are not as frequent (to say the least). About twice a year.

This is a significant feature. My guess is that it will be used extensively, having been such a long time coming. And since WebKit does not support service workers yet, my guess is that writeable streams will be created from scratch and not piped to (but I might be completely misunderstanding the world).

Did the WebKit folks show public support for those breaking specification changes?

☆PhistucK

[Adam Rice]

This is not a good sign... That means that the implementation will ship while already non-interoperable.

Blink is more interoperable with Safari with WritableStream available than it is without it. I am glad Safari shipped it, even though it's an older version which will cause some developer pain. Because it will also cause some developer joy when they can benefit from these new features.

Philosophically speaking, I think a Living Standard implies that browsers being out of sync is going to be more common than not. That's the price we pay for a web platform that actually gets better with time.

[Domenic Denicola]

On Mon, Apr 3, 2017, 21:37 Adam Rice <ri...@chromium.org> wrote:

This is not a good sign... That means that the implementation will ship while already non-interoperable.

Blink is more interoperable with Safari with WritableStream available than it is without it. I am glad Safari shipped it, even though it's an older version which will cause some developer pain. Because it will also cause some developer joy when they can benefit from these new features.

Philosophically speaking, I think a Living Standard implies that browsers being out of sync is going to be more common than not. That's the price we pay for a web platform that actually gets better with time.

Note, however, that this is a somewhat unusual case, as Safari shipped a section of the spec which until six days ago had a big red warning that it was not ready to ship: https://github.com/whatwg/streams/pull/710. Indeed at the time they last updated their now-shipping implementation, the warning was even more severe; see e.g. https://streams.spec.whatwg.org/commit-snapshots/69aaf14eefbe89b4e90bcf0c71a1832a54392955/#ws.

I'm more inclined to chalk this up to accident or bad communication than to an inherent feature of living standards.

☆PhistucK

☆PhistucK

☆PhistucK

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[PhistucK]

No, because developers expect to write code once. Some will benefit from it, if the feature exists and some will not, if it does not. A lot of advocates for progressive enhancement. Having a feature implemented differently at its core is not a good thing, living standards or not. It is the old way of the web, not (or should not be) the new way of the web.

And with the note of Domenic, yes, I think they should have not shipped it. But this is not the topic here.

☆PhistucK

[Philip Jägenstedt]

+Youenn Fablet to whom the WebKit bug is assigned.

Can someone summarize the differences between what just shipped in Safari 10.1 and what the spec+tests currently require? If it is just difference in error handling that might be manageable, although still not great.

☆PhistucK

☆PhistucK

☆PhistucK

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Adam Rice]

The most noticeable difference is that the current standard has a getWriter() method on WritableStream that returns a WritableStreamDefaultWriter object. This provides exclusive write-access to the stream and close() and write() methods. In Safari the close() and write() methods exist directly on the WritableStream and there is no concept of exclusive access.

I think the majority of WritableStream web-platform-tests use getWriter(), so that's going to make them fail right away.

I don't know what version of the standard Safari's WritableStream is based off, so I can't speak authoritatively, but:

My guess is that the next biggest set of changes is abort(). The API hasn't changed, but in old versions of the standard the abort() sink method could be called while another sink method was executing.

Error handling is probably the next biggest area of change. This is mostly about what happens if the same stream has two different errors, or an error and an abort, so it's quite esoteric for many people.

☆PhistucK

☆PhistucK

☆PhistucK

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Philip Jägenstedt]

With close() and write() on different objects, I guess that means that people will have to write different code for this to work with both WebKit's implementation and what is now in Blink?

Youenn, can you comment on the prospects of getting WebKit updated?

Adam et al., this thread already has 3xLGTM, so when you're comfortable shipping given any new facts, please go ahead.

☆PhistucK

☆PhistucK

☆PhistucK

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Adam Rice]

With close() and write() on different objects, I guess that means that people will have to write different code for this to work with both WebKit's implementation and what is now in Blink?

I'd rather people use pipeTo(), since it doesn't require hacks to achieve interoperability, but in the common case where you only use one writer you can paper over the difference by replacing the usual

const writer = writableStream.getWriter();

with

const writer = writableStream.getWriter ? writableStream.getWriter() : writableStream;

Just don't put my name on it.

☆PhistucK

☆PhistucK

☆PhistucK

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Philip Jägenstedt]

On Tue, Apr 4, 2017 at 5:46 PM Adam Rice <ri...@chromium.org> wrote:

With close() and write() on different objects, I guess that means that people will have to write different code for this to work with both WebKit's implementation and what is now in Blink?

I'd rather people use pipeTo(), since it doesn't require hacks to achieve interoperability, but in the common case where you only use one writer you can paper over the difference by replacing the usual

const writer = writableStream.getWriter();

with

const writer = writableStream.getWriter ? writableStream.getWriter() : writableStream;

Just don't put my name on it.

Might it be possible to set WritableStream.prototype.getWriter to something that makes the difference invisible?

[youenn]

Hi all,

Yes, an old version of WritableStream was shipped in Safari 10.1.

Plan is for WebKit to update it to the latest spec.

This is not a high priority task at the moment though.

y

[Adam Rice]

Might it be possible to set WritableStream.prototype.getWriter to something that makes the difference invisible?

Yes. You could emulate WritableStreamDefaultWriter to whatever degree of fidelity you needed. If you also wrapped ReadableStream.prototype.pipeTo you could even emulate the exclusive write locking.

☆PhistucK

☆PhistucK

☆PhistucK

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Dimitri Glazkov]

As a separate concern, are y'all checking in with +John Abd-El-Malek and the mojo folks to make sure that the design is technically optimal for cases when we're dealing streams across process boundaries?

:DG<