Intent to Implement: Streams API

[Kenji Baheux]

Blinkeuses et Blinkeurs! 

We hereby announce our Intent to Implement "Streams API" (the reading part of it for a start).

Primary eng/PM emails

tyos...@chromium.org (kenji...@chromium.org)

Spec

The Streams API (Editor’s Draft) https://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm

XMLHttpRequest (Editor’s Draft) https://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html

Ongoing discussion at W3C http://lists.w3.org/Archives/Public/public-webapps/

Summary

An API for representing binary data in web applications as a Stream object, as well as programmatically building and reading its contents. It’s designed to be used in conjunction with other web platform APIs such as the XMLHttpRequest.

The extension for XMLHttpRequest proposed in the spec enables reading an HTTP response body and writing an HTTP request body progressively via the XMLHttpRequest. For the first step, we’re going to implement reading part, i.e. add Stream, StreamReader and the new responseType “stream” to XMLHttpRequest.

(This Intent to Implement doesn’t cover the writing part, i.e. StreamBuilder and overloading of send() method for Stream type)

Motivation

To implement interactive webapps like chat, video streaming, etc. we’ve been using the XMLHttpRequest with some hack such as COMET as a realtime-bidirectional messaging infrastructure. XMLHttpRequest was originally designed with a focus on request-response model communication. There’s no method available to send messages in the same HTTP transaction. This is important to improve the efficiency of applications which generates data to be streamed to the server. Streams API solves this exact problem by providing an interface to pass streamed data to the XMLHttpRequest by sending it as the body of the HTTP request.

Furthermore, to implement server based notifications, server-push with chunked response and polling has been the traditional approach. On the other hand, Streams API’s StreamReader makes it possible to receive the body of an HTTP response streamed by a server using asynchronous API similar to how FileReader works.

Compatibility Risk

No new compatibility break. In the future, it’s possible that the API surface will be changed. For the initial release, the API will be hidden behind a flag.

Ongoing technical constraints

None.

OWP launch tracking bug?

crbug.com/240603

Row on feature dashboard?

Yes.

Requesting approval to ship?

No (behind runtime flag. We’ll send intent to ship again when it’s time)

[Darin Fisher]

Perhaps this feedback is better sent to the webapps list, but can we avoid StreamReader?  How about just using FileReader?  We use FileReader to read Blobs, so why not also use FileReader to read Streams?

It seems like a Stream should just be the new base class of Blob.  A Stream is a Blob of unknown length.

Given the above, we should be able to just start by generalizing our Blob infrastructure to really be Stream infrastructure.  Blob would just be a special case of a Stream, which happens to have a known length.

We could make XHR vend a Stream, and then ensure that it can be passed to both createObjectURL and FileReader.  Voilà, no?

-Darin

[Zach Kuznia]

The internal way of reading Streams already closely matches Blobs, so using FileReader shouldn't require any major change.  FileReaderLoader already has all the functionality needed to read Streams behind the ENABLE(STREAM) flag.

The only caveat is that FileReader would need the optional maxSize parameter added to each function, and readAsBlob.

[Aaron Colwell]

Darin,

I talked to Adrian Bateman (Microsoft) at the last W3C F2F about the possibility of making Blob inherit from Stream and he said that they initially proposed doing that, but I think he said that someone from Mozilla was afraid that making such a change so late in the game for the File API spec would derail things. Obviously this is only one data point so it may still be worth starting a thread about making this change on the webapps list and see what happens.

FWIW, Kenji you've got my LGTM for at least the XHR returning a Stream object part. We should coordinate on this implementation because I'll need to integrate with this functionality for my Media Source Extensions work.

Aaron

[Takeshi Yoshino]

One concern I come up with is that when FileReader evolves in the future e.g. to be able to seek (Blob.slice() then read is enough, so maybe bad example though), we may need to complicate FileReader interface for reasons rooted in the difference between Blob and Stream. Difference between File and Blob seems much smaller.

The required changes to FileReader that Zach listed look not problematic. maxSize makes sense for File and Blob, too. readAsBlob() that accepts Blob is a bit strange but ok.

On Thu, May 16, 2013 at 11:48 PM, Aaron Colwell <acol...@chromium.org> wrote:

Darin,

I talked to Adrian Bateman (Microsoft) at the last W3C F2F about the possibility of making Blob inherit from Stream and he said that they initially proposed doing that, but I think he said that someone from Mozilla was afraid that making such a change so late in the game for the File API spec would derail things. Obviously this is only one data point so it may still be worth starting a thread about making this change on the webapps list and see what happens.

OK. I'll do that.

[Anne van Kesteren]

On Thu, May 16, 2013 at 3:48 PM, Aaron Colwell <acol...@chromium.org> wrote:
> I talked to Adrian Bateman (Microsoft) at the last W3C F2F about the
> possibility of making Blob inherit from Stream and he said that they
> initially proposed doing that, but I think he said that someone from Mozilla
> was afraid that making such a change so late in the game for the File API
> spec would derail things.

I'm not sure stability of the File API specification should be a huge concern.

> Obviously this is only one data point so it may
> still be worth starting a thread about making this change on the webapps
> list and see what happens.

Yeah.


--
http://annevankesteren.nl/

[Takeshi Yoshino]

I started a thread to discuss the overlap issue at public-webapps: http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0706.html.

Anne suggested that we redesign the streaming API completely, and Jonas proposed an alternative API using Future. We need to reach consensus there before launching this.

OTOH, I know some developers who want to tryout request/response body streaming for webapps to understand its efficiency and protocol level issues asap, but are not interested in how the API surface will be like. There should be some early adopters other than them but like them.

If the discussion is going to take so long, I'd like to start with making streaming available even with unstable API but behind a flag.

Thanks

Takeshi

[Ojan Vafai]

LGTM to start implementation behind a flag before the spec discussion has resolved. We'll want more confidence in the stability of the API before shipping of course. I see you're already proactive in pushing the spec discussion forward. I encourage you try to resolve that as quickly as possible so you're not stuck with an finished implementation that can't ship.

As far as the implementation, are you thinking to implement the FileReader version or the Futures version? The latter would be nicer if you're willing, but would also mean that you'd have to figure out how we implement Futures. So far, we have no APIs that use Futures, but it seems very likely that many new APIs will.

[Greg Simon]

+1 to doing some leg-work with futures. There is also a font loading API that we want to support looking for Futures. Seems we need a bootstrap here.

[Tab Atkins Jr.]

On Mon, May 20, 2013 at 10:57 AM, Greg Simon <greg...@chromium.org> wrote:
> +1 to doing some leg-work with futures. There is also a font loading API
> that we want to support looking for Futures. Seems we need a bootstrap here.

Yes, Futures get useful with network effects, so we need to just jump
in and start using them, even if we may still slightly tweak them as
the spec matures.

~TJ

[Kenji Baheux]

Shall we handle Futures as its own thing?

In other words, a separate track for the Blink launch process which we would start with a separate "intent to implement" announcement?

2013/5/21 Tab Atkins Jr. <jacka...@gmail.com>

[Michael Nordman]

> Given the above, we should be able to just start by generalizing our Blob infrastructure to really be Stream infrastructure.  Blob would just be a special case of a Stream,

> which happens to have a known length.

I think that's right.

On Thu, May 16, 2013 at 12:07 AM, Darin Fisher <da...@chromium.org> wrote:

[Takeshi Yoshino]

Thanks for comments, all.

I did some prototyping today to estimate size of task.

I'll start writing a separate but probably short intent-to-implement for Future. If it turned out to be better to have one doc, I can merge them.

Takeshi

[Takeshi Yoshino]

On Tue, May 21, 2013 at 9:24 PM, Takeshi Yoshino <tyos...@chromium.org> wrote:

Thanks for comments, all.

I did some prototyping today to estimate size of task.

I'll start writing a separate but probably short intent-to-implement for Future. If it turned out to be better to have one doc, I can merge them.

Done

https://groups.google.com/a/chromium.org/forum/?fromgroups#!topic/blink-dev/9q5kP0eMQc8

[Darin Fisher]

I get the motivation for developing a new stream reading API based on Futures, but I think in parallel we should be able to bring up Stream support.  Streams should be usable in places where Blobs are usable today.  For example, you should be able to pass a Stream to createObjectURL and FileReader.  I don't understand why we shouldn't do those in parallel to inventing a Futures-based stream reading interface.

-Darin

On Mon, May 20, 2013 at 10:51 AM, Ojan Vafai <oj...@chromium.org> wrote:

[Anne van Kesteren]

On Thu, May 23, 2013 at 8:47 PM, Darin Fisher <da...@chromium.org> wrote:
> I get the motivation for developing a new stream reading API based on
> Futures, but I think in parallel we should be able to bring up Stream
> support. Streams should be usable in places where Blobs are usable today.
> For example, you should be able to pass a Stream to createObjectURL and
> FileReader. I don't understand why we shouldn't do those in parallel to
> inventing a Futures-based stream reading interface.

Just curious: are you thinking of Blob-without-synchronous-size here
or actual streams (not keeping all data around)? (I agree we should
sort this out, it's way overdue.)


--
http://annevankesteren.nl/

[Darin Fisher]

I think Stream should be introduced as the base-class of Blob.  Blob adds a size attribute.

We should also want to explore the notion of Streams that retain their data versus Streams that do not.  That is, some Streams may be readable only once.  This might be important for use cases where the Stream size is effectively unbounded (e.g., radio station use case).

Maybe we could explore applying this read once notion to both Streams and Blobs.  It might be nice to enable the UA to discard the backingstore for a Blob once it has been read once.

-Darin

 

--
http://annevankesteren.nl/

[Takeshi Yoshino]

So, your suggestion is that we should have another interface aspect to provide read once (consume) semantics than giving that role to Stream class?

Blob and Stream would be both "read-any-number-of-time", but

- Blob: bounded && size-visible and fixed

- Stream: possibly unbounded && size-invisible and may grow until eof is set (reached)

And maybe Stream should be immutable as well as Blob? Then, we should want to introduce some new storage class for read-once semantics rather than having some reader or function that can "consume" data from Stream and Blob which are immutable.

// immutable data

interface Stream

interface Blob : Stream

// mutable data

interface ReadOnceStream

Since ReadOnceStream is mutable, I prefer not to use FileReader like approach but just put read methods directly on it.

readOnceStream.read().then(...);

readOnceStream.read().then(...);

than

reader1.readAsFooBar(readOnceStream);

reader2.readAsFooBar(readOnceStream);

It's ok but doesn't look intuitive to me which read operation takes place first as the operation is asynchronous.

 

[Darin Fisher]

That's not quite what I had in mind.

I was suggesting it could be a property of a Stream (and hence a Blob) that its data may only be available to be consumed once.  This property would be set at instantiation time and be fixed thereafter.  A File would probably always be read many.

-Darin

[Takeshi Yoshino]

On Sat, May 25, 2013 at 12:24 PM, Darin Fisher <da...@chromium.org> wrote:

That's not quite what I had in mind.

I was suggesting it could be a property of a Stream (and hence a Blob) that its data may only be available to be consumed once.  This property would be set at instantiation time and be fixed thereafter.  A File would probably always be read many.

I see. What do we do with slice()? Maybe having it clone the sub-contents for the new Blob?

[Darin Fisher]

I think I would keep slice() on Blob.  That avoids messy issues related to size of stream being unknown.

As for read-once blobs, I don't think slice() needs to copy.  I think the read-once property should apply to the backing data for the blob.  A blob slice is just another reference to the same "read-once" data.

Slicing a Blob happens synchronously, so it can't be a "deep copy" type of operation anyways.

-Darin

[Takeshi Yoshino]

On Mon, May 27, 2013 at 11:07 PM, Darin Fisher <da...@chromium.org> wrote:

I think I would keep slice() on Blob.  That avoids messy issues related to size of stream being unknown.

Sounds fine. Processing partial data could be done by just skipping "load" event for uninterested region until it reaches the target position.

interface Stream {

  // No slice, no size.

  readonly attribute bool isReadOnce;

  readonly attribute DOMString type;

  void close(); 

};

interface Blob : Stream {

  readonly attribute unsigned long long size;

  Blob slice(optional long long start,

                 optional long long end,

                 optional DOMString contentType);

};

 

As for read-once blobs, I don't think slice() needs to copy.  I think the read-once property should apply to the backing data for the blob.  A blob slice is just another reference to the same "read-once" data.

So, let's have them hold partial reference to the backing data of the origin Stream object. Not cloning backing data.

The API needs to maintain a list of intervals for the backing data synchronously. When any asynchronous operation is invoked for the backing data, the list will be passed to the browser process.

For simplicity, how about making read-once Streams neutered once it's passed to reader classes? This makes it clear that the right to consume the data is passed to the reader object. To allow for multiple readers case, we could use slice() (only for Blob) or create a new read once Stream (Blob) to hold multiple references.

  var readOnce = true;

  reader1.readAsFooBar(new Stream(stream, readOnce));

  reader2.readAsFooBar(stream);  // blob is neutered

  var readOnce = true;

  reader1.readAsFooBar(blob.slice(10, readOnce));

  reader2.readAsFooBar(blob);  // blob is neutered

The Stream needs to buffer read data that is delivered to one reader but not yet to the other.

To allow for reading a Stream with maxSize and continue processing the rest later (which we can do for non-read-once Blob by abort() + slice()), we could add "remaining" argument to the loadend event which is a new Stream object for the remaining data.

createObjectURL() may also have similar effect as readAsFooBar().

  var readOnce = true;

  var url = URL.createObjectURL(blob);  // blob is neutered

Once a reader object (or dereference on Blob/Stream URL) finish reading some region from a read-only Stream, the Stream derefs that region on the backing data. Part of backing store with no reference can be freed. Non-read-once Blob and Stream retain reference to whole backing data.

[Takeshi Yoshino]

To allow MSE team (acolwell@) to move forward, I'd like to split this into XHR integration part and Stream reading API part.

Stream reading API is still in flux but we can make XHR return a Stream object to pass to another API without defining reading API for JS and exposing methods on the Stream. So the integration part could be shipped earlier.

When the time comes, I'll send an intent to ship for the XHR part.

[Takeshi Yoshino]

One more heads up. I'm going to make FileReader support Stream behind flag.

Promises are implemented smoothly it takes a little more time. This FileReader integration can end up in test only feature and may be removed in the future, but we need some JS reading API to ensure a Stream returned by XHR works well by layout tests.

[Takeshi Yoshino]

Heads up about Streams API specification.

I and Feras (the original editor of W3C Streams API) merged proposals and replaced ED with it.

The ED has been promoted to WD but during WD CfC, a counter proposal was made by Domenic.

- W3C Streams API: WD, ED, Experimental

- WHATWG Streams API edited by Domenic

I'd like to continue implementation on Blink to find issues, but can be blocked by spec discussion. Your comments are welcome.

 

[Takeshi Yoshino]

Update about Streams.

We had meeting and agreed on spec conversion. See the post [1] by Feras on public-webapps for detail. We're still on the way to reorganize documents, but I started participating standardization of core primitives at whatwg repo.

I plan to propose changes there (or somewhere else after reorg) and resume implementing core primitive methods in Blink based on the spec.

[1] http://lists.w3.org/Archives/Public/public-webapps/2014JanMar/0218.html

[crouzet...@gmail.com]

Hello,

Is there any news or ETA of the Streams API ?

Thanks ! 

[Takeshi Yoshino]

Hi,

Please take a look at https://github.com/whatwg/streams for progress of standardization.

See https://github.com/whatwg/streams/tree/master/reference-implementation/lib for prototype implementation in ES6

See crbug.com/240603 (and its blocking bug list) for progress of prototype implementation in Blink.

Takeshi