Shaka startup performance

[Daniel Baulig]

Hi Shaka Player Users,

I've been playing around with Shaka a bit lately and I noticed that it is pretty slow in regards to startup time. From instantiating a new Player and DashVideoSource instance until the video actually starts playing usually multiple seconds pass.

I looked into what is taking the player so long to start playing and I found several issues that I would like to outline and gauge what the projects stance on fixing these or alternatively taking patches with fixes is. Some of these issues are very specific to the scenario I was using, but others are more general and affect probably most users of Shaka. I give a proposed solution and try to give a rough estimate on the complexity of proposed solution and the potential impact on startup time.

1) Fetching MPD

Issue: Before Shaka can start any significant work it obviously needs to fetch the manifest. In my specific case I had the MPD as a JS string. I created a Blob URL from that string and passed it into Shaka which subsequently fetches the Blob URL using an AsyncRequest. Fetching the Blob URL using AsyncRequest is not as fast as one might assume. It took several hundred milliseconds on most of my tests. See: http://jsfiddle.net/bLc9zLm6/
Proposed solution: Allow Shaka to (optionally) take a shaka.dash.mpd.MPD instead of a URL.

Complexity: low

Impact: medium

2) Setting up MediaSource

The MediaSource API is not exactly blazingly fast. From creating and assigning to the video src attribute until opensource fires several hundred milliseconds can pass. In fact I've seen it even take multiple seconds on rare occasions (that may be a browser bug though). See http://jsfiddle.net/9dhrd2gq/

Proposed solution: Add an (optional?) video argument to StreamVideoSource.prototype.load. Create and assign the MediaSource URL to the given video node if provided, so the browser can spend those milliseconds while fetching and parsing the manifest. Synchronize with StreamVideoSource.prototype.attach and StreamVideoSource.prototype.createAndStartStreams_ through a promise.
Complexity: medium

Impact: low to medium

3) Fetching initData

Fetching initData happens in Stream.prototype.switch (streamInfo.segmentInitSource.create()) as part of StreamVideoSource.prototype.startStreams_ and it blocks resolving the promise for StreamVideoSource.prototype.createAndStartStreams_. This also means that it blocks BufferSourceManager and fetching of the first segment. The first (and only) use of initData however is in BufferSourceManager.prototype.fetch. Other than this step logically being part of the stream startup and a failure in this step causing a failure of the stream startup, there's no reason for this step to block fetching of the first segment.

Proposed solution: instead of passing initData into BufferSourceManager.prototype.fetch, pass in the promise returned by streamInfo.segmentInitSource.create() and wait for it to resolve as part of the initData append task.
Complexity: medium

Impact: medium

4) Fetching and appending first segment [somewhat related to 3]

The task queue in BufferSourceManager.prototype.fetch is very conservative. Some of the steps in it can (and probably should) be parallelized. E.g. fetching of the segment can start while still waiting for the initData append task to finish.

Proposed solution: Start fetching the segment outside of the task queue and make the segment append task wait for the segment fetch promise to resolve.

Complexity: low

Impact: medium

5) Using streaming fetch API

Fetching of segment in BufferSourceManager.prototype.fetch uses a simple XHR. This requires the entire segment to be loaded before we can run the append task and append it to the sourceBuffer, which - depending on segment size and network conditions - may take quite some time. 

Proposed solution: Use the streaming fetch API and append buffers as we get additional data from the fetch API. This probably requires additional refactoring of BufferSourceManager.prototype.fetch and possibly even other parts of the code base and may likely be quite complex. Not entirely sure on the exact scope.
Complexity: high

Impact: medium to high

I'd appreciate feedback on these. I'd like to point out that I have prototypal implementations for all of these except for #5 and am willing to flesh them out a little more with subsequent pull requests if the project shows interest. I am considering playing around with #5 too, but would like to get other opinions on that first, because it would probably mean a non-trivial amount of work.

Bests,

- Daniel

[Joey Parrish]

Hi Daniel,

Thanks for your detailed analysis.  Low latency is something we tried to accomplish in our design, but failed to instrument or stay on top of.

We are currently designing Shaka v2.0, which will have many of the core components rewritten, including our MSE, streaming, and networking.

In general, I'm happy to review pull requests.  However, since we are in a period of transition, things are a bit more complicated right now.  We're trying to finish v1.6.0, and I would like to avoid any major architectural or other risky changes in that release.  And some of these issues may be addressed by the design of v2.0.

That said, let me try to address these one at a time and ask some clarifying questions.

#1: Fetching MPD from string.

a) You mention "AsyncRequest".  We don't have a class by that name.  Do you mean "AjaxRequest"?

b) You created a blob URL, and let XHR handle it.  AjaxRequest has built-in support for data URIs and doesn't use XHR for them.  I would be interested to know if this is faster or slower than XHR+blob.  If it's faster, this could be a workaround for you.  Would you mind trying it and letting me know?

c) I would prefer not to allow an MPD structure directly passed into DashVideoSource.  This is an internal type and we are currently able to assume that an MPD object has come out of our parser and processor and is therefore structured in a way we know we can handle.

d) Networking in Shaka v2.0 will not be centered on XHR.  Rather, all network requests go through plugins based on the URI scheme.  In the case of http/https, the built-in plugin for that will use XHR.  So there should be no need to go through XHR for string literals in Shaka v2.0.  It could be as fast as Promise resolution (which appears to be 0-3ms for me in Chrome).

#2: Preloading MediaSource.

Interesting idea, but one of the points of Shaka Player is to handle MediaSource so the app developer doesn't have to.  So I'm reticent to complicate things by having the app developer handle that.  As an alternative, what if we initialized MediaSource when the Player is initialized, rather than when the VideoSource is loaded?  This will be easier in v2.0, when we drop HttpVideoSource and only support MediaSource-based playbacks.

In any case, I'd be interested to see your patch for this.  Can you send a pull request?  I can't guarantee that we will land it in v1.6, but I'm sure it is worth discussing further.

#3: Fetching initData.

We haven't done the best job of this yet, either in terms of latency or cross-browser support.  When we fetch init data is going to have to change in Shaka v2.0 in order to better support more browsers (long story), and we should try to minimize latency as well.

As for 1.x, I'd like to see a PR for this as well.  It sounds like you are only proposing changes to the internals, so I'm hopeful we can work out a clean way to implement this change before 2.0.

#4: Fetch and append in parallel.

This sounds like an obvious win.  I'd be happy to review the specifics in a PR, in particular since you rate the complexity of your fix as low.

#5: Streaming fetch.

Can you clarify what you mean by streaming fetch API?  Are you talking about XHR's progress event?

Thanks,

Joey

--
You received this message because you are subscribed to the Google Groups "Shaka Player Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-player-us...@googlegroups.com.
To post to this group, send email to shaka-pla...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-player-users/0dcb871f-b1a0-449f-8a46-141a0d485369%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[don...@jwplayer.com]

Hi Joey,

#5: Streaming Fetch

    I believe he is referring to this : https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

    It is, however not supported in IE/safari.

-Donato

[Joey Parrish]

Ah, I see.  Yes, because the Fetch APIs are not supported on IE11 or Safari, we won't be adopting them at this time.

Thanks,

Joey

--
You received this message because you are subscribed to the Google Groups "Shaka Player Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-player-us...@googlegroups.com.
To post to this group, send email to shaka-pla...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-player-users/72bfda79-1d90-43d5-b26f-30572d261259%40googlegroups.com.

[Daniel Baulig]

Hi Joey, 

Sorry for the long delay in my response, since I am currently traveling I only have sporadicly time. I should be back soon though and could take on some of the things outlined here.

Is there an ETA for release of Shaka 2.0? It seems like it might bring plenty of new features that I may be able to benefit from and might save me doing some additional work.

Am Donnerstag, 22. Oktober 2015 19:57:09 UTC+2 schrieb Joey Parrish:

Hi Daniel,

Thanks for your detailed analysis.  Low latency is something we tried to accomplish in our design, but failed to instrument or stay on top of.

We are currently designing Shaka v2.0, which will have many of the core components rewritten, including our MSE, streaming, and networking.

In general, I'm happy to review pull requests.  However, since we are in a period of transition, things are a bit more complicated right now.  We're trying to finish v1.6.0, and I would like to avoid any major architectural or other risky changes in that release.  And some of these issues may be addressed by the design of v2.0. 

That said, let me try to address these one at a time and ask some clarifying questions.

#1: Fetching MPD from string.

a) You mention "AsyncRequest".  We don't have a class by that name.  Do you mean "AjaxRequest"?

 

Yes, sorry for the confusion! 

b) You created a blob URL, and let XHR handle it.  AjaxRequest has built-in support for data URIs and doesn't use XHR for them.  I would be interested to know if this is faster or slower than XHR+blob.  If it's faster, this could be a workaround for you.  Would you mind trying it and letting me know?

I can certainly do that.

 

c) I would prefer not to allow an MPD structure directly passed into DashVideoSource.  This is an internal type and we are currently able to assume that an MPD object has come out of our parser and processor and is therefore structured in a way we know we can handle.

I basically only exported shaka.dash.mpd.parseMpd method and call that from my own code. In DashVideoSource I then short circuit if an Mpd structure is provided instead of a string in the constructor, skipping fetching and parsing of the MPD.

 

d) Networking in Shaka v2.0 will not be centered on XHR.  Rather, all network requests go through plugins based on the URI scheme.  In the case of http/https, the built-in plugin for that will use XHR.  So there should be no need to go through XHR for string literals in Shaka v2.0.  It could be as fast as Promise resolution (which appears to be 0-3ms for me in Chrome).

That sounds great! Will this plugin API allow for streaming APIs? That is will it be able to continuously provide more data as it becomes available or will it only provide a single event/callback/promise for when all of the data is available? Will there be a way to change the default handler for HTTP/HTTPS as well?

 

#2: Preloading MediaSource.

Interesting idea, but one of the points of Shaka Player is to handle MediaSource so the app developer doesn't have to.  So I'm reticent to complicate things by having the app developer handle that.  As an alternative, what if we initialized MediaSource when the Player is initialized, rather than when the VideoSource is loaded?  This will be easier in v2.0, when we drop HttpVideoSource and only support MediaSource-based playbacks.

In any case, I'd be interested to see your patch for this.  Can you send a pull request?  I can't guarantee that we will land it in v1.6, but I'm sure it is worth discussing further.

Yeah, sure. I'll do that.

 

#3: Fetching initData.

We haven't done the best job of this yet, either in terms of latency or cross-browser support.  When we fetch init data is going to have to change in Shaka v2.0 in order to better support more browsers (long story), and we should try to minimize latency as well.

As for 1.x, I'd like to see a PR for this as well.  It sounds like you are only proposing changes to the internals, so I'm hopeful we can work out a clean way to implement this change before 2.0.

Yeah, no public facing APIs need to be changed for this. It does screw a little with the startup process though and at what stage certain errors bubble up to the developer. With the current implementation if fetching of the initData fails, this will cause the startup process to fail (I think it's the StreamVideoSource.load promise). With this change this will no longer be the case because StreamVideoSource.load will no longer block for the initData promise. This only happens at a later step.

 

#4: Fetch and append in parallel.

This sounds like an obvious win.  I'd be happy to review the specifics in a PR, in particular since you rate the complexity of your fix as low.

I'll put out a pull request for this one.

 

#5: Streaming fetch.

Can you clarify what you mean by streaming fetch API?  Are you talking about XHR's progress event?

As Donato has pointed out this is referring to fetch Web API. The fetch API is the first browser HTTP API that has a well defined streaming interface that allows for incrementally receiving data without hacks like using iframes. using this API could decouple Shaka startup time from segment size. If I have a segment size of 10 seconds the current implementation needs to download the entire 10 seconds of the first segment - that is the entire segment - before playback can start. With using an incremental/streaming API as the fetch API, playback can start immediately after enough data was streamed (e.g. a second or two). This should decrease time to playback significantly. It is correct though that the API is not fully implemented in all browser platforms.

 

Thanks,

Joey 

To unsubscribe from this group and stop receiving emails from it, send an email to shaka-player-users+unsub...@googlegroups.com.

[Joey Parrish]

On Mon, Nov 2, 2015 at 2:45 AM, Daniel Baulig <daniel...@gmx.de> wrote:

Is there an ETA for release of Shaka 2.0? It seems like it might bring plenty of new features that I may be able to benefit from and might save me doing some additional work.

A very tentative one, yes.  Here's our timeline so far, which is very much subject to change as things evolve:

1) Design docs published - hopefully next week

2) Preview branch on github showing our work so far - some time in November

3) Shaka 2.0 beta release - some time in December

c) I would prefer not to allow an MPD structure directly passed into DashVideoSource.  This is an internal type and we are currently able to assume that an MPD object has come out of our parser and processor and is therefore structured in a way we know we can handle.

I basically only exported shaka.dash.mpd.parseMpd method and call that from my own code. In DashVideoSource I then short circuit if an Mpd structure is provided instead of a string in the constructor, skipping fetching and parsing of the MPD.

In Shaka 2, you'll be able to provide plugins for manifests, as well.  If you wished, you could have the manifest info as a JSON object in your page and your "manifest parser" could be function() { return Promise.resolve(preloadedManifest); }

 

d) Networking in Shaka v2.0 will not be centered on XHR.  Rather, all network requests go through plugins based on the URI scheme.  In the case of http/https, the built-in plugin for that will use XHR.  So there should be no need to go through XHR for string literals in Shaka v2.0.  It could be as fast as Promise resolution (which appears to be 0-3ms for me in Chrome).

That sounds great! Will this plugin API allow for streaming APIs? That is will it be able to continuously provide more data as it becomes available or will it only provide a single event/callback/promise for when all of the data is available? Will there be a way to change the default handler for HTTP/HTTPS as well?

The plugin API will be based on a Promise to a complete response.  Continuous streaming would conflict with other parts of the design, such as the request/response filtering APIs.  Also, since we never make Range requests that cover multiple segments, we don't believe there's any benefit to continuously streaming, as it would imply providing less than one segment to SourceBuffer.prototype.appendBuffer().

You will be able to change the default http/https handler if you wish.

#5: Streaming fetch.

Can you clarify what you mean by streaming fetch API?  Are you talking about XHR's progress event?

As Donato has pointed out this is referring to fetch Web API. The fetch API is the first browser HTTP API that has a well defined streaming interface that allows for incrementally receiving data without hacks like using iframes. using this API could decouple Shaka startup time from segment size. If I have a segment size of 10 seconds the current implementation needs to download the entire 10 seconds of the first segment - that is the entire segment - before playback can start. With using an incremental/streaming API as the fetch API, playback can start immediately after enough data was streamed (e.g. a second or two). This should decrease time to playback significantly. It is correct though that the API is not fully implemented in all browser platforms.

The filtering system we are putting together for v2.0 would be much more complex if it had to support streaming responses.  I'm not sure the complexity in the library is warranted when you consider the fact that developers of streaming services generally control the details of the content.  I don't think it makes sense to have a minBufferTime of 1.5s with 10s segments.

Maybe I'm being naive, though.  Please tell me if that's the case.  :-)

Thanks,

Joey