Intent to Implement and Ship: Media Source SourceBuffer mode and 'sequence' appendMode support

104 views
Skip to first unread message

Matt Wolenetz

unread,
Feb 11, 2016, 10:26:59 PM2/11/16
to blin...@chromium.org
Contact emails

Spec

Summary
Add 'mode' attribute to SourceBuffer, and support the ability to use and switch among 'sequence' and 'segments' appendMode.
'segments' is the default (and current) behavior. 'sequence' enables web developers to simplify some types of media streaming apps that use Media Source Extensions (MSE).

Motivation
'sequence' appendMode allows more flexibility to maintain a gap-free buffered media timeline for web apps that know less metadata about the media they play using MSE.
It also allows an simpler migration path for current MSE players who may wish to switch from an MSE bytestream which uses 'generate timestamps flag' (*) (audio/mpeg or audio/aac) to other bytestreams which require more app-directed control of where media appends occur in the timeline to obtain a similar result as 'sequence' enables.
(*) Note that current Chrome parsers mimic the 'sequence' mode behavior inside the media engine for audio/mpeg and audio/aac bytestreams. Enabling 'sequence' appendMode at the Blink layer will allow later refactoring of the media engine to use that mode explicitly for these bytestreams.

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Yes.

Interoperability Risk
Firefox: Stabilizing
Edge: Shipped
Microsoft is actively engaged with Google in driving the MSE spec towards W3C "PR" status. SourceBuffer.mode is not an at-risk feature in the spec. See mailing list and issue tracker.

Compatibility Risk
Compatibility risk is low because 'segments' is the default append mode. The intent is to add support for a non-default 'sequence' append mode. 'sequence' mode behavior (or equivalent auto-update of timestampOffset behavior) is required for only a subset of MSE bytestreams (see (*), above), and the equivalent behavior (minus the exposure of SourceBuffer.mode attribute) for those specific bytestreams has already shipped in Chrome.

Ongoing technical constraints
None.

OWP launch tracking bug
Note, this was originally part of https://code.google.com/p/chromium/issues/detail?id=239506 (along with several other still-unimplemented portions of MSE in Chrome), but needs to ship in M-50 independently of those other features ideally to unblock at least one major MSE user that has requested it. It's been stable in Layout Tests, but hidden behind an experimental flag in Chrome, for well over a year now.

Link to entry
Small change.

Requesting approval to ship?
Yes (See p.s.'s below)

p.s. This is my first attempt at doing this process. I would appreciate any and all help understanding if a TAG review or a chromestatus entry are warranted.
p.p.s. It's currently available by --enable-experimental-web-platform-features via the 'MediaSourceExperimental' RuntimeEnabled flag. However, so are some other incomplete features covered by the original OWP launch bug 239506. So I expect I would need to create a new RuntimeEnabled flag specifically for SourceBuffer mode support to allow it to (independently of those other MediaSourceExperimental features) be enabled by default.
p.p.p.s. I would further like to understand if this is simply too late for M50, and if not too late, what are the immediate requirements to keep it on track for M50.
--

Matthew Wolenetz

unread,
Feb 11, 2016, 10:42:51 PM2/11/16
to blin...@chromium.org
[This msg is just to get my @chromium.org address onto this mail thread.]

Philip Jägenstedt

unread,
Feb 16, 2016, 7:36:05 AM2/16/16
to Matthew Wolenetz, blink-dev
LGTM1, this is a very reasonable feature and I can imagine that it's tricky to get the timestamps exactly right for gapless playback without it.

About p.s.'s:

TAG review for incremental changes to existing features isn't common, and in this case Edge has already shipped, so it would kind of be too late anyway.

A chromestatus.com entry would still be good, it's a feature that some web developers will care about after all.

Unless you think there's a non-trivial risk that this has to be reverted and that reverting would be conflict-prone without a flag, that seems like overkill. If [RuntimeEnabled=MediaSourceExperimental] in SourceBuffer.idl is all that hides this features from web developers, then it'll certainly be easy to revert.

It's not too late for M50, https://www.chromium.org/developers/calendar predicts that the branch will be cut on Friday next week.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Chris Harrelson

unread,
Feb 16, 2016, 1:11:28 PM2/16/16
to Philip Jägenstedt, Matthew Wolenetz, blink-dev
LGTM2

Dimitri Glazkov

unread,
Feb 16, 2016, 1:14:12 PM2/16/16
to Chris Harrelson, Philip Jägenstedt, Matthew Wolenetz, blink-dev
LGTM3

Matthew Wolenetz

unread,
Feb 16, 2016, 2:19:55 PM2/16/16
to Dimitri Glazkov, Chris Harrelson, Philip Jägenstedt, Matthew Wolenetz, blink-dev
Thanks for the responses. Regarding the current [RuntimeEnabled=MediaSourceExperimental], do I understand correctly that the recommended approach (assuming, as I believe is correct, that there is very little risk of need for revert) is to just make the SourceBuffer mode feature *not* behind *any* runtime flag (rather than introducing some new runtimeenabled flag for just this feature, to distinguish it from others that need to remain off-by-default at the moment)?

Matt

Philip Jägenstedt

unread,
Feb 16, 2016, 2:25:22 PM2/16/16
to Matthew Wolenetz, Dimitri Glazkov, Chris Harrelson, blink-dev
That is my recommendation, at least. Entries in RuntimeEnabledFeatures tend to accumulate and have a non-zero cost in binary size etc. After going to stable the only purpose they serve (that I know of) is reversibility, and that already seems trivial in this case.

Matthew Wolenetz

unread,
Feb 16, 2016, 2:27:03 PM2/16/16
to Philip Jägenstedt, Matthew Wolenetz, Dimitri Glazkov, Chris Harrelson, blink-dev
Ok. Thank you for the clarification. I'll set about getting the chromestatus entry entered, and a CL to move SourceBuffer.mode out from behind a RuntimeEnabled flag.

Matt

Matthew Wolenetz

unread,
Feb 16, 2016, 4:27:21 PM2/16/16
to Matthew Wolenetz, Philip Jägenstedt, Dimitri Glazkov, Chris Harrelson, blink-dev
Updates:
chromestatus entry: https://www.chromestatus.com/features/5645695201574912
CL to make the implementation "always on" in Chrome: https://codereview.chromium.org/1701173002/
-Matt

Reply all
Reply to author
Forward
0 new messages