Intent to Extend Origin Trial: MSE in Workers

[Matthew Wolenetz]

Hello blink-dev,

We'd like to ask for an extension to our Origin Trial, from M99 to M103.

There are two reasons extension is necessary:

1) A breaking error discovered in the middle of the trial (see https://crbug.com/1268614) caused participation to drop before the 2021 holiday period. While the fix landed by early 2022 and at least one participant has reported excellent improvement in metrics in their experiment since then, further stabilization is desired.

2) An API change for this feature (using srcObject for MSE-in-Workers attachments instead of baking in objectURL attachment further) was agreed with the media workgroup last September [1] following the previous objectURL-based attachment design getting more recent feedback [2], and I'm working on getting that into both the feature spec and Chromium's experimental implementation currently for rapid experimentation during the extended trial.

[1] https://www.w3.org/2021/09/14-mediawg-minutes.html

[2] https://github.com/mozilla/standards-positions/issues/547

Please find the Chrome status template, below:

Contact emails

wole...@chromium.org

Explainer

https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md

Specification

https://github.com/w3c/media-source/pull/282

Summary

Enable Media Source Extensions (MSE) API usage from DedicatedWorker contexts to enable improved performance of buffering media for playback by an HTMLMediaElement on the main Window context. By creating a MediaSource object on a DedicatedWorker context, an application may then create an ObjectURL for it and postMessage that URL to the main thread for use in attaching to an HTMLMediaElement. The context that created the MediaSource object may then use it to buffer media.

Blink component

Blink>Media

Search tags

MSE, MediaSource, MediaSourceExtensions

TAG review

https://github.com/w3ctag/design-reviews/issues/656

TAG review status

Issues addressed

Risks

Interoperability and Compatibility

Main interoperability risk is that other browsers may not implement it. Compatibility risk is mitigated by unchanged same-thread MSE API support and proactive feature-detectability of MSE-in-Workers with a new canConstructInDedicatedWorker attribute on the MediaSource interface.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/547)

WebKit: No signal (https://lists.webkit.org/pipermail/webkit-dev/2021-June/031916.html)

Web developers: No signals

Other signals:

Ergonomics

DedicatedWorker, WorkerGlobalScope and postMessage/onmessage availability is integral to using this feature.

Activation

The primary risk is the potential difficulty in refactoring existing MSE web applications to (conditionally, depending on browser support) perform buffering in a different execution context from the Window context that hosts the DOM and the media element. The benefit of potentially improved buffering performance by offloading the MSE API usage to a worker context provides motivation to overcome this challenge.

Security

Unpredictability of racing context destruction of either the main thread hosting the media element (and owning the underlying MSE media demuxer and player) or the dedicated worker thread hosting the interface to the MSE demuxer when using MSE from a worker context motivated careful design of a refcounted, outside-of-Oilpan, attachment abstraction to ensure safety of operations using locks and having a lifetime that exceeds those execution contexts. Preventing use-after-free and similar was a primary complexity of implementation and a focus area of added tests.

Goals for experimentation

* Obtain developer feedback on API ergonomics and implementation stability. * Increase confidence in benefits of feature (exposing and enabling MSE usage from DedicatedWorker contexts) to support its standardization.

Reason this experiment is being extended

There are two reasons extension is necessary: 1) A breaking error discovered in the middle of the trial (see https://crbug.com/1268614) caused participation to drop before the 2021 holiday period. While the fix landed by early 2022 and at least one participant has reported excellent improvement in metrics in their experiment since then, further stabilization is desired. 2) An API change for this feature (using srcObject for MSE-in-Workers attachments instead of baking in objectURL attachment further) was agreed with the media workgroup last September [1] following the previous objectURL-based attachment design getting more recent feedback [2], and I'm working on getting that into both the feature spec and Chromium's experimental implementation currently for rapid experimentation during the extended trial. [1] https://www.w3.org/2021/09/14-mediawg-minutes.html [2] https://github.com/mozilla/standards-positions/issues/547

Ongoing technical constraints

Debuggability

Feature consists of exposing existing interfaces also on a DedicatedWorker, and addition of a WebIDL method for proactively detecting support for this feature, and are covered by basic tooling.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

MSE and DedicatedWorker already exist on all six Blink platforms. This feature lets MSE work from DedicatedWorker contexts.

Is this feature fully tested by web-platform-tests?

Yes

DevTrial instructions

https://github.com/w3c/media-source/issues/175#issuecomment-721395481

Flag name

MediaSourceInWorkers

Requires code in //chrome?

False

Tracking bug

https://crbug.com/878133

Estimated milestones

OriginTrial desktop last	99
OriginTrial desktop first	95

OriginTrial android last	99
OriginTrial android first	95

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5177263249162240

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/CNRywDqgKjY/m/F0nnA4tTAwAJ
Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/XZMyanniH9E

This intent message was generated by Chrome Platform Status.

[Yoav Weiss]

LGTM to continue experimenting till M103. Note that this will bring the OT to 9 releases, so it'd be good to wrap up experimentation by this extension's end.

[Matthew Wolenetz]

Thank you. The results so far have been very encouraging, so I expect no further extension would be necessary beyond 103.

[Matthew Wolenetz]

I need to request a further extension of this origin trial, to end in M104 instead of the current end of 103.

While I've just sent the intent-to-ship for this feature (with target milestone 105), I have just received a request from another major web developer for the origin trial to be extended into 104.

The full implementation and stabilization of this feature didn't land until early in the 105 milestone due to some API changes requested by participants in the media workgroup discussions of this feature.

While these changes in 105 are not expected to be part of the origin trial, the bulk of the MSE-in-Workers implementation remains usable via the origin-trial-feature "MediaSourceInWorkers" using the older API shape.

The primary change in the API shape is to use a MediaSourceHandle for attaching a worker MediaSource instead of a MediaSource object URL.

Extending the OT into 104 will enable experimentation as requested by the web developer.

Please review this request and advise on next steps. Note, this is not intended to replace the intent to ship in 105 for this feature, but instead to let the feature continue to be experimented with in 104.

Thank you,

Matt

[Yoav Weiss]

LGTM to continue the experiment through M104.