Intent to Prototype: Media Source Extensions for WebCodecs

[Matthew Wolenetz]

Contact emails

wole...@chromium.org

Explainer

https://github.com/wolenetz/mse-for-webcodecs/blob/main/explainer.md

Specification

None

Summary

Extends the Media Source Extensions API (MSE) to enable buffering containerless WebCodecs encoded media chunks with MSE for low-latency buffering and seekable playback.

Blink component

Blink>Media

Motivation

MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE. As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE. Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.

Initial public proposal

https://github.com/w3c/media-source/issues/184#issuecomment-720771445

TAG review

None

TAG review status

Pending

Risks

Interoperability and Compatibility

None

Gecko: No signal

WebKit: No signal

Web developers: No signals

Is this feature fully tested by web-platform-tests?

No

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1144908

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5649291471224832

This intent message was generated by Chrome Platform Status.

[yo...@yoav.ws]

On Saturday, November 21, 2020 at 4:01:43 AM UTC+1 wole...@chromium.org wrote:

Contact emails

wole...@chromium.org

Explainer

https://github.com/wolenetz/mse-for-webcodecs/blob/main/explainer.md

Specification

None

Summary

Extends the Media Source Extensions API (MSE) to enable buffering containerless WebCodecs encoded media chunks with MSE for low-latency buffering and seekable playback.

Blink component

Blink>Media

Motivation

MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE. As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE. Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.

Initial public proposal

https://github.com/w3c/media-source/issues/184#issuecomment-720771445

TAG review

None

 

Can you clarify why?

Was this covered by another TAG review already? 

[Matthew Wolenetz]

On Mon, Nov 23, 2020 at 1:22 AM yo...@yoav.ws <yo...@yoav.ws> wrote:

On Saturday, November 21, 2020 at 4:01:43 AM UTC+1 wole...@chromium.org wrote:

Contact emails

wole...@chromium.org

Explainer

https://github.com/wolenetz/mse-for-webcodecs/blob/main/explainer.md

Specification

None

Summary

Extends the Media Source Extensions API (MSE) to enable buffering containerless WebCodecs encoded media chunks with MSE for low-latency buffering and seekable playback.

Blink component

Blink>Media

Motivation

MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE. As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE. Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.

Initial public proposal

https://github.com/w3c/media-source/issues/184#issuecomment-720771445

TAG review

None

 

Can you clarify why?

Was this covered by another TAG review already? 

No it was not yet covered by a TAG review. Today, I have filed a w3ctag early design review request: https://github.com/w3ctag/design-reviews/issues/576.