Intent to Prototype: Media Source Extensions for WebCodecs

161 views
Skip to first unread message

Matthew Wolenetz

unread,
Nov 20, 2020, 10:01:43 PM11/20/20
to blink-dev, Husain Bengali, Dale Curtis, mlam...@chromium.org

Contact emails

wole...@chromium.org

Explainer


https://github.com/wolenetz/mse-for-webcodecs/blob/main/explainer.md

Specification

None

Summary

Extends the Media Source Extensions API (MSE) to enable buffering containerless WebCodecs encoded media chunks with MSE for low-latency buffering and seekable playback.



Blink component

Blink>Media

Motivation

MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE. As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE. Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.



Initial public proposal

https://github.com/w3c/media-source/issues/184#issuecomment-720771445

TAG review

None

TAG review status

Pending

Risks



Interoperability and Compatibility

None



Gecko: No signal

WebKit: No signal

Web developers: No signals


Is this feature fully tested by web-platform-tests?

No

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1144908

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5649291471224832

This intent message was generated by Chrome Platform Status.

yo...@yoav.ws

unread,
Nov 23, 2020, 4:22:40 AM11/23/20
to blink-dev, Husain Bengali, Dale Curtis, mlam...@chromium.org, You can't view member email addresses for this group (5)
On Saturday, November 21, 2020 at 4:01:43 AM UTC+1 wole...@chromium.org wrote:

Contact emails

wole...@chromium.org

Explainer


https://github.com/wolenetz/mse-for-webcodecs/blob/main/explainer.md

Specification

None

Summary

Extends the Media Source Extensions API (MSE) to enable buffering containerless WebCodecs encoded media chunks with MSE for low-latency buffering and seekable playback.



Blink component

Blink>Media

Motivation

MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE. As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE. Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.



Initial public proposal

https://github.com/w3c/media-source/issues/184#issuecomment-720771445

TAG review

None
 
Can you clarify why?
Was this covered by another TAG review already? 

Matthew Wolenetz

unread,
Nov 23, 2020, 6:45:40 PM11/23/20
to yo...@yoav.ws, Husain Bengali, Dale Curtis, mlam...@chromium.org
On Mon, Nov 23, 2020 at 1:22 AM yo...@yoav.ws <yo...@yoav.ws> wrote:


On Saturday, November 21, 2020 at 4:01:43 AM UTC+1 wole...@chromium.org wrote:

Contact emails

wole...@chromium.org

Explainer


https://github.com/wolenetz/mse-for-webcodecs/blob/main/explainer.md

Specification

None

Summary

Extends the Media Source Extensions API (MSE) to enable buffering containerless WebCodecs encoded media chunks with MSE for low-latency buffering and seekable playback.



Blink component

Blink>Media

Motivation

MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE. As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE. Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.



Initial public proposal

https://github.com/w3c/media-source/issues/184#issuecomment-720771445

TAG review

None
 
Can you clarify why?
Was this covered by another TAG review already? 

No it was not yet covered by a TAG review. Today, I have filed a w3ctag early design review request: https://github.com/w3ctag/design-reviews/issues/576.
Reply all
Reply to author
Forward
0 new messages