Intent to Extend Origin Trial: WebCodecs

Skip to first unread message

Husain Bengali

Nov 17, 2020, 6:19:00 PM11/17/20
to, Chris Cunningham, Dan Sanders

Contact emails,,






Design docs



Provides efficient, low-level access to built-in (software and hardware) media encoders and decoders.


Blink component



TAG review


TAG review status

Issues addressed




Interoperability and Compatibility

The main risk is that other browsers do not implement it.


Gecko: Positive co-editing the spec (Paul Adenot)


WebKit: No signal ( "We have some concerns about WebCodecs, but don’t think we’ve stated a public position on it. Concerns about fingerprinting and design of the API, but I don’t think it’s fair to say “negative”. " - Myles Maxfield


Web developers: Positive ( Wikipedia: 





The proposed API shape enables the core use cases (see explainer) in a performant manner. We've left room for future optimizations and generally minimized complexity, even if that means we don't support all codec features. We intend for this shape to be friendly to both WASM and JS users. 


Decoder outputs will typically be rendered with Canvas and Web Audio. Developers have asked for this low level rendering control. The Canvas rendering path also allows VideoFrames to be manipulated via WebGL. The WebAudio rendering path will often leverage AudioWorklet. The AudioDecoder output uses Web Audio primitives to make this easy.


Encoder inputs will often come from getUserMedia() and getDisplayMedia(). 




WebCodecs would benefit from having libraries built on top that do things such as: 

+ containerize (mux) and de-containerize (demux) to/from media file formats (e.g. mp4, webm).

+ render media with low latency from an unreliable media transport (in other words, a jitter buffer)

+ RTC client logic / signaling 


WebCodecs could benefit somewhat from polyfills to experiment with the API, but that would not bring any of the performance benefits or access to hardware codecs. Significant documentation and outreach would likely be helpful, especially for advanced uses of codecs, such as spatial and temporal scalability.




The implementation is thoroughly fuzz tested. There may be marginally increased fingerprinting surface due to support for detecting the presence of hardware encode capabilities. This has been reviewed and deemed acceptable by the privacy sandbox team.



Goals for experimentation

We need feedback on most of the API surfaces, which roughly amounts to decode for video, audio and images, and encode for video and audio. 


Multiple partners have committed (and shown a great deal of eagerness) to participate in our origin trial and we will look for their feedback and validation of the appropriateness of the API shape for their use cases (e.g. real time communications, low latency streaming).



Experimental timeline

The initial goal of the origin trial was to run for 3 milestones, from M86 Stable thru till M88 Stable. We are now looking to extend the origin trial thru till M91 Stable


Reason this experiment is being extended

The origin trial has been successful in generating strong interest and valuable feedback. We need additional time to fill API gaps (e.g. worker support, pixel format conversion, audio encoding support) and collect feedback on those areas. This additional time will also allow us to land performance improvements.


Ongoing technical constraints



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?



Is this feature fully tested by web-platform-tests?

Yes. We have coverage for all basic uses of the API, including exception and error states, which should ensure a base-level uniformity across browser implementation of the API surface. Some deeper aspects of the API's implementation, such as which codecs are supported, vary from browser to browser and hence may not need to be tested by WPTs. Test status can be found at:


Tracking bug


Launch bug


Link to entry on the Chrome Platform Status


Links to previous Intent discussions

Intent to Experiment:


This intent message was generated by Chrome Platform Status.

Yoav Weiss

Nov 19, 2020, 1:13:37 AM11/19/20
to Husain Bengali, blink-dev, Chris Cunningham, Dan Sanders
LGTM to extend experimentation till M91, but not beyond.
In case that won't be sufficient, please reach out to discuss further.

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit

Thomas Guilbert

Jan 15, 2021, 7:10:08 PM1/15/21
to Yoav Weiss, Husain Bengali, blink-dev, Chris Cunningham, Dan Sanders

As suggested by the OT core group, continuing a discussion here:

We've made design changes to WebCodecs, we decided that portions of the Breakout Box project should replace a small portion of WebCodecs, to deduplicate functionalities and to allow better performance in workers long term.

Specifically, before WebCodec ships, we will deprecate VideoTrackReader/VideoTrackWriter in favor of using TrackProcessor/TrackGenerator

The Breakout Box launch bug has passed through Security and Privacy reviews, and is pending legal review and final approvals.

We're looking to bring those primitives under the WebCodecs OT flag, since our launches are now coupled, and so partners can experiment ahead of time. Ideally, we'd like to merge the flag changes back to M89 (pending launch approvals), to get feedback from the Stable channel during our M91 bug fix push.

Can you advise as to what needs to happen next?


Yoav Weiss

Jan 24, 2021, 11:10:37 PM1/24/21
to Thomas Guilbert, Husain Bengali, blink-dev, Chris Cunningham, Dan Sanders
Apologies for missing this...

Merging back those changes makes sense, but as it exposes a new API, an "Intent to Experiment" for that new API seems warranted. Once approved, you can expose that new API under the old OT.

Guido Urdaneta

Jan 27, 2021, 1:31:00 PM1/27/21
to Yoav Weiss, Thomas Guilbert, Husain Bengali, blink-dev, Chris Cunningham, Dan Sanders
I just sent an Intent to Experiment to blink-dev. 

Reply all
Reply to author
0 new messages