This feature defines an API surface for manipulating raw media carried by MediaStreamTracks such as the output of a camera, microphone, screen capture, or the decoder part of a codec and the input to the decoder part of a codec. It uses WebCodecs interfaces to represent raw media frames and exposes them using streams, similarly to the way the WebRTC Insertable Streams spec exposes encoded data from RTCPeerConnections.
TAG review statusPending
Interoperability and Compatibility
As with all new features, the main interoperability risk is that other browsers do not implement it. There is no compatibility risk since the existing behavior is unaffected if the feature is not used.Gecko
: No signalEdge
: No signalWebKit
: No signalWeb developers
* Are there any other platform APIs this feature will frequently be used in tandem with?
This API will be used together with other MediaStream and WebRTC related APIs, such as getUserMedia, getDisplayMedia, and RTCPeerConnection.
* Could the default usage of this API make it hard for Chrome to maintain good performance (i.e. synchronous return, must run on a certain thread, guaranteed return timing)?
* Will it be challenging for developers to take advantage of this feature immediately, as-is?
* Would this feature benefit from having polyfills, significant documentation and outreach, and/or libraries built on top of it to make it easier to use?
This feature introduces more ergonomic ways to expose data that is already exposed in Chrome via other APIs (e.g., MediaStreams in combination with media elements, canvas and Web Audio). In this sense, it does not introduce new attack surfaces other than the API implementation itself plus the proposed explicit signaling streams.
An application may try to DoS a MediaStream source by issuing many control signals, but since these signals are just hints they are not a reliable way to launch a DoS attack.
This feature relies on existing security mechanisms for MediaStreamTracks. For example, all camera or screen-capture tracks require user authorization, and element-capture tracks do not allow access to cross-origin data.
All the code for this feature runs in the sandboxed renderer process and therefore respects the Rule of Two.
Goals for experimentation
We are looking for feedback with regards to the API shape and its performance, especially for the video processing use case. We also want to learn if the WebCodecs interfaces for raw data need adjustment to better support the use cases enabled by this API. A number of partners have expressed interest in participating in the Origin Trial.
Since this API can be seen as an extension to WebCodecs, we would like to reuse the WebCodecs Origin Trial token in order to make this API easier to use for existing participants in the ongoing WebCodecs trial.
Ongoing technical constraints
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?YesNo, but WPTs will be added before an intent to ship.
Link to entry on the Chrome Platform Statushttps://chromestatus.com/feature/5499415634640896
Links to previous Intent discussionsIntent to prototype: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/gkBkLvGKuzc