This feature defines an API surface for manipulating raw media carried by MediaStreamTracks such as the output of a camera, microphone, screen capture, or the decoder part of a codec and the input to the decoder part of a codec. It uses WebCodecs interfaces to represent raw media frames and exposes them using streams, similarly to the way the WebRTC Insertable Streams spec exposes encoded data from RTCPeerConnections.
As with all new features, the main interoperability risk is that other browsers do not implement it. There is no compatibility risk since the existing behavior is unaffected if the feature is not used.
Gecko: Negative. Their main objections are audio support and allowing processing on the Window scope. See:
WebKit: Negative. They object to using the streams API, supporting audio and allowing processing on the Window scope. See:
Web developers: Positive so far, but more feedback being collected.
* Are there any other platform APIs this feature will frequently be used in tandem with?
This API will be used together with other MediaStream and WebRTC related APIs, such as getUserMedia, getDisplayMedia, and RTCPeerConnection.
* Could the default usage of this API make it hard for Chrome to maintain good performance (i.e. synchronous return, must run on a certain thread, guaranteed return timing)?
* Will it be challenging for developers to take advantage of this feature immediately, as-is?
* Would this feature benefit from having polyfills, significant documentation and outreach, and/or libraries built on top of it to make it easier to use?
This feature introduces a new MediaStreamTrack sink and source.
The sink (MediaStreamTrackProcessor) exposes data that is already exposed in Chrome by other sinks (e.g., media elements, Web Audio, peer connection. In this sense, it does not introduce new attack surfaces other than the API implementation itself. Security relies on existing security mechanisms for MediaStreamTrack sources. For example, all camera or screen-capture tracks require user authorization and element capture does not allow access to cross-origin data.
The source (MediaStreamTrackGenerator) allows users to create new tracks using data already available to the Web page. In this case, security relies on the same-origin policy, as the document can only create media frames using data that is already visible.
All the code for this feature runs in the sandboxed renderer process and therefore respects the Rule of Two.
We are looking for feedback with regards to the API shape and its performance, especially for the video processing use case. We also want to learn if the WebCodecs interfaces for raw data need adjustment to better support the use cases enabled by this API.
Since this API can be seen as an extension to WebCodecs, we are reusing the WebCodecs Origin Trial token in order to make it easier for existing participants in the WebCodecs trial to use this API.
This feature depends on WebCodecs,, which itself has had experimentation extended (see the WebCodecs Intent To Extend Origin Trial). The trial extension will also provide more time to improve our implementation and seek cross UA consensus. This feature is already controlled by the WebCodecs origin trial.
This experiment is controlled by the WebCodecs origin trial. It started on M90, and was initially expected to conclude with the release of M92 stable. We are now requesting to continue to use the WebCodecs trial for the additional 2 milestones (M93 and M94) for which that trial will run, concluding with the release of M94 stable.
Intent to prototype: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/gkBkLvGKuzc
Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/fyJfqEwP1FY
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAP4M0WbPpy6wrrPWvkxtKABZB_8FjondUZj8GaMz9oVouXoNNw%40mail.gmail.com.
Web developers: Positive so far, but more feedback being collected.Linkable positive signals: https://twitter.com/search?q=url%3Ahttps%3A%2F%2Fweb.dev%2Fmediastreamtrack-insertable-media-processing%2F%20-from%3A%40tomayac&src=typed_query&f=live (see the ReTweets and Favorites).
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALgRrLn2YfMpPVTjEeee1izDFwxvH_vdvAoU7ALKaXuKznXgTg%40mail.gmail.com.
Do we have concrete users and concrete examples for the contention points? (Window scope, audio processing, etc)
Web developers: Positive so far, but more feedback being collected.Linkable positive signals: https://twitter.com/search?q=url%3Ahttps%3A%2F%2Fweb.dev%2Fmediastreamtrack-insertable-media-processing%2F%20-from%3A%40tomayac&src=typed_query&f=live (see the ReTweets and Favorites).Any signals from current OT participants?
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfUtiB5kbmtbozO1JG7W_3Ox4gj_E%3Dct5RvrsJ8oh%3Dwo0w%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxZ1wLMnqj9C1_NLVB6TTmXgaW-f_sFC%2BjtByPux1rzn0Q%40mail.gmail.com.