Intent to implement: Web Codecs
Paul Adenot
All,
We've started the implementation already, but I realized we never sent the email, so here it is, better late than never.
Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=webcodecs
Specification: https://w3c.github.io/webcodecs/
Standards Body: W3C, on standards track
Platform coverage: All platforms Firefox runs on
Preference: dom.media.webcodecs.enabled, will be enabled soon in Nightly (see below for more info)
DevTools bug: No specific need anticipated outside of regular and existing tools
Link to standards-positions discussion: https://github.com/mozilla/standards-positions/issues/209
Other browsers:
- Blink: shipped on all platforms since version 94 (21st September 2021), intent to ship: https://groups.google.com/a/chromium.org/g/blink-dev/c/7UlTzFMbTFs
- WebKit: currently implementing, some interfaces available in Safari Tech Preview (e.g. https://webkit.org/blog/13575/release-notes-for-safari-technology-preview-157/)
web-platform-tests: https://wpt.fyi/results/webcodecs?label=master&label=experimental&aligned&q=webcodecs, rather comprehensive test suite. We're contributing tests to this suite while implementing.
Web Codecs aims at offering a low-level and high performance audio/video
encoding/decoding interface on the Web. Additionally, it provides image decoding
capabilities.
This essentially exposes to the Web capabilities of Web browsers that have been long available with high level and sometimes inflexible interfaces (e.g. HTMLMediaElement, MediaRecorder, bits of WebRTC, parts of the Web Audio API). This unlocks new use-cases for the Web, and/or allows significant performance improvements, enough to in turns allow new types of applications to be viable on the Web. Production and experimental use case as of today include video conferencing, low-latency broadcast, cloud gaming or desktop, professional media production applications, creative software, video game engine. Web Codecs is also used to optimize a number of applications that were using other interfaces that weren't ideal for the task at hand (e.g., getting the pixel data for an image could be done by drawing it to a canvas and reading it back, but can now be done directly).
We've been involved in the spec from day one, and I've been an editor from the start. The spec is now very stable, and of good quality. The specification has been designed in a way to allow addition of functionality without breaking changes (an example would be to control the encoding quality of a media), and part of the specifications are implementable and usable independently for each other. It's for example very useful to only have the VideoDecoder and VideoFrame interfaces on their own, or to support the VideoEncoder interface without
all encoding settings implemented. For this reason, we intend to enable interfaces in Nightly builds as we implement them, to get as much feedback as early as possible. Shipping in release will happen at a later date, when a coherent feature set will be implemented.
The specification has two of sibling documents: a first registry, containing information pertaining to each codec that Web Codecs allows using and their specific settings, and a second registry containing possible metadata a VideoFrame object can carry (currently empty).
Conforming implementions don't have to implement all the items of those registries to be conformant. It's for example possible to ship without support for decoding and encoding all codecs.
The specification has been reviewed by the TAG (two reviews: first, second).
The security and privacy section of the specification are rather detailed, and have received attentions from various parties, implementers, researchers and users alike.
The specification repository contains a website containing live examples, showing example usage and various common scenarios: https://w3c.github.io/webcodecs/samples/.
Let me know if you have any questions,
Paul.