WebCodecs Shipping in Chrome 94: Breaking changes and new features
Chris Cunningham
To WebCodecs origin trial / experimental users:
Chrome 94 is expected to release to stable channel on Sep 21st. Please see below for breaking changes and new features for WebCodecs in this release.
WebCodecs will ship in M94, which means this is the final email of this sort. Breaking changes in future releases will be rare and will include a deprecation period. Thank you all for your patience as we iterated on the design!
Breaking changes in M94
General
VideoDecoder, VideoEncoder, AudioDecoder, AudioEncoder, and ImageDecoder are now only exposed in secure contexts (generally https or localhost).
Chrome will reclaim decoders and encoders (calling error() callback w/ QuotaExceeded) that are inactive (no codec API calls, no outputs generated) for 1+ minutes. Developers are encouraged to avoid constructing new codecs until new activity is imminent. This frees up system resources (including for use by the same site running in a separate tab).
In a later release (no sooner than 96), we will reclaim codecs more aggressively, per the rules in https://www.w3.org/TR/webcodecs/#resource-reclamation. This includes narrow cases of active codecs running in background tabs. These rules are crafted to avoid breaking core use cases (RTC, media production, audio rendering, etc...). Please have a look at this plan and raise concerns in our github. Additionally, please update sites to prepare for this change.
Site developers can use the command line flag --disable-features=ReclaimInactiveWebCodecs to disable reclamation locally.
VideoEncoder
VideoEncoderConfig.latencyMode defaults to "quality". Apps that want "realtime" (e.g. RTC) should take care to set this attribute accordingly. In previous releases the default behavior was platform/codec specific.
VideoEncoderConfig.hardareAcceleration attribute's values are changed: "allow", "deny", and "require" are now "no-preference", "prefer-software", and "prefer-hardware" respectively. The semantics are unchanged in Chrome (we treat the preferences as strict requirements).
VideoDecoder
VideoDecoderConfig.hardwareAcceleration attribute's values changed, matching the change for VideoEncoder above.
VideoFrame
VideoFrame.allocationSize() will throw an InvalidStateError exception if called on a closed VideoFrame.
VideoFrame.colorSpace is no longer nullable.
AudioData
AudioData.allocationSize() will throw an InvalidStateError exception if called on a closed AudioData.
Noteworthy additions in M94
AudioDataCopyToOptions.format allows for sample format conversion to "f32-planar" for easy integration with WebAudio's AudioBuffer and AudioWorklet.
Added SharedArrayBuffer support for all "BufferSource" usage (AudioData.copyTo(), VideoDecoderConfig.description, ImageDecoder, etc...)
When hardwareAcceleration = no-preference, VideoEncoder will automatically fallback to software if hardware encoding hits an error.
Reduced copies when passing canvas-backed frames to VideoEncoder.
Added a 'duration' attribute to EncodedAudioChunk interface
Best,
Chris