WebCodecs Origin Trial in Chrome 92: Breaking changes and new features
Chris Cunningham
Breaking changes
VideoFrame
PlaneInit dict no longer has 'rows', adds an optional 'offset'
VideoFrame.createImageBitmap() was removed. Use self.createImageBitmap(frame) (VideoFrame is a CanvasImageSource).
VideoFrame.destroy() was removed. Use VideoFrame.close().
VideoFrame(planes, ...) constructor no longer has format arg. Format is now a required member of VideoFramePlaneInit dictionary.
AudioFrame
Renamed to AudioData. In M92 this is simply a rename; no changes to interface members. In a later release we will make larger breaking changes to AudioData (decoupling from AudioBuffer) to match the current spec.
AudioDecoder/VideoDecoder
Users must provide a key chunk (frame) after calling flush(). Failing to do so may break decoding on some platforms and not others. More rigorous enforcement (break everywhere) will be added in M93.
ImageDecoder
Removed ImageDecoder.decodeMetadata() in favor of ImageTrackList.ready
Noteworthy additions
New VideoFrame "regions" API for simplified pixel copying
New API VideoFrame.copyTo(destination, { region: ..., layout: ... }). This will replace planes.readInto() in a later release.
New VideoFrame.codedRegion and visibleRegion properties for convenience when calling copyTo().
The spec now uses the name "rectangle" (or rect) to describe regions, but "regions" is the name shipped in M92.
VideoFrame is now transferrable. Putting a frame in a transfer list close()s the source frame and transfers the underlying data to a new object in the destination realm.
Added chrome://tracing support (media category) for encoders and decoders.
Žanetta Berķe
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALG6eSorbF4ObPUjiD6HrqU_WnUL4u6sKc_LtaKQd25vY4mNtA%40mail.gmail.com.
Chris Cunningham
Correction: My earlier message mentions the addition of VideoFrame.copyTo(). In M92, this method is named VideoFrame.readInto() (matching Plane.readInto()). I'll use that name in the text below. In a M93, this name will change to VideoFrame.copyTo(), as already reflected by the spec.
Other notes:
The PlaneInit offset member indicates where the source pixel data begins as the UA copies the data from the source buffer into its internal memory. Internally, the UA may store the data however it prefers, optionally trimming off padding bytes prior to the offset.
We did not add an offset attribute to the Planes interface (e.g. videoFrame.planes[i].offset). The Plane.readInto() method will always write data to the destination BufferSource starting with the 0th byte.
NOTE: In a later release, the Planes interface will be removed entirely in favor of videoFrame.readInto().
By default, the new videoFrame.readInto() method will also write to the destination buffer starting with the 0th byte and each plane will be tightly packed. For convenience, callers can determine the PlaneLayout (offsets and strides) of the tightly packed planes by inspecting the returned promise's value.
Callers may override the default tight packing by providing their own sequence of plane layouts when calling videoFrame.readInto(). The offsets and strides in these layouts may be totally different from those that may have been given in the PlaneInit dict during construction.
Callers may use videoFrame.allocationSize(), providing options as desired, to determine the appropriate allocation size for the destination buffer provided to videoFrame.readInto().
In this release, videoFrame.readInto() does not accept a SharedArrayBuffer as a destination. This is an oversight on our part, remedied in M93. Apps that were already using SharedArrayBuffer may opt to continue using the videoFrame.planes[i].readInto() for M92.
Best,