Mysterious WebCodecs and ServiceWorker behavior

[Cris van Pelt]

Hi chromium-discuss,

I've been playing around with webcodecs and serviceworkers a bit, mainly in the context of developing Chrome extensions. I ran into some strange situations that may be bugs or a classic PEBKAC situation.

To narrow things down, I also tried to reproduce it in Safari-- I had different but similar, issues there.

These are things I'm seeing:

1.  When I create a ReadableStream containing a VideoFrame and use it as a Transferable with postMessage to transfer it to the service worker, different things happen in Chrome and Safari:
   (a) Chrome: Chrome will post the stream successfully. The service worker will receive the stream, but then it will throw.a "chunk could not be cloned" on read. (a) Chrome: Chrome will post the stream successfully. The service worker will receive the stream, but then it will throw.a "chunk could not be cloned" on read.
   (b) Safari refuses to even post the ReadableStream, and immediately throws a data clone error.
2. When I use postMessage to transfer a VideoFrame object, which should be Transferable, again, one of two things happens.
   (a) Chrome: the message ... evaporates. The service worker's 'message' listener is never called.
   (b) Safari: the message arrives, but its 'data' property is null.
3. When I use 'Object.assign({}, videoFrame)' the result is '{}'.
4. When copying the VideoFrame into an ArrayBuffer and storing its metadata in an object, using postMessage fails, as the "colorSpace" property can't be cloned. Serializing colorSpace to JSON and parsing it back creates a postable object, with a transferable ArrayBuffer.
5. Not quite as relevant to chromium-dev, but deserializing the VideoFrame using the ArrayBuffer and metadata works fine in Chrome -- Safari barfs on it.

As I am not too experienced in this area, I would lean toward user error, but given Chrome's and Safari's (seemingly) bizarre behavior, I'm not so sure.

I wrote some test code and put it in this repo: https://github.com/crisvp/videoframe-sw/

This all surfaced when using MediaStreamTrackProcessor and friends, but it seems like a simple VideoFrame, and ReadableStream are enough to repro.

My Chrome version: 116.0.5792.0 (Official Build) canary (arm64)

Full version info: https://gist.github.com/crisvp/f267557134a2a00fc68c931c1305740c

Thanks! Any. thoughts or suggestions would be much appreciated.