Intent to ship: Web Codec API

[Paul Adenot]

As of Firefox 130 I intend to turn Web Codecs API on by default on all desktop platforms. It has been developed behind the dom.media.webcodecs.enabled preference, and the part of the API related to images is not shipping at this time, but has been implemented, it is likely to ship later this year. Audio and Video, and encoding and decoding is supported. The API has been shipping in Chromium for some time, and implementation in WebKit is underway (https://wpt.fyi/results/webcodecs shows the state of things).

Bug to turn on by default: http://bugzilla.mozilla.org/show_bug.cgi?id=1908572 is the bug to enable, and https://bugzilla.mozilla.org/show_bug.cgi?id=webcodecs is the meta bug for this.

Standard: https://w3c.github.io/webcodecs/, in the W3C Media Working Group

This feature was previously discussed in this "Intent to prototype" thread: https://groups.google.com/a/mozilla.org/g/dev-platform/c/3g0fnn6682A

Let me know if you have any questions,

Paul.

[Tom Ritter]

Sorry for not replying to the intent to prototype email, I've been out
for a bit. I was trying to figure out how much, if anything, this
exposed from a fingerprinting point of view - it seems to be almost
nothing, less actually than MediaCapabilities, but I did find the
following I wanted to check on:

- https://searchfox.org/mozilla-central/rev/ea91f336d0004ca28c909da948cb363f3e560877/dom/media/platforms/PEMFactory.cpp#80
- This will expose if the user has EME enabled, which is not very
concerning, you can check that lots of ways

- https://searchfox.org/mozilla-central/rev/ea91f336d0004ca28c909da948cb363f3e560877/dom/media/platforms/ffmpeg/FFmpegDataDecoder.cpp#310-313
- It looks like this defers codec checking to libav, which is
subclassed based on the version of the library the user has? I doubt
this is different in practice, right?

- https://searchfox.org/mozilla-central/rev/ea91f336d0004ca28c909da948cb363f3e560877/dom/media/platforms/android/AndroidEncoderModule.cpp#19-26
- This is probably the most worrisome to me, it seems like you'd be
able to determine if a user has hardware support for these codecs.

-tom

> --
> You received this message because you are subscribed to the Google Groups "dev-pl...@mozilla.org" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dev-platform...@mozilla.org.
> To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-platform/CANWt0Wpc1t1%3Dam19oQt%3DKDpts_Fbv7nqMZ3VB2KepPBUJQrWqw%40mail.gmail.com.

[Paul Adenot]

On Fri, Aug 23, 2024 at 10:22 PM Tom Ritter <t...@mozilla.com> wrote:

Sorry for not replying to the intent to prototype email, I've been out
for a bit.  I was trying to figure out how much, if anything, this
exposed from a fingerprinting point of view - it seems to be almost
nothing, less actually than MediaCapabilities, but I did find the
following I wanted to check on:

 - https://searchfox.org/mozilla-central/rev/ea91f336d0004ca28c909da948cb363f3e560877/dom/media/platforms/PEMFactory.cpp#80
- This will expose if the user has EME enabled, which is not very
concerning, you can check that lots of ways

Web Codecs doesn't deal with EME at all, only with clear media.

 

 - https://searchfox.org/mozilla-central/rev/ea91f336d0004ca28c909da948cb363f3e560877/dom/media/platforms/ffmpeg/FFmpegDataDecoder.cpp#310-313
- It looks like this defers codec checking to libav, which is
subclassed based on the version of the library the user has?  I doubt
this is different in practice, right?

On all OSes, this classes will use a subsetted version of FFmpeg that is vendored, for all royalty-free codecs. On Linux Desktop, this will load the system's libraries, and use them for non-royalty-free codec (downstream can make it so some royalty-free codecs will use the system libraries as well). The version itself isn't exposed.

 

 - https://searchfox.org/mozilla-central/rev/ea91f336d0004ca28c909da948cb363f3e560877/dom/media/platforms/android/AndroidEncoderModule.cpp#19-26
- This is probably the most worrisome to me, it seems like you'd be
able to determine if a user has hardware support for these codecs.

This isn't shipped (desktop only for now), and is going to return the same info as https://developer.mozilla.org/en-US/docs/Web/API/MediaCapabilities. We've added something based on the resist fingerprinting pref on the decode side a long time ago, we can very much do the same thing on the encode side, and verify it's all good at the same time. I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1914839 to address this.

Thanks,

Paul.