Intent to Ship: H26x Codec support updates for MediaRecorder

[Markus Handell]

Contact emails:

zhu...@bytedance.com

hand...@google.com

Explainer:

Explainer: Codec support updates for MediaRecorder

crbug.com/40276507

TAG review:

N/A, small incremental change

Summary:

Support for HEVC platform encoding was added in WebCodecs in Chromium M130. As a follow-up, support has been added to the MediaRecorder API in Chromium. The API now supports both MP4 and Matroska muxer types with different HEVC and H.264 mime type specifications. HEVC encoding is only supported if the user's device and operating system provide the necessary capabilities.

Link to “Intent to Prototype”

N/A, small incremental change and going directly for shipping.

Risks

Interoperability and Compatibility

Gecko: Firefox 136 recently added support for decoding HEVC videos, see https://bugzilla.mozilla.org/show_bug.cgi?id=1924066. Standards position request link: https://github.com/mozilla/standards-positions/issues/1174

WebKit: No official signal but note that MacBooks/iPhones have >95% HW encode/decode support of H265 and that Safari has added support for this codec in adjacent web APIs such as WebRTC. Standards position request link: https://github.com/WebKit/standards-positions/issues/460

Web developers: Positive. Support implemented by ByteDance & Intel.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

The support is currently limited to Windows, macOS and Android hosts. ChromeOS and Linux support may follow in the future when/if their platform codec availability allows.

Is this feature fully tested by web-platform-tests?

Yes, the existing test suite was extended with cases for the new codec strings.

Tracking bug

crbug.com/40276507

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6375884229181440

[Markus Handell]

CC Dale Curtis as well.

[Mike Taylor]

Could you please request the various security, privacy, enterprise, etc. bits in your chromestatus entry?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/edc31811-00da-4410-bfc7-b224bb825e4fn%40chromium.org.

[Ashley Gullen]

As a web developer interested in Chrome's video encoding capabilities with both WebCodecs and MediaRecorder, I had no idea until reading this that Chrome actually already shipped support for HEVC encoding with WebCodecs in M130 - it doesn't appear to be listed on chromestatus.com. Could Google make sure significant updates like this are noted there? Otherwise important updates can pass us right by!

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/cb683232-ef5e-47f4-93fa-3e8809a0d2b5%40chromium.org.

[Dale Curtis]

Sorry about that. Historically video codec changes were handled via internal processes since they were aspects of the underlying Chromium platform. Especially so in the case of HEVC where it requires hardware support. Going forward all video codec changes will use the Blink process.

- dale

[Markus Handell]

> Could you please request the various security, privacy, enterprise, etc. bits in your chromestatus entry?

Done!

[Philip Jägenstedt]

Hi,

The API owners (Dan, Alex, Vlad, Yoav, Mike, Chris, Daniel, me) discussed this today. We'd like to understand a bit better where HEVC is already supported and what contexts don't yet support it.

As mentioned, HEVC (encode and decode) is already supported in WebCodecs. The tests on wpt.fyi are failing, but presumably that's because the VMs the tests run in don't have the required hardware support. I tried a few of the tests and they pass on my own Mac machine.

Is decoding HEVC in a plain <video> supported? I tried `document.createElement('video').canPlayType('video/mp4; codecs=hvc1')` which returns "probably" in Safari, but the empty string in Chrome. I can't tell from https://caniuse.com/hevc if I should expect it to be supported or not.

And how about in MSE?

Are there contexts where HEVC is not yet supported that aren't covered by this intent? WebRTC comes to mind, but are there any other?

Note that HEVC is called out as an example in the Blink principles of web compatibility. Wherever royalty-free codecs like AV1 can be used, that's definitely preferable for the Chromium project and the web. Do we already support AV1 in every context where HEVC is supported or is planned?

Best regards,

Philip

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJjiFfG4zq4_mUknwsG%2BLetoM_RE1fiqFKVjkAyLJ4GpSUUkCw%40mail.gmail.com.

[Markus Handell]

I'll need to defer definitive answers regarding <video> and MSE questions to Dale. But selecting video/mp4;codecs=hvc1.1.6.L186.B0,opus as format on https://webrtc.github.io/samples/src/content/getusermedia/record/ with MediaRecorderHEVCSupport enabled yields a video that plays back in the same window.

> Are there contexts where HEVC is not yet supported that aren't covered by this intent? WebRTC comes to mind, but are there any other?

Correct, WebRTC doesn't yet have support but there's activity happening to progress towards support though. What's the context of the question though, interoperability? Recorded content from MediaRecorder can't immediately be used with WebRTC connections.

> Do we already support AV1 in every context where HEVC is supported or is planned?

Both WebRTC and MediaRecorder have had AV1 support for a long time. I'll need to defer MSE & video support statements to Dale.

[朱思达]

Is decoding HEVC in plain <video> supported? I tried `document.createElement('video').canPlayType('video/mp4; codecs=hvc1')` which returns "probably" in Safari, but the empty string in Chrome. I can't tell from https://caniuse.com/hevc if I should expect it to be supported or not.

And how about in MSE?

The correct codec string should be `video/mp4;codecs="hev1.1.6.L120.90"`, that's why 'video/mp4; codecs=hvc1' is rejected on Chromium. You could try the code below to test the support:

```javascript

const video = document.createElement('video'); 

if (video.canPlayType('video/mp4;codecs="hev1.1.6.L120.90"') === 'probably') { 

  console.log('HEVC is supported!');

}

if (MediaSource.isTypeSupported('video/mp4;codecs="hev1.1.6.L120.90"')) { 

  console.log('HEVC is supported!');

}

```

For video decoding context, there are four major API FYI: 

1. <video> + File.
2. <video> + MSE.
3. WebCodecs.
4. WebRTC.

The HEVC support for the first three APIs has been enabled by default in M107 (On all platforms including: Windows, macOS, Linux, Android, ChromeOS). For the WebRTC API, there's activity happening to progress towards.

Are there contexts where HEVC is not yet supported that aren't covered by this intent? WebRTC comes to mind, but are there any others?

For video encoding context, there are three major API FYI:

1. WebCodecs.
2. MediaRecorder.
3. WebRTC.

The HEVC support for the WebCodecs API has been enabled by default in M130 (On WIndows, macOS, Android), and for the WebRTC API, there's activity happening to progress towards.

From: "Markus Handell"<hand...@google.com>

Date: Thu, Feb 20, 2025, 03:40

Subject: [External] Re: [blink-dev] Re: Intent to Ship: H26x Codec support updates for MediaRecorder

To: "Philip Jägenstedt"<foo...@chromium.org>

Cc: "Dale Curtis"<dalec...@chromium.org>, "Ashley Gullen"<ash...@scirra.com>, "Mike Taylor"<mike...@chromium.org>, "blink-dev"<blin...@chromium.org>, "朱思达"<zhu...@bytedance.com>

[Philipp Hancke]

https://webrtc.github.io/samples/src/content/getusermedia/record/

just got an "upload locally" button (thanks Harald!) which makes tinkering easier - download in one browser, upload in the other.

"interop" for MediaRecorder is an both an issue and an interesting concept.

The issue has gotten "better" when it comes to container formats since Chromium started supporting MP4 in addition to webm mid-2024-ish.

For audio codecs there is still a "divide" with Safari insisting on AAC from what I can see (https://webkit.org/blog/11353/mediarecorder-api/; despite supporting Opus in WebRTC).

Unlike in WebRTC there is no codec negotiation protocol (apart from humans) when you send files recorded with MediaRecorder so

quite often a file upload service in the path may do transcoding anyway.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYcNt017garh4Ps-LLJRmO1DbsHvt7yZT19Yj-feXifmDA%40mail.gmail.com.

[Alex Russell]

For transparency, I'm disinclined to make the proprietary vs. open codecs situation worse just because Apple is harming the ecosystem in an unaccountable way.

It would be helpful in making a decision about this if we understood the full matrix of what is supported where, and under what conditions. Does such a spreadsheet exist?

Thanks,

Alex

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/edc31811-00da-4410-bfc7-b224bb825e4fn%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/cb683232-ef5e-47f4-93fa-3e8809a0d2b5%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJjiFfG4zq4_mUknwsG%2BLetoM_RE1fiqFKVjkAyLJ4GpSUUkCw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Dale Curtis]

Roughly https://www.chromium.org/audio-video/ covers the codecs that should be _possible_ to work throughout Chrome APIs (modulo WebRTC):

• Video decode and encode support should ideally be (and has historically been) symmetric.
• HEVC has been a temporary exception here as it requires work from WebRTC and MediaRecorder to integrate.
• Android doesn't ship a software H.264 encoder, so is limited to working Android implementations.
• Not all platforms have a working / performant H.265 encoder.
• Audio encode is limited to Opus (everywhere) and AAC (where OS support exists).
• Note: The above is for Chrome only, Chromium builds don't enable any proprietary codecs (even when supported by hardware) by default due to licensing requirements. I.e., Chromium does not ship AAC, H.264, or H.265 support.
• WebRTC is more complicated, but mostly covers the codecs above plus some speech specific codecs IIRC.

- dale

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/edc31811-00da-4410-bfc7-b224bb825e4fn%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/cb683232-ef5e-47f4-93fa-3e8809a0d2b5%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJjiFfG4zq4_mUknwsG%2BLetoM_RE1fiqFKVjkAyLJ4GpSUUkCw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Philip Jägenstedt]

Thank you Sida for that breakdown, it's good to know we're not overlooking any encoding or decoding surface.

The most important question here is the one of royalty-free codecs vs non-RF codecs, and the API owners will discuss this again alongside the WebRTC intent.

Poking a bit at the details, however, is it really the case that even when Chrome and Safari support playing the same file, there's no codecs strings that they both claim support for? Given your example, it seems like `video/mp4;codecs="hev1.1.6.L120.90"` is supported by Chrome but not Safari, while the opposite is true for `video/mp4;codecs="hvc1"`.

For MediaRecorder.isTypeSupported(), will we have the same situation? Are these strings clearly defined by specs and tested in WPT?

Best regards,

Philip

[朱思达]

Given your example, it seems like `video/mp4;codecs="hev1.1.6.L120.90"` is supported by Chrome but not Safari, while the opposite is true for `video/mp4;codecs="hvc1"`.

AFAIK, Safari doesn't support HEVC MediaRecorder encoding. If you are talking about decoding, for decoding, safari supports both  `video/mp4;codecs="hvc1"` and  `video/mp4;codecs="hvc1.1.6.L120.90"` mimetypes, but technically, the "full" mimetypes is an ISO standard (Please refer to: ISO/IEC 14496-15:2019 - Annex E), and `video/mp4;codecs="hvc1"` is not incorrect.

For MediaRecorder.isTypeSupported(), will we have the same situation? Are these strings clearly defined by specs and tested in WPT?

They are defined in https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/mediacapture-record/MediaRecorder-canvas-media-source.https.html;l=16 and ISO/IEC 14496-15:2019 - Annex E.

Best regards,

Sida

From: "Philip Jägenstedt"<foo...@chromium.org>

Date: Tue, Mar 4, 2025, 17:18

Subject: Re: [External] Re: [blink-dev] Re: Intent to Ship: H26x Codec support updates for MediaRecorder

To: "朱思达"<zhu...@bytedance.com>

Cc: "Markus Handell"<hand...@google.com>, "Dale Curtis"<dalec...@chromium.org>, "Ashley Gullen"<ash...@scirra.com>, "Mike Taylor"<mike...@chromium.org>, "blink-dev"<blin...@chromium.org>

[Henrik Boström]

For reference, I had some hardware availability numbers over in the I2S thread for H265 in WebRTC.

[Philip Jägenstedt]

The API owners discussed this again last week, considering it alongside H265 (HEVC) codec support in WebRTC. We invited Dale Curtis, Frank Galligan, and Henrik Boström to better understand the situation.

The case for supporting HEVC is that it can result in a better user experience if supported in hardware, and Henrik provided some numbers on that in the WebRTC thread. From a platform perspective, one might also say that web apps are at a disadvantage to native apps if they can't use the same device capabilities.

It remains the case that we prefer royalty-free (RF) codecs and don't want to disincentivize their adoption/use. AV1 has already been supported for a long time and is in wide use, so the risk in this case seems small. An important point is that web developers cannot depend on HEVC support exclusively, given the support numbers that Henrik shared.

To avoid setting a precedent for future non-RF codecs, approval for HEVC in the remaining contexts comes with some caveats:

• The approval is only for exposing OS-provided HEVC support, not for other OS-provided codecs, and not for browser-provided HEVC.
• That there is a well established alternative in AV1 is important.
• We're open to mitigations like Finch-disabling support for a percentage of users to ensure that HEVC support cannot be taken for granted.
  

LGTM1 with those caveats.

[Chris Harrelson]

LGTM2

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYcX26Vuu5_MuRGJuqk3TWUo6RVQCvP_pGPAJp_UBdn5MA%40mail.gmail.com.

[Yoav Weiss (@Shopify)]

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.