Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Intent to Ship: H265 (HEVC) codec support in WebRTC

801 views
Skip to first unread message

Henrik Boström

unread,
Mar 3, 2025, 5:36:23 AMMar 3
to blink-dev
Contact emails
Specification

But there are no new API surfaces, just a new mime type addition exposed in existing APIs - the new mime type ("video/H265") is queryable via MediaCapabilities API:
https://w3c.github.io/media-capabilities/#dom-videoconfiguration-contenttype

If supported it will show up in SDP generated by WebRTC and sender/receiver capabilities which can be used to set codec preferences (existing WebRTC API). Note that the codec is only used if successfully negotiated, meaning if the other endpoint does not support the codec then a different one is automatically used instead.

Summary

This newer codec has increased compression efficiency (higher quality per bitrate) relative to VP8/VP9/H264 and very strong hardware support going back over a decade. This translates into improved visual experience, increased battery life and reduced risk of performance issues. The codec is already industry standard and we should support it in WebRTC when provided by the platform, i.e. if it is available in hardware. This codec is already available in WebCodecs (M130) and MediaRecorder APIs (soon?). Support will be queryable via MediaCapabilities API. Safari has already shipped support in WebRTC.


H265 encoding is only available if the user's device and operating system provide the necessary capabilities as we will not provide a software implementation to fall back to.

Blink component
Blink>WebRTC>Video

TAG review/status
N/A, small incremental change

Risks
Interoperability and Compatibility

No significant backwards compat risk since the codec is only used if both endpoints negotiate it. Interop risk is also very small for the same reason - note that while this codec is new, needing to negotiate support else falling back to a different format is not a new dynamic: old codecs like H264 (that's ...4, not ...5) for example come in different flavors (profile IDs) where support is also HW and implementation specific and need to be negotiated, to that regard this is not fundamentally any different.


Gecko: Standards position link: https://github.com/mozilla/standards-positions/issues/1188
(Firefox has some non-WebRTC support for H265 already such as media playback).

WebKit: Shipped, including WebRTC support.

Web developers: Strongly positive. This includes both internal and external developers who have both contributed code and emails asking for timelines.

WebView application risks

None


Debuggability

N/A


Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?
Yes

Is this feature fully tested by web-platform-tests?
Yes

Flag name on about://flags
No new flags are added specific to this codec but there are existing flags to disable  HW acceleration (any codec) which would also disable H265.

Finch feature name
WebRtcAllowH265Send for H265 encoding support and
WebRtcAllowH265Receive for H265 decoding support

Requires code in //chrome?
False

Tracking bug
https://crbug.com/391903235

Estimated milestones
Shipping on desktop
136
Shipping on Android
136
Shipping on WebView
136

Anticipated spec changes

None


Link to entry on the Chrome Platform Status

Alex Russell

unread,
Mar 3, 2025, 11:15:16 AMMar 3
to blink-dev, Henrik Boström
Hey Henrik,

I'd like to understand the case for the API OWNERS approving more support for patent encumbered codecs when we have AV1. Apple continues to use its hardware to push closed solutions in this space, and I understand the instinct, but I'd like to see quantified benefits.

Best,

Alex

On Monday, March 3, 2025 at 2:36:23 AM UTC-8 Henrik Boström wrote:
Contact emails

Diego Perez Botero

unread,
Mar 3, 2025, 1:36:40 PMMar 3
to blink-dev, Alex Russell, Henrik Boström
Alex - AV1 isn't an option for all WebRTC use cases due to hardware limitations. For instance, for Xbox Cloud Gaming, we do not have AV1 encoding capabilities on our server hardware and would benefit substantially from having H265 decode support on browser clients. We can already get that working end-to-end with Safari, so Chromium-based browsers essentially have a feature gap in their WebRTC implementation at this point. Also note that the H265 support being tracked here is only for when the client has hardware support for the codec, so the licensing implications are mitigated.

On Monday, March 3, 2025 at 8:15:16 AM UTC-8 Alex Russell wrote:
Hey Henrik,

I'd like to understand the case for the API OWNERS approving more support for patent encumbered codecs when we have AV1. Apple continues to use its hardware to push closed solutions in this space, and I understand the instinct, but I'd like to see quantified benefits.

Best,

Alex

On Monday, March 3, 2025 at 2:36:23 AM UTC-8 Henrik Boström wrote:
Contact emails

Henrik Boström

unread,
Mar 4, 2025, 5:34:29 AMMar 4
to blink-dev, Diego Perez Botero, Alex Russell, Henrik Boström
AV1 is great from a quality perspective, but it is not an option for use cases that require hardware acceleration. Diego mentioned potential server-side requirements, on the client-side estimates are only 8% Windows endpoints and 0% macOS endpoints have HW accelerated AV1 support. For mobile clients (including native apps which need to be able to interop with web clients in video conferencing use cases), it's 0% Android/iOS as well. As web apps are becoming more taxing on performance and battery (e.g. video effects processing becoming default) and increased expectations on video quality, hardware accelerated alternatives is important for both older and newer devices running modern web apps and for interop with mobile clients.

H.265 is a different story: HW support is 75% Windows, 99% macOS, 86% Android, 90% iOS. This goes back to devices from a decade ago which are still common in the wild. From a quality perspective, this is the only codec that is close to AV1. H.264 does not fit that bill. Hardware acceleration has been measured to reduce power consumption by 20% in basic calling use cases and above 30% when video effects processing is enabled. Combine this with the fact that video conferencing use cases draws >10X more power than when the laptop not doing heavy work (like lite web browsing) and you can imagine this can have a material impact on your device's overall battery life, not to mention improving the performance, unblocking higher resolutions and more CPU budget for features as well as unblocking HW acceleration in mobile-to-web calling and higher resolution use cases for the future.

I do not expect the need for AV1 to go away, but I think a compliment is needed for the hardware/quality balance.
On Monday, March 3, 2025 at 7:36:40 PM UTC+1 Diego Perez Botero wrote:
Alex - AV1 isn't an option for all WebRTC use cases due to hardware limitations. For instance, for Xbox Cloud Gaming, we do not have AV1 encoding capabilities on our server hardware and would benefit substantially from having H265 decode support on browser clients. We can already get that working end-to-end with Safari, so Chromium-based browsers essentially have a feature gap in their WebRTC implementation at this point. Also note that the H265 support being tracked here is only for when the client has hardware support for the codec, so the licensing implications are mitigated.

On Monday, March 3, 2025 at 8:15:16 AM UTC-8 Alex Russell wrote:
Hey Henrik,

I'd like to understand the case for the API OWNERS approving more support for patent encumbered codecs when we have AV1. Apple continues to use its hardware to push closed solutions in this space, and I understand the instinct, but I'd like to see quantified benefits.

Best,

Alex

On Monday, March 3, 2025 at 2:36:23 AM UTC-8 Henrik Boström wrote:
Contact emails

Pooja K

unread,
Mar 7, 2025, 4:28:26 PMMar 7
to blink-dev, Henrik Boström, Diego Perez Botero, Alex Russell
Our team has met with Alex Russell and we are aligned on HEVC support being limited to hardware (no software fallback). With that in mind, I wanted to provide some key conclusions/justifications that will solidify why HEVC is so important to our product and mission.

What is Xbox Cloud Gaming: Microsoft's cloud gaming service that allows users to stream Xbox games directly to their devices over the internet, without requiring a console or high-end PC. Devices include smartphones, tablets, TVs, and web browsers. The game itself runs on a datacenter server and the gameplay (audio/video) is streamed over WebRTC media streams.

Why We Need HEVC on WebRTC
We want to provide the best gaming experience to our customers by getting the most out of their client endpoint capabilities.
  • Our cloud gaming scenario requires low latency to ensure a seamless experience for players. Thus, hardware-accelerated encoding and decoding are essential as a baseline requirement (our quality bar isn't met by software encode/decode). 
  • HEVC enables us to provide a high-quality experience not only when streaming to our own hardware (Xbox Consoles) but also when streaming to Smart TVs, Windows, Android, Chrome OS, and Linux endpoints since many of them support HEVC hardware decode.
  • HEVC allows us to stream at lower bitrates than AVC while maintaining an even higher quality bar and keeping streaming costs lower.
  • We do want to support AV1 as well, but as Henrik Boström pointed out, the client hardware support for HEVC exceeds that of AV1 right now.
  • Regarding software vs hardware decode on the client side outside of the HEVC-specific conversation, we understand that MediaCapabilities APIs allow us to query the HW/SW capabilities of the client device, but there is no guarantee that those capabilities will take effect during the actual game stream. Xbox Cloud Gaming and similar scenarios care deeply about what happens during the actual stream (aka we know the device supports HW decode, but was it used?), since we need that data point to make mid-stream decisions (switch codec/resolution/FPS). Thus, we hope that Exposing decode errors / SW fallback as an event · Issue #146 · w3c/webrtc-extensions  is also investigated. Without a reliable way to detect software fallback, our best bet is to monitor decode times (via requestVideoFrameCallback) and have heuristics around what SW and HW decode should look like for a particular device, which is error prone.

On Tuesday, March 4, 2025 at 2:34:29 AM UTC-8 Henrik Boström wrote:
AV1 is great from a quality perspective, but it is not an option for use cases that require hardware acceleration. Diego mentioned potential server-side requirements, on the client-side estimates are only 8% Windows endpoints and 0% macOS endpoints have HW accelerated AV1 support. For mobile clients (including native apps which need to be able to interop with web clients in video conferencing use cases), it's 0% Android/iOS as well. As web apps are becoming more taxing on performance and battery (e.g. video effects processing becoming default) and increased expectations on video quality, hardware accelerated alternatives is important for both older and newer devices running modern web apps and for interop with mobile clients.

H.265 is a different story: HW support is 75% Windows, 99% macOS, 86% Android, 90% iOS. This goes back to devices from a decade ago which are still common in the wild. From a quality perspective, this is the only codec that is close to AV1. H.264 does not fit that bill. Hardware acceleration has been measured to reduce power consumption by 20% in basic calling use cases and above 30% when video effects processing is enabled. Combine this with the fact that video conferencing use cases draws >10X more power than when the laptop not doing heavy work (like lite web browsing) and you can imagine this can have a material impact on your device's overall battery life, not to mention improving the performance, unblocking higher resolutions and more CPU budget for features as well as unblocking HW acceleration in mobile-to-web calling and higher resolution use cases for the future.

I do not expect the need for AV1 to go away, but I think a compliment is needed for the hardware/quality balance.
On Monday, March 3, 2025 at 7:36:40 PM UTC+1 Diego Perez Botero wrote:
Alex - AV1 isn't an option for all WebRTC use cases due to hardware limitations. For instance, for Xbox Cloud Gaming, we do not have AV1 encoding capabilities on our server hardware and would benefit substantially from having H265 decode support on browser clients. We can already get that working end-to-end with Safari, so Chromium-based browsers essentially have a feature gap in their WebRTC implementation at this point. Also note that the H265 support being tracked here is only for when the client has hardware support for the codec, so the licensing implications are mitigated.

On Monday, March 3, 2025 at 8:15:16 AM UTC-8 Alex Russell wrote:
Hey Henrik,

I'd like to understand the case for the API OWNERS approving more support for patent encumbered codecs when we have AV1. Apple continues to use its hardware to push closed solutions in this space, and I understand the instinct, but I'd like to see quantified benefits.

Best,

Alex

On Monday, March 3, 2025 at 2:36:23 AM UTC-8 Henrik Boström wrote:
Contact emails

Philipp Hancke

unread,
Mar 9, 2025, 4:04:51 PMMar 9
to Henrik Boström, blink-dev, Diego Perez Botero, Alex Russell
through the IETF but it has been extremly valuable for shaping the implementation already.

Speaking as an individual below.

API owners: this sounds like an opportunity to elaborate the Chromium position a bit more than the very brief statement cited in

Am Di., 4. März 2025 um 02:34 Uhr schrieb Henrik Boström <hb...@chromium.org>:
AV1 is great from a quality perspective, but it is not an option for use cases that require hardware acceleration. Diego mentioned potential server-side requirements, on the client-side estimates are only 8% Windows endpoints and 0% macOS endpoints have HW accelerated AV1 support. For mobile clients (including native apps which need to be able to interop with web clients in video conferencing use cases), it's 0% Android/iOS as well. As web apps are becoming more taxing on performance and battery (e.g. video effects processing becoming default) and increased expectations on video quality, hardware accelerated alternatives is important for both older and newer devices running modern web apps and for interop with mobile clients.
H.265 is a different story: HW support is 75% Windows, 99% macOS, 86% Android, 90% iOS. This goes back to devices from a decade ago which are still common in the wild.
 
Are those numbers encode, decode or both? The motivator in this space is typically streaming movies which has created a lot of asymmetry between decoding and encoding.
Take this from edge://gpu on my Windows machine (Intel and NV 3060 GPU)
image.png
AV1 encode support is missing, decode support is there. Hardware encoders for RTC are extremly tricky (which is the reason OpenH264 is still very much needed)

From a quality perspective, this is the only codec that is close to AV1.

No. HEVC is generally considered to be the equivalent of VP9 so comparing it to AV1 is slightly off.

H.264 does not fit that bill. Hardware acceleration has been measured to reduce power consumption by 20% in basic calling use cases and above 30% when video effects processing is enabled. Combine this with the fact that video conferencing use cases draws >10X more power than when the laptop not doing heavy work (like lite web browsing) and you can imagine this can have a material impact on your device's overall battery life, not to mention improving the performance, unblocking higher resolutions and more CPU budget for features as well as unblocking HW acceleration in mobile-to-web calling and higher resolution use cases for the future. 
I do not expect the need for AV1 to go away, but I think a compliment is needed for the hardware/quality balance.

Hard to argue with the power consumption win for decoding.

But as my friends from xcloud say an event for when fallback (or due to lack of SW fallback: failure and freezes) happens would be essential to have.
I have never seen a follow-up on the "privacy concerns" cited in the meeting notes for
  https://github.com/w3c/webrtc-extensions/issues/146#issuecomment-1822368254

Philipp 
who prefers to spend his time improving WebRTC/AV1 even on the web
 
On Monday, March 3, 2025 at 7:36:40 PM UTC+1 Diego Perez Botero wrote:
Alex - AV1 isn't an option for all WebRTC use cases due to hardware limitations. For instance, for Xbox Cloud Gaming, we do not have AV1 encoding capabilities on our server hardware and would benefit substantially from having H265 decode support on browser clients. We can already get that working end-to-end with Safari, so Chromium-based browsers essentially have a feature gap in their WebRTC implementation at this point. Also note that the H265 support being tracked here is only for when the client has hardware support for the codec, so the licensing implications are mitigated.

On Monday, March 3, 2025 at 8:15:16 AM UTC-8 Alex Russell wrote:
Hey Henrik,

I'd like to understand the case for the API OWNERS approving more support for patent encumbered codecs when we have AV1. Apple continues to use its hardware to push closed solutions in this space, and I understand the instinct, but I'd like to see quantified benefits.

Best,

Alex

On Monday, March 3, 2025 at 2:36:23 AM UTC-8 Henrik Boström wrote:

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ecdcb150-2719-4040-99d3-3a18fbcbe230n%40chromium.org.

Henrik Boström

unread,
Mar 10, 2025, 4:48:57 AMMar 10
to blink-dev, philipp...@googlemail.com, blink-dev, Diego Perez Botero, Alex Russell, Henrik Boström
The numbers I quoted are HW encoder availability; HW decode availability is sometimes a little higher than encode which can result in an asymmetry but that is more of an exception than the rule, e.g. 77% decode instead of 75% encode on Windows but enc/dec numbers are the same on macOS. Both AV1 and HEVC are better than VP9 for low bitrate and high complexity videos IMO but if you disagree then regarding VP9 as an alternative that also lacks HW support. So the relevant codec to compare for HW acceleration would be H.264 which is not in the same league as HEVC. The MediaCapabilities query already convey supported configuration which lets you rule out that reason for fallback if you choose to go beyond what the "main profile" supports. Agree that an event would be useful but when your decoder stops working this can already be detected in getStats. None of this is codec specific so I don't see how this is relevant to this I2S. And we have not pulled back on our AV1 investments nor do we intend to, it is the preferred codec when HW is not a deciding factor.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Harald Alvestrand

unread,
Mar 10, 2025, 7:13:08 AMMar 10
to Philipp Hancke, Henrik Boström, blink-dev, Diego Perez Botero, Alex Russell
Thanks for pushing the profile document! It's a bit late to get it on the agenda for AVTCORE at IETF 122, but it should be possible to continue discussing it in email / trackers.

(for the benefit of listeners: RFC 7798, published in 2016, is the basic spec for H.265 in RTP.
draft-ietf-avtcore-hevc-rtp is an attempt to profile that rather expansive format in a way appropriate for use with WebRTC applications.)

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKi%2BecT1SQuQVdMX6nTANCsHw6xnCrTWwTxZOkY3onkPafw%40mail.gmail.com.

Jianlin Qiu

unread,
Mar 10, 2025, 11:57:36 AMMar 10
to blink-dev, Harald Alvestrand, Henrik Boström, blink-dev, Diego Perez Botero, Alex Russell, Philipp Hancke
> AV1 encode support is missing, decode support is there. Hardware encoders for RTC are extremely tricky (which is the reason OpenH264 is still very much needed)

AV1 HW encoding: On Intel it requires Core Ultra series or above, or A-series discrete graphics; On NV it requires RTX 4000+ series; On AMD it you need Zen5+.
For HEVC HW encode, Edge does not enable this feature, so you'll not see it in the HW accelerator list in edge://gpu.

Philip Jägenstedt

unread,
Mar 10, 2025, 12:33:03 PMMar 10
to Jianlin Qiu, blink-dev, Harald Alvestrand, Henrik Boström, Diego Perez Botero, Alex Russell, Philipp Hancke
LGTM1 with the same caveats as for MediaRecorder:
  • The approval is only for exposing OS-provided HEVC support, not for other OS-provided codecs, and not for browser-provided HEVC.
  • That there is a well established alternative in AV1 is important.
  • We're open to mitigations like Finch-disabling support for a percentage of users to ensure that HEVC support cannot be taken for granted.

It sounds like making the decision about which codec to use might still be challenging in some cases, but that seems like a preexisting issue and as Henrik says it's not codec-specific. If there are additional things that could be exposed to make this easier that would be great as a separate I2S.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/16ccc499-3a25-4a85-a9c5-c56f0d6f489an%40chromium.org.

Chris Harrelson

unread,
Mar 10, 2025, 2:33:15 PMMar 10
to Philip Jägenstedt, Jianlin Qiu, blink-dev, Harald Alvestrand, Henrik Boström, Diego Perez Botero, Alex Russell, Philipp Hancke
LGTM2

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYfHZTE4GLr6YP7_o56KJGXP_r8KiYtqv9w1HOiHvXF0vg%40mail.gmail.com.

Yoav Weiss (@Shopify)

unread,
Mar 11, 2025, 11:42:12 AMMar 11
to blink-dev, Chris Harrelson, Jianlin Qiu, blink-dev, Harald Alvestrand, Henrik Boström, Diego Perez Botero, Alex Russell, philipp...@googlemail.com, Philip Jägenstedt
LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/16ccc499-3a25-4a85-a9c5-c56f0d6f489an%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Alex Russell

unread,
Mar 12, 2025, 11:31:52 AMMar 12
to blink-dev, Yoav Weiss, Chris Harrelson, Jianlin Qiu, blink-dev, Harald Alvestrand, Henrik Boström, Diego Perez Botero, Alex Russell, philipp...@googlemail.com, Philip Jägenstedt
Thanks to everyone for taking the time to get this to a place where we can accept it. As there are already enough LGTMs to ship, I only want to add one condition to what Philip documented (that this is only for HW-supported decode, that there may be new work to do to detect and block decode/encode on devices w/ OS-level software support, and that we prefer AV1 everywhere): this won't happen again.

We are not creating precedent that can be used in the next generation of codecs, and everyone should be aware that the API OWNERS are not inclined to allow new proprietary codecs (either in hardware or software).

Best,

Alex


Reply all
Reply to author
Forward
0 new messages