Intent to Ship: H.264 software encoder/decoder in Chrome for WebRTC

[Henrik Boström]

Contact emails

hb...@chromium.org, h...@chromium.org, to...@chromium.org

Spec

https://tools.ietf.org/html/draft-ietf-rtcwforeb-video-06

TAG review: N/A, just adding another codec.

Summary

Include a H.264 video codec encoder and decoder in Chrome for use with WebRTC. At IETF in late November 2014, a compromise was reached with the main contributors to WebRTC to ship both VP8 and H.264. This intent to impl is to follow up in this public commitment. OpenH264 (same lib as Firefox uses) is used for encoding and FFmpeg (which is already used elsewhere in Chrome) is used for decoding.

Encoding/decoding is working in WebRTC. We are in the process of enabling this in Chrome (not Chromium) behind a runtime flag. With this Intent to Ship, we hope to remove the flag before the M50 cut.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/rsokc4bi8R4

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

All except Android/Android WebView, where H.264 decoding with FFmpeg is not supported.

Interoperability and Compatibility Risk

No identified risk.

(H.264 is only used if both endpoints support it and listed and chosen from the SDP offer. VP8, which is the one used today, will continue to be offered and supported.)

OWP launch tracking bug

https://code.google.com/p/chromium/issues/detail?id=468365

Link to entry on feature dashboard

https://www.chromestatus.com/feature/6417796455989248

[Rick Byers]

LGTM1

Can you elaborate on the situation in Android?  Would WebRTC normally just use hardware H.264 decoding there?  Or is there a non-trivial interop issue in practice - where Chrome on Android would fail to use WebRTC in a real-world scenario where Chrome on desktop works fine?

Thanks,

   Rick

[Henrik Boström]

I'm investigating why the FFmpeg H264 decoder is not included on Android (there is a config file that excludes it, question is why and if we can/are allowed to include it). Emailing owners. For now the assumption is it is not supported with FFmpeg.
But the goal is to support H.264 on all platforms. There are multiple solutions we have yet to investigate on Android. Currently, H.264 is only supported on Android if there is hardware support (using MediaCodec and white list).

First order of business is to get it working on all other platforms. (For cross-platform calls, increasing the support on other platforms increases the possibility to use H.264 on android too.)

If not supported, H.264 will not be used no, but calls can still be made with VP8 (or VP9?) which is the case today. This does not break any interoperability.

On Tuesday, February 2, 2016 at 7:42:15 PM UTC+1, Rick Byers wrote:

LGTM1

Can you elaborate on the situation in Android?  Would WebRTC normally just use hardware H.264 decoding there?  Or is there a non-trivial interop issue in practice - where Chrome on Android would fail to use WebRTC in a real-world scenario where Chrome on desktop works fine?

Thanks,

   Rick

On Wed, Feb 3, 2016 at 5:26 AM, Henrik Boström <hb...@chromium.org> wrote:

Contact emails

hb...@chromium.org, hta@chromium.org, tommi@chromium.org

[Henrik Boström]

Note: The SDP offer/answer contains all supported codecs from both ends, you decide which one to use based on that. This only increases the H.264 availability.

[Henrik Boström]

+hta

[Philip Jägenstedt]

You say "We are in the process of enabling this in Chrome (not
Chromium) behind a runtime flag." Will this be behind the
proprietary_codecs=1 and ffmpeg_branding=Chrome GYP flag, or a new
flag? For those Chromium-based browsers that do currently not use
H.264 from FFmpeg, would it be possible to use OpenH264 for both
encoding and decoding, or is that a whole lot of code that you have no
reason to write for Chrome?

> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+...@chromium.org.

[Harald Alvestrand]

That's code we have no intention og writing for chrome.
We will support multipler encoders (for hardware reasons), and the code probably isn't that large, but our security folks did not like the idea of security-analyzing another decoder -these are, by definition, complex pieces of code working on complex data from an untrusted source. (Encoders are simpler - their input format is simple.)

[Henrik Boström]

We just decided to make use of |proprietary_codecs|. (Either we use it directly but everywhere test "proprietary_codecs && !android" or we use our own rtc_use_h264 flag and default it to "proprietary_codecs && !android").

That way proprietary_codecs=1 and ffmpeg_branding=Chrome will do the trick.

[Henrik Boström]

Bump. Need more LGTMs.

Also update: The reason H264 is not included for FFmpeg decoding on Android is it adds a lot of binary size and there may be other libraries we could use. We would either have to convince folks that it should be included or implement H264 on Android using other means. We will look into this in the future.

[Philip Jägenstedt]

Since this is gated behind proprietary_codecs=1 (not used in Opera Desktop) and Android is also excluded from this intent, I think I'll have to abstain on this one. I think that in principle Opera should be able to use OpenH264 for WebRTC (encoding and decoding) but that's not in the scope of this intent.

[Chris Harrelson]

LGTM2

[Harald Alvestrand]

Looking for LGTM3 to satisfy the procedure.

We don't seem to have found any technical objection so far.

[Dimitri Glazkov]

LGTM3.