PSA: setCodecPreferences requires codecs from the RTCRtpReceiver.getCapabilities (M124)

[Philipp Hancke]

As of

  https://chromiumdash.appspot.com/commit/db2f52ba88cf9f98211df2dabb3f8aca9251c4a2

RTCRtpTransceiver.setCodecPreferences may throw an error if you configure it with input from

  RTCRtpSender.getCapabilities('video').codecs

instead of taking the input from

    RTCRtpReceiver.getCapabilities('video').codecs

Usually this should not make much of a difference but it turns out that on MacOS Chromium exposes a H264 encoder with Level 5.2 while it only claims to support Level 3.1.

See https://issues.chromium.org/issues/324930413

This can be an issue if you do something along the lines of

  const codecs = RTCRtpSender.getCapabilities('video').codecs;

  transceiver.setCodecPreferences(codecs.filter(c => c.mimeType === 'video/H264')

In general setCodecPreferences dictates what you prefer to *receive* which was the reason for making this change.

It should be easier to work with, see .e.g  the changes in the samples repository:

  https://github.com/webrtc/samples/pull/1640

Chromium 124 ships in mid-april, see the dashboard:

  https://chromiumdash.appspot.com/schedule

[[MSFT] Diego Perez Botero]

 Historically, some Android devices were hitting the following error when trying to setCodecPreferences with only the RTCRtpReceiver.getCapabilities('video') output:

        InvalidModificationError: Failed to execute 'setCodecPreferences' on 'RTCRtpTransceiver': Invalid codec preferences: Missing codec from send codec capabilities.

So we had to add the following workaround to append RTCRtpSender.getCapabilities('video') output before calling setCodecPreferences(...):

        // Even though our transceiver is 'recvonly', LibWebRTC demands that it always has something
        // to offer regardless of transceiver.direction. Thus, we must not forget to add the RTCRtpSender's
        // codecs to the very end. Some devices don't support H264 for *outgoing* video and will hit
        // InvalidModificationError during the setCodecPreferences(...) call if we don't do this.
        //
        // See VerifyCodecPreferences at https://chromium.googlesource.com/external/webrtc/+/master/pc/rtp_transceiver.cc
        const supportedSenderVideoCodecCapabilities = RTCRtpSender.getCapabilities('video')?.codecs ?? [];
        codecPreferences.push(...supportedSenderVideoCodecCapabilities);

        videoTransceiver.setCodecPreferences(codecPreferences);

From my reading of this PSA, it sounds like the ugly workaround is now going to blow up in our face. However, my reading of the M124 change indicates that the original Android problem that we were hitting is also going away with M124+, since the VerifyCodecPreferences implementation doesn't care about the sender codecs anymore. Could you please confirm that my understanding is accurate? :)

My plan moving forward would be to:

1. Leave the workaround for M123 and down

2. No longer attempt the workaround for M124 onwards

Thanks!

-Diego

[Philipp Hancke]

That sounds correct. The lack of H264 SW on Android is what caused all this mess :-|

--
This list falls under the WebRTC Code of Conduct - https://webrtc.org/support/code-of-conduct.
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/90810e3e-c85a-4d74-b7af-90b475e260den%40googlegroups.com.

[witaly...@gmail.com]

Imo it's not Android, but setCodecPreferences itself - it should have been exposed as RTCRtpReceiver and RTCRtpSender API. Adding it to RTCRtpTransceiver not only messed up asymmetrical cases, but also created this little know gotcha that setCodecPreferences actually stands for setReceiveCodecPreferences

[witaly...@gmail.com]

I guess it was done the way it was due to SDP limitations, but don't get me started on SDP...

[Philipp Hancke]

not going to argue with that but it is kinda hard to fix mistakes like that on the web platform

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/5c6a991b-0a3a-411a-8b72-4579749f2322n%40googlegroups.com.

[Khun V]

Yes, changing APIs used so extensively is not an easy feat. Thank you for your webrtc service over the years!

Anyway at this point I've decided to get rid of setCodecPreferences completely as it's way too unreliable and go commando. This JS snippet seems to work in my case (enforce H264), old school SDP munging it is: https://gist.github.com/tnoho/948be984f9981b59df43

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CADxkKiKmjD%3D5Pvgp%2B-TGgQG5tKeobPgeqB3-0zhJMO66SWpkHw%40mail.gmail.com.