Simulcast on Android Chrome doesn't work when Hardware Encoding/Decoding is enabled.

[Vinayak V Kulkarni]

Hi Team,

On Android Chrome it is observed that simulcast doesn't work when using Hardware Encoding/Decoding. 
We try to set the simulcast parameters in the encodings of videoRtpSender using setParameters and addTransceiver method, but it is not taking any effect.
We try to scale down the resolution by 2, 4 and 8 in the 3 simulcast layers but the android chrome client still sends the same resolution in which the video was captured.

We see there is a bug already raised.
https://bugs.chromium.org/p/webrtc/issues/detail?id=12328
"HardwareVideoEncoder cannot scale incoming frames, thus breaking simulcast configuration."

When we disable Hardware Encoding/Decoding on the android mobile using chrome://flags - we see simulcast is working fine. (Please see attached screenshot)

Can you please let us know how to use simulcast when using the default configuration  on Android Chrome browser(i.e Hardware Encoding/Decoding)
Also if there is any way to ask chrome to use software encoder/decoder programmatically, please let us know. 

PS: This issue is not seen on IOS devices.
Any info on this aspect would be appreciated.

Thanks,
Vinayak

[Angelika Serwa]

Hi, we have similar issue. For software encoders everything works fine. For hardware encoders on Android all layers have the same resolution, as original resolution. We reproduced this in m104, m97 versions. On m93 layers have correct resolutions.

Moreover, when switching layers, in some configurations video is not being sent. For example:
- send 3 layers initially (h, m, l resoulutions)
- disable h and m layers (by setting encoding.enabled = false in peer connection)
- l resolution should still be sent, instead no layer is sent

@vinay we set software / hardware encoders programatically using this code https://github.com/livekit/client-sdk-android/blob/288c770297f776b5f8ac9091914f1f276b8e7360/livekit-android-sdk/src/main/java/io/livekit/android/webrtc/SimulcastVideoEncoderFactoryWrapper.kt
you can pass SoftwareVideoEncoderFactory as primary encoder in SimulcastVideoEncoderFactory constructor

[Cameron Lucas]

We have noticed the same issue Though, it looks like Chrome (on Android), specifically, seems to be affected by this bug https://bugs.chromium.org/p/chromium/issues/detail?id=1351371. 

- Cameron

[Vinayak V Kulkarni]

@angelika
Thanks for the info. The code snippet you shared is for Native Android Apps I believe. Is there any way to achieve the same on browsers? Please share if you have any info.