SignalEncoderTimedOut Triggered on Remote Desktop Streams

[Mikko Koivisto]

We are using a WebRTC implementation (built from AOSP) to stream the UI of a virtualized Android Cuttlefish device to a remote client. This setup is characterized by variable frame rate: During inactivity (static screen/home screen) the encoder receives frames only when the screen updates (e.g., once every 60 seconds for a clock tick). During activity (animation/input) the encoder outputs bursts of complex frames at 30-60 FPS.

Immediately following a long period of inactivity (idle screen), the system seems to fail to transition correctly to the active state, leading to a bitrate collapse:

1. The check_encoder_activity_task_ runs during silence and finds no activity.

2. It triggers SignalEncoderTimedOut() because the time since the last frame production is greater than kEncoderTimeOut.

3. total_requested_max_bitrate seem to be reset to very low value like 54kbps

4. This causes severe lag and high packet retransmission rates when the animation burst finally begins.

I think the timeout is a false positive because the encoder is not hung; it is correctly idle and waiting for the next frame. The timeout logic seems inappropriate for variable frame rate sources where silence is expected.

Should the encoder activity check be disabled when the stream is known to be a screen-share?

Best Regards,

Mikko Koivisto

[Vitaly Ivanov]

I think the easiest workaround is to capture at at least 1 fps (kEncoderTimeout is 2s) to keep the whole thing alive and up to date, probing the channel to avoid the BW estimate collapse.

Those static frames will take very little BW and CPU.

--
This list falls under the WebRTC Code of Conduct - https://webrtc.org/support/code-of-conduct.
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/discuss-webrtc/37a3f864-be25-45a1-b98f-2d6253228e46n%40googlegroups.com.

[Lambros Lambrou]

This sounds exactly like a problem we had in Google Chrome Remote Desktop.

If the screen-capturer does not produce new frames, we re-encode and send the previous frame at 1-second intervals. An exception is when "top-off" is in progress; top-off frames continue to be encoded at the usual framerate.

To address the issue of low BW estimation during idle periods:

• We enable ALR probing.
• We set the min and max bitrates. The minimum value comes from our website, so I can't link to it, but we use 1Mbps. The maximum is 100Mbps, which was chosen arbitrarily.

I'm not a core WebRTC engineer, so I don't know if there are any plans to improve the logic in WebRTC for the screen-share case.

Hope this helps!

--

[Mikko Koivisto]

Thanks for the tips,

I'll try feeding 1fps constant frame stream to encoder on idle. I have already tested

WebRTC-Pacer-PadInSilence/Enabled/

...which I think is somewhat similar to periodic_alr_bandwidth_probing and tries to prevent the bwe from decaying during media silence.

- Mikko