Low FPS for one of the simulcast encodings in Chrome

[Michal Śledź]

Hi,

we are adding support for simulcast tracks in Membrane Framework and I noticed that one of the encodings Chrome sends to our SFU is produced in low FPS.

Encoding 1280x720 is produced in 12 FPS (very often even ~10FPS) and has 9 554 187 bytes sent whereas encoding 320x180 is produced in 25 FPS and has 36 670 042 bytes sent.

None of the encodings is limited by bandwidth. I mean qualityLimitationReason is "none"

I can't get why HD encoding is in such a low quality. I removed any maxBitrate constraints from RTCRtpTransceiverInit.

[Michal Śledź]

One more thing I noticed in chrome logs

[169942:5:0303/151402.878051:INFO:video_stream_encoder.cc(1069)] ReconfigureEncoder:
Simulcast streams:
0: 1280x720 min_kbps: 150 target_kbps: 500 max_kbps: 700 max_fps: 60 max_qp: 56 num_tl: 3 active: true
1: 320x180 min_kbps: 600 target_kbps: 2500 max_kbps: 2500 max_fps: 60 max_qp: 56 num_tl: 3 active: true

Why my hd encoding has lower target and max kbps?

I feel like I am missing something important in the whole simulcast concept

[Sergio Garcia Murillo]

try reversing the encoding array before calling setParameters

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/971674b8-eb70-429c-ac47-d28affc9b314n%40googlegroups.com.

[Philipp Hancke]

well, how are you configuring it in the transceiver init?

--

[Michal Śledź]

I have been configuring it in the following way so far

// const TEMPORAL_LAYERS_COUNT = 2;

export const simulcastConfig: RTCRtpTransceiverInit = {

direction: "sendonly",

sendEncodings: [

{

rid: "h",

active: true,

// maxBitrate: 4_000_000,

// scalabilityMode: "L1T" + TEMPORAL_LAYERS_COUNT,

},

{

rid: "l",

active: true,

// maxBitrate: 4_000_000,

scaleResolutionDownBy: 4,

// scalabilityMode: "L1T" + TEMPORAL_LAYERS_COUNT,

},

],

};

but as Sergio Garcia Murillo suggested I changed the order in sendEncodings array and I am getting now Simulcast streams:
0: 320x180 min_kbps: 150 target_kbps: 500 max_kbps: 700 max_fps: 60 max_qp: 56 num_tl: 3 active: true
1: 1280x720 min_kbps: 600 target_kbps: 2500 max_kbps: 2500 max_fps: 60 max_qp: 56 num_tl: 3 active: true It looks like it helped a little but when I join to my room from two chrome tabs I can see something like this FPS produced by the first participant FPS produced by the second participant

[Philipp Hancke]

Am Do., 3. März 2022 um 15:42 Uhr schrieb Michal Śledź <michal...@swmansion.com>:

I have been configuring it in the following way so far

// const TEMPORAL_LAYERS_COUNT = 2;

export const simulcastConfig: RTCRtpTransceiverInit = {

direction: "sendonly",

sendEncodings: [

{

rid: "h",

active: true,

// maxBitrate: 4_000_000,

// scalabilityMode: "L1T" + TEMPORAL_LAYERS_COUNT,

},

{

rid: "l",

active: true,

// maxBitrate: 4_000_000,

scaleResolutionDownBy: 4,

// scalabilityMode: "L1T" + TEMPORAL_LAYERS_COUNT,

},

],

};

but as Sergio Garcia Murillo suggested I changed the order in sendEncodings array and I am getting now

yeah. Best to explicitly write the resolution downscale ratios as in

https://w3c.github.io/webrtc-pc/#rtcrtpencodingspatialsim-example*

otherwise it gets too confusing.

 

Simulcast streams:
0: 320x180 min_kbps: 150 target_kbps: 500 max_kbps: 700 max_fps: 60 max_qp: 56 num_tl: 3 active: true
1: 1280x720 min_kbps: 600 target_kbps: 2500 max_kbps: 2500 max_fps: 60 max_qp: 56 num_tl: 3 active: true It looks like it helped a little but when I join to my room from two chrome tabs I can see something like thi

from the initial stats you seem to produce a very high number of keyframes (374 in relation to actual frames (~2000) and receive a lot of FIRs (186).

That is unlikely to produce good results.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/5c32575e-ba0d-44a9-8ca5-e1d88836b175n%40googlegroups.com.

[Michal Śledź]

So you are suggesting that because of a lot of FIR requests browser has to generate a lot of keyframes and it can't fit into bandwidth estimation so it limits FPS?

But I am running this on localhost and the second peer is working pretty good or at least significantly better than the first one.

Also, in qualitiLimitationReason there is "none" 

Regarding FIR requests. I think we are sending them every one second. This might be naive approach but we wanted to make sure that a new participant can almost instantly display video from other people. Perhaps we should implement more complex logic so that we send FIR request only when a new participant joins.

Anyway, I will try to manipulate FIR request timeout and see if this helps.

[Michal Śledź]

I played a little with the number of FIR requests we are sending and you were right



I was generating FIR request every 20 seconds. 
Every time browser gets FIR request there is a significant change in framesEncoded/s. 
But that is not the case fo each encoding. For some of them FIR requests don't introduce any changes in FPS.

IIRC FIR request should produce a new keyframe. This means that movement should result in similar FPS drop as FIR request does. 

Isn't this strange? I mean any significant user movement like waving hands in front of monitor should result in FPS drop too.

[Michal Śledź]

After experimenting a little it turned out that fixing the order in the encoding array and limiting FIR requests solved my problem. Thanks a lot for help!