Frame-rate sometimes drops below the min getUserMedia constraints

[konst...@24sessions.com]

Hi,

It seems that after Chrome 85 (or 84) frame-rate sometimes drops below the minimal constraint specified in getUserMedia calls.

Here are the constraints that are used:

meaning frame-rate should never drop below 15, but sometimes such metrics are observed in webrtc internals:

Any ideas what could be causing it? 

Experience it on the resent (pre arm) Macbook pro.

Would appreciate any help. Thanks!

[konst...@24sessions.com]

Images didn't come through, so will add those values below.

getUserMedia constraints:

• {width: {exact: 640}, height: {exact: 360}, frameRate: {min: 15, max: 20}}

webrtc internals outgoing stats:

• framesPerSecond: 9

[re...@webinargeek.com]

I observe the same metrics since M85.

[Odin Hørthe Omdal]

These are different things. The constraints you give to getUserMedia is used to actually get a stream.

Those tracks you then give to the peer connection and it will do the adjustments it needs to work. That is within the resource constraints that exist, which is not the same as the constraints you just set to get a video stream locally.  The spec says this[0]:

The encoding and transmission of each MediaStreamTrack SHOULD be made such that its characteristics (width, height and frameRate for video tracks; sampleSize, sampleRate and channelCount for audio tracks) are to a reasonable degree retained by the track created on the remote side. There are situations when this does not apply, there may for example be resource constraints at either endpoint or in the network or there may be RTCRtpSender settings applied that instruct the implementation to act differently.

The sender settings you can change with `setParameters`[1]  -- particularly you'd update the encodings [2]. There isn't any mode to set the min framerate, though it seems there's a maxFramerate though, but it isn't in the spec [3], so don't know how it actually works.

You can test around a bit and see how the implementations behave. Even trying to take the track of the sender and trying `applyConstraints` on it. I wouldn't expect it to work. You could try rather restricting the size by setting scaleResolutionDownBy, so that the bits you have on the network and your cpu can be used to keep a higher framerate?

Someone might come with something smarter or more correct, but this should at least help a bit :)

0: https://w3c.github.io/webrtc-pc/#rtp-media-api

1: https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters

2: https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSendParameters/encodings

3: https://w3c.github.io/webrtc-pc/#rtcrtpencodingparameters

Odin

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/8e1444b3-954c-4c21-91e9-b8465aeb8b36n%40googlegroups.com.

[konst...@24sessions.com]

Weirdly enough, this happens only in Chrome on Mac and not on Windows.

And it seems to not be related to the peer connection, as if I open GetUserMedia test page (https://webrtc.github.io/samples/src/content/getusermedia/gum/) on another tab -- the media stream there will also have a low framerate, but after both tabs are closed get GUM page opened again -- frame rate is back to normal.

Seems to be a bug in Chrome or Mac internals?