Video resolution changing during a call?

[Lorenzo Miniero]

Hi all,

playing with media and bandwidth constraints, we recently noticed that apparently the video resolution of streams can change during a call. In particular, when asking for a 640x480 video and forcing a bandwidth of up to 300kbps, video often switched from 640x480 to 512x384 and back. This is causing us some problems with interaction with a few legacy devices, and with recordings, as the playout in some players even crashes.

I guess this is a recent update to Chrome (this only seems to happen in Beta, not Canary) that tries to do so in order to take into account limited bandwidth and react accordingly, which is reasonable. What I don't think is expected is this happening even when we force minimum and maximum video constraints to be the same: if maximum and minimum are 640x480, resolution should never change dynamically (framerate or video quality should be affected instead), which instead it does.

That said, we've tried to understand what's the current policy Chrome Beta uses to decide for the resolution switch, but we haven't understood all of it. Apparently, if available bandwidth is 500kbps the resolution never drops, but this at times also seemed to happen for lower values so we're not sure. Is there anybody that encountered the same behaviour and that may shed some light upon this?

Thanks,

Lorenzo 

[Benjamin Schwartz]

> if maximum and minimum are 640x480, resolution should never change dynamically (framerate or video quality should be affected instead)

The constraints on getUserMedia, like minWidth and maxWidth, control the camera opening parameters.  Once opened, the camera stream's parameters (e.g. resolution) will not change (at least in current implementations).

The VP8 bitstream resolution produced by the encoder is not required to match the input frame resolution.  When encoding a video stream with WebRTC, you must trust the browser to balance psychovisual quality, CPU time, and available bandwidth.  I would urge you to think of VP8 bitstream resolution as just one of the thousands of VP8 encoding choices that the browser makes every second in order to strike the right balance.

Receivers that cannot handle the resulting stream are not spec-compliant. See http://tools.ietf.org/html/rfc6386#section-9.1 for details on resolution changes in VP8.

--
 
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Lorenzo Miniero]

Benjamin,

thanks for the clarification, I'll look into it.

Lorenzo

[Jesús Leganés Combarro]

I don't find bad that the browser reduce the resolution when bandwidth is constrained, but should it not do an upscale of the received image so it can be the same size of the requested one? It would easier things a lot...

[Oscar Divorra]

IMO, the web developper or whatever app lays on top of the browser, shall not need to know what decisions the encoder and/or infrastructure takes to optimize visual quality-vs-information-cost-vs-network-resilience at the stream level.

Good infrastructure shall take care of that transparently…

--

[Sergio Garcia Murillo]

Changing of the video image size is something normal in all video conference equipments, and you should be ready to support it. Also, I believe Lorenzo is more concerned about server side, on the client, if I am not wrong the video element would handle that automatically for you and rescale the video to the actual element size (adding extra black bars if needed).

Moreover, as we have not included any signaling for the camera orientation, I would expect the video to change from WxH to HxW to compensate a mobile phone orientation change. If not it will be quite difficult to rotate a pc monitor to be able to watch the video.. ;)

Best regards
Sergio

[Lorenzo Miniero]

As Sergio said, my issue is more on the server side, yes, specifically the recordings. Apparently the webm files we get out of these resolution changing videos crashes video players like VLC. Considering Benjamin's point on the VP8 bitstream, we may be doing something wrong in writing the webm headers and files in the first place, so we're investigating in that direction.

That said, I want to clarify the fact that I was not blaming anyone for introducing the feature. As I said, I completely understand it's a useful and needed feature that I support, which as Sergio said has been around forever anyway. I was just concerned about the fact that it appeared to be unpredictable in a few cases, and at least for WebRTC unexpected for us as it was not announced (or maybe I just missed it).

Lorenzo

[mparis]

I also have the same problem recording with webm muxer.
Do I need to configure something of the muxer (e.g: headers) to support frame size changing in a same video file?

[Lorenzo Miniero]

Just as an update, it looks it is mostly an issue in media players. If I understood correctly, the webm container can give a hint on the resolution (I guess the maximum one), but then the VP8 bitstream can signal (and does) dynamic resolution changes. The same can happen with other codecs, e.g., H.264. Many players seem to have problems dealing with this. VLC, for instance, either crashes or freezes, at least in the version I have. Gstreamer doesn't crash but does not resize the resolution. FFplay has issues to, but apparently using FFmpeg to re-encode the file it results in a static resolution file. Mplayer deals with it correctly.

The same problem happens with the integrated players in browsers. Such a webm with a resolution changing VP8 stream did not play in a <video> element in Chrome, while it mostly worked in Firefox.

I guess that, in order to have such a recording work everywhere, the only solution is postprocessing the video to get a static resolution one.

Lorenzo

[michael.gle...@smartblondeco.com]

Hi This is a very old thread, but one of the few that directly focuses on the issue of the browser/codec changing the resolution.

We are using a media server that reacts to the change in resolution by resizing the image within a fixed tile size. We are not able to modify this behavior in. the server.

As a result, people in conference see their images change size.

With some people's connections being 1Gbps up and down,  and our codec is H.264 with 1Mbps bitrate in the constraints, they still see this resolution change. We cannot explain why.

Is there anything currently that will prevent Chrome from changing the resolution size? There is already variable bitrate.

We know about the encoding parameter degradationPreference but it is not implemented in Chrome.

Thank you!