VideoRoom RTP-Forward: Big visual artifacts when low bandwidth publishers change sent resolution

[Ramon Blanquer]

Hello,

I wanted to ask you how do you tackle RTP-forwarded streams that have dynamically changing resolutions due to poor bandwidth conditions.

I am using with the VideoRoom plugin and I RTP-forward the participants so I can process them and compose their contributions into an enriched video scene (with graphics, animations, ...).

Strong visual artifacts ocur when a publisher has a low bandwidth because that triggers resolution changes on the connection and introduces artifacts that are visible if you consume (with VLC or fflpay) that RTP-forwarded stream or any manipulation done upon it (gstreamer, or others...).

I was investigating how to mitigate those issues, and the approaches I thought about were:

- (not possible) Try to set a constant resolution. Is it possible to set the video to a constant resolution? #1775

- (unsuitable) Play with the configure requests passing along a lower bitrate (I don't think it would solve the problem of receiving a resolution-unstable stream). (janus google group) Strategy to handle slowlink, or (janus google group) Janus, NACKS, and REMBs ...

I guess it’s pretty common to RTP-forward a participant to post-process that video further down the pipeline... What would be the best way to protect yourself from those cases? Do you do any post-processing to “sanitize” those artifacts (e.g. freezing the image until the bitrate is good again)?

I am sorry if you think this question is out of Janus’ scope, but I believe it’s very related since it’s about getting out its media streams from it.

Thank you!

[Lorenzo Miniero]

Forcing a constant resolution should be possible using the WebRTC JavaScript APIs (so nothing Janus related), but I can't remember how You'll probably find the right syntax if you check related posts on the discuss-webrtc group.

L.

[Lorenzo Miniero]

For what concerns us, we're fine with changing resolutions, since we use RTP forwarders to either simply forward the stream to other servers (scalability) or to pass them to our videomixer, which can deal with changing resolutions just fine.

L.

[Ramon Blanquer]

Thanks for your clarification.

I guess then efforts must be made on the video mixer side in order to ingest the stream even if it's got errors caused by low bandwidth and sudden resolution changes.

In your reply you speak of "our videomixer", do you refer to one of of your private products? Or does Janus have an opensource video mixer that can be used along too?

Thank you Lorenzo!

[Lorenzo Miniero]

I think even when you work with FFmpeg/GStreamer as they are, they should be able to cope with varying resolutions now, but of course that depends on the version and how they're used. The videomixer I mentioned is a private product that we sell, yes, and that we use for RTMP re-broadcasting and/or live recording.

L.

[Ramon Blanquer]

Thanks Lorenzo :)