Differences in quality between .mjr files and RTP forwarding to GStreamer

[nic...@ubble.ai]

Hi all,

We have set-up Janus so as to receive video streams from a webapp, which are then RTP forwarded to GStreamer (using the videoroom plugin) for further processing frame by frame. 

.mjr files are also recorded and converted to webm or mp4 depending on the encoding (thanks meetecho for the help !)

This is all working very well in most cases.

However, I have noticed that the frames read by gstreamer differ sometimes dramatically from the .mjr video files, with some noise and blocking appearing (attached an example).

Here is the pipeline we use for h.264: udpsrc port=17004 ! application/x-rtp,clock-rate=90000,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink

Do you have suggestions about where the problem could be coming from?

I have suspicions that either 

• the network between Janus and Gstreamer machine have issues, which seems surprising given they are in the same VPC on AWS
  
• the stream is not read fast enough
• other?

Do these assumptions make sense? 

Are there ways to overcome this, maybe with buffering?

Thanks a lot !

nicolas

[Lorenzo Miniero]

The postprocessor has access to all the packets, and reorders them before saving them to file. If the live stream had out of order packets (e.g., due to retransmissions) then I guess this might confuse gstreamer. You should try and add some sort of jitter buffer to the pipeline and see if that improves things.

Lorenzo

[nic...@ubble.ai]

Thanks Lorenzo,

We'll try to use rtpjitterbuffer and see if it helps.

best,

nicolas

[nic...@ubble.ai]

Hi all,

We added Gstreamer jitterbuffer option in the pipeline. It looks like this for h264:

```udpsrc port=17004 ! application/x-rtp,clock-rate=90000,payload=96 ! rtpjitterbuffer drop-on-latency=true latency=100 faststart-min-packets=10 !  rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! app```

I am not sure what is wrong but it does stalls our reading part in our application.

Do you have any idea what could be the culprit here?

Thanks, 

nicolas

[Lorenzo Miniero]

Sorry but I'm not that familiar with gstreamer's internals. You may want to ask on their mailing list.

Lorenzo

[Kaplan]

This is what I use...  rtpjitterbuffer latency=3000 do-retransmission=false drop-on-latency=true ! queue ! rtph264depay..... etc

I still do get those artifacts . Note the ! queue  element there in between (that I think ads another thread to the pipeline).  

[nic...@ubble.ai]

Thanks Kaplan, 

I will try out those !

Have you noticed an improvement, even though the problem does not disappear totally?

Thanks,

nicolas

[nic...@ubble.ai]

Hi Kaplan, Lorenzo & all,

We've (finally) made a series of tests on the GStreamer pipeline, as well as running GStreamer in the same machine as Janus. 

We ended up with a setup that seems to yield a good quality for our use case: 
 - adding a rtpjitterbuffer as suggested (without playing with latency for now) 
 - adding threads to the vp8dec element (threads=4) 
 - running GStreamer natively instead of within Python 

This uncovered another issue... we're streaming 3 successive sequences. In between each sequence, the publisher stops the stream, then relaunch a new one, forwarded in a similar way to GStreamer. GStreamer is unchanged. 

This works fine with all setups we tested *except* when we use a Jitterbuffer. When a Jitterbuffer is used, nothing is received for the second scene. 

Do you have any idea where the issue might be coming from? 

Thanks, 
nicolas 

[Lorenzo Miniero]

Il giorno mercoledì 3 ottobre 2018 14:56:35 UTC+2, nic...@ubble.ai ha scritto:

Hi Kaplan, Lorenzo & all,

We've (finally) made a series of tests on the GStreamer pipeline, as well as running GStreamer in the same machine as Janus. 

We ended up with a setup that seems to yield a good quality for our use case: 
 - adding a rtpjitterbuffer as suggested (without playing with latency for now) 
 - adding threads to the vp8dec element (threads=4) 
 - running GStreamer natively instead of within Python 

This uncovered another issue... we're streaming 3 successive sequences. In between each sequence, the publisher stops the stream, then relaunch a new one, forwarded in a similar way to GStreamer. GStreamer is unchanged. 

This works fine with all setups we tested *except* when we use a Jitterbuffer. When a Jitterbuffer is used, nothing is received for the second scene. 

Do you have any idea where the issue might be coming from? 

When recreating publishers, the whole RTP context (SSRC, timestamps, sequence numbers) changes, which probably confuses the jitterbuffer (it sees a huge jump in all values). You can try not restarting publishers, or using a new pipeline anytime you restart the publisher.

Lorenzo