Camera video stream blinking

[Ivan Lukomskiy]

Hi!

I have a camera that streams video through LTE via RTSP protocol to Janus server. Janus converts the stream to WebRTC to make use of it in the web application.

In browser the same thing constantly happens: video stream plays for a couple of seconds, then disappears for some time (black screen), then plays again and this loop repeats infinetly.

This is nicely illustrated with browser://webrtc-internals. Check out how FPS goes zero periodically - it's when the video disappeared (I use testRTC for better graphs).

The strange thing here is that when I connect directly to RTSP camera stream via VLC player with caching long enough (600 ms for example), the video runs smoothly. And when I reduce this cache I experience the same behavior: black screen ~50% of the time.

That makes me think that probably if I had bigger jitter buffer in browser I will experience black screen no more.

So my questions are:

* Is there a way to give browser a clue about desired jitter buffer size?

* Is there any other ways how I can debug/fix black screen issue?

I use Google Chrome, but tried Firefox and Safari as well. Behaviour is the same though in Safari instead of black screen video just freezes. The codec is H264 baseline profile.

[Ivan Lukomskiy]

Figured that out after several days of debugging.

The thing was that browser handles webrtc packet loss in a weird way when using h264. When it detects a lost packet it just drops all following frames until the next keyframe.

In my infrastructure I constantly have a small percentage of udp packets loss, usually about 3%. And by default, my RTP stream encoded video with quite large delay between keyframes. 

So when browser encountered lost package video stopped playing and continued only after next keyframe which comes in 2-3 seconds. That was the cause of blinking and I easily reproduced it locally inflicting package loss on the RTP stream source. 

Unfortunately I'm not able to use the retransmission mechanisms here, 'cause keeping latency as low as possible is my priority goal.

So I solved it like that:

* In my Gstreamer source I disabled automatic adjusting of keyframes interval

* Set it to send keyframe each 0.33 second (1 keyframe to 2 usual frames in my case)

That worked good enough for me: blinking stopped, only a tiny freezes occur sometime when the network goes bad. Quality of video is slightly lower than before, but it's totally fine for my goals.