ffmpeg->rtp/udp->Janus->WebRTC stream stuttering

2,982 views
Skip to first unread message

Alexander Novikov

unread,
Jun 28, 2019, 2:21:03 AM6/28/19
to meetecho-janus
Hello!

I have a problem roughly explain in this SO post:


in addition, i checked that sending file locally through RTP, works:

ffmpeg -re -thread_queue_size 4 -stream_loop -1 -i ./small.mp4 -an -s 852x480 -c:v libx264 -profile:v baseline -b:v 1M -r 24 -g 60 -bf 0 -sdp_file saved_sdp_file -f rtp rtp://127.0.0.1:4998


ffmpeg -protocol_whitelist "file,rtp,udp" -i saved_sdp_file -strict -2  -an -frames:v 1000 -y ./saved_video_file.mp4


produces a valid looking file


What shall i look into more to fix this? This is becoming a pressing business issue.


Thanks!


Alexander

Alexander Novikov

unread,
Jun 28, 2019, 2:27:20 AM6/28/19
to meetecho-janus
If there is an expectation that switch to TCP streaming will help
(ultra low latency is not a critical requirement in this application),
can you please explain how to switch to it? Just filtering out UDP ICE
candidates doesn't seem to help, and disabling UDP relay on a TURN
server just stops transmission.
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/36bdce96-4833-4dad-afc6-cc2a0a11a954%40googlegroups.com.

Mirko Brankovic

unread,
Jun 28, 2019, 3:12:50 AM6/28/19
to meetecho-janus
I'm guessing that -b:v 1M is the one producing the b=AS:1000 bandwith limit, but still it should be more than enough.
Do you know what are the rest of the options doing/ mainly -r 24 -g 60 -bf 0 ?

Maybe it is best to see on Admin api what is going on on receiving PC of Janus and also in browser PC, if there is any packet loss 

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/CAH92vQJAjErnho16NO7jGJLuvV8%3DSP5WtncxOqjX5mssj357BA%40mail.gmail.com.


--
Regards,
Mirko

Lorenzo Miniero

unread,
Jun 28, 2019, 3:24:41 AM6/28/19
to meetecho-janus
As Mirko said, it's very likely packet loss on the WebRTC side, introducing either missing packets on the RTP forwarder side (unrecovered loss) or packets out of order (recovered packet). As such, it makes little sense comparing it to a local ffmpeg generating a consistent RTP stream. Not sure if ffmpeg has some sort of jitter buffer to cope with that.

Lorenzo


Il giorno venerdì 28 giugno 2019 09:12:50 UTC+2, Mirko Brankovic ha scritto:
I'm guessing that -b:v 1M is the one producing the b=AS:1000 bandwith limit, but still it should be more than enough.
Do you know what are the rest of the options doing/ mainly -r 24 -g 60 -bf 0 ?

Maybe it is best to see on Admin api what is going on on receiving PC of Janus and also in browser PC, if there is any packet loss 

On Fri, Jun 28, 2019 at 8:27 AM Alexander Novikov <alexander...@gmail.com> wrote:
If there is an expectation that switch to TCP streaming will help
(ultra low latency is not a critical requirement in this application),
can you please explain how to switch to it? Just filtering out UDP ICE
candidates doesn't seem to help, and disabling UDP relay on a TURN
server just stops transmission.

On Fri, Jun 28, 2019 at 9:21 AM Alexander Novikov
<alexander...@gmail.com> wrote:
>
> Hello!
>
> I have a problem roughly explain in this SO post:
>
> https://stackoverflow.com/questions/56788387/h264-webrtc-video-streamed-from-ffmpeg-through-janus-is-very-choppy-on-playback
>
> in addition, i checked that sending file locally through RTP, works:
>
> ffmpeg -re -thread_queue_size 4 -stream_loop -1 -i ./small.mp4 -an -s 852x480 -c:v libx264 -profile:v baseline -b:v 1M -r 24 -g 60 -bf 0 -sdp_file saved_sdp_file -f rtp rtp://127.0.0.1:4998
>
>
> ffmpeg -protocol_whitelist "file,rtp,udp" -i saved_sdp_file -strict -2  -an -frames:v 1000 -y ./saved_video_file.mp4
>
>
> produces a valid looking file
>
>
> What shall i look into more to fix this? This is becoming a pressing business issue.
>
>
> Thanks!
>
>
> Alexander
>
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/36bdce96-4833-4dad-afc6-cc2a0a11a954%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.


--
Regards,
Mirko

Alexander Novikov

unread,
Jun 28, 2019, 3:25:25 AM6/28/19
to meetecho-janus


On Friday, June 28, 2019 at 10:12:50 AM UTC+3, Mirko Brankovic wrote:
I'm guessing that -b:v 1M is the one producing the b=AS:1000 bandwith limit, but still it should be more than enough.

Yes 1M is the bandwidth limit, and no it is not limiting anything - pretty much standard bitrate for a stream like that, although i tried to switch it between 256K, 512K, 1M, 2M and 4M, and nothing changed apart from stream quality (although 2M is not distinguishable from 4M and almost not distinguishable from 1M). Choppiness is just the same. I would say it pauses/jerks a bit LESS frequently on lower bitrates - although even on 256k it does once every 10-15 seconds which is still not acceptable - and the quality is of course awful, that is too little of a bitrate.
 
Do you know what are the rest of the options doing/ mainly -r 24 -g 60 -bf 0 ?


-r 24 is they frame rate
-g 60 is they keyframe interval
-bf 0 means 'not using any b-frames' which is natural for streaming video. although nothing seems to change if i remove bf option.
 
Maybe it is best to see on Admin api what is going on on receiving PC of Janus and also in browser PC, if there is any packet loss 


ffmpeg and janus are on the same machine, it is streamed there through localhost.
browser PC - chrome://webrtc-internals does reveal some packet loss yes, but low (fractions of a percent, 0.2-0.5%) - pretty much normal for UDP.
not acceptable? then how do i switch to TCP? that will work for me, just have no idea how.
 
On Fri, Jun 28, 2019 at 8:27 AM Alexander Novikov <alexande...@gmail.com> wrote:
If there is an expectation that switch to TCP streaming will help
(ultra low latency is not a critical requirement in this application),
can you please explain how to switch to it? Just filtering out UDP ICE
candidates doesn't seem to help, and disabling UDP relay on a TURN
server just stops transmission.

On Fri, Jun 28, 2019 at 9:21 AM Alexander Novikov
<alexande...@gmail.com> wrote:
>
> Hello!
>
> I have a problem roughly explain in this SO post:
>
> https://stackoverflow.com/questions/56788387/h264-webrtc-video-streamed-from-ffmpeg-through-janus-is-very-choppy-on-playback
>
> in addition, i checked that sending file locally through RTP, works:
>
> ffmpeg -re -thread_queue_size 4 -stream_loop -1 -i ./small.mp4 -an -s 852x480 -c:v libx264 -profile:v baseline -b:v 1M -r 24 -g 60 -bf 0 -sdp_file saved_sdp_file -f rtp rtp://127.0.0.1:4998
>
>
> ffmpeg -protocol_whitelist "file,rtp,udp" -i saved_sdp_file -strict -2  -an -frames:v 1000 -y ./saved_video_file.mp4
>
>
> produces a valid looking file
>
>
> What shall i look into more to fix this? This is becoming a pressing business issue.
>
>
> Thanks!
>
>
> Alexander
>
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetech...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/36bdce96-4833-4dad-afc6-cc2a0a11a954%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetech...@googlegroups.com.


--
Regards,
Mirko

Alexander Novikov

unread,
Jun 28, 2019, 3:27:38 AM6/28/19
to meetecho-janus
Can i switch Janus to use TCP for WebRTC between itself and the client browser? How?


On Friday, June 28, 2019 at 10:24:41 AM UTC+3, Lorenzo Miniero wrote:
As Mirko said, it's very likely packet loss on the WebRTC side, introducing either missing packets on the RTP forwarder side (unrecovered loss) or packets out of order (recovered packet). As such, it makes little sense comparing it to a local ffmpeg generating a consistent RTP stream. Not sure if ffmpeg has some sort of jitter buffer to cope with that.

Lorenzo


Il giorno venerdì 28 giugno 2019 09:12:50 UTC+2, Mirko Brankovic ha scritto:
I'm guessing that -b:v 1M is the one producing the b=AS:1000 bandwith limit, but still it should be more than enough.
Do you know what are the rest of the options doing/ mainly -r 24 -g 60 -bf 0 ?

Maybe it is best to see on Admin api what is going on on receiving PC of Janus and also in browser PC, if there is any packet loss 

On Fri, Jun 28, 2019 at 8:27 AM Alexander Novikov <alexande...@gmail.com> wrote:
If there is an expectation that switch to TCP streaming will help
(ultra low latency is not a critical requirement in this application),
can you please explain how to switch to it? Just filtering out UDP ICE
candidates doesn't seem to help, and disabling UDP relay on a TURN
server just stops transmission.

On Fri, Jun 28, 2019 at 9:21 AM Alexander Novikov
<alexande...@gmail.com> wrote:
>
> Hello!
>
> I have a problem roughly explain in this SO post:
>
> https://stackoverflow.com/questions/56788387/h264-webrtc-video-streamed-from-ffmpeg-through-janus-is-very-choppy-on-playback
>
> in addition, i checked that sending file locally through RTP, works:
>
> ffmpeg -re -thread_queue_size 4 -stream_loop -1 -i ./small.mp4 -an -s 852x480 -c:v libx264 -profile:v baseline -b:v 1M -r 24 -g 60 -bf 0 -sdp_file saved_sdp_file -f rtp rtp://127.0.0.1:4998
>
>
> ffmpeg -protocol_whitelist "file,rtp,udp" -i saved_sdp_file -strict -2  -an -frames:v 1000 -y ./saved_video_file.mp4
>
>
> produces a valid looking file
>
>
> What shall i look into more to fix this? This is becoming a pressing business issue.
>
>
> Thanks!
>
>
> Alexander
>
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetech...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/36bdce96-4833-4dad-afc6-cc2a0a11a954%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetech...@googlegroups.com.


--
Regards,
Mirko

Lorenzo Miniero

unread,
Jun 28, 2019, 3:28:28 AM6/28/19
to meetecho-janus
Il giorno venerdì 28 giugno 2019 09:27:38 UTC+2, Alexander Novikov ha scritto:
Can i switch Janus to use TCP for WebRTC between itself and the client browser? How?



No, and we don't plan to add that.

Lorenzo

Lorenzo Miniero

unread,
Jun 28, 2019, 3:30:08 AM6/28/19
to meetecho-janus
Ah, sorry, you meant on the WebRTC side, not the RTP forwarder side. Then TCP is there already, if you use TURN/TCP on the user side (notice I said the *user* side, not Janus: don't configure TURN in janus.jcfg). You can try enabling ICE TCP as well, but I have no idea how well that perform sin libnice now. That said, I don't think it would change anything: if you're getting loss in localhost, you have other problems to solve first.

L.

Alexander Novikov

unread,
Jun 28, 2019, 3:37:45 AM6/28/19
to Lorenzo Miniero, meetecho-janus
Again, i think this is a miscommunication. I am not getting loss in
localhost. My setup is:

ffmpeg taking video from a file on a local instance storage on an EC2
instance where the Janus itself runs
ffmpeg sends it as RTP over localhost to Janus (streaming plugin)
Janus sends it over WebRTC (UDP) to a user on a remote machine running
Chrome. And the video comes out choppy.

If instead of Janus i use another ffmpeg on the same EC2 instance,
pulling out that (localhost) RTP stream from the first ffmpeg, it's
fine then.

So i am having loss between Janus and browser. And would like to
switch to TCP there, so i am having WebRTC over TCP and don't have any
packet loss (i am fine with a slight increase in delay, this is not a
low-latency application). How? Is there any definite doc on it?
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/24be0606-3acc-4bb1-b79c-c57867089950%40googlegroups.com.

Alessandro Amirante

unread,
Jun 28, 2019, 3:42:35 AM6/28/19
to Alexander Novikov, Lorenzo Miniero, meetecho-janus
Switching to TCP on the webrtc side won't bring to any improvement. TCP retransmissions don't help for real time flows as late packets are discarded by the browser. Furthermore webrtc already has its own retransmissions mechanism via NACKs, which works for udp as well.

A.

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/CAH92vQLQQ8cDQuuN8dR-PgU%2B0cmkSJxVA-T2autb%2Bw6LEqDvKg%40mail.gmail.com.

Lorenzo Miniero

unread,
Jun 28, 2019, 3:44:00 AM6/28/19
to meetecho-janus
I think TCP is a red herring and a non-solution. Janus supports retransmissions, so losses would be recovered. Maybe packets are too large? FFmpeg to FFmpeg would be plain RTP, but WebRTC uses SRTP, which adds an extra overhead, so packets may be exceeding the MTU. It's good practice to set a maximum size for RTP packets when you encode them, whatever the tool. Besides, I noticed that if we launch our test gstreamer 0.10 script we get some stuttering video, which is not there when using gstreamer 1.0, meaning that something in encoders/RTP packetization may have been bugged there; maybe something similar was in some FFmpeg versions as well? (just thinking out loud).

L.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.

Амнон Исраэли

unread,
Jun 28, 2019, 4:08:13 AM6/28/19
to meetecho-janus
try to remove that options:
-thread_queue_size 4
-r 24
if you don't put -r option you actually tell FFmpeg use source fps.

I would try to transcode file with needed parameters and will try to stream without transcoding and limit packet size:
ffmpeg -re -stream_loop -1 -i ./small.mp4 -an -c:v copy -sdp_file saved_sdp_file -f rtp rtp://127.0.0.1:4998?pkt_size=1100

Mirko Brankovic

unread,
Jun 28, 2019, 4:45:27 AM6/28/19
to meetecho-janus
Indeed I was about to suggest to play with different frame rate settings

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/0248fb5c-adf7-4d18-b382-7757afae2c2b%40googlegroups.com.


--
Regards,
Mirko

Alexander Novikov

unread,
Jun 28, 2019, 5:05:11 AM6/28/19
to Lorenzo Miniero, meetecho-janus
Bingo. That was it. I reduced packet size to 1300 bytes and it started
working. Very occasionally stream still stucks for a short time, but
the problem reduced by at least 10x and i'm no longer bothered.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
>> > To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/24be0606-3acc-4bb1-b79c-c57867089950%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/3822bb6f-70f4-41ba-98aa-ec120c6c744d%40googlegroups.com.

Mirko Brankovic

unread,
Jun 28, 2019, 5:34:32 AM6/28/19
to meetecho-janus
Nice catch Lorenzo ;)
Try lowering to 1200 and leave 300 bytes fir encryption

Regards,
Mirko

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/CAH92vQKSQTSqYJpWiD9RNEddpzJXdOacU9Jj-z_unoGBZdj8OA%40mail.gmail.com.

Lorenzo Miniero

unread,
Jun 30, 2019, 11:24:42 AM6/30/19
to meetecho-janus
Maybe acknowledge where the solution came from? https://stackoverflow.com/a/56823716

L.

>> > To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.
>> > To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/24be0606-3acc-4bb1-b79c-c57867089950%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.

Alexander Novikov

unread,
Jun 30, 2019, 11:27:30 AM6/30/19
to Lorenzo Miniero, meetecho-janus
Just did. Thank you again!
>> >> > To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
>> >> > To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/24be0606-3acc-4bb1-b79c-c57867089950%40googlegroups.com.
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
>> > To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/3822bb6f-70f4-41ba-98aa-ec120c6c744d%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/65442f95-7cd4-49c5-8e15-19cfb2ca84cc%40googlegroups.com.
Message has been deleted

Manu

unread,
Jul 13, 2020, 1:57:46 PM7/13/20
to meetecho-janus
Hi 

I am trying to read the stream from RTSP to do some process(AI) then push the frame to Janus using FFMPEG.  
Currently, I am using libvpx (Vp8) format for this. but my Video is very choppy on playback. It is breaking a lot. I am adding by FFmpeg pipeline.  not sure what is the issue. I am new to this. I found Janus really simple to use. but this is giving me a headache. it is like breaking/freezing so badly. My setup is Janus running in a container with a network as the host . I use WebSocket + HTTP to play stream through UI 

"ffmpeg",
"-y",
"-f",
"rawvideo",
"-vcodec",
"rawvideo",
"-pix_fmt",
"bgr24",
"-s",
f"{width1}x{height1}",
"-i",
"-",
"-c:v",
"libvpx",
'-pix_fmt', 'yuv420p',
#'-maxrate', '2000k',
'-bufsize', '1500k',
'-g', '60',
"-crf","30",
"-b:v","800K",
'-g', '60',
"-r","16",
"-f",
"rtp", 'rtp://192.168.1.47:10027?pkt_size=1300']

Henry S

unread,
Dec 29, 2020, 4:51:38 AM12/29/20
to meetecho-janus
I am having the same problem with streamingtest demo.
Reply all
Reply to author
Forward
0 new messages