Streaming with simultcast problems.

[trunki 3]

Hi,

I am testing the streaming plugin with simulcast with 3 qualities. This is my config:

testmulti: {
type = "rtp"
id = 404
description = "Test Multicast"

audio = true
audioport = 17204
audiopt = 111
audiortpmap = "opus/48000/2"

video = true
videoport = 17104
videopt = 100
videortpmap = "VP8/90000"
#videobufferkf = true
videosimulcast = true
videoport2 = 18104
videoport3 = 19104
}

I use the streamingtest.html supplied with the repo to check the result, but the video does not look good, it only shows one frame every few seconds. The resolution and bitrate information is correct when I change the quality with SL0, SL1 or SL2 buttons.

The same 3 streams see perfect if I split them into three separate configurations with distinct Id.

What am I doing wrong?
Thank you.-

[Bang He]

is this  simultcast ?

[Lorenzo Miniero]

I just made a test with the configuration you shared, and it works just fine for me. Maybe you're sending all three resolutions to the same port, instead of sending them to the different video ports? In your example, high res must be sent to port 19104, middle res to 18104, and low res to 17104.

L.

[trunki 3]

I tesetd again with your suggested quality order with the same result.:

ffmpeg -threads 4 -f decklink -i "DeckLink SDI (4)" \

-codec:v libvpx -deadline realtime -r 20 -s:v 720x440 -pix_fmt:v yuv420p -b:v 500k -minrate 500k -maxrate 500k -vf yadif -an -f rtp rtp://127.0.0.1:19104?pkt_size=1200  \

-codec:v libvpx -deadline realtime -r 15 -s:v 312x176 -pix_fmt:v yuv420p -b:v 200k -minrate 200k -maxrate 200k -vf yadif -an -f rtp rtp://127.0.0.1:18104?pkt_size=1200 \

-codec:v libvpx -deadline realtime -r 10 -s:v 312x176 -pix_fmt:v yuv420p -b:v 100k -minrate 100k -maxrate 100k -vf yadif -an -f rtp rtp://127.0.0.1:17104?pkt_size=1200 \

-codec:a libopus -b:a 20k -vn -f rtp rtp://127.0.0.1:17204?pkt_size=1200

With the "echo test" in the same machine with "?simulcast=true" option and work fine.

[Lorenzo Miniero]

I tried your ffmpeg sample (with webcam/mic since I don't have a decklink) and it does look horrible, but that seems to be caused by how that ffmpeg pipeline is constructed (I see a ton of warnings on the console). I tried with a simple gstreamer pipeline and it was smooth, which means the problem is in your script and not Janus.

L.

[trunkline]

The same ffmpeg pipeline simultaneous to 3 independent mount-points (only one with audio) works fine. And I don't know how to test it with a webcam apart from the "echo test" that works well.

The simulcast wIth Firefox shows only one frame every few seconds, sems keyframes only. In chrome it shows "No remote video available" and the webrtc-internals shows a 0 framesReceived/s but a vàlid bytesReceived_in_bits/s:

packetsReceived	7212
[packetsReceived/s]	17.017015782401895
bytesReceived	5966589
[bytesReceived_in_bits/s]	98642.6354859419
headerBytesReceived	201936
[headerBytesReceived_in_bits/s]	3811.8115352580244
lastPacketReceivedTimestamp	1653996605938
[lastPacketReceivedTimestamp]	31/5/2022, 13:30:05
jitterBufferDelay	1.657
[jitterBufferDelay/jitterBufferEmittedCount_in_ms]	0
jitterBufferEmittedCount	37
framesReceived	38
[framesReceived/s]	0

My current versión of janus:  janus 0.11.6

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/c72fc75e-ca9c-4bd9-84b9-840aab357c98n%40googlegroups.com.

[Lorenzo Miniero]

If you check this GStreamer pipeline, you'll see it works fine with simulcast:

gst-launch-1.0 -ev \
    audiotestsrc ! audioresample ! audio/x-raw,channels=1,rate=16000 ! \
        queue ! opusenc bitrate=20000 ! \
            rtpopuspay ! udpsink host=127.0.0.1 port=17204 \
    v4l2src device=/dev/video0 ! videoconvert ! videoscale ! \
    video/x-raw,width=1280,height=720,framerate=30/1 ! tee name=t \
        t. ! queue ! \
            vp8enc target-bitrate=750000 deadline=1 cpu-used=16 error-resilient=1 ! \
                rtpvp8pay ! udpsink host=127.0.0.1 port=19104 \
        t. ! queue ! videoconvert ! videoscale ! video/x-raw,width=640,height=360 ! \
            vp8enc target-bitrate=400000 deadline=1 cpu-used=16 error-resilient=1 ! \
                rtpvp8pay ! udpsink host=127.0.0.1 port=18104 \
        t. ! queue ! videoconvert ! videoscale ! video/x-raw,width=320,height=180 ! \
            vp8enc target-bitrate=150000 deadline=1 cpu-used=16 error-resilient=1 ! \
                rtpvp8pay ! udpsink host=127.0.0.1 port=17104

This seems to confirm some problem in the ffmpeg pipeline you wrote. If as you say it works when the same pipeline is sent to different mountpoints, that's weird, since if we did anything that could break RTP traffic, it wouldn't work with the GStreamer pipeline either. If you want to check what happens with webcam captured from a browser, open the VideoRoom demo on the same server with ?simulcast=true, and then type this on the JavaScript console:

sfutest.send({message: {
    request: "rtp_forward",
    room: myroom,
    publisher_id: myid,
    secret: "adminpwd",
    host: "127.0.0.1",
    audio_port: 17204,
    video_port: 17104,
    video_port_2: 18104,
    video_port_3: 19104
}})

which will forward the three substreams sent to the browser to the simulcast mountpoint you created. That results in smooth playback too. This is why I think there's something in what ffmpeg is generating that's causing the stream to break (maybe something to do with the PictureID we have to overwrite when using simulcast?)

L.

[trunkline]

You are right, with gstreamer the video looks smooth.

But the quality doesn't seem as good at the same bitrate.

I'll have to investigate a little more.
Thank you very much.

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/3bf3dfc1-5a6e-4e71-940d-2d21f3860092n%40googlegroups.com.

[Lorenzo Miniero]

In the meanwhile I'm investigating why it doesn't work with FFmpeg. As I suspected it seems to have something to do with overwriting PictureID: commenting that part out makes the stream fluid, but then changing substreams breaks video as overwriting PictureID is required. I'll check if there's something wrong in our parser or in the VP8 data we receive, and keep you posted.

L.

[Lorenzo Miniero]

Eureka! Solved the issue finally... as I thought it was a parsing error, where we were not returning the value of PictureID when it was only 7-bits: as a consequence, the simulcast code in Janus thought it was 0 in all packets, and so was not updating it correctly when it actually increased. With browser and GStreamer very likely the PictureID was always 15-bit, which is why this went unnoticed for a while.

I've pushed the fix on both the 0.x branch (the one you'll probably want to test, since you were using 0.11.6) and master.

L.