feed RTP to gstreamer with Janus Gateway

[Benjamin Trent]

I am trying to relay rtp and rtcp to a gstreamer pipeline. I am doing this by sending the buffers over a port and receiving them on a udpsrc element.

struct sockaddr_in addr;

addr.sin_family = AF_INET;

addr.sin_addr.s_addr = inet_addr("127.0.0.1");

if (video && session->mountpoint->streaming_ports.video_rtp_fd) {

addr.sin_port = htons( 5006 );

//JANUS_LOG(LOG_VERB, "Forwarding video rtp to %d\n", session->mountpoint->streaming_ports.video_src_port);

sendto(session->mountpoint->streaming_ports.video_rtp_fd, buf, len, 0, (struct sockaddr*)&addr, sizeof(struct sockaddr_in));
}

My Gstreamer pipeline

gst-launch-1.0 udpsrc port=5006 caps="application/x-rtp, media=video, payload=100, encoding-name=VP8-DRAFT-IETF-01" ! rtpvp8depay ! vp8dec ! ffmpegcolorspace ! xvimagesink

Any clues? It would seem that rtpvp8depay cannot depay the stream? I have verified that my udpsrc is getting the packets by linking against a fakesink and seeing them come in.

This shows a stead stream of packets:

gst-launch-1.0 udpsrc port=5006 caps="application/x-rtp, media=video, payload=100, encoding-name=VP8-DRAFT-IETF-01" ! fakesink silent=false

This pipeline shows no output(seeming that packets don't get past the depay process)

gst-launch-1.0 udpsrc port=5006 caps="application/x-rtp, media=video, payload=100, encoding-name=VP8-DRAFT-IETF-01" ! rtpvp8depay ! fakesink silent=false

Anybody got any ideas? Do I need to modify the header before pushing it to my gstreamer pipeline?

[Lorenzo Miniero]

Hi Benjamin,

nice to see you're playing with the plugin code and do something new!

Not sure whether you need additional caps-filters to decode the stream, but yours seems fine. Is 100 actually the payload type of the video stream coming from the browser? Have you checked what's getting to gstreamer with Wireshark, just to be sure you're relaying the right stuff?

Lorenzo

[Lorenzo Miniero]

PS: make also sure gstreamer is getting a key frame to start with. Gstreamer may be receiving just intra-frames, and so it would need the next upoming key-frame to have something to display: before that, you would get nothing to display as there would be no valid image. Normally Chrome sends one every 100 seconds, but you can try and force one by sending a FIR/PLI yo the browser to force it to send one anyway. You can have a look to the videoroom plugin to see how it's done there.

Lorenzo

[Benjamin Trent]

So, fun fact. Apparently my plugin is returning an answer that is wanting the RED feed (payload 116) and not the VP8 mapping. At least, that is the RTP type for the video packets I am getting from the browser. What on earth? My code is here
It currently is an abhorrent mess and I relied heavily on the SIP, Stream, and Echo plugins. 

Also, Here is my create offer JS

streaming.createOffer(

                    {

                      media: { video: "lowres" },

                      success: function (jsep) {

                        console.log("Create offer attached");

                        var body = { "request": "start", "mode": 2 };

                        var toSend = { "message": body, "jsep": jsep };

                        console.log(JSON.stringify(toSend, null, 4));

                        streaming.send(toSend);

                      },

                      error: function (error) {

                        alert(error);

                      }

                    }

                  );

Thanks for the reply. I will take a look at the video room plugin momentarily. 

[Lorenzo Miniero]

Yes, Chrome tried to negotiate RED and ulpfec by default. Your plugin just sends the SDP back as it is (swapping offer and answer) as the Eho Test does:

https://github.com/Computician/janus-gateway/blob/master/plugins/janus_bidirectionalstreaming.c#L847

Since the offer contains RED+ulpfec, the answer does too, and it's used. Trying your plugin with Firefox would work instead as Firefox uses plain VP8.

To avoid this, you need to remove the RED+ulpfec stuff from the SDP before sending it back: this can be done either in JavaScript (that is, you remove it before sending it to Janus) or in the plugin.

Lorenzo

[Benjamin Trent]

Those two steps worked! 

Forcing the keyframe was absolutely needed and I also removed the unwanted codecs from the SDP before sending it. Was able to get a feed just fine!

Thanks a bunch! After I clean up the plugin and do some more testing, would you want it to be merged in or it would be better just to keep it separate? I know my code formatting does not really fit the rest of the plugins.

[Lorenzo Miniero]

Good to know it worked eventually! And excited to see one of the first new plugins in action :-) Hope you didn't find the API plugin too awkward or uncomfortable to use so far! Feedback on how easy (or not) it is to play with the plugins to do what you want would be really welcome, so if you have anything to share on what could be made less complicated that wouldbe great.

About merging, I'd keep things as they are for now, but simply because we'd like having some kind of repository of third-party Janus plugins in the future. A separate project just for the plugin (or plugins, if you write more than one) would be cool as well, as we'd be able to simply reference it somewhere for everyone interested in Janus to find. I'll definitely keep track of your efforts, as I'm always interested in new things being done with Janus :-)

Thanks!

Lorenzo

[Laurent Denoue]

Hi Benjamin,

Have you published your plugin somewhere?

I'm interested in decoding VP8 frames in order to apply some processing (like grayscale, background-subtraction, etc.) and then re-encoding them so the other peers can see the result.

I could base the plugin on the echotest plugin, but I don't know how to decode and re-encode the VP8 frames.

Laurent.

[Амнон Исраэли]

Tell me please if forcing key-frame implemented?

From my test i see that it's does not but maybe i do something wrone...

среда, 20 августа 2014 г., 23:10:15 UTC+3 пользователь Lorenzo Miniero написал: