WebRTC -> KMS -> RtpEndpoint -> ffmpeg - SDP issues

[Idan]

I am trying to consume a stream from RtpEndpoint using FFMPEG.
What I've done is getting the published WebRTC to a WebRtcEndpoint
From there, connect it to RtpEndpoint, create a ffmpeg sdp offer file manually and running ffmpeg with the RtpEndpoint sdp answer.

The ffmpeg offer I'm manually processing in server side is:

var ffmpegOffer = "v=0\
o=- 0 0 IN IP4 127.0.0.1\
s=\
c=IN IP4 127.0.0.1\
t=0 0\
m=video 11191 RTP/AVP 100\
a=rtpmap:100 H264/90000\
a=recvonly"


This sdp string goes into RtpEndpoint.processOffer() and the sdpAnswer I get from the offer processing is going to a file ffmpeg can read (ffmpeg -i answer.sdp) .

For some reason I can't get the stream in FFmpeg. I can see it doing the sdp handshake and trying to get the media but nothing comes out at the process ends with an empty stream error.

What am I missing / doing wrong?

Thanks,

Idan.

[t.vand...@sping.nl]

I'm not sure why you would feed the answer SDP from KMS to FFMpeg. That's only useful when you want to send media from FFMpeg to KMS. And from your post I understand you want to send media from KMS to FFMpeg. I have no experience with that use case, but I'd expect you have to tell FFMpeg to listen on certain ports on a network interface. The SDP you created manually and send to KMS should then contain the connection parameters for receiving media by FFMpeg.

Hope this helps,

Tom van der Geer

[Idan Yael]

I would like to run ffmpeg after I get a webrtc broadcast and not before. Thats why i would like ffmpeg to initiate the connection.

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/lJ5i1OzrOv0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+unsubscribe@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/a74a021e-bca8-44e2-969a-7715213ee99c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[t.vand...@sping.nl]

Are KMS and FFMpeg on the same machine? The SDP contains the local host IP address (127.0.0.1)

Still not sure why you would want to get the media flowing from FFMPEG to KMS as well.

Can you post the full FFMpeg command line?

Are you trying to record the video stream (without audio)?

An other thing that comes to mind is that you can set a VIDEO_ONLY parameter when you connect the WebRTCEndpoint sink to the RTPEndpoint source.

Best regards,

Tom van der Geer

[Mario Gasparoni Junior]

Hi, Idan.

As a proof of concept for something similar as you are trying to do, i developed a simple application based on kurento-hello-world example (NodeJS version).
In this example i use VLC to play the RTP flow, but ffmpeg should work as well.

Here is the link:
https://github.com/mariogasparoni/kurento-tutorial-node/tree/master/kurento-rtp-relay

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+unsubscribe@googlegroups.com.

To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAN6U2DUoAaLCvqxNL9EYCkbVNZM29rTLFEskPxtqm3UehtgN-Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Att. Mário Gasparoni.

www.inf.ufrgs.br/~mcgjunior

[Idan]

Hi Mario,

You are sending the RTP stream out of KMS. You are getting the sdp offer in advance ... this is not what I'm trying to accomplish. 

I need the ffmpeg to run undependably of KMS and consume its RtpEndpoint output.

My ffmpeg command is (this is an example for re-transmitting, but it can be anything like transcoding, scaling, remuxing, etc...):

ffmpeg -loglevel debug -protocol_whitelist file,udp,rtp -re -analyzeduration 2048M -probesize 2048M -i rtp.sdp -y -vcodec libx264 -vprofile baseline -acodec aac -ar 44100 -ac 1 -f flv rtmp://rtmp-server:1935/path

where rtp.sdp is the SdpAnswer returned from the RtpEnpoint.processoffer() function.

On Friday, July 28, 2017 at 4:40:59 PM UTC+3, Mario Gasparoni Junior wrote:

Hi, Idan.

As a proof of concept for something similar as you are trying to do, i developed a simple application based on kurento-hello-world example (NodeJS version).
In this example i use VLC to play the RTP flow, but ffmpeg should work as well.

Here is the link:
https://github.com/mariogasparoni/kurento-tutorial-node/tree/master/kurento-rtp-relay

Hope this helps.

2017-07-28 10:16 GMT-03:00 Idan Yael <idan...@gmail.com>:

I would like to run ffmpeg after I get a webrtc broadcast and not before. Thats why i would like ffmpeg to initiate the connection.

On Jul 28, 2017 3:30 PM, <t.vand...@sping.nl> wrote:

I'm not sure why you would feed the answer SDP from KMS to FFMpeg. That's only useful when you want to send media from FFMpeg to KMS. And from your post I understand you want to send media from KMS to FFMpeg. I have no experience with that use case, but I'd expect you have to tell FFMpeg to listen on certain ports on a network interface. The SDP you created manually and send to KMS should then contain the connection parameters for receiving media by FFMpeg.

Hope this helps,

Tom van der Geer

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/lJ5i1OzrOv0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to kurento+u...@googlegroups.com.

To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/a74a021e-bca8-44e2-969a-7715213ee99c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "kurento" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.

To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAN6U2DUoAaLCvqxNL9EYCkbVNZM29rTLFEskPxtqm3UehtgN-Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Idan]

Hi Tom,

Sent the command.

I'm not trying to record, just trying to make RtpEndpoint an input stream for ffmpeg so I can manipulate and run different video tasks later (it's just first stage)

You are right that the current SDP is video only, I got another example with audio as well, but it seems like it doesn't matter. Any SDP configuration I tried won't result in FFmpeg getting the stream.

[Mario Gasparoni Junior]

I am still trying to figure out your use case, perhaps i am misunderstanding  it, so could  you please clarify the media flow of your application ?

Something like (some code is also welcome):
Browser's WebCAM -> KMS WebRtcEndpoint -> KMS RTPEndpoint -> FFmpeg (translates rtp into rtmp, with some re-encoding) -> RTMP server -> ...

I noticed that you said:

var ffmpegOffer = "v=0\
o=- 0 0 IN IP4 127.0.0.1\
s=\
c=IN IP4 127.0.0.1\
t=0 0\
m=video 11191 RTP/AVP 100\
a=rtpmap:100 H264/90000\
a=recvonly"


This sdp string goes into RtpEndpoint.processOffer() and the sdpAnswer I get from the offer processing is going to a file ffmpeg can read (ffmpeg -i answer.sdp) .

That won't work, because you need to use your offer.sdp (var ffmpegOffer) as input of FFmpeg (ffmpeg -i offer.sdp ... ), instead of answer.sdp. The offer.sdp tells FFmpeg where it should listen for incoming video (since it is recvonly), while the answer.sdp tells what local ip/port Kurento choose for sending media to your FFmpeg (this answer.sdp is not really important for you at this moment, since your FFmpeg is not sending media to Kurento).

--

You received this message because you are subscribed to the Google Groups "kurento" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kurento+unsubscribe@googlegroups.com.

To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/eab0d193-19de-4f48-947b-8af2bbb4123e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[t.vand...@sping.nl]

Yes, Mario is right. That's what I tried to tell you as well. You're getting the SDP's mixed up. Probably because of what you know of RTSP servers. In telecommunication, when an endpoint is sending you an SDP it's telling you what its receiving capabilities and connection parameters are.

In your case you need to tell KMS where the RTPEndpoint should start sending its media packets to. That's in the offer SDP. And you need to tell FFMPEG on what ports it should listen; also in the offer SDP.

The answer SDP from the RTPEndpoint is not relevant in your use case.

Best regards,

Tom

[Idan]

Thanks for the info Tom and Mario.

As I understand, the offer I'm creating needs to go both to RtpEndpoint.processOffer() AND to "ffmpeg -i offer.sdp". for this to work...

Meaning, ffmpeg should run and wait for the RtpEndpoint to send it's data ... so I should run ffmpeg BEFORE I run processOffer() ? is that correct? 

Is there any way I can reverse the flow and make RtpEndpoint wait for ffmpeg to connect to it and just then will it start sending the stream? (trying to avoid the race condition)

[Mario Gasparoni Junior]

... so I should run ffmpeg BEFORE I run processOffer() ? is that correct?

Not really. Usually FFmpeg will be able to play it if you start listening after Kurento start sending the video.

You may have problem if you start FFmpeg too early (1+ mins earlier, i guess), because FFmpeg closes when not receiving media for a period of time.

--

You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+unsubscribe@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/52789748-2888-4257-8dcb-a3996c240589%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[t.vand...@sping.nl]

Hey Idan,

Yes, you need to send the offer with the FFMpeg connection parameters to KMS. But that's what you did already. I have no experience with starting FFMpeg with an SDP. I assume that is intended for RTSP streaming use cases. So I'm not sure if it works for your use case. I'm interested in your results. Make sure to run KMS and FFMpeg on the same system though, since you're using the local interface (127.0.0.1).

One other thing, since you seem to be relying on timing of FFMpeg; do you expect KMS to buffer the media? And that it will start from the "beginning" of the media once FFMpeg is connected? Because KMS won't do that for you. As a matter of fact KMS doesn't even know if FFMpeg is listening or not.

[Idan Yael]

So I made the changes and now using the manually offer as an input file for FFmpeg. 

Seems like it's getting some data and failing on conversion... so first of all - THANKS!

Running FFmpeg before or after does not makes a difference, both timings work. If there is no stream while ffmpeg starts it will wait for it (probably can change the timeout using some flag, did not check). If there is already a stream running it will also get it - probably only from when the connection was made but I can't be sure before I'll get ffmpeg to run.

Now for the failing part. This is the error I'm getting from FFmpeg:

[h264 @ 0x1f55480] Current profile doesn't provide more RBSP data in PPS, skipping

Have no idea what that means and did not find anyone mentioning this in this context.

The command I'm using: 

ffmpeg -loglevel debug -analyzeduration 2048M -probesize 2048M -i /tmp/test.sdp -y -c:v libx264 -strict -2 -c:a aac -ar 44100 -ac 1 -f flv rtmp://<rtmp host>/<path...>

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/lJ5i1OzrOv0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+unsubscribe@googlegroups.com.

To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/7bcdd2cb-471a-4e3e-b2bd-444e87f9baa0%40googlegroups.com.

[t.vand...@sping.nl]

Hi Idan,

Good to hear you got the media flowing to FFMpeg. Is there any particular reason why you are using H.264 video between KMS and FFMpeg ? Usually between the WebRTC client (browser) and KMS there's VP8 video, so KMS will need to transcode the video.

If you do want to use H.264 video between KMS and FFMpeg you might need to play around with the fmtp attributes in the SDP to instruct the video encoder in KMS to use a certain H.264 profile. For instance, this is part of a SDP from a Cisco Jabber Video client:

a=rtpmap:126 H264/90000
a=fmtp:126 profile-level-id=428028;max-mbps=245760;max-fs=8192;max-smbps=323500;packetization-mode=1

[Idan Yael]

Thanks, I'll try this out.

Currently, I'm getting errors in ffmpeg because I see it miss RTP packets and crash (??) - very weird. 

After some repeated errors like this:

Delay between the first packet and last packet in the muxing queue is 10120000 > 10000000

It just crashes.

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/lJ5i1OzrOv0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+unsubscribe@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/bc45f432-cdc0-40f0-a2bf-3b713c1f8984%40googlegroups.com.