How to connect Kurento RTPEndPoint to FFMpeg

[piwpiw]

I have a Kurento pipeline with several WebRTCEndPoints. The EndPoints are merged in a Composite Element.

I would like to stream the merged video to ffmpeg in order to stream it to Facebook. (For the moment I'm trying only to save the video to a file using ffmpeg to first make sure the connection between Kurento and ffmpeg works properly)

Here is what I've done so far :

• Step 1 : WerRTCEndPoints(n) -> Connect() -> Composite Element (This is working fine. I can watch the merged video on the browser)
• Step 2 : Composite Element -> Connect() -> RTPEndPoint 
• Step 3 : Create manually an SDP Offer : 

v=0

o=- 3831587299 3831587299 IN IP4 172.31.46.122

s=Kurento Media Server

c=IN IP4 172.31.46.122

t=0 0

m=audio 20000 RTP/AVPF 96 0 97

a=setup:actpass

a=rtpmap:96 opus/48000/2

a=rtpmap:97 AMR/8000

a=rtcp:27193

a=sendrecv

a=mid:audio0

m=video 20001 RTP/AVPF 102 103

a=setup:actpass

a=rtpmap:102 VP8/90000

a=rtpmap:103 H264/90000

a=fmtp:103 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

a=rtcp:56189

a=sendrecv

a=mid:video0

• Step 4 : Make the RTPEndPoint Process the offer and get the response (172.31.46.122  is the local machine address)  :

v=0

o=- 3831591840 3831591840 IN IP4 172.31.46.122

s=Kurento Media Server

c=IN IP4 172.31.46.122

t=0 0

m=audio 65218 RTP/AVPF 96 0 97

a=sendrecv

a=mid:audio0

a=rtcp:65219

a=rtpmap:96 opus/48000/2

a=rtpmap:97 AMR/8000

a=setup:active

a=ssrc:3738415442 cname:user3362844230 @ host-21a7b952

m=video 48128 RTP/AVPF 102 103

a=sendrecv

a=mid:video0

a=rtcp:48129

a=rtpmap:102 VP8/90000

a=rtpmap:103 H264/90000

a=setup:active

a=fmtp:103 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

a=ssrc:1289105171 cname:user3362844230 @ host-21a7b952

• Step 5 : Call ffmpeg on the same machine on this response (saved to the file a.sdp) to record the stream

ffmpeg -loglevel debug -protocol_whitelist file,crypto,udp,rtp -re -vcodec libvpx -acodec opus -i a.sdp -vcodec libx264 -acodec aac -y output.mp4

After that I'm getting the error : 

[udp @ 0x563e77ab6060] bind failed: Address already in use

Does anyone know if I'm doing things correcly ? I'm not very clear about SDP offer/process.

Thank you very much for your help.

Regards,

[Juan Navarro]

This might help to learn more about FFmpeg: https://www.kurento.org/blog/rtp-ii-streaming-ffmpeg

I'd say, use something like port 20000 for audio's RTP and 20001 for audio's RTCP.
Similarly, use 20002 and 20003 for video. This is because FFmpeg doesn't really support "a=rtcp", if I recall correctly.

Apart from that... if you read the blog post carefully, you'll learn what is the problem here: there are TWO semantically opposite versions of SDP.

1) The SDP used in WebRTC and Kurento (even for the RtpEndpoint) is one for SDP Offer/Answer, where every endpoint states the ports where they are listening. So, if Kurento replies with an SDP Answer saying "m=video 48128", it means Kurento is listening on port 48128.

2) The "imperative" SDP (for a lack of a better name...) that tells some program where it must access a stream. And this is the type of SDP supported by FFmpeg!

So if you get the SDP Answer from Kurento (type 1) and pass it to FFmpeg (which assumes type 2), what is happening is:

* Kurento says "I'm listening on port 48128"
* You tell FFmpeg "hey, open a socket on port 48128" (which is already in use by Kurento)

The way this should work is that you pass FFmpeg the SDP Offer that you gave Kurento. This way, I believe you'd be close to make it work, because Kurento would be sending data to the ports stated in the SDP Offer, and FFmpeg would be getting data from that same port.

Regards

--
Juan Navarro
Kurento developer
@j1elo at GitHub & Twitter

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/5a52b24d-c744-4c5c-a51f-ac35cf395e5fn%40googlegroups.com.

[piwpiw]

Thanks for your answer Juan.

Actually to split the issues, I checked whether the RTPEndPoint media state is connected. No matter what I do, It's always Disonnected.

Here is my configuration : 

• Composite Element -> Connect() -> RTPEndPoint 

In logs I'm getting : 

• MediaElementImpl.cpp:940 connect()  Connecting 978b4a9f-9700-4858-99e0-3f5cf72bb93d_kurento.MediaPipeline/83bfc261-0a9c-4822-80c0-5ecf95876073_kurento.Composite/4f0784ba-1c6c-4e12-9884-7ddbe0da77b6_kurento.HubPort -> 978b4a9f-9700-4858-99e0-3f5cf72bb93d_kurento.MediaPipeline/9f933dba-103e-4cfd-a2f0-1c8293eb3d79_kurento.RtpEndpoint params AUDIO default default
• MediaElementImpl.cpp:940 connect()  Connecting 978b4a9f-9700-4858-99e0-3f5cf72bb93d_kurento.MediaPipeline/83bfc261-0a9c-4822-80c0-5ecf95876073_kurento.Composite/4f0784ba-1c6c-4e12-9884-7ddbe0da77b6_kurento.HubPort -> 978b4a9f-9700-4858-99e0-3f5cf72bb93d_kurento.MediaPipeline/9f933dba-103e-4cfd-a2f0-1c8293eb3d79_kurento.RtpEndpoint params VIDEO default default
• MediaElementImpl.cpp:940 connect()  Connecting 978b4a9f-9700-4858-99e0-3f5cf72bb93d_kurento.MediaPipeline/83bfc261-0a9c-4822-80c0-5ecf95876073_kurento.Composite/4f0784ba-1c6c-4e12-9884-7ddbe0da77b6_kurento.HubPort -> 978b4a9f-9700-4858-99e0-3f5cf72bb93d_kurento.MediaPipeline/9f933dba-103e-4cfd-a2f0-1c8293eb3d79_kurento.RtpEndpoint params DATA default default

Can you please tell me if from the logs the RTPEndPoint is connected properly to the composite port hub ? Should I ignore the media state of the RTP Endpoint ?

Thanks.

[Neil Young]

IMHO ffmpeg is somehow allocating two consecutive ports. So you can't use 20000 for audio and 20001 for video. Video needs to be minimum 20002

[Neil Young]

Backed up by FFMPEG doc  (https://ffmpeg.org/ffmpeg-protocols.html):

Important notes:

1. If rtcpport is not set the RTCP port will be set to the RTP port value plus 1.
2. If localrtpport (the local RTP port) is not set any available port will be used for the local RTP and RTCP ports.
3. If localrtcpport (the local RTCP port) is not set it will be set to the local RTP port value plus 1.