JSON Message flow in between 2 rtpendpoints.

[PIYUSH BADKUL]

• Operating system (Ubuntu 16.04, 18.04, etc.) --> OS is Ubuntu 16.04
  
• Installation method (apt-get, Docker, AWS, build from sources) --> Installation method was building from source (kms-omni-build Github repo)
• Kurento version --> 6.13

We have a simple requirement, We want the RTP of a normal SIP call flowing through Kurento Media Server. For this, we have integrated a WebSocket library which will connect to the kurento media server on startup through the GET request. When the Phone A will initiate a call and as soon as an INVITE will reach our server, then 

1) Send a Describe request to the Kurento Media server to get a session ID in response.

2) Send a request for the Creation of a Media Pipeline to the kurento.

3) Send a request for the creation of a Media RTPEndpoint to the kurento.

4) Send a Process Offer (INVOKE) and send the SDP received in the INVITE message of A and received a response of the SDP which are accepted to KMS.

5) Forward the Invite from A to B with the SDP of KMS.

On receiving the 200 OK response (Answer) from B, we would,

6) Send a request for the Creation of a Media RTPEndpoint to the kurento(For B).

7) Send a Process Offer (INVOKE) and send the SDP received in the 200OK message of B and received a response of the SDP which are accepted to KMS.

8) Send a Connect request (INVOKE) with values of (SDP accepted of A by KMS) in the object field and (SDP accepted of B by KMS) in the sink field and send another CONNECT request with values vice versa.

9) Forward the 200OK to A and call will be established.

We can see that the RTP messages are going through kurento but the Voice is not coming. It feels like the Voice of B is coming to B itself and the voice of A is coming to A itself but we were not able to verify it using Wireshark also.

We feel that there is a problem in the Flow of a sequence of JSON messages to the KMS through our application. Hence, we want to ask what is the correct flow of messages in this particular case where both are the RTP endpoints.

Also, we did not use IceCandidates and other messages because we felt that there was no use of Video for us and this was just a 

simple call. Also, we are creating the JSON message in the C during Runtime itself. There was no error observed in the response of these requests. Needless to say, the sessionID in each request was the same, and ID in each request was different.

Please let us know if there is anything else that we can provide to you.

Thanks and hope you are in good health

[Anthony Alba]

I would modify your sequence slightly

> 1) Send a Describe request to the Kurento Media server to get a session ID in response.
> 2) Send a request for the Creation of a Media Pipeline to the kurento.
> 3) Send a request for the creation of a Media RTPEndpoint to the kurento.
> 4) Send a Process Offer (INVOKE) and send the SDP received in the INVITE message of A and received a response of the SDP which are accepted to KMS.
> 5) Forward the Invite from A to B with the SDP of KMS.
>
> On receiving the 200 OK response (Answer) from B, we would,
> 6) Send a request for the Creation of a Media RTPEndpoint to the kurento(For B).
> 7) Send a Process Offer (INVOKE) and send the SDP received in the 200OK message of B and received a response of the SDP which are accepted to KMS.
> 8) Send a Connect request (INVOKE) with values of (SDP accepted of A by KMS) in the object field and (SDP accepted of B by KMS) in the sink field and send another CONNECT request with values vice versa.
> 9) Forward the 200OK to A and call will be established.

1) Get session id for JSON-RPC

2) Create MediaPipeline and two RtpEndpoint ForA and ForB

3) Connect ForA to ForB and ForB to ForA

4) SIP INVITE, extract SDP(A-OFFER), send to
KMS:ForA.processOffer(A-OFFER), save ForA-ANSWER (!!!!don't send to
B!!!!)

5) KMS:ForB.createOffer(): get ForB-OFFER SDP, replace the SDP in the
SIP INVITE, i.e, A-OFFER with ForB-OFFER
and send to B (adjust Content-Length header, etc etc)

6) 200 OK from B, take B-ANSWER SDP from 200 OK , and send to
KMS:ForB.processAnswer(B-ANSWER).
At this point: B and ForB will start exchanging RTP/RTCP (even before
ACK), and you can verify this on wireshark,

ForB will show MediaConnectionStateChange DISCONNECTED -> CONNECTED

7) Replace B-ANSWER SDP in 200 OK with ForA-ANSWER, from step 4), and send to A.
A and ForA will start exchanging RTP/RTCP.

8) Send ACK as per normal

9) Summary
i) B sees SIP INVITE with ForB-OFFER
ii) A sees 200 OK with ForA-ANSWER
iii) KMS INVOKE: ForA.processOffer(A-OFFER), ForB.generateOffer(),
ForB.processAnswer(B-ANSWER)
There should be 3 INVOKEs

Note:
* A exchanges RTP with ForA, B exchanges RTP with ForB, and KMS
internally will bridge media between ForA and ForB.
* when you obtain ForB-OFFER SDP in step 5), you can remove all lines
from m=video onwards to have an audio-only SDP.
You should also remove any a=group lines that you see
(usually something like a=group audio0 video0; this is not needed when
you have only one media stream)

[PIYUSH BADKUL]

On Sunday, June 7, 2020 at 6:32:28 PM UTC+5:30, Anthony Alba wrote:

I would modify your sequence slightly

> 1) Send a Describe request to the Kurento Media server to get a session ID in response.
> 2) Send a request for the Creation of a Media Pipeline to the kurento.
> 3) Send a request for the creation of a Media RTPEndpoint to the kurento.
> 4) Send a Process Offer (INVOKE) and send the SDP received in the INVITE message of A and received a response of the SDP which are accepted to KMS.
> 5) Forward the Invite from A to B with the SDP of KMS.
>
> On receiving the 200 OK response (Answer) from B, we would,
> 6) Send a request for the Creation of a Media RTPEndpoint to the kurento(For B).
> 7) Send a Process Offer (INVOKE) and send the SDP received in the 200OK message of B and received a response of the SDP which are accepted to KMS.
> 8) Send a Connect request (INVOKE) with values of (SDP accepted of A by KMS) in the object field and (SDP accepted of B by KMS) in the sink field and send another CONNECT request with values vice versa.
> 9) Forward the 200OK to A and call will be established.


1) Get session id for JSON-RPC

2) Create MediaPipeline and two RtpEndpoint ForA and ForB

3) Connect ForA to ForB and ForB to ForA

4) SIP INVITE, extract SDP(A-OFFER), send to
KMS:ForA.processOffer(A-OFFER), save ForA-ANSWER (!!!!don't send to
B!!!!)

5) KMS:ForB.createOffer():  get ForB-OFFER SDP, replace the SDP in the
SIP INVITE, i.e, A-OFFER with ForB-OFFER
and send to B (adjust Content-Length header, etc etc)

>>> After process offer, when i do generateoffer(), then in that case i get the error - SDP_END_POINT_ALREADY_NEGOTIATED

Error message -  Response: {"error":{"code":40208,"data":{"type":"SDP_END_POINT_ALREADY_NEGOTIATED"},"message":"Endpoint already negotiated"},"id":"glxAvxX7Qpf8cYzI3Pu5fg..19","jsonrpc":"2.0"}

[Anthony Alba]

>>> After process offer, when i do generateoffer(), then in that case i get the error - SDP_END_POINT_ALREADY_NEGOTIATED

Error message -  Response: {"error":{"code":40208,"data":{"type":"SDP_END_POINT_ALREADY_NEGOTIATED"},"message":"Endpoint already negotiated"},"id":"glxAvxX7Qpf8cYzI3Pu5fg..19","jsonrpc":"2.0"}

1. You are doing generateOffer on the second RtpEndpoint (that hasn't been used at all).?

2. You have created two RtpEndpoints  ForA and ForB?

3. The processOffer(..) is invoked on ForA, but the generateOffer is invoked on ForB?

[Raman Kumar]

And what happens when A add C to  group call. Can you explain sequence of sdp negotiation in this case?

[PIYUSH BADKUL]

Thank you for such a fast reply.

The solution provided by you worked like a charm.

However, we wanted to test something rather complex with the kurento media server.

Description - 

Subscriber A calls subscriber  B and their call goes through a sip server. Call between subscriber A and subscriber B is established and RTP flows in between the endpoint points of  A and B. Now, A decides to call the third subscriber C and as soon as the call is picked up, we want the RTP of all the 3 subscribers should flow through the KMS.

We ran the Group call tutorial in kurento to find out the JSON request and responses sent and recieved by the server respectively,

To our knowledge, we found out the following - 

1- As soon as 1 subscriber Joins the room 

    1 Create RTPENDPOINT 

    1 Process Offer

2- As soon as 2nd Subscriber joins the room

     3 CREATE RTPENDPOINT

     3 Connect RTPENDPOINT

     3 Process Offer

3- As soon as 3rd Subscriber joins the room

     5 CREATE RTPENDPOINT

     9 Connect RTPENDPOINT

     5 Process Offer

For each subscriber, kurento will create N RTPENDPOINTs (N total number of members in the room) out of which 1 RTPENDPOINT will be sendonly and others remaining will be recvonly. Is this observation Correct?

Here is something we have thought will work,

Assuming the RTPENDPOINT for subscriber A are denoted in an array A[N] and similarly B[N] and C[N] for subscriber A and C respecitvely.

Assuming for keeping things simple, let say N=3

1- We will GenerateOffer for A[0], B[1] and  C[2].

2- We will send these SDP in the Re-INVITE message to A, B and C.

3- For the SDP recieved in the response of 200OK from A, we will do ProcessOffer(B[0]), ProcessOffer(C[0]), ProcessAnswer(A[0])   

4- For the SDP recieved in the response of 200OK from B, we will do ProcessOffer(A[1]), ProcessOffer(C[1]), ProcessAnswer(B[1])

5- For the SDP recieved in the response of 200OK from C, we will do ProcessOffer(A[2]), ProcessOffer(B[2]), ProcessAnswer(C[2])

We believe that after all these processing, the rtp for subscriber A, B and C will flow through kurento.

Are we missing something or is this correct? 

[Juan Navarro]

I don't know about SIP so I cannot comment much here. But, regarding the many-to-many architecture you propose, that's correct and it should work fine.

You just have to make sure your handling logic for all those Endpoints is correct, because it is easy to make mistakes and leave some Endpoint without connection.

However I don't agree with this:

2- As soon as 2nd Subscriber joins the room

     3 Connect RTPENDPOINT

3- As soon as 3rd Subscriber joins the room

     9 Connect RTPENDPOINT

If my maths are correct, for each new subscriber entering the room, there should be (N-1)+(N-1) = 2N-2 connections.
So, 0 connections when N=1,
2 connections when N=2,
4 connections when N=3,
etc.

This architecture might not be optimal because it uses a lot of Endpoints in a single direction (and in theory the Endpoints can be bi-directional but in practice there might be unsolved bugs or issues with that approach), but it definitely works very well.

In fact, this is the architecture that OpenVidu uses for handling group calls (OpenVidu is an advanced Application Server that uses KMS to simplify making group calls):

--
Juan Navarro
Kurento maintainer & developer
j1elo @ GitHub and Twitter

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/1d64595b-44d9-4528-b4f3-df0f171309b7o%40googlegroups.com.