How to correctly manage a looped call

[Benoit Panizzon]

Hi

Our Set-Up:

Simple Kamailio Registrar Server running rtpengine (as they communicate with the CPE)

Kamailio 'Core' Server doing the routing and interconnection.

Now I'm having an issue in the situation when a CPE is calling another CPE on the same registrar.

Call Flow:

CPE-A => Registrar => Core => Registrar => CPE-B

This 'sort of' works, when I set 'loop-protec' as the DSP in the looped invite is then ignored.

'sort of' meaning: I end up sending the SDP to CPE-B which I intended to use towards the core and interconnections.

So i removed 'loop-protect'. Now the SDP is managed twice, but not the way I spected.

On the first pass through the registrar, the c/m lines are replaced with the ip and port of the rtpengine towards the core.

On the second pass the c/m lines are again replaced with the ip and a new port of rtpengine towards the cpe, but the inbound sdp of the second pass has ip and port originating from the rtpengine itself and I assume it's the same call ID, I suppose rtpengine considers this as an SDP update and replaces the source ip and port of CPE-A with the source ip and port of that second pass of the invite, thus with it's own ip and port.

Subsequently it attempts to exchange RTP with itself and not with CPE-A anymore.

How can I solve this situation? I attempted to use via-branch=auto-next or to use labels to indicate the direction of the call, but none worked yet.

-Benoît-

[Richard Fuchs]

Rtpengine uses the call ID and the from/to-tags to uniquely identify
media flows. I'm guessing in your case both call IDs and from/to-tags
are the same between the two invocations of rtpengine for the two call legs?

You need to be able to distinguish the two invocations of rtpengine and
communicate this difference somehow. One option is to use two instances
of rtpengine. Another is to modify the call ID, or the from/to-tags. Not
in the SIP messages but just in the communication to rtpengine (e.g..
append a fixed suffix from any invocation in the second call leg).

There are other options that avoid the duplicate internal media loop,
but that would require reworking your signalling flow. All this is
assuming that your "core" is actually involved in media flow in some way
- if it isn't then it should be possible to just invoke rtpengine once
per call (but again requiring reworking your signalling logic).

Cheers

[Benoit Panizzon]

Hi Richard

Correct, Call ID, ToTag FromTag are identical in both invocations of rtpengine the looping call. It is the same call.

I also don't know how I can tell both legs apart to use two different rtpengines. A Re-Invite for example could be initiated by either side so just differentiating if it originates from the core ip or not does not help much I guess.

https://rtpengine.readthedocs.io/en/latest/ng_control_protocol.html

would from-label to-label replace the to/from tags? Or what are those labels exactly for? Somehow I undersood that could be used exactly for that, to 'label' different legs of the same call and keep them appart? I would like to use the kamailio rtpengine module 'as is' so I fear there is no way to alter call id to and from tag from the module.

Avoiding the internal medial loop, yes that is somehow the last option. But what if I use a separate 'internal' protected network for my interconnections? If I don't invoke rtpengine a second time, I would end up sending that internal IP to the destination and that would not work either.

-Benoît-

[Richard Fuchs]

On 26/04/2024 09.20, Benoit Panizzon wrote:
> https://rtpengine.readthedocs.io/en/latest/ng_control_protocol.html
>
> would from-label to-label replace the to/from tags? Or what are those
> labels exactly for? Somehow I undersood that could be used exactly for
> that, to 'label' different legs of the same call and keep them appart?

No, those are mostly just for informational purposes, to show up in logs
for example. They can be used to refer to individual call parties in
some cases (e.g. media playback), but ultimately call ID and tags are
the source of truth.

> I would like to use the kamailio rtpengine module 'as is' so I fear
> there is no way to alter call id to and from tag from the module.

Sure there is. You can put your own custom or modified call ID or tags
in the offer/answer messages. Just put "call-id=foobar" or
"from-tag=foobar" etc to override the values taken from the SIP message.
Use in combination with Kamailio scripting and string manipulation.

> Avoiding the internal medial loop, yes that is somehow the last
> option. But what if I use a separate 'internal' protected network for
> my interconnections? If I don't invoke rtpengine a second time, I
> would end up sending that internal IP to the destination and that
> would not work either.

You'd have to special-case the situation with no internal network involved.

Cheers

[Benoit Panizzon]

Hi Richard

I managed to append an identifier to the looping calls.

The endpoints in the SDP look correct now.

But I still get no audio through when rtpengine should loop audio to itself.

Example CPE-IP-A Port 1 <=> IP-RTPEngine Port 2 <=> IP-RTPEngine Port 3 <=> CPE-IP-B Port 4

It looks like rtpengine is never sending audio to it's own IP on the diffent port.

I wonder if rtpengine is employing some NAT detection techniques I know from our SBC or from Asterisk, aka 'only start sending RTP when packets are being received' and if this can be circumnavigated.

This would explain why the two 'internal' ports never talk to each other when each one waits for the other one to talk first.

-Benoit-

[Benoit Panizzon]

asymmetric does not solve the issue either.

neither does table -1

[Richard Fuchs]

On 03/05/2024 11.58, Benoit Panizzon wrote:
> Hi Richard
> I managed to append an identifier to the looping calls.
> The endpoints in the SDP look correct now.
>
> But I still get no audio through when rtpengine should loop audio to
> itself.
>
> Example CPE-IP-A Port 1 <=> IP-RTPEngine Port 2 <=> IP-RTPEngine Port
> 3 <=> CPE-IP-B Port 4
>
> It looks like rtpengine is never sending audio to it's own IP on the
> diffent port.
>
> I wonder if rtpengine is employing some NAT detection techniques I
> know from our SBC or from Asterisk, aka 'only start sending RTP when
> packets are being received' and if this can be circumnavigated.

No, not at all. The only NAT detection is to change the destination
address/port if media from a different source address/port is received.
But that's not a requirement. If no media is received then the
destination is whatever was given in the SDP. And for internal call legs
the addresses from the SDP should be correct. We routinely use internal
media loops like this ourselves and they definitely work.

Cheers

[Den4t]

Hi !

If you have identical call-id and from/to tags, the core seems to be а SIP poxy/router, in this case 

you have a spiraled call here as described in 3261, it is not a loop. It is not easy to implement the spiral processing in scenario, i do save initial SDP from CPE-A in htable (i use kamailio), then do offer to Core,

when core decide send this call back and the registrar detect this call as spiraled i do rtpengine_delete and make the new offer with saved SDP to CPE-B, so i close the media

at registrar. Of course, this is a very rough description of the process to reflect the principle.

 

пятница, 26 апреля 2024 г. в 15:09:23 UTC+3, benoit....@gmail.com:

[Benoit Panizzon]

Hi Den4t

Hmm, may be worth considering.

I still have no success with appeding an index to the callID and I start seeing where it goes wrong but I don't have a solution yet.

Ip have CPE A <=> RTP-Engine >=> RTP-Engine <=> CPE B

So in the final packet stats, I would expect to see 3 media connections. But I get four: One to each CPE. Those look fine.

But then I get two media connections for the connection between rtpengine itself and they are crossed over and when I look at the ports advertised in the SDP exchanged between rtpengine, it's clear they do not connect.

So I start to wonder, if there is more than the call-id which is considered to corellate the calls. Do I also have to add an index to the from and to tags to make them unique?

-Benoit-

[Richard Fuchs]

On 07/05/2024 06.15, Benoit Panizzon wrote:
> Hi Den4t
>
> Hmm, may be worth considering.
> I still have no success with appeding an index to the callID and I
> start seeing where it goes wrong but I don't have a solution yet.
>
> Ip have CPE A <=> RTP-Engine >=> RTP-Engine <=> CPE B
>
> So in the final packet stats, I would expect to see 3 media
> connections. But I get four: One to each CPE. Those look fine.

If you have two different call IDs, then you should see two "final
packet stats" (one for each call), with two call legs each, and one leg
should point to one of the CPEs and the other leg to the other internal
call leg (rtpengine's own port).

> But then I get two media connections for the connection between
> rtpengine itself and they are crossed over and when I look at the
> ports advertised in the SDP exchanged between rtpengine, it's clear
> they do not connect.
>
> So I start to wonder, if there is more than the call-id which is
> considered to corellate the calls. Do I also have to add an index to
> the from and to tags to make them unique?

No, it's definitely the call ID. Tags don't have to be unique between
different calls.

Cheers

[Benoit Panizzon]

Hi

I'm still not managing to get a call to loop via rtpengine.

It looks like in this situation, rtpengine is neither sending any rtp packets out to the clients, nor sending ip packets to it's own ip address.

When looking with tcpdump -i any portrange 30000-40000 which is the range configured, I only see packets received from the clients, nothing sent back to them.

Also no packets sent from rtpengine to itself on the same ip address.

Also cranking up log-level to 7 does not reveal what on earth is causing the issue.

When looking at the the final packet stats, I can't spot an issue...

A had initiated the call, from 172.29.255.11 via rtpengine running on  157.161.23.4 which then loops the call to itself appeding -B to the call ID towards B at: 172.29.255.3

May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] Final packet stats:
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] --- Tag 'a43f8e142d', created 0:09 ago for branch ''
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] ------ Media #1 (audio over RTP/AVP) using unknown codec
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] --------- Port    157.161.23.4:31974 <>   172.29.255.11:3000 , SSRC 0, in 0 p, 0 b, 0 e, 9 ts, out 0 p, 0 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] --------- Port    157.161.23.4:31975 <>   172.29.255.11:3001  (RTCP), SSRC 0, in 0 p, 0 b, 0 e, 9 ts, out 0 p, 0 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] --- Tag '89ff23da133638a8i0', created 0:09 ago for branch ''
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] ------ Media #1 (audio over RTP/AVP) using unknown codec
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] --------- Port    157.161.23.4:32386 <>    157.161.23.4:31592, SSRC 0, in 0 p, 0 b, 0 e, 8 ts, out 2 p, 1529 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [core] --------- Port    157.161.23.4:32387 <>    157.161.23.4:31593 (RTCP), SSRC 0, in 0 p, 0 b, 0 e, 8 ts, out 2 p, 1527 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [crypto] DTLS data received after handshake, code: 2
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-A]: [crypto] DTLS data received after handshake, code: 2
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] Final packet stats:
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] --- Tag 'a43f8e142d', created 0:09 ago for branch ''
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] ------ Media #1 (audio over RTP/AVP) using unknown codec
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] --------- Port    157.161.23.4:31592 <>    157.161.23.4:32386, SSRC 0, in 0 p, 0 b, 0 e, 9 ts, out 2 p, 894 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] --------- Port    157.161.23.4:31593 <>    157.161.23.4:32387 (RTCP), SSRC 0, in 0 p, 0 b, 0 e, 9 ts, out 2 p, 896 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] --- Tag '89ff23da133638a8i0', created 0:09 ago for branch ''
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] ------ Media #1 (audio over RTP/SAVP) using unknown codec
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] --------- Port    157.161.23.4:36250 <>    172.29.255.3:16476, SSRC 0, in 0 p, 0 b, 0 e, 8 ts, out 0 p, 0 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [core] --------- Port    157.161.23.4:36251 <>    172.29.255.3:16477 (RTCP), SSRC 0, in 0 p, 0 b, 0 e, 8 ts, out 0 p, 0 b, 0 e
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [crypto] DTLS data received after handshake, code: 2
May 24 15:05:25 dev-cpereg01 rtpengine[2439098]: INFO: [e42dbc3f9707f231-B]: [crypto] DTLS data received after handshake, code: 2

Any idea how to further dig into the issue?

-Benoît-

[Benoit Panizzon]

Interesting additional observation.

=> Both CPE on IPv6 => Same issue.

=> One CPE on IPv6 the other on IPv4 => We get RTP!

To me, this looks very much like rtpengine having some kind of loop protection which prevents it to send rtp packets to it's own ip address

[Richard Fuchs]

The forwarding chain looks OK, but none of these show any media received
at all. Not even the external facing call legs show anything received.
So you're dealing with a more fundamental problem that isn't directly
related to back-to-back forwarding. (There wasn't anything to forward.)

> => Both CPE on IPv6 => Same issue.
> => One CPE on IPv6 the other on IPv4 => We get RTP!
>
> To me, this looks very much like rtpengine having some kind of loop

> protection which prevents it to send rtp packets to it's own ip address
That is most certainly not the case. I think you're looking at a more
basic networking / network connectivity problem.

Cheers

[Benoit Panizzon]

Hi Richard

I have now the same kamailio version and config on two machines (one Ubuntu 22 jammy the other Ubuntu 24 noble)

One running 12.4 experiencing the 'no rtp' issue on a spiraling calls.

The other one running 12.5 on which rtp is passed on without issue on spiraling calls.

Thank you for the hint on using the backport script to get rid of the liburing dependency. I will revert the snapshot of machine I upgraded to Ubuntu 24 for testing and re-compile 12.5 on jammy in a day or two.

Then I will be able tell if it is indeed version 12.5 which solves the issue.

[Benoit Panizzon]

Confirming: Ubuntu 22 / kernel 5.15.0 with now successfully compiled latest 12.5 without liburing

=> Spiraling calls have audio.

With previously used 12.3 (and 12.4) we had no RTP traffic on looping calls.