Large delete payloads

[Alex Balashov]

Hi,

We use the UDP transport for the RTPEngine NG control socket on mr13.5+.

Occasionally, stream `delete` operations fail. I've traced this to very large responses that RTPEngine cannot send, even over loopback, because they exceed the maximum UDP payload size:

[pid 1262923] sendmsg(6, {msg_name={sa_family=AF_INET, sin_port=htons(43780), sin_addr=inet_addr("10.3.64.8")}, msg_namelen=16, msg_iov=[{iov_base="0_797794318_2992077481", iov_len=22}, {iov_base=" ", iov_len=1}, {iov_base="d7:createdi1782318030e10:created"..., iov_len=84739}], msg_iovlen=3, msg_control=[{cmsg_len=28, cmsg_level=SOL_IP, cmsg_type=IP_PKTINFO, cmsg_data={ipi_ifindex=0, ipi_spec_dst=inet_addr("10.3.64.8"), ipi_addr=inet_addr("0.0.0.0")}}], msg_controllen=32, msg_flags=0}, 0) = -1 EMSGSIZE (Message too long)

This is because RTPEngine, in response to a very rich and noisy RTCP environment, tries to compile labouriously articulated MOS score time series into the `delete` response.

Combined with certain long-running calls' unusually large number of SSRC pivots, this leads to a stupefyingly large payload -- stuff like this, of course rendered as JSON from the bencoded payload, for readability:

"SSRC": {
"623290207": {
"bytes": 516,
"packets": 3,
"last RTP timestamp": 35550,
"last RTP seq": 57411,
"cumulative loss": 5,
"average MOS": {
"MOS": 43,
"round-trip time": 2665,
"round-trip time leg": 1467,
"jitter": 0,
"packet loss": 0,
"samples": 25
},
"lowest MOS": {
"MOS": 43,
"round-trip time": 2820,
"round-trip time leg": 1730,
"jitter": 2,
"packet loss": 0,
"reported at": 1782338476
},

[... I'll spare you the remaining 80 kB or so ...]

As far as I can tell, there's no actual way to suppress these giant QoS payloads using any NG protocol flag or RTPEngine CLI option. I've looked at `--measure-rtp` and `--mos` and there just doesn't seem to be a way to turn these things off.

That leaves switching to WebSocket as the control socket transport for RTPEngine, and that's what we're going to do. As far as I can tell, UDP and WS[S] are the only supported NG listeners and the only transports supported by the Kamailio RTPEngine module, in any case. This second part is a little unclear: it also seems that it supports UNIX domain socket, but I assume that's for the legacy protocol?

Anyway, just wondering if I'm missing anything. It'd be far easier to turn this noise off than to switch NG transports.

Many thanks!

-- Alex

--
Alex Balashov
Principal Consultant
Evariste Systems LLC
Web: https://evaristesys.com, https://www.csrpswitch.com
Tel: +1-706-510-6800

[Richard Fuchs]

On 24/06/2026 19.53, 'Alex Balashov' via Sipwise rtpengine wrote:

This is because RTPEngine, in response to a very rich and noisy RTCP environment, tries to compile labouriously articulated MOS score time series into the `delete` response. 

Combined with certain long-running calls' unusually large number of SSRC pivots, this leads to a stupefyingly large payload -- stuff like this, of course rendered as JSON from the bencoded payload, for readability:

Is it just a large number of SSRCs that you are seeing? Or is it the number of MOS report blocks per SSRC?

AFAICR the number of tracked SSRC is limited, but I think the number of report blocks might not be.

As far as I can tell, there's no actual way to suppress these giant QoS payloads using any NG protocol flag or RTPEngine CLI option. I've looked at `--measure-rtp` and `--mos` and there just doesn't seem to be a way to turn these things off. 

Yeah, that's because at the moment the `delete` is done, we don't yet know if Kamailio wants to see/use the stats later on.

I'll add a flag to the delete handling to tell rtpengine that the stats won't be needed.

That leaves switching to WebSocket as the control socket transport for RTPEngine, and that's what we're going to do. As far as I can tell, UDP and WS[S] are the only supported NG listeners and the only transports supported by the Kamailio RTPEngine module, in any case. This second part is a little unclear: it also seems that it supports UNIX domain socket, but I assume that's for the legacy protocol?

No, Unix sockets are not supported for any protocol. I guess the control module supports them, as a legacy holdover from the original (rtpproxy) code base, but rtpengine itself doesn't.

And you shouldn't have to switch protocols just for this, so if we can determine what exactly it is that makes the payload grow so large, then we can fix it.

Cheers

[Alex Balashov]

Hi Richard,

Thank you for looking into this and for your thoughts. The answer seems to be both: relatively large number of SSRCs and huge MOS blocks.

Here's the complete JSON rendering of the bencoded delete response that illustrates this problem:

https://gist.github.com/abalashov/e6132db3e381fa2a4f2520c12454373d

-- Alex

> --
> You received this message because you are subscribed to the Google Groups "Sipwise rtpengine" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rtpengine+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/rtpengine/810c816a-e731-420b-9f28-03748dbb2699%40sipwise.com.

[Richard Fuchs]

On 25/06/2026 10.27, 'Alex Balashov' via Sipwise rtpengine wrote:

Thank you for looking into this and for your thoughts. The answer seems to be both: relatively large number of SSRCs and huge MOS blocks.

Here's the complete JSON rendering of the bencoded delete response that illustrates this problem:

https://gist.github.com/abalashov/e6132db3e381fa2a4f2520c12454373d

The most obvious angle of attack would be to eliminate the duplication between the per-media statistics and the global "SSRC" hash/list. That should reduce the size considerably.

But this will have to be an option, as some users may depend on the statistics being in one place or the other. Kamailio for example still requires the global list. With the default being to retain existing behaviour, this can then be backported to LTS branches.

There are more improvements that can be made here, but this should fix the immediate problem.

I'll also add a new flag to completely skip returning of the stats in case they're not needed.

Cheers

[Alex Balashov]

Thank you very much for that! That said, we are using this from Kamailio. Are you saying that we might not be able to use the full suppression option because the Kamailio RTPEngine control module wants the stats anyway, even if zeroed out?

—

Sent from mobile, apologies for brevity and errors.

On Jun 25, 2026, at 11:54 AM, 'Richard Fuchs' via Sipwise rtpengine <rtpe...@googlegroups.com> wrote:



--

You received this message because you are subscribed to the Google Groups "Sipwise rtpengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rtpengine+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/rtpengine/f8782c8e-1297-4bbe-9f85-e7c0b0efcc86%40sipwise.com.

[Richard Fuchs]

On 25/06/2026 12.29, 'Alex Balashov' via Sipwise rtpengine wrote:
> Thank you very much for that! That said, we are using this from
> Kamailio. Are you saying that we might not be able to use the full
> suppression option because the Kamailio RTPEngine control module wants
> the stats anyway, even if zeroed out?

No, what I meant was that the default behaviour cannot be changed from
the current one (especially not in a backport) because it may break
existing use cases.

If you don't need the statistics in Kamailio then it's fine to turn off
reporting with a new option.

Cheers