WebRTC - Delay increase in audio calls

[Henrique Rosa]

Hello,

I've been working on WebRTC support for Mobicents Media Server (MMS).
I already succeeded at interop between MMS and Firefox/Chrome.

Unfortunately, while testing conference calls I noticed that there is a tendency for RTT times and Delay to increase as the call goes on (on both browsers).

On Chrome, the delay ranges from 3 to 10 seconds for a conference call between two participants with duration of 3-4 minutes.
On Firefox the results are much better: the delay ranges from 1 to 3 seconds in the same scenario.
Note that while testing with regular SIP clients (jitsi, linphone, xlite) there is no delay whatsoever, so the issue is related with WebRTC calls.

My question is: what can possibly cause such delay?

Any help is greatly appreciated.

Thanks in advance,
- H

[Nils Ohlmeier]

Hi,

The one obvious difference between WebRTC and "regular SIP clients" is
the additional media encryption. But should not take seconds, at least
if you are using modern desktop PC.

The other question is how much delay FF and Chrome add between capturing
audio and actually sending them out on the wire. And on the other end
how delay there is between receiving audio packets and rendering to your
audio device.

Best regards
Nils Ohlmeier

[Henrique Rosa]

Hi Nils,

Thank you for taking you time, really appreciate it.

The media encryption is not the issue, I debugged that and its fast.

I'm sending some logs from the test call I did on Firefox v32.
Firefox will make a call to an IVR application that plays announcements for 40 seconds.
In the meantime, I had the Media Server tracing the RTCP reports and RTT calculations.
I remind you it's a local call, so network congestion shouldn't be a problem either.

- Media Server log: https://dl.dropboxusercontent.com/u/47489176/webrtc-mux-ffox32/mms-log.txt
- Firefox webrtc log: https://dl.dropboxusercontent.com/u/47489176/webrtc-mux-ffox32/about-webrtc.txt
- Wireshark dump: https://dl.dropboxusercontent.com/u/47489176/webrtc-mux-ffox32/mms-ffox32.pcapng

Performing same test with SIP clients has no delay and RTT values range from 5 to 25ms approx.
Reading the Media Server log, you can see the RTT growth rate is as follows:

16:35:25,169 INFO [RtpMember] rtt=3242011459 - 3241960996 - 47825 = 2638 => 40ms
16:35:29,570 INFO [RtpMember] rtt=3242299883 - 3242208002 - 87674 = 4207 => 64ms (+24ms)
16:35:36,989 INFO [RtpMember] rtt=3242786095 - 3242723311 - 57543 = 5241 => 79ms (+15ms)
16:35:42,508 INFO [RtpMember] rtt=3243147788 - 3242997252 - 144437 = 6099 => 93ms (+14ms)
16:35:46,891 INFO [RtpMember] rtt=3243435032 - 3243274862 - 152405 = 7765 => 118ms (+25ms)
16:35:52,248 INFO [RtpMember] rtt=3243786108 - 3243546968 - 229851 = 9289 => 141ms (+23ms)
16:35:55,109 INFO [RtpMember] rtt=3243973607 - 3243796725 - 166610 = 10272 => 156ms (+15ms)
16:36:01,929 INFO [RtpMember] rtt=3244420562 - 3244165365 - 243715 = 11482 => 175ms (+19ms)

If you want to compare results with a similar test I did on Chrome Canary v40, you can refer to the following thread:

https://groups.google.com/forum/#!topic/discuss-webrtc/DmZ2eYKGPWA

Lately I've been reading about RTP usage on WebRTC and came across RTP/SAVPF profile and RTCP XR report. At the moment we only support RTP/SAVP and regular RTCP reports. Do you think this can have an impact on RTT calculations and delay?

Best Regards,
- H

[Randell Jesup]

On 10/16/2014 7:16 AM, Henrique Rosa wrote:
> On Monday, October 13, 2014 11:10:02 PM UTC+1, Nils Ohlmeier wrote:

> Hi Nils,
>
> Thank you for taking you time, really appreciate it.
>
> The media encryption is not the issue, I debugged that and its fast.
>
> I'm sending some logs from the test call I did on Firefox v32.
> Firefox will make a call to an IVR application that plays announcements for 40 seconds.
> In the meantime, I had the Media Server tracing the RTCP reports and RTT calculations.
> I remind you it's a local call, so network congestion shouldn't be a problem either.
>
> - Media Server log: https://dl.dropboxusercontent.com/u/47489176/webrtc-mux-ffox32/mms-log.txt
> - Firefox webrtc log: https://dl.dropboxusercontent.com/u/47489176/webrtc-mux-ffox32/about-webrtc.txt
> - Wireshark dump: https://dl.dropboxusercontent.com/u/47489176/webrtc-mux-ffox32/mms-ffox32.pcapng
>
> Performing same test with SIP clients has no delay and RTT values range from 5 to 25ms approx.
> Reading the Media Server log, you can see the RTT growth rate is as follows:
>
> 16:35:25,169 INFO [RtpMember] rtt=3242011459 - 3241960996 - 47825 = 2638 => 40ms
> 16:35:29,570 INFO [RtpMember] rtt=3242299883 - 3242208002 - 87674 = 4207 => 64ms (+24ms)
> 16:35:36,989 INFO [RtpMember] rtt=3242786095 - 3242723311 - 57543 = 5241 => 79ms (+15ms)
> 16:35:42,508 INFO [RtpMember] rtt=3243147788 - 3242997252 - 144437 = 6099 => 93ms (+14ms)
> 16:35:46,891 INFO [RtpMember] rtt=3243435032 - 3243274862 - 152405 = 7765 => 118ms (+25ms)
> 16:35:52,248 INFO [RtpMember] rtt=3243786108 - 3243546968 - 229851 = 9289 => 141ms (+23ms)
> 16:35:55,109 INFO [RtpMember] rtt=3243973607 - 3243796725 - 166610 = 10272 => 156ms (+15ms)
> 16:36:01,929 INFO [RtpMember] rtt=3244420562 - 3244165365 - 243715 = 11482 => 175ms (+19ms)
>
> If you want to compare results with a similar test I did on Chrome Canary v40, you can refer to the following thread:
>
> https://groups.google.com/forum/#!topic/discuss-webrtc/DmZ2eYKGPWA
>
> Lately I've been reading about RTP usage on WebRTC and came across RTP/SAVPF profile and RTCP XR report. At the moment we only support RTP/SAVP and regular RTCP reports. Do you think this can have an impact on RTT calculations and delay?

SAVPF is needed for any sort of reasonable error recovery for video, and
bandwidth adaptation (right now we don't adapt audio (Opus), but likely
will at some point). We aren't using XR reports yet.

However: your SDP answer appears to have SAVPF for audio. It so happens
that it's safe to mix SAVP and SAVPF generally (see the AVPF spec) - the
SAVP side just won't send as many RTCPs as it could.

RTT increase normally is a result of being over-bandwidth (if you're
sending video and not returning bandwidth reports, it might remain
over-bandwidth). But that doesn't seem to be happening here.

THe PCAP shows the first ~7 packets in one direction are all right on
top of each other (~2ms in-between them). Wireshark reports this as
~100ms of 'skew' which then remains for the rest of the capture. From
the port numbers, this appears to be the media gateway generating the
burst of packets. Note this burst will cause the Firefox jitter buffer
to grow initially (to around 100ms or a bit more), and it will slowly
bleed the delay out of the jitter buffer. about:webrtc shows a 50ms
jitter buffer, which is quite high for a local-LAN connection, but
matches the wireshark.

You can see this jitter-buffer delay due to the burst in the stats
graphs for Chrome you have as well.

Something is wrong with your RTCP I think, or maybe the SRTCP encryption
messing with the values. Also: if your NTP values are messed up, that
might confuse both receivers (playback) and RTCP calculations.

You can also try blocking RTCP (run open-loop) and see if there's delay.

--
Randell Jesup, Mozilla