Can WebRTC be configured to ensure that audio is only played if received in a 'timely' fashion?

[Jonathan H]

I am exploring the possibility of using WebRTC Native (C++) for the transmission of audio for a music application. I am looking to understand whether it is an appropriate technology for this purpose. My understanding is that Opus supports fullband stereo audio but I also need to ensure that the audio is only played on the receiving end if the audio packets are received in a 'timely' fashion, i.e. if the packets don't arrive on time, perhaps FEC can be used, but if not then no audio is better than delaying the output. Delaying the output could result in a gap in audio and change to the rhythm or beat. Is it possible to configure WebRTC to account for this requirement?

Any help here, or alternative suggestions, would be greatly appreciated. 

Thanks,

Jon

[Jonathan H]

I think I should have said packet loss concealment (PLC) here rather than FEC.

Can anyone shed any light on this, or point to any appropriate resources? 

Thanks,

Jon

[James Inkster]

Hi Jon,

I just yesterday posted to the group asking what seems to be the identical question...

Did you ever figure anything out?

Thanks!

[Harald Alvestrand]

I think you need to limit the size of the jitter buffer to get the effect you want.

If you want even tighter control, you could also insert a step in your processing chain using encoded insertable streams or breakout box that drops packets you don't want decoded - doing so is still experimental, but might work.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/d5dfc496-636f-43a3-8c78-71d6e9e519aan%40googlegroups.com.

[guest271314]

> Delaying the output could result in a gap in audio and change to the rhythm or beat.

Have you experience that result thus far in experiments?

Technically individual audio data can be encoded as JSON and streaming consecutively, or as suggested, "Breakout Box" can be used. I have not observed audio latency to the degree that gaps and glitches occur using WebTRC Insertable Streams https://plnkr.co/edit/1HsvQh08tYb24810?preview.

[James Inkster]

After Harald's comment about jitter buffers, I noticed 

PeerConnectionInterface::RTCConfiguration's bool:

// Whether to use the NetEq "fast mode" which will accelerate audio quicker
// if it falls behind.
bool audio_jitter_buffer_fast_accelerate = false;

Interestingly, I'd say this is exactly what I'm hearing. BUT -- I have this bool set to false... 

[Henrik Lundin]

Fast mode is not the culprit. It may exaggerate the unwanted effect you hear, but it will be there also without fast mode.

The fundamental problem is that with real-time communication, you will sometimes have to adjust the timing of the playout. Even if you never lost any packets, and had a perfect network for transport, you would still suffer clock drift between sender and receiver. This will have to be compensated by sometimes bending the playout time a bit, speed- or slow-playing.

There is a setting in NetEq::Config called for_test_no_time_stretching. The clue is in the name,  "for_test" only. Here be dragons. If you build your own C++ application, you should be able to flip this on, but don't rely on it, don't expect it will work or even be there in the future, and don't expect it will ever be promoted to anything but a test config. And it may result in other fun things happening to your audio.

Good luck!

/Henrik

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/8ee583fe-0f97-439a-9ef5-24231dd414b4n%40googlegroups.com.

[James Inkster]

Hi,

Yes, i'm familiar with drift, what I'm hearing is not directly caused by drift as it's quite a bit more exaggerated (and only happens in conjunction with dropped packets)... With a particularly poor connection, I'm hearing speed/pitch changes for a matter of seconds. 

Anyway, I was hoping the fast mode might be contributing but apparently not -- thanks for the headsup, I won't waste any more time there!

Thanks!