low latency (fast path) audio and VoIP

[Jiri Kral]

Hello,

we make VoIP software for Android and I ran into problems with severe audio glitches when testing the app on Google Nexus 4 device. After the analysis, I found that we hit the setting for low-latency audio (fast-path) for that device accidentally (mono, 48KHz sampling rate, 16bit samples).

The call for android.media.property.OUTPUT_FRAMES_PER_BUFFER suggests buffer for 5ms of audio. If I use this buffer size, audio playback is very glitchy - I cannot guarantee that the callback will finish quickly and always take roughly the same time, because of rather complex audio processing (jitter buffer, noise suppression, AGC, echo cancellation etc.) which also needs locking in our multithreaded environment.

So I thought the solution will be easy - I will set up playback buffers which are 5x the OUTPUT_FRAMES_PER_BUFFER. This will even out the dropouts and delay of 25ms is still acceptable for our application.

I was surprised that the quality did not improve at all. It seems that the audio engine silently consumes 9 buffers out of 10 and only then it asks for more data, so I am exactly where I was before - if I take more than a few milliseconds in the callback, there will be dropout.

This would make low-latency audio pretty much unsuitable for our app, we wouldn’t be able to use it without heavy refactorization of our audio code. Before we jump into that - is this really the expected and desired behaviour?

Jiri Kral

[sweisgerber.dev]

I assume you are using OpenSL ES?

Try not inserting 5 separate buffer with 5ms each, try inserting a single 20-25ms buffer everytime the AudioPlayercallback calls.

Also start making timing/performance measurements.
The calback must finish insertion of new frames very exactly. This means a logfunction e.g. should log very periodic every 23-27ms.
Perhaps you have some locks in your application, that lets the callback wait for some other functions before it can insert a new frame.

[Halsafar]

The OpenSL audio callback should run both in constant time and without locking.  The OpenSL audio thread is often high priority but if you are locked waiting on a lower priority thread then you have just introduced priority inversion.  

So the OpenSL audio callback thread should basically just enqueue some already available samples and nothing more, no locks, use a single-reader single-writer ring buffer.

[Taesoon Park]

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at http://groups.google.com/group/android-ndk.
For more options, visit https://groups.google.com/d/optout.

[pps]

I'm working on the same stuff. All I can recommend: forget about opensl, that's the biggest BS in android world regarding low latency audio playback. There is no low latency audio playback in android (other than some options that aren't applicable to voip). I even wrote special test app that measures loopback delay: I write some frequency sound to loudspeaker and measure time before I can see the wave on the microphone input. Delays are around 200ms for 90% of the android phones on the market (I tested pretty much every popular android phone). For most of the phones, contrary to the popular BS suggestion, open sl measured slower by a tiny bit. The fastest audio was on sony experia. I didn't test latest models (in last half a year), cause it's all waste of time. There is no fast audio on android (unlike iOS). Period. On top of that opensl is crippled API (just like everything that comes out from khronos), it takes years before things start working properly and you need to reprogram your brain to understand and use that api.

[Joel Anderson]

Currently the company I am working for is building a tablet.  I need a very low latency Audio chip to recommend. My Z1 tablet is pretty fast and the sony xperia is apparently fast.  Does anyone know which audio chip is responsible for this improvement?

From these articles I gather it's the WCD9310 chip that is responsible.

http://www.chipworks.com/en/technical-competitive-analysis/resources/blog/inside-the-sony-xperia-tablet-z/

http://www.chipworks.com/en/technical-competitive-analysis/resources/blog/inside-the-xiaomi-mi-2-familiar-on-the-outside-different-on-the-inside/

It's still not really fast enough.  Is there anything better out there?

Cheers

[mike digioia]

ok submit me thanks

--