AudioTrack is not suitable for real time voice communication

[pps]

We use AudioTrack for real time voice call and one of the major complaints from one of the clients is that the delay.
I wrote test to check what the source of the delay is and it looks that in best case scenario AudioTrack added 180ms of delay, up to 380ms of delay in latest android-4.2.1  (on galaxy nexus in particular).

Can anybody recommend what's the best possible way to do it? I wouldn't mind to access raw device if it was possible and if I could reduce that insane buffer introduced by the audiotrack. Is there any other solution other than opensl for this problem? What kind of delay I could expect from opensl (that is, if I write 100ms of audio to waveout, in how much time I should he able to hear it from the speakers?)

If I go with OpenSL, can I also access the same functionality that I can using java api? Can I also access all the goodies that some phones provide (built-in aec, agc etc). In terms of functionality, how opensl compares to java api? Before investing time in opensl I wanted to ensure that I won't waste time on something useless.

Thank you.

[VectorRalf]

To test OpenSL vs AudioTrack latency, I'd recommend using FMOD.  It's a clean/efficient native API which supports both AudioTrack and OpenSL (configurable during initialization).

[Doug]

Your customer is complaining about 180ms delay in voice communications?  That's just 1/5 of a second.  That is hardly a perceivable amount of time during a real time voice conversation.  I would expect that the network connection would introduce far more latency than whatever an AudioTrack might introduce.

If you really think AudioTrack is the problem, you'll need to say something about how you are configuring it.  I imagine you will get lower latency if you configure it with the smallest possible buffer size and write into it using small data buffers as well.

Doug

On Thursday, February 7, 2013 8:57:54 PM UTC-8, pps wrote:

[pps]

The customer (a cell phone carrier) has special testing procedures, where calls with delays of 400+ get unacceptable delay rating.
We have no such issues with iOS, so I wanted to verify with a test what the delay is on Android.

To my surprise, it performed way worse than I expected. Best result I got on a SONY phone: 150ms. Samsung Galaxy S2 for example is 300ms!!
OpenSL (contrary to all suggestions) produces exceptionally bad results! Almost in all cases it gave me slower results that AudioTrack+JNI! OpenSL on Samsung GS2 produced good result of "only" 150ms.

For testing I used audio_device from webrtc that I built with and without opensl. In my test I playout some sound and record it at the same time through the microphone. By analyzing the recorded data I can see at what delay I get the sound back.
I'll run the same test on iOS or Windows to check what delay I get there.

[pps]

Basically, transmission+jitter buffer can be assumed as constant, e.g. I cannot reduce them. With 200-300 delay on waveout it's pretty darn impossible to do low delay audio call. Personally I find a call with 500ms delay is perfectly OK, but some people happen to disagree with my opinion. Real CS call for example has almost no delay compared to voip call using android api.

[pps]

I ran a few tests on different phones. I found one phone that does "exceptionally" low delay audio with regular JNI AudioTrack. One particular version of Samsung Galaxy S3 (ICS 4.0.4) I'm getting 40-60ms total delay (speaker+mic). Which the value I'm hopping to achieve. OpenSL on this phone produces 100ms delay.

Another Samsung Galaxy S3 but with different firmware (ICS 4.1.2) suddenly became "fixed" and gives 200ms delay (I cannot say for sure that hardware is the same between these two SGS3 phones, since they came from different parts of the planet). OpenSL on this device didn't work at all (I didn't get callbacks to write to waveout at all).

On Friday, February 8, 2013 2:32:58 PM UTC-5, Doug wrote:

[jeff shanab]

I do not know much about these libraries, but a thought occured to me. Do you control packet size?
Maybe this is an issue of configuration.
The delay is related to the packet size, which is why on ip phones it is so darn small.
Makes sense really, if a packet represents 100ms you cannot start play until you recieve it.
(if you disagree, you are actually using a smaller packet :-) A buffer is a collection of packets.)

Weird truths. So a higher fidelity stream, may have each packet represent a smaller time frame and play better than a low fidelity one. So some adjustments are not always intuitive.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at http://groups.google.com/group/android-ndk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[pps]

No, in my test I write 1 second of test audio in one shot, that is, it starts to play it right away and doesn't need to wait for entire 100ms chunk to be filled.