Low latency audio - the real problem

[Louise Cypher]

Hi

I'm facing the following problem in my engine (I target Android 2.2
and up - rather high end devices with OpenGL ES 2.0):

On android 2.2 I use AudioTrack (in streaming mode) and my own
software mixe to do audio processing - lantency of such solution is
horrible (Minimal buffer size returned by audiotrack is ~60 - 80ms
(depending on device) but the real latency
measured from the time the data is passed to the buffer to the time
that i hear the sound in headphones is about 400ms - horrible :/)

IF i use AudioTrack in 'static' mode the latency is indeed around
~100ms with is acceptable ... but there is another problem - i cannot
'reload' static stream content on certain devices (around 50 - 60% of
devices with android 2.2 have this problem) -> I hit the bug described
here http://code.google.com/p/android/issues/detail?id=3197
my sequence of calls is [i use all cals thru JNI - exception handlig
stripped for clarity of code]:

// the AudioTrack for 'low latency' audio channels is
created as 22050Hz, 16bit mono pcm with constant buffer size (128KB) -
they are use only to play short sounds (shots, clicks, etc)
// there is 8 such a streams and sound engine pools the
one that is stopped and tries to reload its content
int PState = sAudioTracksPool[i]->getPlayState();
if (PState == JAudioTrack::PLAYSTATE_PAUSED || PState ==
JAudioTrack::PLAYSTATE_PLAYING)
sAudioTracksPool[i]->stop();
sAudioTracksPool[i]->flush();
sAudioTracksPool[i]->write(...);
sAudioTracksPool[i]->reloadStaticData();
sAudioTracksPool[i]->setLoopPoints(0,
LOW_LATENCY_SOUND_MAX_SIZE / sizeof(int16_t), 0);
sAudioTracksPool[i]->setPlaybackHeadPosition(0);
sAudioTracksPool[i]->play();

on some devices the bug manifest itself in different way -- the OLD
content of audio track is played instead of the new one (but the write
and reloadStaticData operations did not return any error codes or
exceptions :/)

the partial solution is to destroy and create audio tracks each time i
want to play low latency sound - but this again is bugged - since the
shared memory used in audiotrack backed gets fragmented very quicly
and the whole dalvik crashes (on some devices - se my earier post)
or after few minutes i'm unable to create new audio tracks (i never
have more than 8 at the same time) - i'v tried to allocate them with
constant buffer size - but this is not a solution - after few minutes
there is exception in 'play' and no more audiotracks can be created
until the activity ends.

ANY ideas to try on android 2.2 ???

Now on android 2.3 and up I use OpenSLES -- on some devices (around
80ms on motorola xoom for example) the latency is realy low (in
standard of android devices ofcourse - i only could dream about 5 - 8
ms latency from iPhone/iPad)
but on some devices the situation is horrible (on kindle fire for
example the latency is around 500ms -- the buffer size returned by
OpenSLES is THAT SAME on xoom and on kindle, but on kindle the whole
age passes from the submission of buffer to hearing the sound - and no
the system is not overloaded - i'm just playing one PCM buffer from
memory for test - so no file acces, no other stuff in my activity --
just ->play and ->sleep :))

ANY ideas to try on android 2.3 and up in the field of low latency audio ?

thanks for any responses and ideas to try out

How the comercial sound engines (like fmod) are dealing with audio
latency on android - any ideas ?

the 'static audiotrack' solution would be god - (i'v tried it on 2.3
too) - but it suffers from the content reload bug
(http://code.google.com/p/android/issues/detail?id=3197) - and i dont
see other way to reload its content :(

--
Code it black!

[fei Wang]

Louise:

   Would you please let me know how do you test the latency on android device, I am doing apps which need real time play back the recording, I tried the OpenSL ES, the latency is very different across different devices. Xoom and Galaxy perform very well (I don't know how to test, just based what I heard from speaker), but some low end android phone is really bad. 

  I will try static AudioTrack to see what is the result. 

Fei

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.

[Louise Cypher]

well - i have headphones out connected to sound card line in (i use
semi-pro maya44 sound card witch have known latency)
i'v wrote some simple program on pc that just reads maya44 line in
into file and observes left mouse button press
when the left mouse button gets pressed the program starts counter and
watches for a peak wave on the input, if the peak approaches the
counter stops.

the trickest part is to get low latency mouse -- the best is old wired
ps/2 mouse not the crappy modern wireless shit :)
and you could just attach some conducting rubber to the mouse button,
and use that to press button on the device screen (this way the mouse
trigger will be pressed at ~ that same time ;)) - not the ideal
solution but
works -- of course you could spend ~150$ to buy professional latency
meter if you care about every millisecond - but i dont care

follow the thread
http://music.columbia.edu/pipermail/andraudio/2011-January/000018.html
there is a table with measured audio latency for some devices there

xoom, galaxy tab and all other tegra2 devices are the best - their
latency is acceptable and very low in android standards (real latency
around 70 - 80ms)

but for other devices (especially HummingBird and SnapDragon SOCS) the
latency is very bad

--
--
Code it black!