Problems with AudioTrack and AudioRecord interfaces

[Jonathan]

Hi All,

I have some problems using the Audiotrack and AudioRecord APIs in
native land under Android. I am running the 0xlabs (http://0xlab.org/)
port of Android 1.5 on a BeagleBoard. My goal is to integrate a third
party communications stack within the Android framework. This stack
requires low latency full duplex audio and my plan is to integrate it
with the Audio flinger.

I have two basic issues. The first one relates to using the AudioTrack
and AudioRecord API with callbacks. I have written three test
applications using these interfaces. One records, one plays out and
one performs a loopback using both interfaces. The first two apps work
fine in isolation, but if both are run together one of them will stop
working properly. For example if I do the following in separate
shells:

1. Start audio playback application
2. Start audio record application

Once the audio record app is running the audio playback app starts
playing out at a much slower rate. It sounds like an 8kHz file is
being played at about 2kHz. This behaviour is also seen in the audio
loopback app that uses both APIs, the playout callback starts working
normally, but after 20 or so frames starts reducing the amount of data
it asks for, until it is asking for 2 frames at a time. Again the
audio output sounds like it is being played back at significantly
slower than it should. Is this likely to be an Android problem or a
problem with the underlying port?

My second issue is with the Audiotrack.write() API. My understanding
is that if I can call this as many times as I like with audio data and
it will eventually block when the internal buffers are full. My
experiments have shown somewhat different behaviour where each write
allways suceeds but the playback of the audio seems to restart at the
point each write is called. What this means is that if I was writing
out 10 second blocks of audio, I hear the first few seconds of the
first block, and it then seems to jump immediatly to the start of the
second 10 second block almost immediatly Audiotrack.write() is called
for the second time. Is this how it is supposed to behave?

I would appreciate any feedback regarding these issues particularly
how to debug further. If anybody has any suggestions of alternative
methods of getting a low latency full duplex audio stream into a
native library I would be very interested :).

Many thanks in advance.

Jonathan

[Bytes]

Hi

Yes I too join in the club of problems....

Here are the my observations with AudioTrack/AudioRecord - loopback
functionality.

My settings:

AudioRecord:

AudioRecord::MIC_INPUT,
8000, /* SampleRate */
AudioSystem::PCM_16_BIT, /* Format */
1, /* channel count */
10000, /* frameCount - Hopefully I'll avoid freequent overflow
problems */
0, /* Flags */
rec_audioCallback, /* My callback */
NULL,
0); /* Notification frames By default */

AudioTrack:
AudioSystem::VOICE_CALL, /* streamType: */
8000,
AudioSystem::PCM_16_BIT,
1,/* channelCount: */
0, /* Frame Count */
0, /* flags */
play_audioCallback,
NULL,
512);/* Notification frames */


1. Callbacks are very in-consistent.

Record Callback:
For a notification frames of 128, see the time pattern:

Rec Callback 1251356443:329956 Event 0 Size: 256
Rec Callback 1251356443:330810 Event 0 Size: 256
Rec Callback 1251356443:331024 Event 0 Size: 256
Rec Callback 1251356443:331146 Event 0 Size: 256

But ideally, for 8KHz, (8000 frames/sec ), to record 128 frames, it
takes 16 msec,
but I'm seeing callback is called for every 1 milli second.


PlayCallback:
No matter what ever the notification frame ..... as you mentioned,
it is asking for random number of bytes.......
Play Callb 1251356770:397705 Event :0 size:168
Play Callb 1251356770:397796 Event :0 size:266
Play Callb 1251356770:397857 Event :0 size:434
Play Callb 1251356770:397949 Event :0 size:170
Play Callb 1251356770:398040 Event :0 size:264
Play Callb 1251356770:398132 Event :0 size:434
Play Callb 1251356770:399078 Event :0 size:172

Simply callbacks are flooding.

RESULT: Only distrorted voice is heard.

2. Approach II:
Do enable only record callback, and "write" what ever the data
we received immediately to the player using AudioTrack.....

RESULT: Quality is GOOD and clear but as time progress LATENCY is
increasing and NOT ACCEPTABLE.
Observed AudioFlinger( ): AudioRecordThread Overflow.. no
matter what you set the buffer value (Frame Count).


Answers to your issues:
========================
Issue 1: Yes, I do have problem.

Issue 2: I'm referring code verison 1.1 .. look at "write" API in
frameworks/base/media/libmedia/AudioTrack.cpp

*********************
// Calling obtainBuffer() with a negative wait count causes
// an (almost) infinite wait time.
status_t err = obtainBuffer(&audioBuffer, -1);
if (err < 0) {
// out of buffers, return #bytes written
if (err == status_t(NO_MORE_BUFFERS))
break;
return ssize_t(err);
}

size_t toWrite;
if (mFormat == AudioSystem::PCM_8_BIT) {
// Divide capacity by 2 to take expansion into account
toWrite = audioBuffer.size>>1;
// 8 to 16 bit conversion
int count = toWrite;
int16_t *dst = (int16_t *)(audioBuffer.i8);
while(count--) {
*dst++ = (int16_t)(*src++^0x80) << 8;
}
}else {
toWrite = audioBuffer.size;
memcpy(audioBuffer.i8, src, toWrite);
src += toWrite;
}
userSize -= toWrite;
written += toWrite;

releaseBuffer(&audioBuffer);

**********************

If I understood properly, it simply overwriting even after getting the
error 'NO_MORE_BUFFERS'.
Hope this will answer your second issue.

My questions:

1. I'm using Dev 1 Phone, and recently updated to 1.5.
Where do I get the source code for this, I mean I heard so many
version of the code like
*main branch* , *cupcake* , *donut* etc.,
Could any body suggest URL for source code.

2. Does any body successful in using AudioTrack/AudioRecord Java
version ????

3. What is the menaing of Lateny returned by AudioRecord.latency()...
is it initial latency ????
If I increase frame size, its value also increasing.

3. Instead of integrating with AudioFlinger, can I write my own audio
driver and communicate using native code ??? If so, please let me
know the procedure.

4. What is the best way to deal with audio issues to develop a real
time audio application ???

5. If I'm not too greedy, can we have a documentation about these call
flows...

Let me post some more observations/solution as my work progress....

Thanks for your time....

Any pointers really helpful and appreciated.








It seems to be most of the time record callback is called once it has
2048 bytes ready i.e., 1024 frames... no matter how much value we set
for notification frames.
If you set it to 512 frames, its called twice successively..