OpenSLES recorder sample rate randomly changing

[Andres Traumann]

Hello!

I am experiencing a weird issue with OpenSLES. I am developing an Android app that is doing audio playback and recording, everything was working fine, until the threading model changed slightly: previously the audio thread (openSLES init and buffer filling) was spawned from Java code, but now a new pthread is created inside NDK code. After that change the recorder sample rate seems to be changing by itself somehow, sometimes everything works correctly, sometimes the record sample rate becomes two times higher according to the frequency of record callbacks.

In the record and playback callbacks I am doing nothing besides modifying atomic variables to keep track of the number of enqueued buffers. I am using the native sample rate (PROPERTY_OUTPUT_SAMPLE_RATE) for both record and playback, and a non-native buffer size 4096, tried native buffer size also, but the issue remained.

After some debugging I found that it seems to be connected to the fact that in some cases the playback system gets a few seconds of zero buffers i.e. playing silence. Is it something that would break the record system?

Can anyone explain that issue?

[Andres Traumann]

After some more debugging I would like to clarify that the frame rate also changes independantly of playback also. After I turned off the playback entirely, there seems to be 10% chance that the record sample rate becomes wrong. It seems that this is determined at the beginning of the application, does not seem to change later on. Nevertheless, I don't see any errors from OpenSLES init code, I check for errors at every step in init.

[Andres Traumann]

One more day of debugging suggests that it might be hardware specific somehow. I tried the code on 3 more Android phones and it worked flawlessly on each one of them. The only phone where I have have the bug, and have it very often, is Galaxy S6 running Android 6.0.1. Should not be Android version specific, as some of the phones also had 6.0.1 and it worked.

I really would not like to blame hardware, so can anyone suggest something I might be doing wrong to expercience such issues. I compared my code to the example code in the Android repo under frameworks/wilhelm/tests/examples/slesTestRecBuffQueue.cpp and it really seems that I am initing things in the correct way.

On Monday, October 24, 2016 at 5:49:47 PM UTC+3, Andres Traumann wrote:

[Lev Levchenko]

Try stick to 44100 recording sample rate on this device.

[Don Turner]

How are you detecting that the record sample rate is changing? 

[Andres Traumann]

How are you detecting that the record sample rate is changing?

I am detecting sample rate frequency by simply counting how many bufferqueue callbacks I get in one second.

Try stick to 44100 recording sample rate on this device.

Interesting, it seems the problem disappears after changing to 44100. What is the logic behind that? How did you know that? Should I expect similar problems with other devices? I was previously using 48000 that was given by PROPERTY_OUTPUT_SAMPLE_RATE. Also, I get the following warning with other sample rates:
W/AudioRecord: AUDIO_INPUT_FLAG_FAST denied by client; transfer 1, track 44100 Hz, primary 48000 Hz

[Andres Traumann]

Bump... This is an important topic for me...

On Monday, October 24, 2016 at 5:49:47 PM UTC+3, Andres Traumann wrote:

[Glenn Kasten]

I have a question about where the buffers are filled: in the first post, you mentioned

"everything was working fine, until the threading model changed slightly: previously the audio thread (openSLES init and buffer filling) was spawned from Java code, but now a new pthread is created inside NDK code."

but later in the thread you also mentioned using callbacks.

I'm curious: when you say "buffer filling", what do you mean?
Usually the app should Enqueue empty buffers for OpenSL ES recorder to fill;

the app should not be filling those buffers itself.

Also, what operations do you do inside of the record buffer completion callback?

normally the steps should be: copy data out of most recently completed buffer

[you will need to track which buffer that is] to your private area,

then enqueue an empty buffer for recorder to use next.

One last thing, avoid doing I/O or logging or acquiring a mutex inside the callback.

Please post here the exact sequence, and if I spot something fishy I'll let you know.

[Andres Traumann]

Init
1. OpenSLES is initialised.
2. I spawn a processing thread.
3. At some point later, when a user clicks "record", I allocate memory for two buffers of same size. I Enqueue both buffers and call SetRecordState() to change the state to SL_RECORDSTATE_RECORDING.

OpenSLES Callback
The ONLY thing my registered callback function does is set an std::atomic_bool variable to true, this means that a new buffer has been filled and my processing thread can copy its contents.

Processing Thread:
Checks the std::atomic_bool variable and whenever it changes to true, the contents of the finished buffer are copied somewhere else, the buffer itself is Enqueued again. (I have tried moving the Enqueue operation to OpenSLES callback also, does not seem to make a difference)

On Monday, October 24, 2016 at 5:49:47 PM UTC+3, Andres Traumann wrote:

[Glenn Kasten]

I see one possible race condition:  since you are using 2 buffers, but your buffer completion callback function simply sets the bool to true,

then if 2 buffers are completed (filled) in rapid succession, it is possible that your bool will be set to true twice in a row.

Thus you may lose track of one of the buffers.

For this and other reasons, I usually do the next Enqueue within the buffer completion callback handler.
I'm not saying that's the only cause here, but it does seem suspicious to me.

[and you mention you have already tried this but it did not work]

Another thing to try doing is to get a log of both the buffer completion callbacks and Enqueues,

and look for timing errors in that log.

[Andres Traumann]

Sadly, I have tried these things already. Logs show everything is perfectly in sync. I am still interested to know if Lev Levcehnko had some clever reason to believe 44100 sample rate would work (it did seem to fix the problem). Are some sample rates better than others, despite being reported as PROPERTY_OUTPUT_SAMPLE_RATE?

I recently found that OpenSLES is built on top of AudioFlinger, so it seems likely that the problem lies there. I haven't started reading the source yet, but it seems a much thicker layer of code.

On Monday, October 24, 2016 at 5:49:47 PM UTC+3, Andres Traumann wrote:

[Lev Levchenko]

I have same problem on this device and found solution experimentally. It seems just a bug.