SL_IID_ANDROIDCONFIGURATION and Fast Audio Path

[Andrés Pérez]

Hi there,

I am trying to configure the audio path to be reproduced through the ear speaker using OpenSL.

In that way, I succeeded by using the approach described in http://stackoverflow.com/questions/21576739/how-to-configure-opensl-to-work-with-ear-speaker-other-than-main-speaker-or-head/21719421#21719421

i.e., by configuring SL_IID_ANDROIDCONFIGURATION to use SL_ANDROID_STREAM_VOICE.

The idea is to capture audio from the mic, and reproduce it through the ear piece. Therefore, I am starting from the audio-echo project.

However, in audio_player.cpp, lines 91-94, there is the following comment:

/*

 * create fast path audio player: SL_IID_BUFFERQUEUE and SL_IID_VOLUME interfaces ok,

 * NO others!

 */

I have made minor changes in audio_player.cpp to force the audio path through ear speaker (lines 94-98 changed for the following snippet)

/*************/

 SLInterfaceID  ids[3] = { SL_IID_BUFFERQUEUE, SL_IID_VOLUME, SL_IID_ANDROIDCONFIGURATION};

 SLboolean      req[3] = {SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE,SL_BOOLEAN_TRUE};

 result = (*slEngine)->CreateAudioPlayer(slEngine, &playerObjectItf_, &audioSrc, &audioSnk,(ids)/sizeof(ids[0]), ids, req);

 SLASSERT(result);

 // Force the use of the earspeaker

 SLAndroidConfigurationItf playerConfig;

 result = (*playerObjectItf_)->GetInterface(playerObjectItf_, SL_IID_ANDROIDCONFIGURATION, (void*)&playerConfig);

 SLASSERT(result);

 SLint32 streamType = SL_ANDROID_STREAM_VOICE;

 result = (*playerConfig)->SetConfiguration(playerConfig, SL_ANDROID_KEY_STREAM_TYPE,streamType, sizeof(SLint32));

 SLASSERT(result);

 /*************/

However, with a LGE Nexus 4 API 22, I can check through dumpsys media.audio_flinger that audio-echo is running on a Fast Track, which IMHO contradices the aforementioned comment.

Snippet of dumpsys media.audio_flinger (audioecho running with pid 4015)

// Output thread

...

4 Tracks of which 1 are active

  Name Active Client Type      Fmt Chn mask Session fCount S F SRate  L dB  R dB    Server Main buf  Aux Buf Flags UndFrmCnt

  F  5     no    701    1 00000001 00000003      36   9065 S 1 48000  -inf  -inf  00002369 0xb8caf900 0x0 0x600         0

  F  3    yes   4015    0 00000001 00000001      51    480 A 1 48000     0     0  00063F60 0xb8caf900 0x0 0x400         0

  F  1     no    547    1 00000001 00000003      49   4512 S 1 48000  -inf  -inf  000011A0 0xb8caf900 0x0 0x600         0

  F  2     no    547    1 00000001 00000003      52   4512 S 1 48000  -inf  -inf  000011A0 0xb8caf900 0x0 0x600         0

0 Effect Chains

// Input Thread

Buffer size: 480 bytes

Fast capture thread: yes

Fast track available: no

...

1 Tracks of which 1 are active

  Active Client Fmt Chn mask Session S   Server fCount SRate

     yes   4015   1 00000010      53 6 00064AA0   2048 48000

0 Effect Chains

Same result achieved with a Motorola Nexus 6 API 23.

Therefore, my first question is about the restrictions to the Fast Mixer Thread when using SL_IID_ANDROIDCONFIGURATION (and potentially other Player Interfaces)

I have not found any documentation about the question.

On the other side, when running the modified audio-echo with a rooted Samsung Galaxy S5 API 21, the OpenSL callback mechanism silently fails to stream audio.

I have added logs just after bqPlayerCallback and bqRecorderCallback, to check the callbacks, and the result is following (buffer queue parameters are same as default):

01-15 01:07:05.805 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.825 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.845 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.855 9722-11419/com.google.sample.echo D/AUDIO-ECHO: PLAYER CALLBACK

01-15 01:07:05.855 9722-11419/com.google.sample.echo D/AUDIO-ECHO: PLAYER CALLBACK

01-15 01:07:05.855 9722-11419/com.google.sample.echo D/AUDIO-ECHO: PLAYER CALLBACK

01-15 01:07:05.865 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.885 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.905 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.925 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.945 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.975 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:05.985 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.005 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.025 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.045 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.065 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.085 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.105 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.125 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.175 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

01-15 01:07:06.175 9722-11421/com.google.sample.echo D/AUDIO-ECHO: RECORDER CALLBACK

My interpretation is: all goes fine until first player callback. In that moment, player callback is called 3 times as expected, but all callbacks happen in the same instant.

After that, 16 successful recorder callbacks, and then there are no more buffers in FreeQueue and audio processing stops.

This situation maybe has something todo with the absence of past path...

Snippet of dumpsys media.audio_flinger (audioecho running with pid 6326):

// Output thread

...

2 Tracks of which 0 are active

  Name Active Client Type      Fmt Chn mask Session fCount S F SRate  L dB  R dB    Server Main buf  Aux Buf Flags UndFrmCnt

     1     no   1218    5 1 00000003   12619   3528 S 1 44100     0     0  00007A56 0xb214a000 0x0 0x400      1768

     0     no   6326    0 1 00000001   12718   3840 S 1 48000     0     0  00000000 0xb214a000 0x0 0x200         0

0 Effect Chains

// Input thread

  No active record clients

  Fast capture thread: no

  Fast track available: no

  ...

  1 Tracks of which 0 are active

    Active Client Fmt Chn mask Session S   Server fCount SRate

        no   6326 1 00000010   12720 7 00004B00   2880 48000

  0 Effect Chains

The same device, when running without (*playerConfig)->SetConfiguration(playerConfig, SL_ANDROID_KEY_STREAM_TYPE,streamType, sizeof(SLint32))

runs ok, with Fast Audio Path, and the first player callbacks are not instantaneous as in the previous case.

Is it a normal behavior to expect, or it has to do with the "special" way of Samsung to deal with things??

Thank you very much.

Best,

Andrés Pérez

[Sylvain Berfini]

Hi Andrés,

Since that time, have you found an answer to this problem ? I have the same one...

Thanks.