Decode MP3 using OpenSL NDK

[Klaser]

Hi!

I'm trying to get decoded PCM data from a mp3 file using the newly
released NDK.
I have tried different combinations of AudioPlayer and AudioRecorder
using audio source and audio sink like this:

// configure audio source

SLDataLocator_AndroidFD loc_fd = {SL_DATALOCATOR_ANDROIDFD, fd,
start, length};

SLDataFormat_MIME format_mime = {SL_DATAFORMAT_MIME, NULL,
SL_CONTAINERTYPE_UNSPECIFIED};

SLDataSource audioSrc = {&loc_fd, &format_mime};





// configure audio sink

SLDataLocator_AndroidSimpleBufferQueue loc_bq =
{SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE, 2};

SLDataFormat_PCM format_pcm = {SL_DATAFORMAT_PCM, 1,
SL_SAMPLINGRATE_16,

SL_PCMSAMPLEFORMAT_FIXED_16, SL_PCMSAMPLEFORMAT_FIXED_16,

SL_SPEAKER_FRONT_CENTER, SL_BYTEORDER_LITTLEENDIAN};

SLDataSink audioSnk = {&loc_bq, &format_pcm};





However I keep getting errors in the log saying that this combination
is not supported.

Does anyone have any ideas how to get PCM data from a MP3 file using
OpenSL? Maybe it is not possible?



Cheers

[Glenn Kasten]

Though this particular combination of data source and data sink makes
good sense, unfortunately it is not supported as of Android API level
9 (NDK r5). Please see the figure in section "Objects and interfaces"
of docs/opensles/index.html. You'll see in that chart that a buffer
queue is not allowed as a sink for audio player. The Android media
team is aware of this and would like to expose native decode
(including optional hardware-accelerated decode when feasible), but I
can't commit to a date. As a workaround, I suggest using a 3rd-party
software decoder, either proprietary or open source. Note that you
would be responsible for complying with any appropriate licensing
requirements, depending on the codec.

[bob1at7shore]

I am
looking at the file:///opt/android-ndk-r7/docs/opensles/index.html
document.  The "Decode audio to PCM" section seems to indicate that
this is possible in API 14.  I tried it, but cannot get it to work.
My project build target is Android 4.0.

I modified the asset player creation:

// create asset audio player
jboolean
Java_com_example_nativeaudio_NativeAudio_createAssetAudioPlayer(JNIEnv*
env, jclass clazz,
      jobject assetManager, jstring filename)
{
  SLresult result;

  // convert Java string to UTF-8
  const jbyte *utf8 = (*env)->GetStringUTFChars(env, filename,
NULL);
  assert(NULL != utf8);

  // use asset manager to open asset by filename
  AAssetManager* mgr = AAssetManager_fromJava(env, assetManager);
  assert(NULL != mgr);
  AAsset* asset = AAssetManager_open(mgr, (const char *) utf8,
AASSET_MODE_UNKNOWN);

  // release the Java string and UTF-8
  (*env)->ReleaseStringUTFChars(env, filename, utf8);

  // the asset might not be found
  if (NULL == asset) {
      return JNI_FALSE;
  }

  // open asset as file descriptor
  off_t start, length;
  int fd = AAsset_openFileDescriptor(asset, &start, &length);
  assert(0 <= fd);
  AAsset_close(asset);

  // configure audio source
  SLDataLocator_AndroidFD loc_fd = {SL_DATALOCATOR_ANDROIDFD, fd,
start, length};
  SLDataFormat_MIME format_mime = {SL_DATAFORMAT_MIME, NULL,
SL_CONTAINERTYPE_UNSPECIFIED};
  SLDataSource audioSrc = {&loc_fd, &format_mime};

  // configure audio sink

  SLDataLocator_AndroidSimpleBufferQueue loc_bufq =
{SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE, 2};
  SLDataFormat_PCM format_pcm = {SL_DATAFORMAT_PCM, 2,
SL_SAMPLINGRATE_44_1,
      SL_PCMSAMPLEFORMAT_FIXED_16, SL_PCMSAMPLEFORMAT_FIXED_16,
      SL_SPEAKER_FRONT_LEFT|SL_SPEAKER_FRONT_RIGHT,
SL_BYTEORDER_LITTLEENDIAN};
  SLDataSink audioSnk = {&loc_bufq, &format_pcm};

  result = (*engineEngine)->CreateAudioPlayer(engineEngine,
&fdPlayerObject, &audioSrc, &audioSnk,
          0, NULL, NULL);
  assert(SL_RESULT_SUCCESS == result);

  // realize the player
  result = (*fdPlayerObject)->Realize(fdPlayerObject,
SL_BOOLEAN_FALSE);
  assert(SL_RESULT_SUCCESS == result);

  // get the play interface
  result = (*fdPlayerObject)->GetInterface(fdPlayerObject,
SL_IID_PLAY, &fdPlayerPlay);
  assert(SL_RESULT_SUCCESS == result);

  // get the buffer queue interface
  result = (*fdPlayerObject)->GetInterface(fdPlayerObject,
SL_IID_BUFFERQUEUE,
          &fdPlayerBufferQueue);
  assert(SL_RESULT_SUCCESS == result);

  // register callback on the buffer queue
//    result = (*fdPlayerBufferQueue)-
>RegisterCallback(fdPlayerBufferQueue, fdPlayerCallback, NULL);
//    assert(SL_RESULT_SUCCESS == result);

  return JNI_TRUE;
}



This returns SL_RESULT_FEATURE_UNSUPPORTED:

  // get the buffer queue interface
  result = (*fdPlayerObject)->GetInterface(fdPlayerObject,
SL_IID_BUFFERQUEUE,
          &fdPlayerBufferQueue);
  assert(SL_RESULT_SUCCESS == result);


Can anyone verify if this should really work or not?
Thank you very much.
Bob

[Steven Merel]

You need to pass in the list of required interfaces to CreateAudioPlayer:

SLInterfaceID ids[] =

        {

            OpenSLES::SL_IID_BUFFERQUEUE,

            OpenSLES::SL_IID_PLAY

        };

SLboolean req[] =

        {

            SL_BOOLEAN_TRUE,

            SL_BOOLEAN_TRUE

        };

result = (*engineEngine)->CreateAudioPlayer(engineEngine, &fdPlayerObject, &audioSrc, &audioSnk, 2, ids, req);