[OpenSL] File decoding to arbitratry-sized buffers

[Domagoj Saric]

Hi everyone,

I'm trying to implement a simple audio file reader class (which decodes
the file on-demand into in-memory raw PCM data):
class file_reader
{
void open( location );
void read( buffer );
}

As usual, on iOS this is of course trivial (you just call
ExtAudioFileRead() and get the result on the same thread right there and
then) while on Android it is an excercise in torture.

There appear to be three (undocumented) requirements:

1) buffer.size() (in the (*pBQ)->Enqueue( pBQ, buffer.begin(),
buffer.size() call) cannot be any arbitrary value, it has to be a
multiple of the internal block size used by the decoder for the
particular type of file opened (e.g. 1152 for MP3 or 1024 for AAC).
If the enqueued buffer is of the wrong size there will be no error or
warning in the log, rather the decoder will simply decode a multiple of
the block-size and silently discard the data that does not fit in the
buffer (or something along those lines - the end effect is that the file
plays back too fast/it skips).

2) Between two calls to read() the file_reader object cannot just "do
nothing and then simply enqueue a new buffer on the next call to
read()", rather it has to block the last
slAndroidSimpleBufferQueueCallback and then unblock it after enqueuing a
new buffer (in the next read() call) - otherwise, again, the decoder
seems to silently decode the file in the background discarding the data
when there is no buffer enqueued (the end effect is again that the file
plays too fast/it skips).

3) If you want to use the SLPrefetchStatusItf interface (it is not
clear, but from the official examples it seems this is the only way to
'reliably' detect errors on file opening/metadata extraction) then at
least two buffers have to be enqueued before setting
SL_PLAYSTATE_PLAYING as otherwise an assertion failure inside OpenSL
(srsly, compiling an OS, a mobile one at that, with assertions
enabled!?) crashes your app...

Now, the second and third point "I can live with sort of", but 'how on
earth' am I supposed to know which buffer size to use in the most
general case (where I don't know the file type in advance, rather the
user chooses it)? Even if I can somehow detect which file type was
opened (the filename extension is of course not the most reliable
mechanism) how can I know the undocumented block size used by the
decoder for the given file encoding/container (e.g. what block size must
one use for simple uncompresed .wav files)?


--
"What Huxley teaches is that in the age of advanced technology,
spiritual devastation is more likely to come from an enemy with a
smiling face than from one whose countenance exudes suspicion and hate."
Neil Postman

[Haojie Xu]

Hi Domagoj,

I'm a little familiar with OpenSL ES on Android, the following document maybe help you.

Within Android NDK folder, docs/Additional_library_docs/opensles/index.html:

Determining the format of decoded PCM data via metadata

The metadata extraction interface SLMetadataExtractionItf is a standard OpenSL ES 1.0.1 interface, not an Android extension. However, the particular metadata keys that indicate the actual format of decoded PCM data are specific to Android, and are defined in header SLES/OpenSLES_AndroidMetadata.h.

I check the OpenSLES_AndroidMetadata.h, It contains ANDROID_KEY_PCMFORMAT_CONTAINERSIZE

#define ANDROID_KEY_PCMFORMAT_NUMCHANNELS   "AndroidPcmFormatNumChannels"

#define ANDROID_KEY_PCMFORMAT_SAMPLERATE    "AndroidPcmFormatSampleRate"

#define ANDROID_KEY_PCMFORMAT_BITSPERSAMPLE "AndroidPcmFormatBitsPerSample"

#define ANDROID_KEY_PCMFORMAT_CONTAINERSIZE "AndroidPcmFormatContainerSize"

#define ANDROID_KEY_PCMFORMAT_CHANNELMASK   "AndroidPcmFormatChannelMask"

#define ANDROID_KEY_PCMFORMAT_ENDIANNESS    "AndroidPcmFormatEndianness"

I'm not sure whether it can solve your first question. If it's wrong, sorry in advance.