How to use OpenSL ES with USB Digital Audio devices (Headphones)

[Rafał Kanus]

Welcome everybody,

I am new on this group and this is my first topic.

We are working on Voip application which is mostly implemented in c/c++ to be multi-platform.
On Android we are using OpenSL ES API to access audio devices.

We don't have problems with Earpiece, Speaker, Bluetooth Headset or Wired Headset devices.
We are able to detect which device is connected and in use using AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate configuration (sample rate, channels etc).


But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation https://source.android.com/devices/audio/usb.html we can read that system automatically routes audio playback and capture appropriately, based on the audio policy rules.


How to detect if USB Audio device is connected and audio is routed through it?
I know that the system route audio automatically to USB but It would be good to know about that and open the device in correct configuration e.g. stereo instead mono.

Did someone had the same problems?


BTW: Nexus devices are working correctly with USB Audio devices but Samsung do not route audio via USB when mode is set to IN_COMMUNICATION. That seems to be broken.


Best regards,
Rafal Kanus

[Glenn Kasten]

Thank you for your post.

Android "M" includes new SDK APIs for explicit routing; see

http://developer.android.com/reference/android/media/AudioTrack.html#setPreferredDevice(android.media.AudioDeviceInfo)

A similar explicit routing API is not available in the "M" NDK (native audio based on OpenSL ES).

Thank you also for your experience with IN_COMMUNICATION on some devices.
I will investigate that.

[Paul Mclean]

I've been informed that there is a problem with consistently connecting USB audio to a Samsung device. Can someone tell me which devices are involved?

- Paul

[Rich Eakin]

On Friday, October 2, 2015 at 10:03:04 AM UTC-4, Glenn Kasten wrote:

Thank you for your post.

Android "M" includes new SDK APIs for explicit routing; see

http://developer.android.com/reference/android/media/AudioTrack.html#setPreferredDevice(android.media.AudioDeviceInfo)

A similar explicit routing API is not available in the "M" NDK (native audio based on OpenSL ES).

I've also been wondering if there are any ways to get more audio device details. Is it possible to utilize AudioTrack.setPreferredDevice() while still running OpenSL / native audio code? Similar to how we are using AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE, for example, to correctly set the samplerate on SLDataFormat_PCM in C / C++.

cheers,

Rich

[Glenn Kasten]

The difference is that AudioManager returns static information, and so information returned using the SDK API is

also useful with the NDK API.

However, AudioTrack is a per-stream object, and so the SDK and NDK APIs are not inter-operable on the same stream.

("stream" here meaning a sequence of audio data, not "stream type" as used for the volume control APIs).

[Rich Eakin]

(Created new thread so as not to hijack the original)

Thanks for the information. Are there any plans to also provide device information to OpenSL users? Currently, I'm wondering how to query whether the device is mono or stereo, but I could also see targeting specific hardware a nice feature as well, such as what you can do in Android M with AudioTrack.

Thanks,

Rich

[Glenn Kasten]

You can use JNI to call SDK APIs from native code.

Note that the OpenSL ES buffer completion callback runs on a non-JNI-capable thread,

so you cannot call SDK APIs from that callback handler and will need to use a JNI-capable thread instead.

If you are requesting a pure native API, please file a feature request here:

https://code.google.com/p/android/issues/entry

I cannot give forward-looking statements, such as about plans. 

[Rich Eakin]

On Sat, Oct 3, 2015 at 11:34 AM, Glenn Kasten <gka...@android.com> wrote:

You can use JNI to call SDK APIs from native code.

Note that the OpenSL ES buffer completion callback runs on a non-JNI-capable thread,

so you cannot call SDK APIs from that callback handler and will need to use a JNI-capable thread instead. 

If you are requesting a pure native API, please file a feature request here:

https://code.google.com/p/android/issues/entry

I cannot give forward-looking statements, such as about plans. 

Not sure if I'm understanding your answer, so I'll be more specific - can I make use of something in java, perhaps android.media.AudioDeviceInfo, to correctly configure the channels of my audio I/O setup in OpenSL?  Right now, in very early stages of my implementation, I'm just using stereo for output and mono for input, but I've also read comments in others' code that even stereo output is not always reliable.  Basically if I could get that info the AudioDeviceInfo and _be sure_ that the device it represents is the same as the one I am configuring with OpenSL, then I'd feel much better about it. Perhaps I can assume that AudioTrack.getRoutedDevice() represents the same output device (SL_DATALOCATOR_OUTPUTMIX) that I'll be configuring in OpenSL? Can I be sure?

Though I don't see why any OpenSL user wouldn't want to have a native API to give them the information they needed, rather then having to jump through an entirely different language to get basic device information, so I've filed a feature request (188502) for that.

Thanks again for the help,

Rich

[Rafał Kanus]

OK. So is it true that if we read from Java AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE we will get sample rate of current primary audio output device?
If I switch to Speaker, I will get sample rate to built-in speaker.
If I plug in USB Headphones, I will get sample rate for USB Headphone device (BTW what if device supports multiple sample rates).
Is this true?

What about Input device (e.g. USB Microphone). The is no equivalent property for microphone sample rate.

Maybe the solution for USB Audio Devices will be to read USB Audio Device Descriptor and get correct sample rate and channels from there.
Or parse /proc/asound/card1/stream0.
Then we can just pass that information to native C/C++ and open Input and Output in correct configuration via OpenSL ES.
What do you think about that?

BTW: Since in Android 5 we don't have possibility to change manually route from/to USB Aidio,  I assume that USB Audio Device become primery right after we plug it in.

[Rafał Kanus]

Hi,

I've observed problems on Samsung S5, S6 and S6 edge with Android 5+.
We are working on Voip client and use OpenSL ES library. When I set mode to IN_COMMUNICATION and then open Input and Output device, the sound is not routed to USB Headphones and USB microphone is alos not use.
Internal Earpiece and Microphone is used. It is like there was no USB headphones connected. I've checked in a shell what is in /proc/asound/cards and device in correctly detected as soundcard ( also data in /proc/asound/card1 are correct).

I've tried to change to mode to NORMAL or IN_CALL. In this configuration the output is working but input is not recording.
When I use any Music Player app which use only Java API USB Headphones are working correctly but recording does not.

[Rich Eakin]

On Mon, Oct 5, 2015 at 10:45 AM, Rafał Kanus <rafal...@gmail.com> wrote:

OK. So is it true that if we read from Java AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE we will get sample rate of current primary audio output device?
If I switch to Speaker, I will get sample rate to built-in speaker.
If I plug in USB Headphones, I will get sample rate for USB Headphone device (BTW what if device supports multiple sample rates).
Is this true?
 

What about Input device (e.g. USB Microphone). The is no equivalent property for microphone sample rate.

Since my original reply, I noticed AudioManager.getDevices(), that returns an AudioDeviceInfo object. Maybe this will help setup usb and input devices? I don't have anything running Android M so can't test myself.

[Glenn Kasten]

Rafal, thanks for the device names.  We're investigating the discrepancy.

[Rafał Kanus]

Hi,

Yes, I will use in on Android M, but this API is not available on Android 5 so I have to get that information from USB Descriptor or directly from Alsa sound card (/proc/asound/card1 )


 

[mike digioia]

Submit me

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at http://groups.google.com/group/android-ndk.
For more options, visit https://groups.google.com/d/optout.