How to reroute audio data to Android's audio input

[gps...@gmail.com]

Hi,

I have a remote device that connects to my android device over BLE.

My remote device is handheld, and can send audio data to android device over radio. My own apps can receive this audio successfully and play it back on android.

I want to enable other (third party) android apps to receive this audio also, e.g. Voice recognition apps, that can be triggered from handheld remote.

To achieve this, my audio must be routed such that it looks like its coming in as input audio to other apps. I need help in achieving this.

Can remote_submix do this? I did find that remote_submix can do exact opposite of this, i.e. capture local playback and give it to app.

I am also open to implementing a new virtual audio cable audio_hw if that's the only way, but I want to consult experts here first.

[Originally asked here: http://stackoverflow.com/questions/39263696/how-to-route-audio-for-other-apps]

Thanks

[gps...@gmail.com]

Hi,

I figured I need a new audio HAL to achieve this, something on the lines of "Virtual Audio Cable [http://vb-audio.pagesperso-orange.fr/Cable/]" for windows.

So I am writing a new HAL, which has one audio output and one audio input.

If an app opens the input, it will receive data currently fed to the output by my app, or blank if my app is not feeding in any data.

Writing the HAL is pretty straightforward, And I have written most of it, specifically read/write/standby/pause handling.

Yet, I do not understand what I need to do for some methods, specifically:

1. adev_get_audio_port() and adev_set_audio_port_config(): do I need to implement ports too?

2. adev_create_audio_patch() and adev_release_audio_patch(): what does it even mean?

3. adev_set_parameters() out_set_parameters() and in_set_parameters(): Do I need to parse, store and use set parameters? 

Is it OK to return -ENOSYS for all the above functions unconditionally?

Apart from this, I have some more questions:

1. How do I tell Android to use my HAL? Is it sufficient to place .so in /system/lib/hw/?

2. Does it use all HALs in that location by default, or only primary? Do I need to replace primary

Any help is appreciated.

Thanks

-Gaurav

[gps...@gmail.com]

Hi,

I could really use any help in this work. Any pointers are helpful.

I have written an Audo HAL module. I have added following (bold items) to audio_policy.conf:

global:

global_configuration {

  attached_output_devices AUDIO_DEVICE_OUT_SPEAKER|AUDIO_DEVICE_OUT_LINE

  default_output_device AUDIO_DEVICE_OUT_SPEAKER

  attached_input_devices AUDIO_DEVICE_IN_BUILTIN_MIC|AUDIO_DEVICE_IN_BACK_MIC|AUDIO_DEVICE_IN_REMOTE_SUBMIX|AUDIO_DEVICE_IN_LINE

}

and under audio_hw_modules

  vloop {

    inputs {

      vloop {

        sampling_rates 16000

        channel_masks AUDIO_CHANNEL_IN_MONO

        formats AUDIO_FORMAT_PCM_16_BIT

        devices AUDIO_DEVICE_IN_LINE

      }

    }

    outputs {

      vloop {

        sampling_rates 16000

        channel_masks AUDIO_CHANNEL_OUT_STEREO

        formats AUDIO_FORMAT_PCM_16_BIT

        devices AUDIO_DEVICE_OUT_LINE

        flags AUDIO_OUTPUT_FLAG_DIRECT

      }

    }

  }

I also added following (bold) to AudioFlinger.cpp

static const char * const audio_interfaces[] = {
    AUDIO_HARDWARE_MODULE_ID_PRIMARY,
    AUDIO_HARDWARE_MODULE_ID_A2DP,
    AUDIO_HARDWARE_MODULE_ID_USB,
    AUDIO_HARDWARE_MODULE_ID_VLOOP
};

I can see that during boot, my HAL gets loaded, and I get these logs:

10-06 06:14:40.365 194-194/? I/AudioFlinger: Using default 3000 mSec as standby time.

10-06 06:14:46.664 194-194/? I/AudioPolicyService: AudioPolicyService CSTOR in new mode

10-06 06:14:46.673 194-194/? I/APM::ConfigParsingUtils: loadAudioPolicyConfig() loaded /system/etc/audio_policy.conf

10-06 06:14:46.681 194-194/? D/audio_hw_primary: adev_open: enter

10-06 06:14:46.797 194-194/? I/AudioFlinger: loadHwModule() Loaded primary audio interface from QCOM Audio HAL (audio) handle 1

10-06 06:14:46.797 194-194/? I/AudioFlinger: openOutput(), module 1 Device 2, SamplingRate 48000, Format 0x000001, Channels 3, flags 2

10-06 06:14:46.797 194-194/? I/AudioFlinger: AudioStreamOut::open(), mHalFormatIsLinearPcm = 1

10-06 06:14:46.798 194-194/? I/AudioFlinger: HAL output buffer size 240 frames, normal sink buffer size 960 frames

10-06 06:14:46.813 194-194/? I/AudioFlinger: Using module 1 has the primary audio interface

10-06 06:14:46.816 194-607/? I/AudioFlinger: AudioFlinger's thread 0xb4140000 ready to run

10-06 06:14:46.816 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2

10-06 06:14:46.818 194-194/? I/AudioFlinger: openOutput(), module 1 Device 2, SamplingRate 48000, Format 0x000001, Channels 3, flags 8

10-06 06:14:46.818 194-194/? I/AudioFlinger: AudioStreamOut::open(), mHalFormatIsLinearPcm = 1

10-06 06:14:46.818 194-194/? I/AudioFlinger: HAL output buffer size 960 frames, normal sink buffer size 960 frames

10-06 06:14:46.818 194-608/? I/AudioFlinger: AudioFlinger's thread 0xb3dc0000 ready to run

10-06 06:14:46.818 194-607/? E/AudioFlinger: no wake lock to update!

10-06 06:14:46.818 194-608/? D/audio_hw_primary: out_set_parameters: enter: usecase(0: deep-buffer-playback) kvpairs: routing=2

10-06 06:14:46.818 194-608/? E/AudioFlinger: no wake lock to update!

10-06 06:14:46.820 194-609/? I/AudioFlinger: AudioFlinger's thread 0xb3c40000 ready to run

10-06 06:14:46.823 194-194/? I/AudioFlinger: loadHwModule() Loaded a2dp audio interface from A2DP Audio HW HAL (audio) handle 7

10-06 06:14:46.828 194-194/? I/AudioFlinger: loadHwModule() Loaded usb audio interface from USB audio HW HAL (audio) handle 8

10-06 06:14:46.832 194-194/? I/r_submix: adev_open(name=audio_hw_if)

10-06 06:14:46.832 194-194/? I/AudioFlinger: loadHwModule() Loaded r_submix audio interface from Wifi Display audio HAL (audio) handle 9

10-06 06:14:46.832 194-194/? D/r_submix: submix_audio_device_create_pipe_l(addr=0, idx=9)

10-06 06:14:46.833 194-610/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run

10-06 06:14:46.833 194-194/? D/r_submix: submix_audio_device_release_pipe_l(idx=9) addr=0

10-06 06:14:46.833 194-194/? D/r_submix: submix_audio_device_destroy_pipe_l(): pipe destroyed

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open: audio_hw_if

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1678

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1685

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1688

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1722

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_init_check(): 1252

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_set_master_volume: 1.000000

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_set_master_mute: 0

10-06 06:14:46.835 194-194/? I/AudioFlinger: loadHwModule() Loaded vloop audio interface from UEI_audio_HW_HAL (audio) handle 11

10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open_input_stream(): 1490

10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_sample_rate(): 979

10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_channels(): 1017

10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_channels: 0x00000001

10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_format(): 1029

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format: 0x00000001

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format(): 1029

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format: 0x00000001

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels(): 1017

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels: 0x00000001

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size(): 1005

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size: 1600

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size(): 1005

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size: 1600

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format(): 1029

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format: 0x00000001

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_sample_rate(): 979

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels(): 1017

10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels: 0x00000001

10-06 06:14:46.838 194-613/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run

10-06 06:14:46.838 194-194/? D/audio_vloop: adev_close_input_stream(): 1570

10-06 06:14:46.839 194-194/? W/APM::AudioPolicyManager: Input device 00020000 unreachable

10-06 06:14:46.839 194-611/? D/audio_vloop: looper_thread(): 216: Entered

10-06 06:14:46.839 194-611/? D/audio_vloop: looper_thread(): 366: Exiting

10-06 06:15:07.137 616-616/? I/InputManager: Initializing input manager, mUseDevInputEventForAudioJack=false

10-06 06:15:10.155 616-616/? I/SystemServer: Audio Service

10-06 06:15:10.222 194-607/? E/AudioFlinger: no wake lock to update!

10-06 06:15:10.222 194-608/? E/AudioFlinger: no wake lock to update!

10-06 06:15:10.224 194-614/? D/audio_hw_primary: adev_set_mic_mute: state 0

10-06 06:15:10.224 194-614/? D/audio_vloop: adev_set_mic_mute: 0

10-06 06:15:14.061 194-614/? D/audio_hw_primary: adev_set_parameters: enter: A2dpSuspended=false

10-06 06:15:14.061 194-614/? D/audio_vloop: adev_set_parameters(): [A2dpSuspended=false]

10-06 06:15:14.084 194-194/? I/AudioFlinger: systemReady

10-06 06:15:16.308 194-194/? D/audio_hw_primary: adev_set_mic_mute: state 0

10-06 06:15:16.308 194-194/? D/audio_vloop: adev_set_mic_mute: 0

10-06 06:15:17.072 194-194/? D/audio_hw_primary: adev_set_parameters: enter: A2dpSuspended=false

10-06 06:15:17.072 194-194/? D/audio_vloop: adev_set_parameters(): [A2dpSuspended=false]

10-06 06:15:25.023 733-733/? W/AudioTrack: AUDIO_OUTPUT_FLAG_FAST denied by client; transfer 4, track 44100 Hz, output 48000 Hz

10-06 06:15:25.032 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2

10-06 06:15:25.043 194-607/? D/audio_hw_primary: select_devices: out_snd_device(2: speaker) in_snd_device(0: none)

10-06 06:15:25.043 194-607/? D/audio_hw_primary: enable_snd_device: snd_device(2: speaker)

10-06 06:15:25.050 194-607/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: low-latency-playback

10-06 06:15:26.431 1150-1298/? I/MicrophoneInputStream: mic_starting com.google.android.apps.gsa.speech.audio.ag@c6eb0e1

10-06 06:15:26.443 194-1585/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run

10-06 06:15:26.447 1150-1298/? I/MicrophoneInputStream: mic_started com.google.android.apps.gsa.speech.audio.ag@c6eb0e1

10-06 06:15:26.457 194-1585/? D/audio_hw_primary: select_devices: out_snd_device(0: none) in_snd_device(38: voice-rec-mic)

10-06 06:15:26.457 194-1585/? D/audio_hw_primary: enable_snd_device: snd_device(38: voice-rec-mic)

10-06 06:15:26.460 194-1585/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: audio-record

10-06 06:15:26.942 1150-1271/? I/AudioController: internalShutdown

10-06 06:15:26.943 1150-1271/? I/MicrophoneInputStream: mic_close com.google.android.apps.gsa.speech.audio.ag@c6eb0e1

10-06 06:15:26.943 1150-1298/? E/AudioRecord-JNI: Error -4 during AudioRecord native read

10-06 06:15:26.986 194-1585/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: audio-record

10-06 06:15:26.987 194-1585/? D/audio_hw_primary: disable_snd_device: snd_device(38: voice-rec-mic)

10-06 06:15:27.066 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2

10-06 06:15:27.100 194-607/? D/AudioFlinger: mixer(0xb4140000) throttle end: throttle time(7)

10-06 06:15:30.257 194-607/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: low-latency-playback

10-06 06:15:30.257 194-607/? D/audio_hw_primary: disable_snd_device: snd_device(2: speaker)

10-06 06:15:30.262 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2

10-06 06:15:30.272 194-607/? D/audio_hw_primary: select_devices: out_snd_device(2: speaker) in_snd_device(0: none)

10-06 06:15:30.273 194-607/? D/audio_hw_primary: enable_snd_device: snd_device(2: speaker)

10-06 06:15:30.280 194-607/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: low-latency-playback

10-06 06:15:30.347 194-607/? D/AudioFlinger: mixer(0xb4140000) throttle end: throttle time(10)

10-06 06:15:31.517 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2

10-06 06:15:31.751 1150-1298/? I/MicrophoneInputStream: mic_starting com.google.android.apps.gsa.speech.audio.ag@dd13203

10-06 06:15:31.762 194-1826/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run

10-06 06:15:31.771 1150-1298/? I/MicrophoneInputStream: mic_started com.google.android.apps.gsa.speech.audio.ag@dd13203

10-06 06:15:31.780 194-1826/? D/audio_hw_primary: select_devices: out_snd_device(0: none) in_snd_device(38: voice-rec-mic)

10-06 06:15:31.780 194-1826/? D/audio_hw_primary: enable_snd_device: snd_device(38: voice-rec-mic)

10-06 06:15:31.783 194-1826/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: audio-record

10-06 06:15:34.695 194-607/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: low-latency-playback

10-06 06:15:34.695 194-607/? D/audio_hw_primary: disable_snd_device: snd_device(2: speaker)

10-06 06:15:34.850 1150-1271/? I/AudioController: internalShutdown

10-06 06:15:34.851 1150-1271/? I/MicrophoneInputStream: mic_close com.google.android.apps.gsa.speech.audio.ag@dd13203

10-06 06:15:34.851 1150-1298/? E/AudioRecord-JNI: Error -4 during AudioRecord native read

10-06 06:15:34.885 194-1826/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: audio-record

10-06 06:15:34.885 194-1826/? D/audio_hw_primary: disable_snd_device: snd_device(38: voice-rec-mic)

Mine is audio_vloop. I can see that Android opens my device, then opens input stream, and then closes the input stream. It never attempts to open output stream. audio_vloop implements both input and output streams. After this, nothing in audio_vloop is ever invoked by Android.

I made a small app that plays audio (from a pcm file for now). I want to redirect this output to my HAL. To achieve this, I believe I need to do a AudioTrack.setPreferredDevice() on my audio track. I found that Audio Manager should have a list of all audio devices.

so I call : 

AudioDeviceInfo aDevInfo[] = am.getDevices(AudioManager.GET_DEVICES_OUTPUTS);

It  only finds 1 device, more info on this device:

10-06 06:37:01.962 3295-3663/? D/AudioPlayer: Have [1] devices

10-06 06:37:01.964 3295-3663/? D/AudioPlayer: devInfo[0]: [Landroid.media.AudioDeviceInfo;@90bd9da

10-06 06:37:01.965 3295-3663/? D/AudioPlayer: getProductName()AOSP on Flo

10-06 06:37:01.965 3295-3663/? D/AudioPlayer: getType()2

10-06 06:37:01.966 3295-3663/? D/AudioPlayer: isSink()true

10-06 06:37:01.966 3295-3663/? D/AudioPlayer: isSource()false

This seemingly is from audioPort which I have not implemented. So its not from my HAL.

I've obviously missed one or more steps before Android will allow my app to talk to my device.

I'd need to be able to send audio from my app into my HAL.

Later, I also need to be able to receive audio from my HAL (through AudioRecord etc). 

If anyone here can help me with what I missed in integrating my HAL into Android, that'd be great.  Do I need to implement Audio Ports? Is something else required?

Thanks

On Friday, 16 September 2016 19:35:54 UTC+5:30, gps...@gmail.com wrote:

[gps...@gmail.com]

I found my answers. Posted specific issue resolutions here:

http://stackoverflow.com/questions/39746221/

Thanks

On Friday, 16 September 2016 19:35:54 UTC+5:30, gps...@gmail.com wrote:

[Oleg B.]

I know that this is an old thread, but did someone find a solution to this?  We need to be able to "feed" our audio stream as a  standard microphone input so that the rest of the Aps (primarily VOIP) can be faked to grab the microphone input and use it.  Any help is greatly appreciated....thanks..

[Darshan Phy]

hello Oleg,

this is the exact requirement that i have as well. did you find any solution. ? If Yes, please guide me!

I have asked Micheal regarding the same

**My comment from stackoverflow**

@Michael Hello Mike, I am having a similar requirement. My requirement is for personal testing purpose. My app wont go to play store. I can change AOSP code, build and run to the device. You can also tell me which device to use, I would get the same device and rooted as well. I need to inject stored audio file into MIC. I would be making a VoIP call and inject the audio file as input to MIC. So that i can test the audio quality of received audio at the other end, over the network. My requirement is to test audio quality. Once VoIP call is setup, the only input to the MIC is my audio file.

[Shahriar Vaghar]

Hi

You usually read from mic audio stream and write to speaker stream.

Look at tiny play and record source code on how to work with alsa.

--
You received this message because you are subscribed to the Google Groups "android-platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-platfo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-platform/05020c5c-ce72-43b6-a682-0e42a6ce2c29%40googlegroups.com.

[apoorv n]

HI Gaurav

Were you able to implement an virtual audio loopback in Android? I am looking for something similar and would be really helpful if you can share your implementation of HAL and the required Audio configuration/policy changes?

Thanks

[Johny Sm]

Hi All,

I am looking for audio virtual driver android interface and hal sample code , please help me to share sample code or related documents related to audio virtual loopback for android .please do the needful.

Thnaks

johny

[Din Assegaf]

Hi Gps,

i seem it more explanation how you solve your idea, this mean you are creating audio_policy vloop, but where is the implementation ? or how to debug it
I wonder vloop is the implementation of modules on kernel snd-aloop ?

Let us know for more knowledge in Universe.