AEC not working on a high-end Android device with USB Camera

[Ömer Furkan Tercan]

Hi,

We have a webrtc enabled service with 2 different endpoints; a web app and a native android app. The android app is installed on a high-end android device with a USB Camera. 

Using the web app on chrome/firefox, pc2pc audio quality is almost perfect. But we want to improve pc2android and android2android audio quality.

Chrome uses acoustic echo cancellation (AEC - conference) for high-end devices. But for Android it forces AECM.  We are not happy with the AECM performance. For our native app, we modify webrc source-code to use AEC instead. But the result is even worse. It acts like echo cancellation is totally disabled and we end up having so much echo and feedback!

According to the following issue AEC should only work with 8k and 16k sample rates and only in high-end devices. 

https://code.google.com/p/webrtc/issues/detail?id=2580

That should be OK. We are using PCMU codec having 8k sample rate and our android device is quite powerful:

Quad core ARM CPU @ 2Ghz

8 core Mali-450MP GPU @ 600Mhz

DDR3 1GB RAM  

Android Kit Kat

If needed, I'm happy to share plots of our echo cancellation performance. 

Is it just not possible to use AEC for mobiles or are we missing something?

- Furkan

[Brave Yao]

Does chrome work on that android device?

Is the audio capture from the USB Camera or built-in Mic? Would an audio-only call work better?

Please help to modify and test WebRTCDemo with AEC and collect the webrtc trace file with TRACE_ALL to us.

[Ömer Furkan Tercan]

The device does not have a built in camera. It is a set-top android box with an external USB Camera.

Tried audio-only call with AEC set on our native android app. It behaves the same as having both audio&video. AEC fails and we get huge echo and feedback.

Modified WebRTCDemo with AEC. TRACE_ALL was already default set via the video_engine. I Attached the log file.

What should I test with chrome? Chrome calls work but the default echo mode for android devices is AECM - speakerphone. We want to us AEC instead. If we can get AEC modes working, we also want to try out the extended filter mode.

The device should be capable of handling the additional load when using AEC. Am I wrong? Is there something I am missing?

[Brave Yao]

Hi Omer,

So the Mic is from the external Camera too? (most of time the audio quality would be bad in my experience.)

The trace shows many 'audio overrun', as the issue 2982/3830 shows that opensles has some issue with KitKat. You can try JNI implementation on that device by setting "'enable_android_opensl%': 0" in trunk/ webrtc/ build/ common.gypi. Or you could try 4.3 or L on that device.

But I don't expect the performance as bad as "like echo cancellation is totally disabled". Maybe you can try to add some log into aec to check if it runs normally.

The last thing to try if none of above helps is to do a debug recording, by API "VoEAudioProcessingImpl::StartDebugRecording".

--

---
You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/01eMuL48atE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ömer Furkan Tercan]

I remember those issues and 3222. In all issues people experience ragged/interrupted/clipped audio. That was exactly what we were having until we increased the kNum10MsToBuffer. After that audio was clean.

In the previous trace file the kNum10MsToBuffer is 2. In that case 'audio overrun' is seen. We increased it to 50 and dont see 'audio overrun'. Now I made 2 additional tests with kNum10MsToBuffer = 6 and then with android_opensl disabled. 

I attached the trace_all for kNum10MsToBuffer=6 version. The result has no 'audio overrun' but some 'audio underrun'. But in both tests echo and feedback was persistent.

enabling android_opensl with increased buffer gives us clean audio. But switching AECM to AEC results in echo and feedback. 

How does disabling android_opensl affect AEC performance while it doesn't affect AECM performance? 

What is the relation between audio overrun/underrun, kNum10MsToBuffer and AEC performance?

[Brave Yao]

I think the openSLES has some performance issue on 4.4, as the issue3830 shows. One possible reason is too long delay which would make AEC having no effect. 

Try JNI, or upgrade to Android L, might help at present.

[Ömer Furkan Tercan]

I already tested using Android AudioTrack as an alternative to openSLES (test case #2 in my previous message). Isn't that what you mean by the 'JNI implementation' ? I can disable openSLES by building webrtc with GYP_DEFINES = "enable_android_opensl=0".

That doesn't make AEC work better in our device.

Here is a plot based on the debug recording. This recording is made with the below settings:

- android2android video&audio call

- using webrtc r7370.

- openSLES enabled with increased  kNum10MsToBuffer (defined in opensles_input.h)

- AEC with conference mode, noise suppression, and highpass filter.

The wave-forms clearly shows huge feedback loop (very load 'beep' sound) happening between seconds: 10-17, 20-30, 45-55, 80-95, 125-135, 145-155. Overall 'beep' sound exists in the background unless you have a very low volume level.

Based on the wave-forms, do you think 'too long delay' is a strong reason for AEC having no effect?

Is there anything else that I can provide to help inspect the issue further?

[Brave Yao]

Seems that some of the signal is saturated, which also would lower the AEC performance. And it matches the time range of feedback loop. Try to lower the capture level a little bit.

And yes, the AudioTrack is the "JNI implementation". Please use it if you have to work on 4.4.