AEC for Android when HW-AEC is not available

[Lucas Lafarga]

We are developing a videocalling app for Android TV that uses webrtc. We've been doing some tests and we've unfortunately seen the echo canceler regularly perform quite poorly. We have some questions related to how webrtc echo cancellation works for Android. Reading the code, we can see that for Android hardware echo cancellation is preferred, but unlike phones, set-top boxes and TVs that use Android TV don't have HW-AEC, that means that we have to use Webrtc's AEC.

In audio_record_jni.cc in AudioRecordJni::OnDataIsRecorded there's a comment that says that AEC will take into consideration the value set in SetVQEData. This value is a constant that is set to 50ms when using OpenSL ES and 150ms when using Java for the playout. It seems that this value is very important for the AEC to work properly, since it's the expected delay between getting the audio and playing it from speakers. This post in discuss-webrtc mentions that "The AECM requires high-quality, external, delay estimates."

Is it so, that for Android the delay value stays fixed at 50ms or 150ms and echo cancelling will stop working properly if the real delay starts to drift from that initially estimated value? What's the range in ms around the estimate where the AEC performs as expected? Are we supposed to update the actual delay using "set_stream_delay_ms(int delay)"?

Other questions:

- A comment in a test file suggests that 500ms is the max delay that the AEC can handle. Is this correct?

- DA-AEC is now the default for Chrome in Mac, LInux and Win, but Chrome for Android, as well as comments in the webrtc code, say that DA-AEC is not available for Android. Is it possible to use it anyway? Is it going to be available soon?

[Lucas Lafarga]

We found one of the answers we were looking for in: webrtc/modules/audio_device/android/audio_common.h

It says that the filter size is 128ms and that AEC should work properly in the the range [50, ~170] ms for OpenSL and [150, ~270] ms for Java.

[Lucas Lafarga]

After investigating a little bit more, in 
webrtc/modules/audio_processing/include/audio_processing.h there's a comment that says that the normal filter size is 48ms. Only when using extended filter the value is 128ms. We can see in logcat that extended filter is disabled by default for Android, does AECM support enabling the extended filter?

[Christoffer Jansson]

Hi,

Re: why AECM is used: AEC is not deemed as good for Android as AECM. Details in https://bugs.chromium.org/p/webrtc/issues/detail?id=4472.

Re: if you can use extended filter with AECM (answer from a colleague): No AECM does not support an extended filter. That is too complex for the mobile platform.

/Chris

On Thu, Mar 31, 2016 at 2:55 PM Lucas Lafarga <lu...@tellybean.com> wrote:

After investigating a little bit more, in 
webrtc/modules/audio_processing/include/audio_processing.h there's a comment that says that the normal filter size is 48ms. Only when using extended filter the value is 128ms. We can see in logcat that extended filter is disabled by default for Android, does AECM support enabling the extended filter?

___________________________________________________

Tellybean. Making lifelike video calling available to everyone.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/62e487f5-bb82-40db-ba3b-3b7994870ac0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

/Chris

[Patrik Johnson]

Re: why AECM is used: AEC is not deemed as good for Android as AECM. Details in https://bugs.chromium.org/p/webrtc/issues/detail?id=4472.

The bug details don't seem to be public, I get "You do not have permission to view the requested page." when opening the above link... It's not available when browsing the issue pages either, at least with my account.

 

Re: if you can use extended filter with AECM (answer from a colleague): No AECM does not support an extended filter. That is too complex for the mobile platform.

/Chris

On Thu, Mar 31, 2016 at 2:55 PM Lucas Lafarga <lu...@tellybean.com> wrote:

After investigating a little bit more, in 
webrtc/modules/audio_processing/include/audio_processing.h there's a comment that says that the normal filter size is 48ms. Only when using extended filter the value is 128ms. We can see in logcat that extended filter is disabled by default for Android, does AECM support enabling the extended filter?

___________________________________________________

Tellybean. Making lifelike video calling available to everyone.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/62e487f5-bb82-40db-ba3b-3b7994870ac0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

/Chris

[Christoffer Jansson]

On Fri, Apr 1, 2016 at 3:45 PM Patrik Johnson <pat...@tellybean.com> wrote:

Re: why AECM is used: AEC is not deemed as good for Android as AECM. Details in https://bugs.chromium.org/p/webrtc/issues/detail?id=4472.

The bug details don't seem to be public, I get "You do not have permission to view the requested page." when opening the above link... It's not available when browsing the issue pages either, at least with my account.

Aha, well looking back at the bug there wasn't any technical details about that anyhow, just that sentence. 

 

Re: if you can use extended filter with AECM (answer from a colleague): No AECM does not support an extended filter. That is too complex for the mobile platform.

/Chris

On Thu, Mar 31, 2016 at 2:55 PM Lucas Lafarga <lu...@tellybean.com> wrote:

After investigating a little bit more, in 
webrtc/modules/audio_processing/include/audio_processing.h there's a comment that says that the normal filter size is 48ms. Only when using extended filter the value is 128ms. We can see in logcat that extended filter is disabled by default for Android, does AECM support enabling the extended filter?

___________________________________________________

Tellybean. Making lifelike video calling available to everyone.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/62e487f5-bb82-40db-ba3b-3b7994870ac0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

/Chris

___________________________________________________

Tellybean. Making lifelike video calling available to everyone.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/780e3aba-dd9f-4fb2-abaa-60a8ad2f0f7b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

/Chris

[Lucas Lafarga]

Thank you for your answers Christofer!

Summarizing, this is the info we've gather so far:

- AECM is the only alternative for Android (Our devices don't have HW-AEC and DA-AEC is not available)

- The filter size for AECM is 48ms, echo cancellation won't work properly if the estimation is off by more than ~48ms. There's no possibility of using extended filter, is too complex for the mobile platform.

One thing that's still not clear for us is the stream delay estimate. As far as we know, it is dependent on the AudioLayer that is in use, and once the AudioLayer is set, that estimate will remain constant (e.g 50 for OpenSL and 150ms for Java). 

- Does AECM calculate new values for the expected delay during a peerconnection or does it rely on the initial constant estimate? 

- Can we provide our own estimates during a peerconnection? There does not seem to be an API for that, but the use of "set_stream_delay_ms" was suggested in an old discuss-webrtc post.