Chromium AEC on out-of-process system audio

[Andrei Gheorghe]

Hey everyone, I'm hoping to get some advice on my AEC scenario.

I'm building a desktop Electron app running Chromium M114 that captures both system audio as well as microphone, to record and transcribe. My challenge is applying AEC when the user isn't wearing headphones, since the system audio coming from the laptop speakers feeds into the microphone.

I was hoping to use Chrome's AEC for this. I have confirmed that an audio file played from within the Chromium instance is correctly cancelled out automatically from the microphone stream, which means I'm hitting kChromeWideEchoCancellation, and I'm trying to figure out how to do the same with the system audio stream I'm acquiring.

Because this system audio (a mix of audio coming from all processes) is not coming from the Chromium process, I'm trying to get AEC to work by piping the AudioBuffers into the AudioContext and playing them. However because of the inherent delay between when the system audio happened and when I get the buffers and play them (~20ms), the audio I'm playing ends up being doubled over the real audio, with this slight offset - not good.

So I guess my questions are:

1. Is there any way to make Chrome consider the system audio buffers when applying AEC without actually playing them out loud through the AudioContext? Can I pipe them to some kind of /dev/null while still having them be picked by the kChromeWideEchoCancellation code? I tried putting a GainNode with gain=0 in-between and it doesn't look like AEC works then.

2. Assuming I get (1) to work, Is a 20ms delay too long to do effective echo cancellation? I don't think I have much control over how many samples I receive at each tick, currently receiving 960 at a frequency of 48k hence every 20ms.

Thanks

[胡海]

I think you should do echo cancellation with the system audio with the audio from webrtc conversation as farend.The two audio streams(one from microphone and one from system sound card) should be processed(echo cancellation) seperately. Process the mircophone audio with all audio out of speakerphone as farend, and process the system audio(from sound card) with webrtc playing audio as farend.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/a18ba378-5fbc-49fa-bc7f-ebc7f8741402n%40googlegroups.com.

[Olga Sharonova]

Hi Andrei,

Please see the answer to your questions in https://bugs.chromium.org/p/chromium/issues/detail?id=687574#c143: the solution proposed in (1) won't work, unfortunately. 

Regards,

Olga

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/5DA13758-C066-4C23-9623-EF6CB974F6C5%40gmail.com.

[Andrei Gheorghe]

Thanks a lot for your responses Olga & 胡海, really appreciate it, you're saving me days of going down a rabbit hole that goes nowhere!

It sounds like using Chrome's instance-wide-aec out of the box is not an option, for a couple of good reasons.

In that case, taking a step back, I would really appreciate any suggestions on how to approach this, because I suspect that my challenge might be simpler that the general problem of "AEC in a video call with a remote participant".

What I'm trying to make happen involves no remote participant, and no microphone feedback loop. Basically imagine you're at your laptop, watching a YouTube video, while also recording a voice note. The issue is that if the user isn't wearing headphones, the youtube audio coming out of the laptop speakers bleeds into the microphone, so I'm double recording it. I have access to these two separate audio streams: the pure system audio (so, just the youtube video) and the microphone, and I think my challenge is essentially just cancelling out the system audio from the microphone stream when necessary.

Is there any library that could do this for me (by feeding it these two audio streams and asking it to cancel A from B), so I don't have to roll my own audio processing code? Can I perhaps leverage the WebRTC AEC (3?) code, if it's not too tightly coupled with the Chromium instance?

Thanks again!

[V I]

You don’t need Chrome/Chromium/Electron for it. You can either use libwebrtc or even extract AEC3 out of it - I did it before and even though it wasn’t the most straightforward process, it worked fine

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/0375874f-38e3-4db3-b834-b4d9314b228an%40googlegroups.com.

[Arun Raghavan]

On Wed, 28 Jun 2023, at 5:35 AM, Andrei Gheorghe wrote:
[...]

> Is there any library that could do this for me (by feeding it these two
> audio streams and asking it to cancel A from B), so I don't have to
> roll my own audio processing code? Can I perhaps leverage the WebRTC
> AEC (3?) code, if it's not too tightly coupled with the Chromium
> instance?

There's this (pending an update to a more recent version of the code, but it works pretty well): https://gitlab.freedesktop.org/pulseaudio/webrtc-audio-processing/

Cheers,
Arun