Intent to Implement: disableHwNoiseCanceller constraint

[os...@chromium.org]

Contact emails

os...@chromium.org

Explainer

https://docs.google.com/document/d/e/2PACX-1vRYEUohUdK-c_l0W8JyyAoYC7luae6fm498OpVHF6wtdVoRUkK57Lzakcb2fJFTSo2awzCFdPlL7a9s/pub

Summary

Provide a constraint that allows getUserMedia callers to disable hardware noise suppression. Primarily, this allows WebRTC to rely on its internal noise suppressor, which should have a less deleterious effect on echo suppression and audio quality. Outside of WebRTC, it will allow users to get unprocessed audio into their pipelines. This constraint will be put through an Origin Trial.

Motivation

Hardware noise suppression, such as that provided by macOS on its built-in microphone, while useful for simpler tasks, gets in the way of the processing performed by communications software, such as WebRTC, and other audio pipelines which expect unprocessed audio as input.

Risks

Interoperability and Compatibility

API-wise, the change only adds a single, optional constraint to MediaTrackConstraintSet, so the risks are inherently low. As noise suppression is currently user-controllable but not web-application controllable, applications are already expected to handle both states gracefully.

Ergonomics

Don’t foresee this interacting with other APIs, outside of, possibly, other MediaTrack constraints.

Activation

Very simple to use.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Although the constraint will be available on all platforms, the underlying functionality will initially only be implemented on macOS.

Link to entry on the feature dashboard

The change itself is very small and shouldn’t require a feature for itself. There are already a couple of features pertaining to the implementation of media constraints.

Requesting approval to ship?

No

[Johnny Stenback]

If this is really only something that OSX does then should this be exposed to the web, or could WebRTC simply just disable the HW noise suppressor when it talks to the HW, period? Is there ever a case where WebRTC wants the HW suppressor enabled?

Cheers,

Johnny

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/7bedab1a-0b90-4120-9b32-fff264e48e7d%40chromium.org.

[Harald Alvestrand]

Please file a spec bug against the "Media capture and streams" spec. The spec bug (preferably a PR) should also include an explanation of how this interrelates with the "noiseSuppression" constraint.

https://w3c.github.io/mediacapture-main/getusermedia.html#def-constraint-noiseSuppression

Since this is an experiment, I assume that 1) we'll be able to ship the experiment before landing the spec change, and 2) that if the spec discussion results in a change of the name, that will be reflected in the implementation.

(Personally I detest abbreviations in names, and constraints that start with "disable"; I'd much prefer to have a constraint named hardwareNoiseSuppression that our implementation can default to true.)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[os...@chromium.org]

Hmm... I must've missed the noiseSuppression constraint - possibly from reading the source but failing to read the specs. :/

I believe what I'm trying to do would fit just as well, if not better, as an implementation of the noiseSuppression constraint; with it defaulting to, as you propose, true and setting it to false will disable any native noise suppression that we have control over. In my mind, activating echoCancellation may add (software) noise suppression as part of that processing, and setting echoCancellation: true, noiseSuppression: false would leave that software noise suppression, but disable native noise suppression.

This, of course, falls down if the currently vendor-specific googNoiseSuppression (which controls software noise suppression) is supposed to be upgraded to the official noiseSuppression. I see the constraint was added in February this year, but none of the reasoning behind its addition. Do you know this, or perhaps where I can find out?

I guess another way forward is to let the OriginTrial itself control this behavior, without altering the API surface: when enabling the trial, echoSuppression: true will disable native noise suppression - also setting noiseSuppression: false would disable software noise suppression as well; finally, setting both to false would disable software signal processing as well as any hardware noise suppression.

WRT to being macOS-specific: other platforms do this as well, but for the initial experiment (which I'm hoping to start by M64), we'll be targeting macOS specifically. We aim to address this on other platforms as well, eventually - wherever audio is processed before getting into our audio pipeline.

[PhistucK]

Then perhaps, "hardwareProcessing", instead? There is more than just noise suppression that processes the audio, I believe, like boosting, virtual surround, room correction, loudness equalization, echo cancellation, DC offset cancellation...

https://www.google.com/search?q=microphone+enhancements&source=lnms&tbm=isch

☆PhistucK

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5b1ac394-decb-4f57-9248-4b4625198ca9%40chromium.org.

[Harald Alvestrand]

We're trying to get rid of the goog constraints as part of the Web-exposed API. The noiseSuppression constraint was added because the standards committee was convinced that being able to control noise suppression was useful.

The classic case for disabling noise suppression is the "911" case - where one end of the call wants to make sure he can hear anything going on at the other side, even when the sounds one is listening for are very weak. It would be surprising (I think) for someone who has disabled noise suppression for that reason to see it turned on because echo cancellation was turned on.

So I'd prefer (I think) to see a new constraint named "hardwareNoiseSuppression" for controlling *which* noise suppression is turned on or off, with "noiseSuppression=false" being reserved for turning off (as far as possible) *all* noise suppression.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5b1ac394-decb-4f57-9248-4b4625198ca9%40chromium.org.

[Harald Alvestrand]

We have another proposal for turning off all processing:

https://github.com/w3c/mediacapture-main/pull/493

(the PR was closed because the submitter was unable to make the IPR commitments required by W3C rules at this time. We hope to have it reopened when that situation changes.)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_%2Bho9TRyg_MXfa8MVMob3TVFtgwXW9spKmQ7wUSoFKqKA%40mail.gmail.com.

[os...@chromium.org]

It sounds like googNoiseSuppression -> noiseSuppression and that both would, if supported, imply that native noise suppression were turned off as well.

I wonder if adding a specific "hardwareNoiseSuppression" constraint under these assumptions is perhaps adding a bit too much detailed control to the user. Realistically, they'd either want noise suppression or not, regardless where it's from.

After this discussion, I believe the best way forward right now would be to add an Origin Trial which changes the behavior of echoCancellation to always disable native noise suppression if enabled.

This can then be followed by one where (goog)NoiseSuppression also has this updated behavior, and the noiseSuppression constraint is properly implemented.

This aligns with how we imagine the API would look after the trial. The new constraint is proposed as a way to control this change in behavior from the application side - though I believe just selectively enabling the trial should be enough for what we want.

If there are no major objections to this, I'll go ahead and re-write the explainer, as well as update my CL with this behavior. As we'd really like to start evaluating this functionality in M64, I'd like to do the echoCanceller behavior changes as a separate step first and based on the evaluation of that possibly go ahead with the (goog)noiseSuppression changes.