Dynamic range compression (volume normalization) on audio stream

[khaled]

hi all,

we are working on a stream solution based on Janus as SFU/MCU and coturn as our TURN servers. Currently, the setup is hosted on AWS. 

The video-source to destination is one-way, and data-channel is two-way. 

our current latency is ~400ms, which is alright for now, as the source-to-destination distance is half-the-globe.

But the challenge we are facing that on source there's usually two persons speaking, one is loud while another one is too quiet due to distance from the single mic. In the source scenario, we have very little control over the environment. However, on the receiving end, both voices are important. So, i think, we need to apply Dynamic range compression(DRC) or, volume normalization on the audio stream to make both voices reasonably loud, at least bring the difference down.

How to approach the problem? 

Does it make sense to do that on MCU (through Janus plugin?) or to do it on the client-side (browser)? If on the MCU, what impact it may have on the latency?

as a note, we are currently using VP8+PCM encoding to make the RTP stream.

any pointer or suggestion will be highly helpful.

// khaled

[Chris Ward]

On client side you could use webaudio dynamicsCompressorNode https://developer.mozilla.org/en-US/docs/Web/API/DynamicsCompressorNode. You could use this node to dynamically reduce the volume of the louder participant. And then you could add in a gain node to increase the total volume back up to a desired level (thus making the quiet speaker louder). 

-Chris

[khaled]

Thanks a lot for the pointer, Chris. 

Here the issue is that we are currently using <video> tag to playback on client browser, hence issues like syncing is managed by the built-in player. 

Is it be possible to handle (or pass-through) the audio separately through a AudioNode ? What'd be a feasible approach ?

If it helps, please note that we don't need mobile-browser support right now, only desktop ones (Chrome & Firefox only).
Thanks again.

// khaled

[ch...@inthelocus.com]

You can still pass the stream to video element, but mute the audio in the video element. Then also bring the stream into webaudio. Then output the audio through audioContext.destination, or to an audio element. I think webaudio ends up adding like 3 ms delay to the audio, fairly minor.

[khaled]

Awesome. Thanks a lot :)