getUserMedia producing out of bounds audio values? What to do?

71 views
Skip to first unread message

Ben Wiley

unread,
Sep 10, 2021, 3:53:15 AMSep 10
to discuss-webrtc
I'm building an application that processes/saves audio from getUserMedia(). I'm noticing that in Chrome, the PCM data read from a ScriptProcessorNode occasionally contains values greater than 1 or less than -1, which is contrary to the spec I believe. I have filed a bug with Chromium (https://bugs.chromium.org/p/chromium/issues/detail?id=1247921&q=component%3ABlink%3EGetUserMedia&can=2), but in the meantime, I'm wondering what I should do for this type of situation. Should I roll all the too-large values down to 1? Should I scale all the PCM values together so the max is 1? Leave it alone? AudioBuffer seems to be happy holding on to out-of-bounds values but I'm not sure exactly what the effect is, or why these values could be out of range.

Thanks

guest271314

unread,
Sep 10, 2021, 9:24:05 PMSep 10
to discuss-webrtc
I have not yet attempted to reproduce the bug described. I have not tried ScriptProcessorNode (which is deprecated). I have tested output of AudioWorklet and MediaStreamTrackProcessor from raw PCM streams multiple times and have not encountered the result described.

Ben Wiley

unread,
Sep 14, 2021, 8:34:00 AM (11 days ago) Sep 14
to discuss-webrtc
Note that I just found a way to implement the recorder with AudioWorkletNode, but I got the same issue as with ScriptProcessorNode. I will try to work on a minimal repro case. Sort of hard since it depends on input from the user.

guest271314

unread,
Sep 14, 2021, 10:05:54 AM (11 days ago) Sep 14
to discuss-webrtc
For others to reproduce exactly what you observe you can record the input you believe produces the bug, then set that recording as the source for getUserMedia()

chrome --use-fake-device-for-media-stream --use-fake-ui-for-media-stream --use-file-for-fake-audio-capture=$HOME/output.wav%%noloop

Ben Wiley

unread,
Sep 17, 2021, 12:10:47 PM (8 days ago) Sep 17
to discuss-webrtc
Here is a repro case. The source is on github and it's deployed to github pages.

You can select either ScriptProcessorNode or AudioWorkletNode and start recording with microphone input. The UI will show the top 10 samples with the greatest magnitude during the recording. I don't have a super scientific method for reproducing this, but you can try to make a really large amount of noise next to your microphone and you can see a value greater than 1 or less than -1. I was able to reproduce this with Chrome and Firefox on Ubuntu Linux 18.04, and Chrome on macOS Mojave.

guest271314

unread,
Sep 17, 2021, 7:23:53 PM (8 days ago) Sep 17
to discuss-webrtc
If you can observe the values then you can 1) store the values for others to view; 2) record the input with parec or arecord.

Check the volume of the microphone you use.

Ben Wiley

unread,
Sep 21, 2021, 11:13:25 AM (4 days ago) Sep 21
to discuss-webrtc
Sorry I'm not sure I understand what you're saying. I'm recording in the browser so I don't see how those utilities would be options for me.

It's sure that I only reproduce this when the microphone volume is at a high level, but I'm not sure what I'm expected to do with values above 1. If I understood better how these out of bounds values are generated and also interpreted by the playback engine, it would help me a lot.

On the recording end:
- A: Are these valid peak values, and the maximum recordable amplitude is misrepresented as 1 but it's actually closer to 1.25?
- B: or, are these values recorded by error and represent invalid audio information?

On the playback end:
- C: Will the playback engine find a way to represent a greater-than-1 value through audio playback without clipping?
- D: or, will the value just be clipped to 1 or -1?

If A and C, then I will just leave everything as-is, I guess, and scale my visual representation to match the peak value
If A and D, I will scale the values so the peak is at 1 or -1 before playback
And otherwise I will probably just clamp the extreme values to -1 or 1

Does this make sense? Unfortunately I just don't understand enough about the intricacies of the browser's audio capture and playback.

Ben

Philipp Hancke

unread,
Sep 21, 2021, 11:23:29 AM (4 days ago) Sep 21
to discuss...@googlegroups.com
sounds a bit like autogaincontrol is leading to overshooting. You might want to check if this goes away if you disable it (see https://stackoverflow.com/a/37332145/4352772 for the getUserMedia constraint)

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/964b4ec8-630a-48f2-b903-976a1f7f6456n%40googlegroups.com.

guest271314

unread,
Sep 21, 2021, 7:43:32 PM (4 days ago) Sep 21
to discuss-webrtc
What I am saying is whatever you are observing locally can be reproduced by others externally, for the ability to concretely suggest solutions - once what you observe is reproduced by someone other than you.

If you are screaming into your microphone you can record that input as a WAV file then supply the file to getUserMedia() for testing.

My understanding is the values should not exceed 1 or -1. 

You can round down to 1 or round up to -1, if that is important.

Ben Wiley

unread,
Sep 21, 2021, 8:40:26 PM (4 days ago) Sep 21
to discuss-webrtc
Wow you were super close, and the StackOverflow answer helped! This bug disappears if I set echoCancellation: false on the audio constraints.

Ben Wiley

unread,
Sep 21, 2021, 10:25:35 PM (4 days ago) Sep 21
to discuss-webrtc
Well that solves it in many cases, but I still consistently get a huge (out of bounds) gain jump when clicking the trackpad on my Linux laptop.
Reply all
Reply to author
Forward
0 new messages