getUserMedia producing out of bounds audio values? What to do?

[Ben Wiley]

I'm building an application that processes/saves audio from getUserMedia(). I'm noticing that in Chrome, the PCM data read from a ScriptProcessorNode occasionally contains values greater than 1 or less than -1, which is contrary to the spec I believe. I have filed a bug with Chromium (https://bugs.chromium.org/p/chromium/issues/detail?id=1247921&q=component%3ABlink%3EGetUserMedia&can=2), but in the meantime, I'm wondering what I should do for this type of situation. Should I roll all the too-large values down to 1? Should I scale all the PCM values together so the max is 1? Leave it alone? AudioBuffer seems to be happy holding on to out-of-bounds values but I'm not sure exactly what the effect is, or why these values could be out of range.

Thanks

[guest271314]

I have not yet attempted to reproduce the bug described. I have not tried ScriptProcessorNode (which is deprecated). I have tested output of AudioWorklet and MediaStreamTrackProcessor from raw PCM streams multiple times and have not encountered the result described.

[Ben Wiley]

Note that I just found a way to implement the recorder with AudioWorkletNode, but I got the same issue as with ScriptProcessorNode. I will try to work on a minimal repro case. Sort of hard since it depends on input from the user.

[guest271314]

For others to reproduce exactly what you observe you can record the input you believe produces the bug, then set that recording as the source for getUserMedia()

chrome --use-fake-device-for-media-stream --use-fake-ui-for-media-stream --use-file-for-fake-audio-capture=$HOME/output.wav%%noloop

[Ben Wiley]

Here is a repro case. The source is on github and it's deployed to github pages.

https://github.com/benwiley4000/web-media-record-out-of-bounds-repro

https://benwiley4000.github.io/web-media-record-out-of-bounds-repro/

You can select either ScriptProcessorNode or AudioWorkletNode and start recording with microphone input. The UI will show the top 10 samples with the greatest magnitude during the recording. I don't have a super scientific method for reproducing this, but you can try to make a really large amount of noise next to your microphone and you can see a value greater than 1 or less than -1. I was able to reproduce this with Chrome and Firefox on Ubuntu Linux 18.04, and Chrome on macOS Mojave.

[guest271314]

If you can observe the values then you can 1) store the values for others to view; 2) record the input with parec or arecord.

Check the volume of the microphone you use.

[Ben Wiley]

Sorry I'm not sure I understand what you're saying. I'm recording in the browser so I don't see how those utilities would be options for me.

It's sure that I only reproduce this when the microphone volume is at a high level, but I'm not sure what I'm expected to do with values above 1. If I understood better how these out of bounds values are generated and also interpreted by the playback engine, it would help me a lot.

On the recording end:

- A: Are these valid peak values, and the maximum recordable amplitude is misrepresented as 1 but it's actually closer to 1.25?

- B: or, are these values recorded by error and represent invalid audio information?

On the playback end:

- C: Will the playback engine find a way to represent a greater-than-1 value through audio playback without clipping?

- D: or, will the value just be clipped to 1 or -1?

If A and C, then I will just leave everything as-is, I guess, and scale my visual representation to match the peak value

If A and D, I will scale the values so the peak is at 1 or -1 before playback

And otherwise I will probably just clamp the extreme values to -1 or 1

Does this make sense? Unfortunately I just don't understand enough about the intricacies of the browser's audio capture and playback.

Ben

[Philipp Hancke]

sounds a bit like autogaincontrol is leading to overshooting. You might want to check if this goes away if you disable it (see https://stackoverflow.com/a/37332145/4352772 for the getUserMedia constraint)

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/964b4ec8-630a-48f2-b903-976a1f7f6456n%40googlegroups.com.

[guest271314]

What I am saying is whatever you are observing locally can be reproduced by others externally, for the ability to concretely suggest solutions - once what you observe is reproduced by someone other than you.

If you are screaming into your microphone you can record that input as a WAV file then supply the file to getUserMedia() for testing.

My understanding is the values should not exceed 1 or -1. 

You can round down to 1 or round up to -1, if that is important.

[Ben Wiley]

Wow you were super close, and the StackOverflow answer helped! This bug disappears if I set echoCancellation: false on the audio constraints.

[Ben Wiley]

Well that solves it in many cases, but I still consistently get a huge (out of bounds) gain jump when clicking the trackpad on my Linux laptop.