Recording mixed audio using the native API

[Urs D.]

Hi,

I'm trying to record (only) audio to a file, using the native webrtc.org API. 

There are StartRecordingPlayout and StartRecordingMicrophone, which work great! However, I would like to have both input (microphone) and output (speaker) mixed together.

There are also TransmitMixer, OutputMixer and AudioConferenceMixer, which sound very promising.

Can anybody give me a hint about how to combine those, to get the desired result, ideally with a good mixing quality?

And what is TransmitMixer::StartRecordingCall supposed to do exactly?  It sounds like what I want, but it calls me back just for writing the file headers, but not for actual audio data.

Urs D.

[Fredrik Solenberg]

Hi, sorry for the late reply (been OOO).

I would discourage you from using TransmitMixer, OutputMixer and AudioConferenceMixer directly. They are regarded as internal an may change without notice. Have a look at the NATIVE_API_DIRS list in PRESUBMIT.py for a list of components/directories you can rely upon.

Mixing recorded and played back audio is not as straightforward as it may seem. For one thing they can be using different sample rates, which would require either or both to be converted before summed together. We don't have support for doing that in VoiceEngine today.

Also, note that the VoEFile APIs you're using today are slated for deprecation, although there will be a reasonable alternative in place once that happens.

Depending on what you want to do, an alternative may be to use VoEAudioProcessing::StartDebugRecording(), which will generate a file in protobuf format which contains the mic and speaker signals in the same file (although not mixed). There is a tool to unpack the file, which you could possibly use or build upon. Beware though that this is not really intended to be used externally, so the state may be somewhat volatile, although the protobuf format will generally be extended rather than changed completely.

Hope that is of any help.

- Fredrik

[Urs D.]

Thanks Fredrik for these insights,

so what we ended up with is to virtually record both microphone and playout to virtual files (using OutStream), with the encoding set to PCM, and then we mix those two streams manually. Not the most elegant solution, but it works fine for now.

Urs D.

[raulf tang]

Hi Urs,

What platform did you work this on?

We are implementing the same on iOS and Android. The recorded voice from microphone is good on iOS but bad on Android(some devices noise, some devices distorted). We got the raw data direct from TransmitMixer::StartRecordingMicrophone.

Can you share some idea about what you did? Thanks in advance.

[Akshay Shah]

Can someone help me with this? https://groups.google.com/forum/#!topic/discuss-webrtc/d5bpay6s0H4