What is the correct way to have multiple audio processors streaming samples to the output buffer?

[Patrick Collins]

I have been struggling trying to understand what the proper way to mix multiple audio sources together is.  I am currently calling emscripten_create_audio_context() multiple times to create multiple contexts, and then calling emscripten_start_wasm_audio_worklet_thread_async with each context..  Similar to the progrmaming example here:

https://emscripten.org/docs/api_reference/wasm_audio_worklets.html

I am basically doing that two times, one for each processor...  So each processor has its own RenderAudio callback which looks like:

EM_BOOL RenderAudioCallback(int numInputs, const AudioSampleFrame *inputs,
                      int numOutputs, AudioSampleFrame *outputs,
                      int numParams, const AudioParamFrame *params,
                      void *userData) {
  AudioPackage *audioPackage = static_cast<AudioPackage *>(userData);
  int frames = 128;

  for (int i = 0; i < numOutputs; i++) {
    audioPackage->sampleStreamable->process(outputs[i].data, frames*outputs[i].numberOfChannels);
  }

  return EM_TRUE;
}

Both of my processors have this "process" method which generates samples and puts them into the outputs[i].data buffer...

The other thing I am doing is, after connecting each worklet node to its audio context destination, I am performing a callback to js, where I take that source and create a visual analyzer for it and draw an oscilloscope.

If I just create ONE audio context for either source, and run this, everything sounds great..  But as soon as I create TWO contexts, (so two RenderAudioCallback functions are running), and from my JS, I instruct one source to start, so again, two RenderAudioCallback functions are running-- but only one of them should be receiving samples..  The visualizer for the source that should not be playing has little blips on it, as if the signal is bleeding through between the two contexts or something...

So, I am wondering, am I doing something wrong creating two contexts / two worklets?  Is there a better way to do this?

Here is a screenshot of my UI..  You can see, on the left, is contextA, which I started  playing its audio source-- and have not instructed contextB to play its audio source..  Yet, the oscilloscope on the right has this little blip in it...  And this is when it is sounding distorted and crackly and weird..

[Patrick Collins]

I figured out my problem..  I think the issue was creating the two contexts.

What I needed to do was, only create one context and then call this with each source:
    emscripten_create_wasm_audio_worklet_processor_async(audioContext, &sourceAOpts, &AudioWorkletProcessorCreated, (void *)sourceA);
    emscripten_create_wasm_audio_worklet_processor_async(audioContext, &sourceBOpts, &AudioWorkletProcessorCreated, (void *)sourceB);

And now the distortion is gone...

[Patrick Collins]

Actually there is still something wrong.. 

It appears if I call emscripten_create_wasm_audio_worklet_processor_async multiple times (with the same audio context), then later when the onClick handler fires and emscripten_resume_audio_context_sync is called, it is throwing:

RuntimeError: null function or function signature mismatch
    at emulator.wasm:0x140f1
    at emulator.wasm:0x1230d
    at emulator.wasm:0x11606
    at emulator.wasm:0x11015
    at emulator.wasm:0x10c88
    at emulator.wasm:0x1eba8
    at emulator.wasm:0x1e4c1
    at emulator.wasm:0x1e24b
    at emulator.wasm:0x4bff7
    at emulator.wasm:0x4d587

Yet audio plays...

If I change it so that only one worklet node is created (one emscripten_create_wasm_audio_worklet_processor_async call), then this error does not happen.

So I still seem to be having issues trying to have multiple worklets.

[Patrick Collins]

It seems that this the callstack sometimes has at WasmAudioWorkletProcessor.process (emulator.aw.js:1:1555) at the end, and sometimes it doesn't..  When it ends with that .aw.js file, the error is coming from:

this.callbackFunction(numInputs, inputsPtr, numOutputs, outputsPtr, numParams, paramsPtr, this.userData)

If I step through this code, this function is called multiple times (as it is producing audio), and it seems the 2nd or 3rd call throws this error, and then all calls after that seem to be fine...

Is this a bug in emscripten? 

[Sam Clegg]

You can add `-g` or `--profiling-funcs` to your build to get a symbols in your stack traces.

Once you have that, it might be worth opening a bug for this.

cheers,

sam

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/1e437c35-d293-4567-b456-bf515886af15n%40googlegroups.com.

[Patrick Collins]

ahhhh thank you!!! --profiling-funcs revealed to me, this was not a bug with enscripten, but rather, a bug with ME!!!!!!!!!!!!!!