Multichannel causes Audio Underflow

[Jason Levine]

Hey Xtm,

In preparation for my upcoming performance at MIT, I've been pushing Extempore harder than usual and getting constant buffer underruns, including some big ones where sound cuts out altogether.  I've been trying to pinpoint the cause, and today I did a test and found that when outputting stereo, Extempore's CPU usage tops out at 99% and I get no underruns. However, using my current setup of 20 outputs,  I am limited to how many sounds I can play at once, and Extempore's CPU usage can exceed 200%.  Does this make sense? Is the number of channels linearly related to CPU usage?  Or is something else going on?

Much thanks,  
--

Jason Levine

new media performer + creative coder

https://www.behance.net/jasonlevine

[Toby Gifford]

are you using the multicore dsp stuff? That can help.

--
You received this message because you are subscribed to the Google Groups "Extempore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to extemporelang+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jason Levine]

good call! I will try to incorporate that.  Although I must say, Extempore is happy to use at least two cores without me doing anything special.

[Ross Bencina]

Jason,

How are you getting these CPU% numbers? And on which platform are you
running?

Am I correct to assume that the difference between stereo and 20-channel
is only where you route your signals, not how much compute you are doing?

Depending on how it's built, PortAudio float->int conversion may be
responsible for some overhead when running 20 channels, probably nowhere
near enough to account for the 100% difference you're seeing, but maybe
there's something there.

Ross

> _https://www.behance.net/jasonlevine_

>
> --
> You received this message because you are subscribed to the Google
> Groups "Extempore" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to extemporelan...@googlegroups.com
> <mailto:extemporelan...@googlegroups.com>.

[Jason Levine]

Hi Ross,

I'm on MacOS 10.13.3 and I am getting the CPU numbers from Activity Monitor.  My machine is a 2012 MacBook Pro with a hyperthreaded quad core processor.  So 8 virtual cores.  So 100% is one virtual core and 200% is 2 virtual cores and presumably 1 physical core.  The only other difference between the stereo code and the 10 stereo channel code is that, in the stereo code, I have one sampler loaded with 1000 samples, whereas in the multichannel one the samples are split between 10 samplers and the code looks like this:

Setup samplers

(bind-sampler tSNEsampler0 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler1 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler2 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler3 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler4 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler5 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler6 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler7 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler8 sampler_note_hermite_c sampler_fx)

(bind-sampler tSNEsampler9 sampler_note_hermite_c sampler_fx)

From the play function:

  (cond

                ((= (get_type index) 0) (play tSNEsampler0 sample vel dur bank pan))

                ((= (get_type index) 1) (play tSNEsampler1 sample vel dur bank pan))

                ((= (get_type index) 2) (play tSNEsampler2 sample vel dur bank pan))

                ((= (get_type index) 3) (play tSNEsampler3 sample vel dur bank pan))

                ((= (get_type index) 4) (play tSNEsampler4 sample vel dur bank pan))

                ((= (get_type index) 5) (play tSNEsampler5 sample vel dur bank pan))

                ((= (get_type index) 6) (play tSNEsampler6 sample vel dur bank pan))

                ((= (get_type index) 7) (play tSNEsampler7 sample vel dur bank pan))

                ((= (get_type index) 8) (play tSNEsampler8 sample vel dur bank pan))

                ((= (get_type index) 9) (play tSNEsampler9 sample vel dur bank pan))

                    (else 0))

DSP

(bind-func dsp:DSP

  (lambda (in time chan dat)

  (cond ((< chan 2)  ;; stereo bus 1

         (tSNEsampler0 in time (- chan 0) dat))

        ((< chan 4)  ;; stereo bus 2

         (tSNEsampler1 in time (- chan 2) dat))

        ((< chan 6)  ;; stereo bus 3

         (tSNEsampler2 in time (- chan 4) dat))

        ((< chan 8)  ;; stereo bus 4

         (tSNEsampler3 in time (- chan 6) dat))

        ((< chan 10)  ;; stereo bus 4

         (tSNEsampler4 in time (- chan 8) dat))

        ((< chan 12)  ;; stereo bus 4

         (tSNEsampler5 in time (- chan 10) dat))

        ((< chan 14)  ;; stereo bus 4

         (tSNEsampler6 in time (- chan 12) dat))

        ((< chan 16)  ;; stereo bus 4

         (tSNEsampler7 in time (- chan 14) dat))

        ((< chan 18)  ;; stereo bus 4

         (tSNEsampler8 in time (- chan 16) dat))

        ((< chan 20)  ;; stereo bus 4

         (tSNEsampler9 in time (- chan 18) dat))

        (else 0.0))))

an email to extemporelang+unsubscribe@googlegroups.com
<mailto:extemporelang+unsubscri...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Extempore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to extemporelang+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jason Levine

new media performer + creative coder

https://www.behance.net/jasonlevine

[Andrew Sorensen]

Jason, 

Extempore utilizes multiple cores for a variety of tasks - but by default audio dsp only utilizes a single core.

As Toby suggested, dsp:set! makes it very easy to add multicore audio dsp support.  See examples/core/mtaudio.xtm for a trivial example that could be easily extended to your case (perhaps 3 or 4 samplers per core - but experiment to taste).

Each instrument instantiation incurs some overhead.  (10 instruments playing 1 note) != (1 instrument playing 10 notes).

A small amount of overhead is involved in instrument "accounting" (scheduling "notes" etc..), but the most significant overhead comes from the instruments FX.  

For your particular case (i.e. sampler), this means a Reverb per sampler (i.e. sampler_fx includes a reverb).  Note that this reverb is always processing, regardless of the number of notes being performed.  So if you instantiate 10 samplers, you'll have 10 active reverbs chewing up CPU (even with no notes sounding).  You can of course write, and use, an FX closure that is a simple pass-through for minimal overhead.  If you *only* did this then you'd still notice a significant cpu drop.  Of course the downside to this is - no reverb!

Of course, an alternative is to wrap all of the 10 * sampler output in a single reverb.  Then you've gone from 10 * reverb to 1 * reverb.  If that suites your particular circumstances (i.e. all samplers sharing a 'global' reverb) then that's a good option for you.  Note that this also works very well in the multicore case and is already demonstrated in the example I shared above.  

However, note that if you do go with the 'shared reverb' approach, you still need to deactivate all the individual instrument reverbs - otherwise you'll go from 10 reverbs to 11!

So, your ideal solution will be to use dspmt with a shared 'global' reverb.

Andrew.

 

[Jason Levine]

Hey Andrew,

Thanks for the tips.  First of all, a quick update.  I a/b tested multichannel vs. stereo almost 10 times back to back and the real difference is max 99% for stereo while multichannel falls somewhere between 130% and 150% with disastrous audio dropouts as you approach 150%.  I wasn't able to reproduce the 200% I mentioned in my original post.

So I removed the reverb, and now the multichannel setup tops out at 125%.  Extempore still warns of Audio Underflow now and then, but not constantly like before and with no audible dropouts or crunchiness.  I also tried setting up multicore DSP but failed and got sporadic loud zipper noises on all 10 tracks and none of the sounds I was actually triggering.

I'm guessing it has something to do with the way I convolved your multichannel example with the multicore example, because the multicore example works fine.

Here is my multicore dsp closure:

bind-func dsp:DSP

  (lambda (in time chan dat)

  (cond ((< chan 2)  ;; stereo bus 1

         (tSNEsampler0 in time (- chan 0) dat))

        ((< chan 4)  ;; stereo bus 2

         (tSNEsampler1 in time (- chan 2) dat))

        ((< chan 6)  ;; stereo bus 3

         (tSNEsampler2 in time (- chan 4) dat))

        ((< chan 8)  ;; stereo bus 4

         (tSNEsampler3 in time (- chan 6) dat))

        ((< chan 10)  ;; stereo bus 4

         (tSNEsampler4 in time (- chan 8) dat))

        ((< chan 12)  ;; stereo bus 4

         (tSNEsampler5 in time (- chan 10) dat))

        ((< chan 14)  ;; stereo bus 4

         (tSNEsampler6 in time (- chan 12) dat))

        ((< chan 16)  ;; stereo bus 4

         (tSNEsampler7 in time (- chan 14) dat))

        ((< chan 18)  ;; stereo bus 4

         (tSNEsampler8 in time (- chan 16) dat))

        ((< chan 20)  ;; stereo bus 4

         (tSNEsampler9 in time (- chan 18) dat))

        (else 0.0))))

(dsp:set! dsp)

and here is the multicore version which doesn't work

(bind-func dsp1:DSP

  (lambda (in time chan dat)

  (cond ((< chan 2)  ;; stereo bus 1

         (tSNEsampler0 in time (- chan 0) dat))

        ((< chan 4)  ;; stereo bus 2

         (tSNEsampler1 in time (- chan 2) dat))

        ((< chan 6)  ;; stereo bus 3

         (tSNEsampler2 in time (- chan 4) dat))

        ((< chan 8)  ;; stereo bus 4

         (tSNEsampler3 in time (- chan 6) dat))

        (else 0.0))))

(bind-func dsp2:DSP

  (lambda (in time chan dat)

  (cond ((< chan 10)  ;; stereo bus 4

         (tSNEsampler4 in time (- chan 8) dat))

        ((< chan 12)  ;; stereo bus 4

         (tSNEsampler5 in time (- chan 10) dat))

        ((< chan 14)  ;; stereo bus 4

         (tSNEsampler6 in time (- chan 12) dat))

        (else 0.0))))

(bind-func dsp3:DSP

  (lambda (in time chan dat)

  (cond ((< chan 16)  ;; stereo bus 4

         (tSNEsampler7 in time (- chan 14) dat))

        ((< chan 18)  ;; stereo bus 4

         (tSNEsampler8 in time (- chan 16) dat))

        ((< chan 20)  ;; stereo bus 4

         (tSNEsampler9 in time (- chan 18) dat))

        (else 0.0))))

(bind-func dspmt:DSPMT 1000000

  (lambda (in time chan dat)

    (+ (* 0.5 (pref in 0))

      (* 0.5 (pref in 1))

      (* 0.5 (pref in 2)))

      0.5 0.5))

(dsp:set! dspmt dsp1 dsp2 dsp3)

Thanks again, and let me know what you think,

Jason

[Andrew Sorensen]

Hint: ((< chan ???) 0.0)

[Jason Levine]

OK, so I updated my dsps as follows:

(bind-func dsp1:DSP

  (lambda (in time chan dat)

  (cond ((< chan 2)  ;; stereo bus 1

         (tSNEsampler0 in time (- chan 0) dat))

        ((< chan 4)  ;; stereo bus 2

         (tSNEsampler1 in time (- chan 2) dat))

        ((< chan 6)  ;; stereo bus 3

         (tSNEsampler2 in time (- chan 4) dat))

        ((< chan 8)  ;; stereo bus 4

         (tSNEsampler3 in time (- chan 6) dat))

        (else 0.0))))

(bind-func dsp2:DSP

  (lambda (in time chan dat)

  (cond ((< chan 8) 0.0)

        ((< chan 10)  ;; stereo bus 5

         (tSNEsampler4 in time (- chan 8) dat))

        ((< chan 12)  ;; stereo bus 6

         (tSNEsampler5 in time (- chan 10) dat))

        ((< chan 14)  ;; stereo bus 7

         (tSNEsampler6 in time (- chan 12) dat))

        (else 0.0))))

(bind-func dsp3:DSP

  (lambda (in time chan dat)

  (cond ((< chan 14) 0.0)

        ((< chan 16)  ;; stereo bus 8

         (tSNEsampler7 in time (- chan 14) dat))

        ((< chan 18)  ;; stereo bus 9

         (tSNEsampler8 in time (- chan 16) dat))

        ((< chan 20)  ;; stereo bus 10

         (tSNEsampler9 in time (- chan 18) dat))

        (else 0.0))))

(bind-func dspmt:DSPMT 1000000

  (lambda (in time chan dat)

    (+ (* 0.5 (pref in 0))

      (* 0.5 (pref in 1))

      (* 0.5 (pref in 2)))

      0.5 0.5))

(dsp:set! dspmt dsp1 dsp2 dsp3)

And now I just get an endless stream of:

!Locked with threads:13 of 3 cnt(6)

!Still locked in 2 cnt(6:6)

!Still locked in 1 cnt(6:6)

!Still locked in 0 cnt(6:6)

!Locked with threads:13 of 3 cnt(6)

!Still locked in 2 cnt(6:6)

!Still locked in 1 cnt(6:6)

!Still locked in 0 cnt(6:6)

!Locked with threads:13 of 3 cnt(6)

!Still locked in 2 cnt(6:6)

!Still locked in 1 cnt(6:6)

!Still locked in 0 cnt(6:6)

!Locked with threads:13 of 3 cnt(6)

!Still locked in 2 cnt(6:6)

!Still locked in 1 cnt(6:6)

!Still locked in 0 cnt(6:6)

!Locked with threads:13 of 3 cnt(6)

!Still locked in 2 cnt(6:6)

!Still locked in 1 cnt(6:6)

!Still locked in 0 cnt(6:6)

!Locked with threads:13 of 3 cnt(6)

Are they competing for channels or something?

[Andrew Sorensen]

try this:

(sys:load "libs/external/instruments_ext.xtm")

(make-instrument sampler0 sampler)

(make-instrument sampler1 sampler)

(make-instrument sampler2 sampler)

(make-instrument sampler3 sampler)

(make-instrument sampler4 sampler)

(make-instrument sampler5 sampler)

(make-instrument sampler6 sampler)

(make-instrument sampler7 sampler)

(make-instrument sampler8 sampler)

(make-instrument sampler9 sampler)

;; turn off reverb (< 0.0001 == no processing)

(let ((amt 0.0))

    (set_reverb_mix sampler0 amt)

    (set_reverb_mix sampler1 amt)

    (set_reverb_mix sampler2 amt)

    (set_reverb_mix sampler3 amt)

    (set_reverb_mix sampler4 amt)

    (set_reverb_mix sampler5 amt)

    (set_reverb_mix sampler6 amt)

    (set_reverb_mix sampler7 amt)

    (set_reverb_mix sampler8 amt)

    (set_reverb_mix sampler9 amt))

;; andrew specific samples

(load-sampler sampler0 "D:/Documents/Samples/piano/")

(load-sampler sampler1 "D:/Documents/Samples/mk2/")

(load-sampler sampler2 "D:/Documents/Samples/greenfairy/")

(load-sampler sampler3 "D:/Documents/Samples/soprano/")

(load-sampler sampler4 "D:/Documents/Samples/tenor/")

(load-sampler sampler5 "D:/Documents/Samples/accordian/")

(load-sampler sampler6 "D:/Documents/Samples/vibes/")

(load-sampler sampler7 "D:/Documents/Samples/vrtel/")

(load-sampler sampler8 "D:/Documents/Samples/orgami/")

(load-sampler sampler9 "D:/Documents/Samples/kit/")

(bind-func dsp1:DSP

  (lambda (in time chan dat)

  (cond ((< chan 2)  ;; stereo bus 1

         (sampler0 in time (- chan 0) dat))

        ((< chan 4)  ;; stereo bus 2

         (sampler1 in time (- chan 2) dat))

        (else 0.0))))

(bind-func dsp2:DSP

  (lambda (in time chan dat)

  (cond ((< chan 4) 0.0)

        ((< chan 6)  ;; stereo bus 3

         (sampler2 in time (- chan 4) dat))

        ((< chan 8)  ;; stereo bus 4

         (sampler3 in time (- chan 6) dat))

        (else 0.0))))

(bind-func dsp3:DSP

  (lambda (in time chan dat)

  (cond ((< chan 8) 0.0)

        ((< chan 10)  ;; stereo bus 5

         (sampler4 in time (- chan 8) dat))

        ((< chan 12)  ;; stereo bus 6

         (sampler5 in time (- chan 10) dat))

        (else 0.0))))

(bind-func dsp4:DSP

  (lambda (in time chan dat)

  (cond ((< chan 12) 0.0)

        ((< chan 14)  ;; stereo bus 7

         (sampler6 in time (- chan 12) dat))

        ((< chan 16)  ;; stereo bus 8

         (sampler7 in time (- chan 14) dat))

        (else 0.0))))

(bind-func dsp5:DSP

  (lambda (in time chan dat)

  (cond ((< chan 16) 0.0)

        ((< chan 18)  ;; stereo bus 9

         (sampler8 in time (- chan 16) dat))

        ((< chan 20)  ;; stereo bus 10

         (sampler9 in time (- chan 18) dat))

        (else 0.0))))

(bind-func dspmt:DSPMT

 (lambda (in time chan dat)

   (+ (* 1.0 (pref in 0))

      (* 1.0 (pref in 1))

      (* 1.0 (pref in 2))

      (* 1.0 (pref in 3))

      (* 1.0 (pref in 4)))))

;; note #f

(dsp:set! #f dspmt dsp1 dsp2 dsp3 dsp4 dsp5)

(define test

  (lambda (beat dur)

    (let ((d 1.0))

        (play sampler0 (random 48 72) 80 (* d dur))

        (play sampler1 (random 48 72) 80 (* d dur))

        (play sampler2 (random 48 72) 80 (* d dur))

        (play sampler3 (random 48 72) 80 (* d dur))

        (play sampler4 (random 48 72) 80 (* d dur))

        (play sampler5 (random 48 72) 80 (* d dur))

        (play sampler6 (random 48 72) 80 (* d dur))

        (play sampler7 (random 48 72) 80 (* d dur))

        (play sampler8 (random 48 72) 80 (* d dur))

        (play sampler9 (random 48 72) 80 (* d dur)))

      (callback (*metro* (+ beat (* .5 dur))) 'test (+ beat dur) dur)))

 

(test (*metro* 'get-beat 4) 1/4)

[Toby Gifford]

What is the first argument to dsp:set! there?  As far as I can decipher from scheme.xtm it is the zerolatency? variable, which i'm guessing determines whether the dsp kernel is single-frame based (zerolatency? = #t) or hardware buffer based (zerolatency? = #f).

examples/core/mtaudio.xtm doesn't give any hint. I'm happy to update the example if i've got this right. Though, its not immediately obvious to me why buffer-based processing would solve thread contention.

[Jason Levine]

Ok so you're example works (and the example is some A+ dodecaphonic musical comedy)

But the only major difference between what I had and this example seems to be the #f param after (dsp:set!  

So I tried adding the #f param and as soon as I eval (dsp:set! ...) I hear a loud pop, all the levels shoot up to peak and then I can't play any sound.  The levels remain at peak until I quit Extempore at which point there is another loud pop and the levels drop back down.  So what is the real difference? The number of samplers per thread?

So then I copied the dsp part of your example into my code (and renamed the samplers) and now I have the thread locking again

Locked with threads:12 of 5 cnt(3)

!Still locked in 3 cnt(3:3)

!Still locked in 1 cnt(3:3)

!Still locked in 0 cnt(3:3)

!Still locked in 4 cnt(3:3)

!Still locked in 2 cnt(3:3)

But then I quit and tried it again, and it worked!  I tested it 5 more times, and it didn't thread locked until the 6th time

Any idea why I might sometimes get thread locking? 

When it does work, there are no more  'Are you pushing Extempore too hard' warnings, despite multiple highspeed TRs at 180bpm,  but I do get a pretty big drop out often when I eval at TR or a (*metro* 'set-tempo ??)

Does that make any sense? It sort of reminds me of when you eval an opengl TR and every freezes for a moment

[Andrew Sorensen]

First arg indicates whether or not the summing (i.e. dspmt) closure runs in the same frame as the dsp 'cores' or in a following frame - i.e. one frame delayed.

The advantage with running delayed (i.e. #f) is that you don't need to wait for the 'cores' before processing the 'sum'.  The downside being 1 frame latency.

Cheers,

Andrew.

[Andrew Sorensen]

Sorry Jason, there is a race there that I can't reproduce - not sure exactly why you're reproducing it so consistently :( I have a pretty good idea of what's going wrong but I need to try to reproduce it to make sure i have a good fix.

Hopefully I'll find some time to look at it tomorrow.

Also note that your original dspmt is returning 0.5 - i.e a constant, which will definitely be causing you trouble!

[Andrew Sorensen]

Jason,

I've pushed a fix for this.  You'll need to build from head.

Let me know how you go.

Cheers,

Andrew.

[Jason Levine]

Thanks Andrew,

The performance is tomorrow evening, so I've decided to stick with a single core but no reverbs (i have some reverb in the DAW), and continue refining the multichannel setup when I get back from MIT.