xtlang temporal recursion question

[Tim Roberts]

So, I'm at a point where I have input and output buffers, and I'm using a frame buffer as storage between the input and output buffers.  For some reason though, the callback time needs to be half of the length of the frame buffer.  I've tried different frame sizes, different input and output buffer sizes, but the callback time still needs to be half of the frame length.

This confuses me.  Shouldn't a callback offset smaller than the number of samples being moved cause the frame read-head to move past where samples are being added to the input buffer.

The code is below.  In terms of improvement external to this issue, moving the store_frame closure to it's own process needs to be done, along with a couple of other things.

store_frame2 was trying to rule out the processing time of double writing.

(bind-val input_buffer float* 44100)

(bind-val output_buffer float* 44100)

(bind-val frame_buffer float* 1024)

(bind-val in_size i64 44100)

(bind-val out_size i64 44100)

(bind-val frame_size i64 1024)

(bind-func dsp:[float,float,i64,i64,float*]*

  (let ((in_offset_ptr:i64 0)

        (out_offset_ptr:i64 0)

        (output_sample:float 0.0))

  (lambda (in time chan dat)

     (pset! input_buffer in_offset_ptr in)                                                     ;store the current sample

     ;(pset! output_buffer out_offset_ptr (pref input_buffer in_offset_ptr)) ;copy the current sample (This was the step before frame buffering)

     (set! output_sample (pref output_buffer out_offset_ptr))                    ;set the output sample

     (set! in_offset_ptr (% (+ in_offset_ptr 1) in_size))                              ;advance the input pointer offset

     (set! out_offset_ptr (% (+ out_offset_ptr 1) out_size))                        ;advance the output pointer offset

     output_sample)))

;This one doesn't work properly (Although it would if frame_size was frame_size/2)

(bind-func store_frame

  (let ((read_head:i64 0)

        (write_head:i64 0))

    (lambda ()

      (let ((n:i64 0))

        (dotimes (n frame_size)

          (pset! frame_buffer n (pref input_buffer read_head))

          (set! read_head (% (+ read_head 1) in_size)))

        (dotimes (n frame_size)

          (pset! output_buffer write_head (pref frame_buffer n))

          (set! write_head (% (+ write_head 1) out_size)))

        (callback (+ (now) frame_size) store_frame)

        ))))

;This one does work properly

(bind-func store_frame2

  (let ((read_head:i64 0)

        (write_head:i64 0))

    (lambda ()

      (let ((n:i64 0))

        (dotimes (n frame_size)

          (pset! output_buffer write_head (pref input_buffer read_head))

          (set! read_head (% (+ read_head 1) in_size))

          (set! write_head (% (+ write_head 1) out_size)))

        (callback (+ (now) 512) store_frame2)

        ))))

[Tim Roberts]

I should also say that the callback delay just needs to be less that half of the frame size.

[Toby Gifford]

Hi Tim. Probably your soundcard is stereo? The DSP (at least the one with the prototype you are using) function gets called for every sample, which means twice for every frame (if you have 2 output channels).

--
You received this message because you are subscribed to the Google Groups "Extempore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to extemporelan...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Roberts]

Yes it is stereo, but if I'm creating a buffer of interleaved samples, and then copying that to the output, it shouldn't matter that it's in stereo. Right?  The samples going out are the same as the samples going in, they have just been delayed through the buffer.

I've just done a mono test, and it works with a delay of frame size.

Interesting.

[Toby Gifford]

The callback function (and also the DSP function) measure time in units of 'frames', where a frame is a set of single samples on each channel. In other words one frame of audio data consists of 2 floats if you have 2 channels. Note that there is potential for confusion here when you talk about a 'frame buffer' (by which you mean a buffer of interleaved frames) and the associated 'frame size' by which you really mean number of floats the buffer can hold, but which would probably more sensibly refer to the number of frames (= buffer-size-in-floats / 2).

The input parameter called 'time' in the callback and DSP is advancing at the so-called 'sample-rate' which should probably actually be called the frame-rate. So when you call store_frame2 it will process 512 frames of stereo interleaved (since your buffer size is 1024).  Thus calling back in 512 frames time is going to work. Calling back in 1024 frames will be too late.

[Tim Roberts]

Yeah that makes sense.

For future reference of anyone finding this thread.

The store_frame closure has been adjusted to:

(bind-func store_frame

  (let ((read_head:i64 0)

        (write_head:i64 0))

    (lambda (t:i64)

      (let ((n:i64 0))

        (dotimes (n frame_size)

          (pset! frame_buffer n (pref input_buffer read_head))

          (set! read_head (% (+ read_head 1) in_size)))

        (dotimes (n frame_size)

          (pset! frame_buffer n (* (pref frame_buffer n) 1.0))) ;To ensure that I can manipulate the frame.

        (dotimes (n frame_size)

          (pset! output_buffer write_head (pref frame_buffer n))

          (set! write_head (% (+ write_head 1) out_size)))

        (callback (+ t (/ frame_size IN_CHANNELS)) store_frame t)

        ))))

($ (store_frame (now)))

Things to note are, dividing by IN_CHANNELS, the lack of single quote in the callback (xtlang rather than Scheme) and calling the closure natively in xtlang with the $ notation.

[Toby Gifford]

For clarity I tend to used nested dotimes to process the channels and frames.

(dotimes (n num_frames)

   (dotimes (c num_channels)

       (pset! output_buffer (+ c (* num_channels n)) groovy_value)))

But it is more efficient to do it as you have.

[Ben Swift]

Hi guys

Glad you figured it out - and I agree that the nomenclature for
frames/channels/samples is a bit confusing. If anyone wanted to write up
a little explanatory paragraph in docs/audio-signal-processing.rst then
they should knock themselves out :)

Cheers,
Ben

[Toby Gifford]

The terminological confusion is endemic to the field, with terms such as sample-rate embedded in the lexicon. And I've yet to find a good term for the hardware 'frame buffer' or 'slice' or 'audio window' or whatever you want to call it – the area of memory that is read/written in and out of the hardware at regular intervals. Add to this the ambiguity of terms like 'size' when applied to 'buffers', sometimes meaning number of frames, sometimes number of samples, sometimes size in bytes, and confusion is sure to follow :)

I hadn't noticed the docs folder. Is that the recommended way to contribute documentation?

[Ben Swift]

Hey Toblerone

Yes, I completely agree that the terminological mess isn't our fault -
I guess the best we can hope to do is provide a specific glossary-ish
thing for what the terms mean in Extempore.

The docs/ folder is new-ish, and I haven't advertised it all that widely
yet, but it's what gets used to generate

http://digego.github.io/extempore/

Which will replace my blog as the go-to place for Extempore docs in the
future. I'll take the warning signs off soon, as part of the 0.7
release.

In particular, it's worth checking out docs/about-this-documentation.rst

Cheers,
Ben

[Ross Bencina]

Hi Toby,

On 18/04/2016 10:06 AM, Toby Gifford wrote:
> The terminological confusion is endemic to the field, with terms such as
> sample-rate embedded in the lexicon. And I've yet to find a good term
> for the hardware 'frame buffer' or 'slice' or 'audio window' or whatever
> you want to call it – the area of memory that is read/written in and out
> of the hardware at regular intervals.

"block" or "buffer" would be my preferred terms.

Keep in mind that hardware buffers (and low-level API buffers) can take
on multiple forms depending on the driver model. For example, the
windows WDM device driver interface defines at least 3 different ways
that these buffers can be structured, something like: ringbuffer,
disjoint scatter/gather buffer chains, ping-pong buffers (Off the top of
my head). Some of these are dictated by certain DMA methods of
transferring data between the device and system memory.

If you're interested, there is a survey of buffering structures for
different user space audio APIs here:

https://www.assembla.com/spaces/portaudio/wiki/BufferingLatencyAndTimingImplementationGuidelines

In particular, the section starting at the heading "User-space native
audio API buffering model latency".

Cheers,

Ross.

[Toby Gifford]

Thanks Ross, interesting! I've mostly been buffered from such considerations by CoreAudio or PortAudio :)

[Ben Swift]

Toby Gifford <toby.g...@gmail.com> writes:

> I've mostly been *buffered* from such considerations by CoreAudio or
> PortAudio :)

I see what you did there :)

[Tim Roberts]

I'm glad I wasn't the only one.