Pyo + Librosa/mir_eval usage

[Francesco Calabrese]

Hi to everyone, I'm new here and I'm a student working on my thesis in Computer Science and Engineering in Politecnico di Milano. I'm trying to do a "real-time" application where I analyze spectral features from a sound and then I generate a sound with some parameters fixed. To do this I'm working with Librosa and mir_eval as libraries, and I was managing the I/O stream with pyaudio. After this I should modify the output changing the timbre and other stuff of the sound, pyo is a very cool library but I'm having a hard time trying to use it in the way I described. I don't get very well how I can manage numpy arrays with this library, I've tried to use the class Data Table and TableRead to read the incoming arrays, but it doesn't seem to work. My questions are: how can I convert a numpy to a pyo object and vice versa, so that I can use my custom function to synthesize a sound? 

[Alexandros Drymonitis]

Hi and welcome!

Can you share some code? Even if it doesn't use Pyo. Still, it would be better if it uses Pyo so we can see what you are trying to do, and how you do it.

On 1/21/24 18:26, Francesco Calabrese wrote:

Hi to everyone, I'm new here and I'm a student working on my thesis in Computer Science and Engineering in Politecnico di Milano. I'm trying to do a "real-time" application where I analyze spectral features from a sound and then I generate a sound with some parameters fixed. To do this I'm working with Librosa and mir_eval as libraries, and I was managing the I/O stream with pyaudio. After this I should modify the output changing the timbre and other stuff of the sound, pyo is a very cool library but I'm having a hard time trying to use it in the way I described. I don't get very well how I can manage numpy arrays with this library, I've tried to use the class Data Table and TableRead to read the incoming arrays, but it doesn't seem to work. My questions are: how can I convert a numpy to a pyo object and vice versa, so that I can use my custom function to synthesize a sound?  --
You received this message because you are subscribed to the Google Groups "pyo-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyo-discuss/a2b00c82-304f-4d98-bc13-1c8826541c0en%40googlegroups.com.

[Olivier Bélanger]

Hi,

I think this example is doing what you want to do:

https://belangeo.github.io/pyo/examples/21-utilities/04-buffer-interface.html

Olivier

To view this discussion on the web visit https://groups.google.com/d/msgid/pyo-discuss/b4269f23-cb3b-4479-ada8-ae917b7671b4%40gmail.com.

[Francesco Calabrese]

Hi to everyone again, I have to elaborate the sound in input to my microphone and then generate a sound (given certain parameters with the library Librosa) that has to be manipulated with sound effects like modulations, filters etc..( the pipeline is: input_data from microphone-> synthesizer function (numpy array) -> sound to be changed by pyo or something similar -> output on speakers). I tried to do something like this, converting the sound from the synthesizer function to a Table and then passing it to the Freeverb(just as an example, to see if it worked). I didn't use the pyo stream with the object Input because I don't even know how to convert from PyoObject to an array. The fact is that I have to do this in real time and I don't know how to do this conversion even inside a callback function. Indeed, I don't even know if I'm managing correctly the session or if it is correct to have the stream from pyaudio and also the session for pyo. I wanted to implement the pipeline using only pyo or pyaudio, but firstly I wanted to understand how to do this conversion at least in one of the 2 ways. I saw also the object Sig that should do a pyo object given numerical values, but I don't know if it can works with actual array of values. Anyway thank you very much for the answers, tomorrow I will try the solution Oliver linked to me, but I don't know if there are some implications that I'm missing. This world of sound programming is very huge and I get lost sometimes

(i truncated various parts of the code that shouldnt be necessary to understand my workflow just for the sake of brevity)

def synthesizer(y, sr, fm, n_fft):
    #synthesize a sound given the input and some parameters
    return y_out

def array_to_table(array, sr=44100):
    num_samples = len(array)
    table = DataTable(size=num_samples, chnls=1)
    table.replace(array.tolist())
    table_read = TableRead(table, freq=1, loop=True)
    return table_read

def callback(in_data, frame_count, time_info, status):
    input_data = np.frombuffer(in_data, dtype=np.float32)
    y_synth = synthesizer(input_data, sr=RATE, fm=500, n_fft=2048)
    t=array_to_table(y_synth,RATE)
    reverb = Freeverb(t, size=0.80, damp=0.50, bal=0.25).out()
    return (y_synth_normalized, pyaudio.paContinue)

if __name__ == '__main__':
    p = pyaudio.PyAudio()
    s = Server().boot()
    s.start()
    stream = p.open(#various device parameters)
    stream.start_stream()

    try:
        while stream.is_active():
            pass
    except KeyboardInterrupt:
        pass
    finally:
        # Ferma lo streaming e termina PyAudio
        stream.stop_stream()
        stream.close()
        p.terminate()

[Alexandros Drymonitis]

The example Olivier posted provides a way to convert Pyo signals to NumPy arrays. If you want to use NumPy arrays to control parameters of PyoObjects, you can simply index your NumPy array and pass a single value of the array to a parameter of a PyoObject. Almost all PyoObjects take both PyoObjects or floats as arguments.

Otherwise, you can create a function or class that takes a number of arguments and call that with an unpacked NumPy array. Here's an example:

```
f = Freeverb()

def freeverb_ctrl(size, damp, bal):
    f.setSize(size)
    f.setDamp(damp)
    f.setBal(bal)

# suppose "a" is a NumPy array with three elements
# call the function by unpacking the list with the asterisk
freeverb_ctrl(*a)
```

To view this discussion on the web visit https://groups.google.com/d/msgid/pyo-discuss/ab4bd0c6-1427-469d-bc87-8fb6393f840fn%40googlegroups.com.

[Olivier Bélanger]

I think the real question here is why you try to mix up all these libs together (pyo, numpy, librosa, pyaudio, ...)?

If you want to do real time DSP, then pyo alone should be able to do anything you want, it was created exactly to do that... You certainly can't mix pyo and pyaudio, they will both try to connect to physical IO.

What does your "synthesizer' function do?

Olivier

To view this discussion on the web visit https://groups.google.com/d/msgid/pyo-discuss/40597d77-896b-42bc-b492-890474b40c8b%40gmail.com.

[Francesco Calabrese]

Hi Olivier, sorry if I've replied only now, I've been busy studying an exam. In this function 'synthesizer' I do an estimation of the pitch, then I use the librosa function f0_harmonics to retreive all the amplitudes of the harmonics given the pitch. I do this to control the amount of harmonics in the sound, recreating the sound just as it is in the original but with less harmonics. The procedure is similar to this one given by the page of Librosa: https://librosa.org/doc/0.10.1/auto_examples/plot_spectral_harmonics.html#sphx-glr-auto-examples-plot-spectral-harmonics-py 

I've seen that in pyo there is function that has the yin algorithm, but I didn't see any other function that do the part of extracting the amplitudes for every harmonics, like f0_harmonics does. After this extraction I have to process the sound to change its timbre and other characteristics. Another question, does yin in pyo work well in real time? Because this algorithm is a little bit unstable in other libraries.

Thank you for your time.

[Olivier Bélanger]

Hi Francesco,

About the Yin algorithm, IMO, it's the best compromise between speed and stability for a real time environment! I did a little bit of exploration, what about something like this (self-documented):

from pyo import *

s = Server().boot()

# Harmonic sound...
a = SfPlayer("/home/olivier/git/pyo/pyo/examples/snds/flute.aif", loop=True)

# Number of harmonics
HARMS = 20

# Extract the base frequency of the sound
bfreq = Yin(a)
# Little smoothing
basefreq = Tone(bfreq, freq=10)

# Harmonic frequencies
freqs = Sig([basefreq * i for i in range(1, HARMS+1)])

# Narrow bandpass filters around harmonic frequencies
filts = Biquadx(a, freq=freqs, q=10, stages=2, mul=0.01)

# Energy per frequency bands
gains = Follower(filts, freq=10)

# Re-synthesis
synth = Sine(freqs, mul=gains).mix(1).out()

s.gui(locals())

Starting with that, we can begin to be more creative:

from random import uniform
from pyo import *

s = Server().boot()

a = SfPlayer("/home/olivier/git/pyo/pyo/examples/snds/flute.aif", loop=True)

HARMS = 20

bfreq = Yin(a)
basefreq = Tone(bfreq, freq=10)

freqs = Sig([basefreq * i for i in range(1, HARMS+1)])
filts = Biquadx(a, freq=freqs, q=10, stages=2, mul=0.01)
energy = Follower(filts, freq=10)

# Independent randoms applied to synthesized frequencies
vars = Randi(min=0.98, max=1.02, freq=[uniform(0.5, 1.5) for i in range(HARMS)])

# Play with the graph to modify the gain of the synthesized frequencies
gaintable = DataTable(size=len(freqs), init=[0.7]*HARMS)
gaintable.graph()

gains = TableIndex(gaintable, index=Sig(list(range(HARMS))))

synth = Sine(freqs * vars, mul=energy*gains).mix(1).out()

s.gui(locals())

Little bonus, the harmonic extraction embedded in its own class:

from random import uniform
from pyo import *

class Harmonics:
    def __init__(self, input, maxharms=20):
        self.bfreq = Yin(input)
        self.basefreq = Tone(self.bfreq, freq=10)
        self.freqs = Sig([self.basefreq * i for i in range(1, maxharms+1)])
        self.filters = Biquadx(input, freq=self.freqs, q=10, stages=2, mul=0.01)
        self.followers = Follower(self.filters, freq=10)
        self.gaintable = DataTable(size=maxharms, init=[0.7]*maxharms)
        self.gainctrl = TableIndex(self.gaintable, index=Sig(list(range(maxharms))))
        self.gains = self.followers * self.gainctrl

    def get(self, identifier):
        if identifier == "freq":
            return self.freqs
        elif identifier == "gain":
            return self.gains
        raise Exception("Harmonics.get: wrong identifier {freq or gain}")

    def graph(self):
        self.gaintable.graph()

s = Server().boot()

a = SfPlayer("/home/olivier/git/pyo/pyo/examples/snds/flute.aif", loop=True)

HARMS = 20

h = Harmonics(a, HARMS)
h.graph()

vars = Randi(min=0.98, max=1.02, freq=[uniform(0.5, 1.5) for i in range(HARMS)])
synth = Sine(h.get("freq") * vars, mul=h.get("gain")).mix(1).out()

s.gui(locals())

Hope that helps!

Olivier

--
You received this message because you are subscribed to the Google Groups "pyo-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyo-discuss...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pyo-discuss/650154dd-d15b-406b-985c-d05326cbc1e9n%40googlegroups.com.

[Francesco Calabrese]

Olivier, thank you very much for this, this actually worked very well. I would have never thought about separating the amplitudes of each frequency with a biquadratic filter, I've always thought about extracting the amplitudes from the magnitude spectrum. I really appreciate your help, you are the best! :)

I have only two curiosities did you use basefreq = Tone(bfreq, freq=10) to cut out some noise? Isn't the cutoff frequency at 10 a little bit too low? 

Instead, also for the line energy = Follower(filts, freq=10), how does it work the cutoff frequency? Is it like a threshold above the filtered frequencies to set a maximum freq each computation to get the amplitude? 

And lastly, if I have more doubts about the library in general can I ask again? 

You received this message because you are subscribed to a topic in the Google Groups "pyo-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pyo-discuss/jLTrXpweImA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pyo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyo-discuss/CAMXBGhRWzRMdb-68ecyYUvXELkep59svv-f4_dBJDsLTUDoKkw%40mail.gmail.com.