Real time modulation, no sound

[Dimitri Vlachos]

Hello everyone! I was hoping someone could help me. I am writing a simple program to modulate my voice in real-time using librosa and I just can't get the audio to work...

This is the callback I am using:

def process_input(self, in_data, frame_count, time_info, flag):

        '''Process the input audio data and play back the modified audio.'''

        # Convert the input audio data to a numpy array

        numpy_array = np.frombuffer(in_data, dtype=np.float32)

       

        # Apply pitch shifting to create a robotic voice effect

        pitch_shifted_audio = librosa.effects.pitch_shift(y=numpy_array, sr=self.RATE, n_steps=4)

       

        # Apply a low-pass filter to attenuate high frequencies

        filtered_audio = librosa.effects.preemphasis(pitch_shifted_audio, coef=0.95)

       

        # Convert the modified audio data back to bytes

        out_data = filtered_audio.astype(np.float32).tobytes()

        # Play back the modified audio

        self.output_stream.write(out_data)

        return None, pyaudio.paContinue

Now if I remove the effects, it plays back the input audio just fine. But the effects don't seem to run? I'm not sure what I am doing wrong. Thanks for your help.

[Graham Coleman]

Hi Dimitri,

Did you get this working? I don't have any experience with pyaudio streaming.

But if I had to debug it, I might try first getting it working with file input/output (librosa.load and soundfile.write). Then, you can check each step, and listen to hear, or use matplotlib.pyplot.plot to view intermediate outputs.

Graham

--
You received this message because you are subscribed to the Google Groups "librosa" group.
To unsubscribe from this group and stop receiving emails from it, send an email to librosa+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/librosa/5381d816-4b90-43bb-ac2a-c2f5f760cc0cn%40googlegroups.com.

[Dimitri Vlachos]

No, not yet sadly!

That's a good idea though, I've been doing it the other way, if I remove the effects my voice is passed straight through to the output no problem. But I'll give working with files a go first and update here when I do.

[Dimitri Vlachos]

It works with file manipulation, no problem. But working with audio streams it seems to hang. It doesn't produce an error it just stops at the pitch_shift line. Does anyone have an idea what to do?

[Brian McFee]

The code looks fine to me, in the sense of correctness.  I suspect what's happening is that the pitch-shift operation is inducing too much latency for your callback to function properly.

My general hunch is that librosa is probably not the right tool for what you want to do.  The pitch-shift function is included as a basic demonstration and utility function, but there are better (faster) options out there for timescale modification, eg https://pypi.org/project/rubberband/

As an aside, if you're going to apply preemphasis in a streaming context like this, you'll probably want to use the `return_zf` function to carry state over between buffers.  (See second code example in the documentation.)

[Dimitri Vlachos]

That makes a lot of sense, I think that that may be exactly what the issue is. I will give rubberband a go! Frustratingly, I am developing on windows, but this is meant to be deployed on a Raspberry Pi so I'll move to developing there so I can run rubberband on Linux.

Thanks for the suggestions!