Live/Real-time analysis of audio stream

[Yoan Mollard]

Hey guys,


I am wondering whether it is possible to analyse an audio stream rather than a static input file, say for instance capturing and analysing Pulse Audio's main speakers output ?
And if not what will be the difficulties and the key code locations to achieve that by deeping into the source code and making the appropriate changes?

Note: I am not looking for recording a files or even short files further proceeded by libROSA but really looking for a real-time analysis such as what ProjectM does :
https://help.ubuntu.com/community/projectM

Thanks

[Brian McFee]

It's definitely possible, but it's not entirely easy.

You can use pyaudio to capture frames from port audio, and either buffer them manually, or analyze each frame individually.  The latency here is pretty bad in my experience, so I wouldn't recommend doing any effects processing.  However, if you're just trying to do something like event detection, it might be good enough.

Here's a blog post describing how to do it in detail.

[Yoan Mollard]

Thanks for the post. Reading audio stream was my main issue but pyaudio makes it very easy. I am just computing FFT on my side, this is pretty fast.

The code snippet misses some declaration, so I'm repasting here a self-sufficient example of grabbing audio data:

import pyaudio
import numpy as np
import time

pa = pyaudio.PyAudio()

def callback(in_data, frame_count, time_info, flag):
    audio_data = np.fromstring(in_data, dtype=np.float32)
    # Instead of printing, process here the audio chunk 'audio_data' with libROSA
    # [...]
    print(audio_data)
    return None, pyaudio.paContinue

stream = pa.open(format=pyaudio.paFloat32,
                 channels=1,
                 rate=44100,
                 output=False,
                 input=True,
                 stream_callback=callback)

stream.start_stream()
while stream.is_active():
    time.sleep(0.25)
stream.close()
pa.terminate(

[Marcello Lussana]

hi,

On Saturday, December 10, 2016 at 12:38:25 AM UTC+9, Brian McFee wrote:

You can use pyaudio to capture frames from port audio, and either buffer them manually, or analyze each frame individually.  The latency here is pretty bad in my experience, so I wouldn't recommend doing any effects processing.  However, if you're just trying to do something like event detection, it might be good enough.

Here's a blog post describing how to do it in detail.

 thanks for this answer. I checked the blog post but to me still unclear how I can analysed the captured frames.
I would like to do an mfcc analysis of the last 2 seconds of incoming audio stream, but how can I do that in real-time?
Latency is not an issue (sofar is not too big), i could even record it and analyse the saved file but maybe there is a more direct (and quicker) option.

[Ghiath Majjanne]

Hi Yoan, how do you prepare the in_data to process it with Librosa? converting it to numpy array is enough? do I have to apply the fft ?

[Sutirtha Chakraborty]

Is it possible to find out the tempo of a song live from music from microphones ? using librosa? 

I am very new in this area! kindly help 

[Sundar Krishna]

I was trying to do a mel-spectogram with this method. As soon as I loaded this into librosa.load I got this error

Traceback (most recent call last):

  File "d:/Python36/Audio_to_nump2.py", line 15, in callback

    audio_data = librosa.load(audio_data)

  File "D:\Python36\lib\site-packages\librosa\core\audio.py", line 129, in load

    with sf.SoundFile(path) as sf_desc:

  File "D:\Python36\lib\site-packages\soundfile.py", line 629, in __init__

    self._file = self._open(file, mode_int, closefd)

  File "D:\Python36\lib\site-packages\soundfile.py", line 1182, in _open

    raise TypeError("Invalid file: {0!r}".format(self.name))

TypeError: Invalid file: array([ 0.0000000e+00,  0.0000000e+00, -3.0517578e-05, ...,

        0.0000000e+00,  0.0000000e+00,  0.0000000e+00], dtype=float32)

Traceback (most recent call last):

  File "d:/Python36/Audio_to_nump2.py", line 45, in <module>

    time.sleep(0.25)

TypeError

It says typeerror , can I load using librosa.load(live_audio) into this?

Thanks!