I need help in dealing with chroma and chord detection in Librosa

[Sebastian Mayer]

Hi there,

I need really urgent help with a more complex issue. But I would be really grateful if someone could help me there

My goal is the following: I want to get with my python program a list of times when there is a change in the harmony of a song. That means, if for example a C-Major chord is played until second 3 (from the beginning), and from then on G-Major, then I want to find out this point in time.

That actually already works.

Only my big problem is now the following: These times are VERY inaccurate and I have already tried many things to make it more accurate. 

I wanted to ask now, whether anyone can give me there tips? 

I use the Chroma-Detection module from Libra (more info: https://librosa.org/doc/main/auto_examples/plot_chroma.html). Also here is a part of my code:

#load the audio file

y, sr = librosa.load(audiofile, offset=10, duration=30, sr=sr)

y = librosa.effects.harmonic(y=y, margin=8)

#Apply Filters

chroma_harm = librosa.feature.chroma_cqt(y=y, sr=sr)

chroma_filter = np.minimum(chroma_harm, librosa.decompose.nn_filter(chroma_harm, aggregate=np.median, metric='cosine'))

chroma_smooth = scipy.ndimage.median_filter(chroma_filter, size=(1, 9))

#Plot the graph

fig, ax = plt.subplots(nrows=2, sharex=True,sharey=True)

librosa.display.specshow(chroma_smooth, y_axis='chroma', x_axis='time', ax=ax[0])

ax[0].set(ylabel='Harmonics')

#write data to list. List has 12 seperate list for all 12 notes

#each one of these lists contains the strength per time (0 to 1)

times=chroma_smooth.tolist()

For example, can I still apply any filters here? Is there any solution?

I would be really grateful if someone could help me with this. 

Thanks so much in advance

Sebastian

[Brian McFee]

I don't think we have enough information to go on here.  You say the times are "very inaccurate" but provide no examples of expected and actual behavior, so it's impossible to tell what's going wrong here.

Two things though:

1. Your call to specshow does not supply the sampling rate, so it will use the default sampling rate of 22050 for unit conversion and axis decoration.  This could be a source of perceived error.
2. It's not clear from your post if you want "chords" or "chroma", and the two are not interchangeable.  We often use chroma features as a representation to detect chords, but there's still a pretty large gap between chroma features and chord labels, which is typically filled by a classification model.  If you want chord detections (not chroma features), you may want to look into using a pre-trained model like crema: https://github.com/bmcfee/crema
   

[Sebastian Mayer]

Hello again. 

Thank you so much for your quick and detailed answer. You are probably right; I should use a module that is specifically for this. I also tried the module directly what you suggested ("Crema"). 

Unfortunately, it did not work for me. I always ran into errors. Can it be that this module is already outdated (I use Python 3.8.7)? 

I get this error, for example (when I use crema from the command line:

...

  File "C:\Users\Kinder\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\keras\utils\generic_utils.py", line 237, in func_load

    code = marshal.loads(raw_code)

ValueError: bad marshal data (unknown type code)

Or this one (when I use crema in a program):

...

  File "C:\Users\Kinder\Downloads\crema-master\crema\models\base.py", line 98, in _instantiate

    self.pump = pickle.load(fd)

EOFError: Ran out of input

Do you know any solution for this problem? It would really help me a lot

P.S.: I know this is actually a forum for Librosa, so please help me only if you want.

Thanks a lot 

Sebastian