Weird error in stft when using Numpy with MKL

[João Felipe Santos]

Hi,

I just spent an awful amount of time debugging a weird issue with the stft implementation in librosa. It turns out the stft is completely wrong when using a Numpy that was linked to Intel MKL (as opposed to ATLAS, which is the "standard" BLAS). The stft results from librosa are completely wrong when using MKL, even though matplotlib's specgram works. The implementations are kind of similar and both use the numpy.fft module for FFTs, so I could not figure out by myself what was going wrong.

I tested it by creating two similar Python environments with conda (the tool from the Anaconda Python distribution) and installing MKL only in one of them.

Do you have any idea on what might have caused this?

Best,

João

[Brian McFee]

On Wednesday, March 25, 2015 at 2:56:47 PM UTC-4, João Felipe Santos wrote:

Hi,

I just spent an awful amount of time debugging a weird issue with the stft implementation in librosa. It turns out the stft is completely wrong when using a Numpy that was linked to Intel MKL (as opposed to ATLAS, which is the "standard" BLAS). The stft results from librosa are completely wrong when using MKL, even though matplotlib's specgram works. The implementations are kind of similar and both use the numpy.fft module for FFTs, so I could not figure out by myself what was going wrong.

Could you please be more precise about what constitutes "completely wrong"?  Maybe provide an example input and output?

 

I tested it by creating two similar Python environments with conda (the tool from the Anaconda Python distribution) and installing MKL only in one of them.

Do you have any idea on what might have caused this?

Having never used intel MKL, I can't speculate about what's going on.

Would you mind testing the current development branch to see if the same issue pops up?  I recently switched over from numpy.fft to scipy.fftpack.  The results should be numerically equivalent, but it would be useful to know if that setup also fails for you.

 

Best,

João

[João Felipe Santos]

Hi Brian, and thanks for your quick reply!

The output simply made no sense, I could not find what was wrong specifically. As an example, you can try running the example notebook 'LibROSA audio effects and playback' and listening to the outputs of the hpss call. I forgot to mention that this was on a Mac. It might be only an issue with the numpy package with MKL bindings on Anaconda, as opposed to something wrong with LibROSA.

I switched to the develop branch and switching to fftpack fixed this issue. scipy.fftpack uses a different implementation so it probably dodges the bug in numpy.fft with MKL.

Thanks again!

[Brian McFee]

On Wednesday, March 25, 2015 at 3:40:48 PM UTC-4, João Felipe Santos wrote:

Hi Brian, and thanks for your quick reply!

The output simply made no sense, I could not find what was wrong specifically. As an example, you can try running the example notebook 'LibROSA audio effects and playback' and listening to the outputs of the hpss call. I forgot to mention that this was on a Mac. It might be only an issue with the numpy package with MKL bindings on Anaconda, as opposed to something wrong with LibROSA.

Well, I don't have a mac, or intel mkl... so I doubt that's gonna work.

At any rate, there's nothing too fancy going on with our use of fft, so I suspect you'll have to take it upstream to the numpy developers.  The only things that come to mind are: 1) some kind of byte alignment issue with dtype=complex64 vs complex128, or 2) something's going wrong with striding in the frame generator.

Do the unit tests pass?

 

I switched to the develop branch and switching to fftpack fixed this issue. scipy.fftpack uses a different implementation so it probably dodges the bug in numpy.fft with MKL.

Ok, i think that rules out being a librosa issue, since those should be roughly api-compatible.  

 

Thanks again!

Welcome!  Good luck debugging upstream.