numpy using mkl libraries

[Ricardo A Corredor J]

Hi everyone,

I have a python code that internally uses numpy (plus other dependencies) to perform some basic numerical operations.

I installed numpy as "conda install numpy" (Windows 10) meaning that it includes and uses the mkl libraries. After running pyinstaller on this code, I have an executable that contains the mkl libraries. I can see those dlls when I run the executable and check the folder created in Temp with all the dependencies uncompressed. Nevertheless, when I run the executable, the program crashes when it tries to use numpy with the optimized mkl libraries (a numpy.dot(..,..) operation). I copied the executable to the folder where I have the dlls of the mkl libraries in my system and the program runs properly. 

Question: why the mkl libraries are not properly detected/used during runtime even if they are included in the executable?  Is there a way to set the .specs to link properly to those dlls instead of copying always the dlls next to the program or adding to the path? If not, is there a way to properly exclude the mkl libs from the executable if in any case I need to use them from a different folder ?

Thanks,

Ricardo

RaC

[Ricardo A Corredor J]

UPDATE:

I checked one by one the dlls I had in the folder where the executable was running and found that there was only one that was required to run the executable properly:

Seems like this omp library is not included in the executable. Any reason why ? 

Thanks in advance!

RaC

[S CartonForever]

I can't answer the question I'm afraid. But did want to say thank you for asking it. I had the exact same error but didn't realise it. When I tried copying the required dll into the folder my compiled program is now working. So thanks, but yeah this seems to be two of us with this bug.

[Ricardo A Corredor J]

UPDATE 2 : In the meantime ... I added the dll to the data variable in specs (as stated here https://stackoverflow.com/questions/62625517/pyinstaller-exe-file-terminates-early-without-an-error-message and here https://stackoverflow.com/questions/58096010/pyinstaller-importerror-with-scipy-dll-load-failed. Is this Ok ? Seems to work ...

data = [('C:/envs/venv/Lib/Library/bin/libiomp5md.dll', '.')]

RaC

[Chris Barker]

Question: why the mkl libraries are not properly detected/used during runtime even if they are included in the executable?

MKL does some run-time magic to detect the processor and the like, so it may be linking in an unusual way that PyInstaller can not detect.

Adding it by hand may be the only solution. But it would be good it have a recipe for this.

-CHB

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[bwoodsend]

The other mkl dlls are manually specified in a hook file inside PyInstaller here. It looks like they just missed that one simply because libiomp5md doesn’t have mkl in its name. In which case it looks like that hook file needs fixing.

In answer to RaC’s last message - strictly speaking it should be binaries=[...] instead of data=[...] for binary files although I’m not entirely sure what the difference is.

Brénainn

​

[Abasi Brown]

include numpy as a hidden library from the command line:

pyinstaller your_script.py --hidden-import=numpy

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyinstaller...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyinstaller/20a0f688-4032-409c-b48f-3e777a32cb57o%40googlegroups.com.

[Ricardo A Corredor J]

Thanks everyone for the replies and suggestions. I was planning to add to the binaries, but followed what others did by setting directly in data. That worked. I didn't try adding numpy to hiddenimports though, because the rest of numpy works and is detected properly.

RaC

On Monday, July 6, 2020 at 4:41:46 PM UTC+2, Ricardo A Corredor J wrote: