Error loading MPI DLL in mpi4py

[aso....@gmail.com]

Hi,

I am trying to use Mpi4py 1.3 with python 2.7 on Windows 7 64bits. I downloaded the installable version from here which includes OpenMPI 1.6.3 so in the installed directory (*/Python27\Lib\site-packages\mpi4py\lib) following libraries exist: libmpi.lib, libmpi_cxx.lib, libopen-pal.lib, and libopen-rte.lib. Now in my codes when trying to import it:

from mpi4py import MPI

It returns following error: ImportError: DLL load failed: The specified module could not be found. I tried to copy above DLL files alongside the */Python27\Lib\site-packages\mpi4py\MPI.pyd and even to Windows/System32, but it didn't work. I appreciate your help on what DLL is missing and how to fix this error?

Thanks,

[Lisandro Dalcin]

On 22 December 2012 14:13, <aso....@gmail.com> wrote:
> Hi,
>
> I am trying to use Mpi4py 1.3 with python 2.7 on Windows 7 64bits. I
> downloaded the installable version from here which includes OpenMPI 1.6.3 so
> in the installed directory (*/Python27\Lib\site-packages\mpi4py\lib)
> following libraries exist: libmpi.lib, libmpi_cxx.lib, libopen-pal.lib, and
> libopen-rte.lib. Now in my codes when trying to import it:
>

Big Disclaimer: I know almost nothing about Windows.

All the *.lib files you mention are not the Open MPI *.dll files. I'm
not sure why these *.lib files are there in the mpi4py tree (or were
you who put them there?), but there are useless at runtime, you really
need the Open MPI *.dll libraries.

So, I would do the following:

1) Install Open MPI from the official installer:
http://www.open-mpi.org/software/ompi/v1.6/downloads/OpenMPI_v1.6.2-2_win64.exe

2) Modify your %PATH% environment variable (just in case, do it
system-wide, not just for the current user) to include
%ProgramFiles%\OpenMPI_v1.6.2-win64\bin (note the last path component,
...\bin, there you should find the Open MPI *.dll files, please
double-check)

PS: Right now I do not remember, but I think the OpenMPI installer has
an option to perform step (2) for you.


After that, enter python and try again "from mpi4py import MPI"

> from mpi4py import MPI
>
> It returns following error: ImportError: DLL load failed: The specified
> module could not be found. I tried to copy above DLL files alongside the
> */Python27\Lib\site-packages\mpi4py\MPI.pyd and even to Windows/System32,
> but it didn't work. I appreciate your help on what DLL is missing and how to
> fix this error?
>

Are sure the libraries were *.dll files? At the beginning, you mention
*.lib files ... again, you really need the DLL's, if you do not have
it, you need to install first Open MPI.



--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

[aso....@gmail.com]

Dear Lisandro,

Many thanks for your reply. I installed OpenMPI from the link you provided, then added the path of the 'C:\Program Files (x86)\OpenMPI_v1.6.2-x64\bin' directory to the system and user environment variables and rerun my Python script with 'from mpi4py import MPI'. Again it returend same error: 'ImportError: DLL load failed: The specified module could not be found.' 

Now what else can I do?

Regards,

Aso Agile

[Lisandro Dalcin]

On 25 December 2012 13:56, <aso....@gmail.com> wrote:
> Dear Lisandro,
>
> Many thanks for your reply. I installed OpenMPI from the link you provided,
> then added the path of the 'C:\Program Files (x86)\OpenMPI_v1.6.2-x64\bin'
> directory to the system and user environment variables and rerun my Python
> script with 'from mpi4py import MPI'. Again it returend same error:
> 'ImportError: DLL load failed: The specified module could not be found.'
>

I'm running out of ideas. FYI, I installed OpenMPI and tthe mpi4py
binaries you mention in previous email on a Windows 7 32bit computer,
and everything worked smoothly. Could it be that you are missing the C
runtime DLLs (msvcrt.dll) ... It is so unhelpful that the error
message says "... The specified module could not be found ..."

Have you contacted the author of the binary mpi4py packages? Perhaps
he knows better than me what's going on...

What about installing MinGW for Win64, and try to build mpi4py from
sources? Installing MinGW should be easy, and it is not a huge
download (at least compared to downloading the MS toolchain).

[Lisandro Dalcin]

On Sun, Jan 6, 2013 at 7:50 AM, <cjgo...@gmail.com> wrote:
>
>>
>> The binaries from http://www.lfd.uci.edu/~gohlke/pythonlibs/#mpi4py
>> contain OpenMPI 1.6.3 (executables, DLLs, libs, etc), specifically built to
>> be compatible with the official CPython distributions. Use the MPI enabled
>> python interpreter installed in
>> sys.prefix\lib\site-packages\mpi4py\bin\python-mpi.exe instead of
>> sys.prefix\python.exe. Do not put any other OpenMPI distribution in the OS
>> PATH!
>>

Why are you requiring users to use python-mpi.exe? Is this a way to
ship a binary that does not require any other task on users side? Have
you tried to do the same but using the stock Python interpreter at
sys.prefix?

[Lisandro Dalcin]

On Tue, Jan 8, 2013 at 12:18 PM, Christoph Gohlke <cjgo...@gmail.com> wrote:
>
>
> What is the purpose of shipping python-mpi.exe with mpi4py
> <http://code.google.com/p/mpi4py/source/browse/setup.py#418>?
>

python-mpi.exe is shipped for old MPI implementations that do require
at MPI_Init() the command line arguments passed to main(). In this
case, it is impossible to use the stock python.exe executable.
However, I do not know any MPI implementation for Windows that has
this issue. Other use case would be to generate a Python binary that
statically links the MPI libraries.

> Users are not required to use python-mpi.exe. It is a shortcut/workaround
> for the reported problem, where OpenMPI is not present or set up correctly.
> Users are free to set OPENMPI_HOME and PATH environment variables to any
> compatible OpenMPI installation.
>

Oh! I see...

> I am considering adding the following code to mpi4py/__init__.py so my
> binaries work with the standard python.exe out of the box.
>
> ```
> def _init_openmpi():
> """Pre-load libmpi.dll and register OpenMPI distribution."""
> import os
> import ctypes
> if os.name != 'nt' or 'OPENMPI_HOME' in os.environ:
> return
> try:
> openmpi_home = os.path.abspath(os.path.dirname(__file__))
> openmpi_bin = os.path.join(openmpi_home, 'bin')
> ctypes.cdll.LoadLibrary(os.path.join(openmpi_bin, 'libmpi.dll'))
> os.environ['OPENMPI_HOME'] = openmpi_home
> os.environ['PATH'] = ';'.join((openmpi_bin, os.environ['PATH']))
> except Exception:
> pass
>
> _init_openmpi()
> ```

If that hackery makes your binaries work out of the box with the stock
python.exe, then go ahead and ship the modified mpi4py/__init__.py
with your binaries. But... Do you really need to preload libmpi.dll ?
I have little knol about dynamic loading on Windows, but AFAIK, the
preloading is useless... the PATH fixing is what really makes it
working.

BTW, will you also ship "mpiexec.exe" in the mpi4py/bin directory?