Windows 64 MPI4PY Intel MPI

[Emre Kartal]

Hi everyone,

I am trying to install and run a python based code called "PyFR" which needs OpenMP.

I installed mpi4py on a Windows 10 64bit machine with both Microsoft MPI (Microsoft HPC Pack 2008 R2) and Intel MPI. With both of them I have the same problem; the answer to the famous mpiexec -n 4 python hello.py command "size  1 rank 0"

when I run mpiexec --version:
Intel(R) MPI Library for Windows* OS, Version 5.1.3 Build 20151210
Copyright (C) 2003-2015, Intel Corporation. All rights reserved.

By the way mpirun command is not recognized.

So, I guess that my machine try to execute mpi commands with Intel MPI (which may not be compatible with OpenMP, could you please confirm it?), and mpi4py didn't have any problem during setup with Intel MPI.

For Intel MPI, I wrote ( there was not an example for the Intel MPI)
[impi]
mpi_dir = D:\IntelSWTools\mpi\5.1.3.180\intel64
include_dirs = %(mpi_dir)s\include
libraries = impi
#library_dirs = %(mpi_dir)s\lib\i386
library_dirs = %(mpi_dir)s\lib\intel64

Should I specify bin folder in the mpi_dir?


If I have to change the system MPI program (mpiexec -- version to Microsoft) how can I do it?


Would it cause any problems with Intel Fortran, C++ compilers and Visual Studio 2015?

For MSMPI, I wrote:

[msmpi]
mpi_dir = $ProgramFiles\Microsoft HPC Pack 2008 R2
include_dirs = %(mpi_dir)s\inc
libraries = msmpi
#library_dirs = %(mpi_dir)s\lib\i386
library_dirs = %(mpi_dir)s\lib\amd64

Thanks, 

Best regards,

Emre

[Lisandro Dalcin]

On 31 March 2016 at 21:28, Emre Kartal <emres...@gmail.com> wrote:
> Hi everyone,
>
> I am trying to install and run a python based code called "PyFR" which needs
> OpenMP.
>

mpi4py does not use Open MP at all, just MPI. I'm not sure why you
mention this requirement, but it is irrelevant to mpi4py.

> I installed mpi4py on a Windows 10 64bit machine with both Microsoft MPI
> (Microsoft HPC Pack 2008 R2) and Intel MPI. With both of them I have the
> same problem; the answer to the famous mpiexec -n 4 python hello.py command
> "size 1 rank 0"
>
> when I run mpiexec --version:
> Intel(R) MPI Library for Windows* OS, Version 5.1.3 Build 20151210
> Copyright (C) 2003-2015, Intel Corporation. All rights reserved.
>
> By the way mpirun command is not recognized.
>

If your MPI does not provide the 'mpirun' command, there is nothing we
can do about it. Ask for it to the provider of your MPI
implementation. BTW, IIRC the 'mpiexec' command was introduced as a
STANDARD recommendation in MPI 2.0 circa 1998. After 18 years, it
would be really good if people stop using 'mpirun'.

> So, I guess that my machine try to execute mpi commands with Intel MPI
> (which may not be compatible with OpenMP, could you please confirm it?),

I have no idea, I'm not a Windows users and I do not work for Intel.
That being said, MPI and OpenMP are rather orthogonal programming
models. As long as your MPI library can work with multiple threads
running (that should be the case for Intel MPI as well as Microsoft
MPI), you should be fine.

And yes, if you built mpi4py with some MPI but you use 'mpiexec' from
other MPI, you will get the "size 1 rank 0" output. Just to confirm,
instead of using "mpiexec", try with the full path
"C:\Path\To\IntelMPI\bin\mpiexec.exe" (just an example, never tried
Intel MPI on Windows).

> and
> mpi4py didn't have any problem during setup with Intel MPI.
>

Why should it have? mpi4py does not care at all about OpenMP, as
mpi4py simply does not use it.

> For Intel MPI, I wrote ( there was not an example for the Intel MPI)
>

I never ever built mpi4py with Intel MPI on Windows, that's the reason
there are no examples.

> [impi]
> mpi_dir = D:\IntelSWTools\mpi\5.1.3.180\intel64
> include_dirs = %(mpi_dir)s\include
> libraries = impi
> #library_dirs = %(mpi_dir)s\lib\i386
> library_dirs = %(mpi_dir)s\lib\intel64
>
> Should I specify bin folder in the mpi_dir?
>

As long as "mpi.h" is in "%(mpi_dir)s\include" and "impi.lib" is in
"%(mpi_dir)s\lib\intel64", that should be enough.

>
> If I have to change the system MPI program (mpiexec -- version to Microsoft)
> how can I do it?
>

I guess you have to modify your %PATH% environment variable such as
the "bin" directory of Intel MPI is listed first. Or otherwise use the
full path to Intel MPI mpiexec.exe command, or write a tiny batch file
"mpiexec_intel.bat" using the full path to mpiexec and pass the rest
of command line args. There are the approaches a Linux or OS X guy
would use, I guess that Windows users would do quite the same.

>
> Would it cause any problems with Intel Fortran, C++ compilers and Visual
> Studio 2015?
>

That would depend on the contents of Intel MPI bin folder, but I guess
you should be fine. Try at your own risk, though (anyway, if something
breaks, you can revert the change to the %PATH% environment variable,
and you should be back to normal).

> For MSMPI, I wrote:
>
> [msmpi]
> mpi_dir = $ProgramFiles\Microsoft HPC Pack 2008 R2
> include_dirs = %(mpi_dir)s\inc
> libraries = msmpi
> #library_dirs = %(mpi_dir)s\lib\i386
> library_dirs = %(mpi_dir)s\lib\amd64
>

You are using a quite dated version of Microsoft MPI, I would
recommend to upgrade.


--
Lisandro Dalcin
============
Research Scientist
Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
Extreme Computing Research Center (ECRC)
King Abdullah University of Science and Technology (KAUST)
http://ecrc.kaust.edu.sa/

4700 King Abdullah University of Science and Technology
al-Khawarizmi Bldg (Bldg 1), Office # 4332
Thuwal 23955-6900, Kingdom of Saudi Arabia
http://www.kaust.edu.sa

Office Phone: +966 12 808-0459