Building mpi4py with Intel MPI on Windows

[maarten.b...@gmail.com]

I'm trying to build mpi4py with Intel MPI on Windows 10. I first tried to install with pip but the installation kept defaulting to MSMPI, even though I removed it entirely. I also tried passing "MPICC=" flag. This didn't help either.

So I cloned the repo from bitbucket and I'm trying to build using setup. I added the following block to mpi.cfg:

[intelmpi]
mpi_dir              = $(I_MPI_ROOT)\intel64
mpicc                = %(mpi_dir)s\bin\mpicc.bat
mpicxx               = %(mpi_dir)s\bin\mpicxx.bat
include_dirs         = %(mpi_dir)s\include
libraries            = impi.lib
library_dirs         = %(mpi_dir)s\lib

where I_MPI_ROOT evaluates to the location of intel MPI on my system.

I called:

python setup.py build --mpi=intelmpi

Building fails. Checking the errors, there's a couple of things that catch my eye.
First, the mpi compiler wrappers are not found:

error: $(I_MPI_ROOT)\intel64\bin\mpicc.bat' not found
error: '$(I_MPI_ROOT)\intel64\bin\mpicxx.bat' not found

I checked, the paths are correct. Using the full absolute path rather than $(I_MPI_ROOT) didn't help.

The script continues anyway, trying to build dependencies using the Visual C compiler on my system. Not sure if that is normal or a result from not finding the MPI compilers. Anyway, it fails on the first library, since library "lmpe.lib", cannot be found. Is this library supposed to be included with the MPI distribution?

Help would be appreciated.
Thanks,
Maarten

[Lisandro Dalcin]

On Wed, 12 Aug 2020 at 14:10, <maarten.b...@gmail.com> wrote:

I'm trying to build mpi4py with Intel MPI on Windows 10.

I never tried that. I'm not a windows user.

 

I first tried to install with pip but the installation kept defaulting to MSMPI, even though I removed it entirely.

Of course, those binary packages are built with MSMPI. They will not work with other MPI implementations.

 

I also tried passing "MPICC=" flag. This didn't help either.

So I cloned the repo from bitbucket and I'm trying to build using setup. I added the following block to mpi.cfg:

[intelmpi]
mpi_dir              = $(I_MPI_ROOT)\intel64
mpicc                = %(mpi_dir)s\bin\mpicc.bat
mpicxx               = %(mpi_dir)s\bin\mpicxx.bat
include_dirs         = %(mpi_dir)s\include
libraries            = impi.lib
library_dirs         = %(mpi_dir)s\lib

where I_MPI_ROOT evaluates to the location of intel MPI on my system.

I called:

python setup.py build --mpi=intelmpi

Building fails. Checking the errors, there's a couple of things that catch my eye.
First, the mpi compiler wrappers are not found:

error: $(I_MPI_ROOT)\intel64\bin\mpicc.bat' not found
error: '$(I_MPI_ROOT)\intel64\bin\mpicxx.bat' not found

I checked, the paths are correct. Using the full absolute path rather than $(I_MPI_ROOT) didn't help.

Try setting mpi_dir this way:

mpi_dir = $I_MPI_ROOT\intel64

If that does not work, then use the full explicit path `mpi_dir = C:\path\to\IntelMPI\intel64`.

 

I'm not really sure you can use that mpicc.bat batch file to build mpi4py. Look at the MSMPI sections in mpi.cfg, perhaps if you explicitly list the include and libraries, it will work.

The script continues anyway, trying to build dependencies using the Visual C compiler on my system. Not sure if that is normal or a result from not finding the MPI compilers. Anyway, it fails on the first library, since library "lmpe.lib", cannot be found. Is this library supposed to be included with the MPI distribution?

It is not a real failure, lack of lmpe.lib lib should not abort the build. It is an optional component.

--

Lisandro Dalcin
============
Research Scientist
Extreme Computing Research Center (ECRC)
King Abdullah University of Science and Technology (KAUST)
http://ecrc.kaust.edu.sa/

[maarten.b...@gmail.com]

Thanks for your reply. I dove into it and found a few issues related to distutil functions.First, the split_quoted() function doesn't handle backslash path separators. It works fine with forward slashes. Second, the find_executable() function only finds .exe files, nothing else. This is a very old issue.

I modified find_exe() in mpiconfig.py to fix the second problem. Now the compiler wrappers are recognized but still the MSVC compiler is used. Looking at the documentation for compilers in distutils, it seems that solving this would either require creating a subclass for intel compilers, or manually specifying the compiler options, like you suggested.

Instead I downloaded and installed Intel distribution for Python. This includes a version of mpi4py built with Intel compilers, which works out of the box (unrelated problems aside). Still, since I don't like being tied to specific distribution, maybe I'll try to fix the problems at some point in the future.

cheers,

Maarten

On Wednesday, August 12, 2020 at 3:57:06 PM UTC+2, Lisandro Dalcin wrote:

[Lisandro Dalcin]

On Thu, 13 Aug 2020 at 10:51, <maarten.b...@gmail.com> wrote:

Thanks for your reply. I dove into it and found a few issues related to distutil functions.First, the split_quoted() function doesn't handle backslash path separators. It works fine with forward slashes. Second, the find_executable() function only finds .exe files, nothing else. This is a very old issue.

I modified find_exe() in mpiconfig.py to fix the second problem. Now the compiler wrappers are recognized but still the MSVC compiler is used.

Yes, because the whole windows branch of my hacks did not accommodate for the usual MPI compiler wrappers we use in POSIX world. Please note that all that code predates the existence of Intel MPI for Windows.

 

Looking at the documentation for compilers in distutils, it seems that solving this would either require creating a subclass for intel compilers, or manually specifying the compiler options, like you suggested.

Can't you use Intel MPI with MSVC? Do you really need to add the Intel compiler to the combo? In short, can't you keep using MSVC to build mpi4py, passing include_dir/library_dir/libraries information?

 

Instead I downloaded and installed Intel distribution for Python. This includes a version of mpi4py built with Intel compilers, which works out of the box (unrelated problems aside).

Well, I guess that even Python itself is built with Intel compilers.

 

Still, since I don't like being tied to specific distribution, maybe I'll try to fix the problems at some point in the future.

Any help most welcome. It seems you really know the business, and the nightmare it is.

As I said, I'm not a Windows developer, the best I could do was to set things up to use all-Microsoft stuff.

[maarten.b...@gmail.com]

You were right, using the MSVC compiler and linking against Intel MPI works fine. I removed the lines for mpicc and mpicxx from mpi.cfg:

[intelmpi]
mpi_dir              = $I_MPI_ROOT/intel64
include_dirs         = %(mpi_dir)s/include
libraries            = impi
library_dirs         = %(mpi_dir)s/lib/release

Now it builds and installs!

Maarten

On Thursday, August 13, 2020 at 11:35:00 AM UTC+2, Lisandro Dalcin wrote:

[maarten.b...@gmail.com]

An addition:

The argument --mpi cannot be passed to setup.py with the "install" command. As a result, setup will default to MSMPI if it finds this, and rebuild if necessary. It seems the only way to install with Intel MPI is to remove MSMPI from the system.

cheers,

Maarten

[Lisandro Dalcin]

On Fri, 14 Aug 2020 at 10:53, <maarten.b...@gmail.com> wrote:

An addition:

The argument --mpi cannot be passed to setup.py with the "install" command. As a result, setup will default to MSMPI if it finds this, and rebuild if necessary. It seems the only way to install with Intel MPI is to remove MSMPI from the system.

Well, the way to do it would be:

python setup.py build --mpi=intelmpi install

Another way:

set MPICFG=intelmpi

python setup.py install

Now that you figured how to make things work in intel MPI, I'm planning to update the example mpi.cfg. 

I have a few questions.

* I noticed you wrote $I_MPI_ROOT/intel64 . Do backward slashes work? 

* In your intel MPI installation, does $I_MPI_ROOT contain an intel32 subdirectory with the stuff required for 32bits builds? Any chance you can try building with a Python 32bit version?

My plan is to add the following to mpi4py's mpi.cfg file:

# Intel MPI for Windows
# ---------------------
[intelmpi-windows-32bit]
mpi_dir              = $I_MPI_ROOT/intel32

include_dirs         = %(mpi_dir)s/include
libraries            = impi
library_dirs         = %(mpi_dir)s/lib/release

[intelmpi-windows-64bit]
mpi_dir              = $I_MPI_ROOT/intel32

include_dirs         = %(mpi_dir)s/include
libraries            = impi
library_dirs         = %(mpi_dir)s/lib/release

Note I'm adding the `-windows-32/64bit` tag. However, you can just keep passing --mpi=intelmpi or set MPICFG=intelmpi, the code in conf/mpiconfig.py will take care of resolving by platform and architecture.

Can you confirm the above configuration works?

BTW, are you ok if I add the following to the git commit message?

Co-authored-by: Maarten Braakhekke <maarten.b...@gmail.com>

Additionally, I could add support for automatically looking for $I_MPI_ROOT in the environment, and then use Intel MPI in favor of MSMPI.

All that should be implemented in conf/mpiconfig.py, However, I would need your help for testing it.

If you are willing to help me, I could eventually put the code in a branch and contact you privately (or in a Bitbucket issue) to keep going.

[maarten.b...@gmail.com]

Sure I'd be happy to help out. However, I've passed the point where I can keep doing this in the boss' time ;-), so it'll be a bit slower. I'll get back to you.

Maarten

On Friday, August 14, 2020 at 10:34:19 AM UTC+2, Lisandro Dalcin wrote:

Co-authored-by: Maarten Braakhekke <maarten....@gmail.com>

[Lisandro Dalcin]

On Fri, 14 Aug 2020 at 12:14, <maarten.b...@gmail.com> wrote:

Sure I'd be happy to help out. However, I've passed the point where I can keep doing this in the boss' time ;-), so it'll be a bit slower. I'll get back to you.

Oh, do not worry. All I would need is you to clone the repository and run `python setup.py build` and confirm it works.

You did not answer my previous question. Does your Intel MPI install contains the directory $I_MPI_ROOT/intel32 with the include/lib subdirectories?

[Maarten Braakhekke]

There is no directory $I_MPI_ROOT/intel32 on my system. During the installation it is suggested that libraries for 32bit would also be installed, but I can't find them. I'll look into it because I also want to be able to build for 32bit.

cheers,

Maarten

[Lisandro Dalcin]

On Fri, 14 Aug 2020 at 15:23, Maarten Braakhekke <maarten.b...@gmail.com> wrote:

There is no directory $I_MPI_ROOT/intel32 on my system. During the installation it is suggested that libraries for 32bit would also be installed, but I can't find them. I'll look into it because I also want to be able to build for 32bit.

Since v5.1, it seems that 32bit libraries are no longer supported:

https://software.intel.com/sites/default/files/managed/cc/5b/intelmpi-5.1-releasenotes-windows.pdf

-- 

[Maarten Braakhekke]

I finally got around to trying building with the settings you posted. As we discussed previously there is no intel32 directory. The last version of Intel MPI to support 32bit is 4.1 and this is not available for download any more. See my post on the Intel forum.

W.r.t. 64bit, the [intelmpi-windows-64bit] section you posted contained a typo--"intel32" should "intel64". Fixing that, it builds fine with passing --mpi=intelmpi. I also tried replacing the forward slashes with backward--this works too. I guess backward slashes are only an issue for specifying the compiler.

Fine with adding my name to the Git commit message--thanks.

If you have a version that searches for the $I_MPI_ROOT env variable I can test it. I think this variable added with the installation. I can check this.

cheers,

Maarten

[Lisandro Dalcin]

On Fri, 21 Aug 2020 at 12:39, Maarten Braakhekke <maarten.b...@gmail.com> wrote:

I finally got around to trying building with the settings you posted. As we discussed previously there is no intel32 directory. The last version of Intel MPI to support 32bit is 4.1 and this is not available for download any more. See my post on the Intel forum.

W.r.t. 64bit, the [intelmpi-windows-64bit] section you posted contained a typo--"intel32" should "intel64". Fixing that, it builds fine with passing --mpi=intelmpi. I also tried replacing the forward slashes with backward--this works too. I guess backward slashes are only an issue for specifying the compiler.

Yes, I catched that typo after my email. Also, it should be 'ia32' and not 'intel32' for the 32bit path.

None: Now I'm using "impi" for the name, kind of what I do for "msmpi". If you have strong objections, I could revise that.

 

Fine with adding my name to the Git commit message--thanks.

Done, see branch below.

 

If you have a version that searches for the $I_MPI_ROOT env variable I can test it. I think this variable added with the installation. I can check this.

 

https://bitbucket.org/mpi4py/mpi4py/branch/feature/impi-windows

 

Now please I need you to teste the following:

* First, make sure the%I_MPI_ROOT% environment variable is set.

* To test the `mpi.cfg` configuration:

python setup.py clean --all

python setup.py build --mpi=impi

* To test automatic configuration based on %I_MPI_ROOT%:

python setup.py clean --all

python setup.py build

Do them both work?

[Maarten Braakhekke]

Both of the commands work.

I've never encountered the abbreviation "impi" before, so it might not be immediately obvious to people that it's Intel. But it's not a big deal--up to you.

Regarding the %I_MPI_ROOT% variable, there's an option "Intel MPI Environment Variables" (or something). I assume this needs to be selected to set it. But this is the case by default.

Cheers & thanks!

Maarten

PS If you need more help with testing on Windows let me know.

[Lisandro Dalcin]

On Tue, 25 Aug 2020 at 21:56, Maarten Braakhekke <maarten.b...@gmail.com> wrote:

Both of the commands work.

I've never encountered the abbreviation "impi" before, so it might not be immediately obvious to people that it's Intel. But it's not a big deal--up to you.

Do you think we should go for "intelmpi" instead? Not a big deal for me either, but I assume you have more experience using Intel MPI than I have. I'll wait for your final comments.

 

Regarding the %I_MPI_ROOT% variable, there's an option "Intel MPI Environment Variables" (or something). I assume this needs to be selected to set it. But this is the case by default.

Remember this is a default. Users installing mpi4py can set the environment variable manually pointing to the right location, then `python -m pip install mpi4py`. Although I understand such environment tweaks is not everyday practice for Windows users.

PS: Please let's decide together on the impi vs intelmpi. I'll wait for your comments.

[Maarten Braakhekke]

Let's go for intelmpi. A google search for "impi" resulted in one page on Intel MPI in the first 6 pages.

cheers,

Maarten

[Lisandro Dalcin]

On Wed, 26 Aug 2020 at 13:35, Maarten Braakhekke <maarten.b...@gmail.com> wrote:

Let's go for intelmpi. A google search for "impi" resulted in one page on Intel MPI in the first 6 pages.

OK, I fixed up my commits, and force pushed to the same branch.

Can you please repeat the two tests I asked before?

Once you confirm everything works, this is good to go, I'll merge into master.

Regards,

 

--
You received this message because you are subscribed to the Google Groups "mpi4py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpi4py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mpi4py/86e5e762-c0d8-4a0c-8283-ad6924ea6a08n%40googlegroups.com.

[Maarten Braakhekke]

With both build commands ("impi" replaced by "intelmpi" for the first) mpi4py builds fine. However, the optional libraries do not build. This was previously also the case but I forgot to mention this. I assume this is OK but I though I'd mention it anyway.

cheers,

Maarten

[Lisandro Dalcin]

On Thu, 27 Aug 2020 at 12:40, Maarten Braakhekke <maarten.b...@gmail.com> wrote:

With both build commands ("impi" replaced by "intelmpi" for the first) mpi4py builds fine.

Great! I'll merge the feature branch into master. Many thanks for your help.

 

However, the optional libraries do not build. This was previously also the case but I forgot to mention this. I assume this is OK but I though I'd mention it anyway.

Of course. These are optional components that require either MPE or VampirTrace libraries.

Warm regards,