Cannot find mpi.h?

[min...@gmail.com]

Hi all,

I'm fairly new to this, so I apologize if this is a bit of a dumb question, but...

I've installed mpich and set my .bashrc as follows:

export MPI_DIR=/Users/minerd/mpich

export PATH=/Users/minerd/mpich/bin:$PATH

export LD_LIBRARY_PATH=/Users/minerd/mpich/lib:$LD_LIBRARY_PATH

export DYLD_LIBRARY_PATH=/Users/minerd/mpich/lib:$DYLD_LIBRARY_PATH

I tried to use pip to install mpi4py. But I get the following:

"

client41-on-cinque:~ minerd$ pip install mpi4py

Downloading/unpacking mpi4py

  Running setup.py egg_info for package mpi4py

    

Installing collected packages: mpi4py

  Running setup.py install for mpi4py

    MPI configuration: [mpi] from 'mpi.cfg'

    MPI configuration: [mpi] from 'mpi.cfg'

    checking for MPI compile and link ...

    checking for header 'mpi.h' ...

    xcrun clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _configtest.c -o _configtest.o

    _configtest.c:2:10: fatal error: 'mpi.h' file not found

    #include <mpi.h>

             ^

    1 error generated.

    failure.

    removing: _configtest.c _configtest.o

    failure.

    error: Cannot find 'mpi.h' header. Check your configuration!!!

    Complete output from command /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools;__file__='/private/var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-build-minerd/mpi4py/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-fD8igT-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_src

running build_py

running build_clib

MPI configuration: [mpi] from 'mpi.cfg'

running build_ext

MPI configuration: [mpi] from 'mpi.cfg'

checking for MPI compile and link ...

checking for header 'mpi.h' ...

xcrun clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _configtest.c -o _configtest.o

_configtest.c:2:10: fatal error: 'mpi.h' file not found

#include <mpi.h>

         ^

1 error generated.

failure.

removing: _configtest.c _configtest.o

failure.

error: Cannot find 'mpi.h' header. Check your configuration!!!

----------------------------------------

Command /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools;__file__='/private/var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-build-minerd/mpi4py/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-fD8igT-record/install-record.txt --single-version-externally-managed failed with error code 1 in /private/var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-build-minerd/mpi4py

Storing complete log in /Users/minerd/.pip/pip.log

client41-on-cinque:~ minerd$ 

"

I found only one reference to such an error in the group archives, and the problem ended up being an issue with mpicc <--> openmpicc, which doesn't seem to apply here. Any thoughts? Thanks.

Regards,

Daniel

[Lisandro Dalcin]

On 3 May 2013 16:55, <min...@gmail.com> wrote:
> Hi all,
>
> I'm fairly new to this, so I apologize if this is a bit of a dumb question,
> but...
>
> I've installed mpich and set my .bashrc as follows:
>
> export MPI_DIR=/Users/minerd/mpich
> export PATH=/Users/minerd/mpich/bin:$PATH
> export LD_LIBRARY_PATH=/Users/minerd/mpich/lib:$LD_LIBRARY_PATH
> export DYLD_LIBRARY_PATH=/Users/minerd/mpich/lib:$DYLD_LIBRARY_PATH
>

Are you able to get the right output from "which mpicc" after setting $PATH?

It seems you are using Homebrew. Why not just "brew install mpich" and
then install mpi4py?



--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

[min...@gmail.com]

I get "/Users/minerd/mpich/bin/mpicc" from "which mpicc." I have not, in fact, been using homebrew (I did a manual install), though I'll give it a shot here if you think it might simplify things...

Regards,

Daniel

[min...@gmail.com]

Running into a problem here with the fortran compiler and linker. First it says it needs a fortran compiler, so I installed gfortran. Now gives up with a fortran linking error. I should probably mention that this is an OSX system.

Regards,

Daniel

[Aron Ahmadia]

Hi Daniel,

It's obviously OS X from the "/Users" bit.

If I may make a recommendation, it is very hard to get a sane Fortran compiling environment on OS X, and you are really best served by installing your base system with either macports or homebrew.

Sean Farley maintains the excellent science ports package for doing computationally-focused software development on macports:

https://bitbucket.org/seanfarley/scienceports/wiki/Installation

I use homebrew (and I think Lisandro uses it as well) because I find that I need a slimmer, less comprehensive solution:

https://github.com/mxcl/homebrew/wiki/Installation

With either of these distributions, you should be able to get a reasonable Fortran stack, and MPI on top of that.

Let us know how you're getting on.

Cheers,

Aron

--
You received this message because you are subscribed to the Google Groups "mpi4py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpi4py+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[min...@gmail.com]

Alright, I've run "brew update" and "brew doctor," followed the doctor's instructions to remove potential problem files, uninstalled and reinstalled gfortran and successfully installed mpich via homebrew. Upon attempting to install mpi4py, I *still* get the mpi.h header error, plus an error regarding something called clang, with which I am not familiar. Any thoughts? The output is below:

Regards,

Daniel

pip install mpi4py

Downloading/unpacking mpi4py

  Running setup.py egg_info for package mpi4py

    

Installing collected packages: mpi4py

  Running setup.py install for mpi4py

    MPI configuration: [mpi] from 'mpi.cfg'

    MPI C compiler:    /usr/local/bin/mpicc

    MPI C++ compiler:  /usr/local/bin/mpicxx

    MPI F77 compiler:  /usr/local/bin/mpif77

    MPI F90 compiler:  /usr/local/bin/mpif90

    MPI configuration: [mpi] from 'mpi.cfg'

    MPI C compiler:    /usr/local/bin/mpicc

    MPI C++ compiler:  /usr/local/bin/mpicxx

    MPI F77 compiler:  /usr/local/bin/mpif77

    MPI F90 compiler:  /usr/local/bin/mpif90

    checking for MPI compile and link ...

    checking for header 'mpi.h' ...

    /usr/local/bin/mpicc clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _configtest.c -o _configtest.o

    clang: error: no such file or directory: 'clang'

    failure.

    removing: _configtest.c _configtest.o

    failure.

    error: Cannot find 'mpi.h' header. Check your configuration!!!

    Complete output from command /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools;__file__='/private/var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-build-minerd/mpi4py/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-pxUvRR-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_src

running build_py

running build_clib

MPI configuration: [mpi] from 'mpi.cfg'

MPI C compiler:    /usr/local/bin/mpicc

MPI C++ compiler:  /usr/local/bin/mpicxx

MPI F77 compiler:  /usr/local/bin/mpif77

MPI F90 compiler:  /usr/local/bin/mpif90

running build_ext

MPI configuration: [mpi] from 'mpi.cfg'

MPI C compiler:    /usr/local/bin/mpicc

MPI C++ compiler:  /usr/local/bin/mpicxx

MPI F77 compiler:  /usr/local/bin/mpif77

MPI F90 compiler:  /usr/local/bin/mpif90

checking for MPI compile and link ...

checking for header 'mpi.h' ...

/usr/local/bin/mpicc clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _configtest.c -o _configtest.o

clang: error: no such file or directory: 'clang'

failure.

removing: _configtest.c _configtest.o

failure.

error: Cannot find 'mpi.h' header. Check your configuration!!!

----------------------------------------

Command /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools;__file__='/private/var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-build-minerd/mpi4py/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-pxUvRR-record/install-record.txt --single-version-externally-managed failed with error code 1 in /private/var/folders/1v/f0jkxc6n10x_wywlx9dvp0v00000gn/T/pip-build-minerd/mpi4py

Storing complete log in /Users/minerd/.pip/pip.log

[Aron Ahmadia]

mpi.h is a red herring here, the true error is being reported here:

/usr/local/bin/mpicc clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _configtest.c -o _configtest.o

clang: error: no such file or directory: 'clang'

I'm not sure why "clang" is showing up in the call to mpicc here, but it shouldn't.  It looks like you have a clean Python installation, and I can't imagine that coming from mpi4py's setup.  Can you report the output of:

printenv | clang

(strip out anything sensitive before pasting it here, I'm interested to see if somehow clang has slipped into one of the environment variables that distutils checks when compiling).

A

[min...@gmail.com]

Nothing much sensitive to strip; the output is simply as follows:

printenv | clang

clang: error: no input files

Thanks for troubleshooting all this with me.

Regards,

Daniel

[Aron Ahmadia]

Sorry, that should have been:

printenv | grep clang

You can find me at aron.a...@gmail.com if you want to try and sort this out over GTalk, we can then summarize our findings to this thread.

A

[min...@gmail.com]

I have to run to a meeting shortly, but in the mean time, I can tell you that printenv | grep clang produces no output at all. If you're around when I get back from the meeting in an hour or so, then let's see if we can work it out over gchat as you propose.

Regards,

Daniel

[Aron Ahmadia]

It turns out that some OS X Python installations have a slightly bogus setup:

CC = xcrun

CFLAGS = clang

We would have been to override normally if CC="xcrun clang", this is just a funny interaction between mpicc, a script that acts as a new compiler but still inherits cflags from the old compiler, and xcrun, another script that acts as an indirection to the actual compiler but has a mandatory flag to work properly.

One option for making this more robust in the future would be to grab the compile/link flags from the MPI installation, then overlay those over distutils. I'm not sure this would be worth the effort, as this problem is likely to be fairly uncommon.  

Perhaps just a check for the CC = xcrun business and then moving clang over from CFLAGS to CC for our run-time?

A

[jal...@lbl.gov]

This actually helps me. 

what I did on NERSC is 

export CC=cc

and it went through.

Best,

Jialin