missing mpi4py in site-packages (install problem on RHEL / scientific linux)

[geoffreyof...@gmail.com]

Hello,

The site-packages dir is missing the mpi4py package directory. 

[ga@sam site-packages]$ pwd

/usr/lib/python2.6/site-packages

[ga@sam site-packages]$ ls mp*

ls: cannot access mp*: No such file or directory

 Also, the error message

 

ImportError: libmpi.so.1: cannot open shared object file: No such file or directory

is shown when this command is executed:

[root@sam mpi4py-1.3]# mpiexec -n 2 python demo/helloworld.py

Perhaps these two problems are related??

Here is all the relevant information which I can imagine might be helpful....

BEFOREHAND, these two commands were executed in directory

 ~/Downloads/mpi4py-1.3/ 

with apparently no errors, it all seemed to run OK:

(as a user) python setup.py build

(as the su) python setup.py install

After encountering these aforementioned problems, I next also attempted as a fix, to run the following command:

yum install mpi4py-openmpi

However it returned the error message:

No package mpi4py-openmpi available.

Can you give some assistance to get the mpi4py helloworld.py program up and running on Red Hat Enterprise Linux or Scientific Linux 6.2?  I am running the latter, which is a clone of RHEL. 

Platform details:

[ga@sam site-packages]$ mpiexec

mpiexec (OpenRTE) 1.5.3

[ga@sam site-packages]$ mpicc --version

gcc (GCC) 4.4.6 20110731 (Red Hat 4.4.6-3)

[ga@sam site-packages]$ python

Python 2.6.6 (r266:84292, Jun 18 2012, 09:57:52) 

[GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2

File libmpi.so.0 is in directory /usr/lib64/compat-openmpi/lib

File libmpi.so is in directory /usr/lib64/openmpi/lib

It's probably something simple to do but I am not very expert as a system admin. I'm learning.

I am surprised that the python sitepackages directory did not have the expected mpi4py files, because the install and build "seemed to work fine" for mpi4py.

Thank you

Geoffrey

[Aron Ahmadia]

Can you show a complete log of your install?  As a general rule, I don't recommend that you ever run anything as root unless you have to.  Just install the packages to some local directory and then append your PYTHONHOME directory appropriately.  I strongly recommend the approach outlined in this SO answer if you'd like to have Python manage your local installs for you using virtualenv:

http://stackoverflow.com/a/5177027/122022

You're showing a couple of different problems which I'm not going to address, let's make sure we can get you installed first :)

A

[Lisandro Dalcin]

For Python 2.6 and up, user-type installs are the quickest way to get
up and running. Just make sure "mpicc" is in your $PATH, (run "which
mpicc" to verify)

$ python setup.py install --user

This uses ~/.local as installation prefix, this path is automatically
added to Python's search path, so no extra configuration required.

BTW, if you are running on a Red Hat system, mpi4py is available in
the Fedora yum repositories.


--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

[Aron Ahmadia]

That's new to me.  Thanks for the pointers Lisandro.

A

[Geoffrey Anderson]

Using the supplied advice, the so file is still not being found, and the yum repository as I said does not contain the package for my OS.

Moreover, I found that to get a clean build and install of the setup.py program required editing the mpi.cfg file to remove the hard-coded version numbers and other mismatches on my OS from the search paths.

Please give some tips on how to get the required so file to be found, if you can.

Thank you,

Geoffrey

Supporting details:

[ga@sam mpi4py-1.3]$ python setup.py install --user

running install

running build

running build_src

running build_py

running build_clib

MPI configuration: [mpi] from 'mpi.cfg'

MPI C compiler:    /usr/lib64/openmpi/bin/mpicc

MPI C++ compiler:  /usr/lib64/openmpi/bin/mpicxx

MPI F77 compiler:  /usr/lib64/openmpi/bin/mpif77

MPI F90 compiler:  /usr/lib64/openmpi/bin/mpif90

running build_ext

MPI configuration: [mpi] from 'mpi.cfg'

MPI C compiler:    /usr/lib64/openmpi/bin/mpicc

MPI C++ compiler:  /usr/lib64/openmpi/bin/mpicxx

MPI F77 compiler:  /usr/lib64/openmpi/bin/mpif77

MPI F90 compiler:  /usr/lib64/openmpi/bin/mpif90

running build_exe

MPI configuration: [mpi] from 'mpi.cfg'

MPI C compiler:    /usr/lib64/openmpi/bin/mpicc

MPI C++ compiler:  /usr/lib64/openmpi/bin/mpicxx

MPI F77 compiler:  /usr/lib64/openmpi/bin/mpif77

MPI F90 compiler:  /usr/lib64/openmpi/bin/mpif90

running install_lib

creating /home/ga/.local/lib/python2.6/site-packages/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/rc.py -> /home/ga/.local/lib/python2.6/site-packages/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/__init__.py -> /home/ga/.local/lib/python2.6/site-packages/mpi4py

creating /home/ga/.local/lib/python2.6/site-packages/mpi4py/include

creating /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/mpi4py.MPI_api.h -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/mpi4py.h -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/mpi4py.MPI.h -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/__init__.pxd -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/MPI.pxd -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/mpi_c.pxd -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/__init__.pyx -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/mpi.pxi -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/include/mpi4py/mpi4py.i -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/include/mpi4py

creating /home/ga/.local/lib/python2.6/site-packages/mpi4py/lib-pmpi

copying build/lib.linux-x86_64-2.6/mpi4py/lib-pmpi/libmpe.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/lib-pmpi

copying build/lib.linux-x86_64-2.6/mpi4py/lib-pmpi/libvt.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/lib-pmpi

copying build/lib.linux-x86_64-2.6/mpi4py/lib-pmpi/libvt-mpi.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/lib-pmpi

copying build/lib.linux-x86_64-2.6/mpi4py/lib-pmpi/libvt-hyb.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/lib-pmpi

copying build/lib.linux-x86_64-2.6/mpi4py/MPI.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/mpi.cfg -> /home/ga/.local/lib/python2.6/site-packages/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/MPE.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py

copying build/lib.linux-x86_64-2.6/mpi4py/dl.so -> /home/ga/.local/lib/python2.6/site-packages/mpi4py

creating /home/ga/.local/lib/python2.6/site-packages/mpi4py/bin

copying build/lib.linux-x86_64-2.6/mpi4py/bin/python-mpi -> /home/ga/.local/lib/python2.6/site-packages/mpi4py/bin

byte-compiling /home/ga/.local/lib/python2.6/site-packages/mpi4py/rc.py to rc.pyc

byte-compiling /home/ga/.local/lib/python2.6/site-packages/mpi4py/__init__.py to __init__.pyc

writing byte-compilation script '/tmp/tmp9EPfYM.py'

/usr/bin/python -O /tmp/tmp9EPfYM.py

removing /tmp/tmp9EPfYM.py

running install_egg_info

Writing /home/ga/.local/lib/python2.6/site-packages/mpi4py-1.3-py2.6.egg-info

[ga@sam mpi4py-1.3]$ python demo/helloworld.py

Traceback (most recent call last):

  File "demo/helloworld.py", line 6, in <module>

    from mpi4py import MPI

ImportError: libmpi.so.1: cannot open shared object file: No such file or directory

[Lisandro Dalcin]

On 3 August 2012 13:56, Geoffrey Anderson

<geoffreyof...@gmail.com> wrote:
> Using the supplied advice, the so file is still not being found, and the yum
> repository as I said does not contain the package for my OS.
>

Are you using Fedora? If not, you can still download the RPMs and
install them with "yum localinstall ..."

> Moreover, I found that to get a clean build and install of the setup.py
> program required editing the mpi.cfg file to remove the hard-coded version
> numbers and other mismatches on my OS from the search paths.
>
> Please give some tips on how to get the required so file to be found, if you
> can.
>

Two ways to get it working, I recommend the second:

1) Add the following line in your ~/.bashrc file:

export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib:$LD_LIBRARY_PATH


2) Add the following line in mpi.cfg under [mpi] section, then rebuild
and install:

runtime_library_dirs = /usr/lib64/openmpi/lib

[Aron Ahmadia]

He mentioned in the subject that he's on Scientific Linux.  I'm a little surprised that they don't have mpi4py in their repo.  Do you think it's worth pinging the maintainers?

A

--
You received this message because you are subscribed to the Google Groups "mpi4py" group.

To post to this group, send email to mpi...@googlegroups.com.
To unsubscribe from this group, send email to mpi4py+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mpi4py?hl=en.

[Lisandro Dalcin]

On 3 August 2012 14:12, Aron Ahmadia <ar...@ahmadia.net> wrote:
> He mentioned in the subject that he's on Scientific Linux. I'm a little
> surprised that they don't have mpi4py in their repo. Do you think it's
> worth pinging the maintainers?
>

As Fedora has all the RPM spec files for packaging mpi4py, it should
be easy for them to provide mpi4py.

[Thomas Spura]

On Fri, Aug 3, 2012 at 7:14 PM, Lisandro Dalcin <dal...@gmail.com> wrote:
> On 3 August 2012 14:12, Aron Ahmadia <ar...@ahmadia.net> wrote:
>> He mentioned in the subject that he's on Scientific Linux. I'm a little
>> surprised that they don't have mpi4py in their repo. Do you think it's
>> worth pinging the maintainers?

Pong.

> As Fedora has all the RPM spec files for packaging mpi4py, it should
> be easy for them to provide mpi4py.

I just requested the el6 branch [1] for EPEL [2].
Currently, it seems el6 doesn't have a mpich2 package yet [3].
@deji, is that branch missing by intention?


Greetings,
Tom


[1] https://bugzilla.redhat.com/show_bug.cgi?id=562504#c37
[2] https://fedoraproject.org/wiki/EPEL
[3] https://admin.fedoraproject.org/pkgdb/acls/name/mpich2

[Aron Ahmadia]

Thanks Tom :)

A

[Thomas Spura]

On Fri, Aug 3, 2012 at 11:42 PM, Thomas Spura <thomas...@gmail.com> wrote:
> On Fri, Aug 3, 2012 at 7:14 PM, Lisandro Dalcin <dal...@gmail.com> wrote:
>> On 3 August 2012 14:12, Aron Ahmadia <ar...@ahmadia.net> wrote:
>>> He mentioned in the subject that he's on Scientific Linux. I'm a little
>>> surprised that they don't have mpi4py in their repo. Do you think it's
>>> worth pinging the maintainers?
>
> Pong.
>
>> As Fedora has all the RPM spec files for packaging mpi4py, it should
>> be easy for them to provide mpi4py.
>
> I just requested the el6 branch [1] for EPEL [2].
> Currently, it seems el6 doesn't have a mpich2 package yet [3].

There is no mpich2 on ppc64 for el6, so there won't be a mpi4py-mpich2
package either.
The branch is up and mpi4py built [4] and an update is waiting at [5].
Please test it and add karma.

Greetings,
Tom

[1] https://bugzilla.redhat.com/show_bug.cgi?id=562504#c37
[2] https://fedoraproject.org/wiki/EPEL
[3] https://admin.fedoraproject.org/pkgdb/acls/name/mpich2

[4] http://koji.fedoraproject.org/koji/taskinfo?taskID=4359282
[5] https://admin.fedoraproject.org/updates/mpi4py-1.3-5.el6