import Error : for MPI.so undefined symbol : MPI_UNWEIGHTED

[kamal sutaria]

Hi

can anyone help me on following error?


Traceback (most recent call last):
  File "helloworld.py", line 6, in <module>
    from mpi4py import MPI
ImportError: /usr/local/lib/python2.7/dist-packages/mpi4py/MPI.so: undefined symbol: MPI_UNWEIGHTED
Traceback (most recent call last):
  File "helloworld.py", line 6, in <module>
    from mpi4py import MPI
ImportError: /usr/local/lib/python2.7/dist-packages/mpi4py/MPI.so: undefined symbol: MPI_UNWEIGHTED
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun.openmpi detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[8323,1],2]
  Exit code:    1
--------------------------------------------------------------------------

[Lisandro Dalcin]

My guess is that you built mpi4py with an MPI that is not the same you
are using for running.


1) What is the output of

$ ldd /usr/local/lib/python2.7/dist-packages/mpi4py/MPI.so

2) What is the output of

$ ompi_info --version

> --
> You received this message because you are subscribed to the Google Groups
> "mpi4py" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mpi4py+un...@googlegroups.com.
> To post to this group, send email to mpi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/mpi4py.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mpi4py/7ab608f5-d8e7-4170-9711-6bfa3acfc5be%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Lisandro Dalcin
============
Research Scientist
Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
Extreme Computing Research Center (ECRC)
King Abdullah University of Science and Technology (KAUST)
http://ecrc.kaust.edu.sa/

4700 King Abdullah University of Science and Technology
al-Khawarizmi Bldg (Bldg 1), Office # 0109
Thuwal 23955-6900, Kingdom of Saudi Arabia
http://www.kaust.edu.sa

Office Phone: +966 12 808-0459

[kamal sutaria]

Thank you for your reply..!!

here is the details that you required..

    vvpadmin@system2:~$ ldd /usr/local/lib/python2.7/dist-packages/mpi4py/MPI.so
    linux-vdso.so.1 =>  (0x00007fff955ce000)
    libmpi.so.12 => /usr/lib/libmpi.so.12 (0x00007f42c0973000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f42c0756000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f42c038c000)
    libibverbs.so.1 => /usr/lib/libibverbs.so.1 (0x00007f42c017d000)
    libopen-rte.so.12 => /usr/lib/libopen-rte.so.12 (0x00007f42bff03000)
    libopen-pal.so.13 => /usr/lib/libopen-pal.so.13 (0x00007f42bfc65000)
    /lib64/ld-linux-x86-64.so.2 (0x000055e817179000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f42bfa61000)
    libhwloc.so.5 => /usr/lib/x86_64-linux-gnu/libhwloc.so.5 (0x00007f42bf827000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f42bf61e000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f42bf41b000)
    libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f42bf210000)
    libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f42bf005000)

vvpadmin@system2:~$ ompi_info --version
Open MPI v1.10.2

http://www.open-mpi.org/community/help/

[Lisandro Dalcin]

Given that mpi4py is on dist-packages, I assume you installed mpi4py
with apt, right? In such case, the mpi4py package you installed is
somehow broken, a does not work with the openmpi package, and there is
nothing easy we can do about it.

My recommendation would be to remove your current installation with
"apt-get remove python-mpi4py" , and install directly from the git
repository with "pip install
https://bitbucket.org/mpi4py/mpi4py/get/master.tar.gz".

> https://groups.google.com/d/msgid/mpi4py/c0eaa7b3-4c63-4466-8eed-90ca3d8f98c4%40googlegroups.com.

[kamal sutaria]

thank you very much sir..!!

As per your recommendation, i have remove current installation of python-mpi4py.

than i tried to use following command to install mpi4py from "https://bitbucket.org/mpi4py/mpi4py/get/master.tar.gz"

$ sudo pip install wget https://bitbucket.org/mpi4py/mpi4py/get/master.tar.gz


but i get following error

Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-LJQAJ0-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-Mt3nJQ-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-LJQAJ0-build/

[Lisandro Dalcin]

On 14 July 2017 at 06:44, kamal sutaria <kamal....@gmail.com> wrote:
> thank you very much sir..!!
>
> As per your recommendation, i have remove current installation of
> python-mpi4py.
>
> than i tried to use following command to install mpi4py from
> "https://bitbucket.org/mpi4py/mpi4py/get/master.tar.gz"
>
> $ sudo pip install wget
> https://bitbucket.org/mpi4py/mpi4py/get/master.tar.gz
>

Did you actually added wget? Please don't, pip is able to download by
itself. Just to be sure, do it this way:

pip install --user Cython # to generate the C wrappers
pip install --user https://bitbucket.org/mpi4py/mpi4py/get/master.tar.gz

Please notice the "--user" flag, this will install the packages under
~/.local , so you do not risk messing up with the system install
directories, also note you DO NOT NEED "sudo" for this, as the
packages will install under your home directory.

[kamal sutaria]

Thank you sir..!!

its works fine, as per your suggestion i have installed mpi4py using pip,

but now i am using single machine,  on that i am trying to execute following command,

$ mpirun.openmpi -np 2 -machinefile /home/kks/pi/mpi_testing/ python  helloworld.py

but i get the following error

"input in flex scanner failed"

[kamal sutaria]

Thank you sir..!!

its works fine, as per your suggestion i have installed mpi4py using pip,

but now i am using single machine,  on that i am trying to execute following command,

$ mpirun.openmpi -np 2 -machinefile /home/kks/pi/mpi_testing/ python  helloworld.py

but i get the following error

"input in flex scanner failed"

[Lisandro Dalcin]

On 18 July 2017 at 06:13, kamal sutaria <kamal....@gmail.com> wrote:
> Thank you sir..!!
>
> its works fine, as per your suggestion i have installed mpi4py using pip,
>
> but now i am using single machine, on that i am trying to execute following
> command,
>
> $ mpirun.openmpi -np 2 -machinefile /home/kks/pi/mpi_testing/ python
> helloworld.py
>
> but i get the following error
>
> "input in flex scanner failed"
>

This is unrelated to mpi4py, should be a problem with your machinefile.

[geof....@gmail.com]

Hi Lisandro --

It appears that both, from the pip I got from apt-get python3-pip, and the python3-mpi4py apt-get package, there seems to be a dependency on ompi_mpi_logical8 that I can't shake.

I'm using MVAPICH2.

I guess I'll try building it.

:(

It looks really cool and easy to write handy python mpi.  It was making my life easier in one area, but the dependency area is hurting.

G

[Lisandro Dalcin]

On Sat, 27 Oct 2018 at 22:00, <geof....@gmail.com> wrote:

Hi Lisandro --

It appears that both, from the pip I got from apt-get python3-pip, and the python3-mpi4py apt-get package, there seems to be a dependency on ompi_mpi_logical8 that I can't shake.

That's because the mpi4py package you are installing was built with Open MPI.

 

I'm using MVAPICH2.

I guess I'll try building it.

:(

It looks really cool and easy to write handy python mpi.  It was making my life easier in one area, but the dependency area is hurting.

And what am I supposed to do about it? If you are using an MPI implementation different than the one your Linux distro provides, then you have to build from sources. Unfortunately, the MPI API was not originally designed to be implementation-independent (i.e. ABI-compatible across different implementations).

If you have the `mpicc` compiler wrapper from MVAPICH2 in your path, then apt uninstall mpi4py  use plain 'pip install mpi4py' and you should get it installed. If that fails, well, come back and we will sort it out.

[Geof Sawaya]

Hi Lisandro, thank you for your response.

I was able to build with pip from source.

My problem now is that I’m getting, at import, ‘auth_munge.so: missing symbol ‘slurm_debug’.

Have you heard of this one before?

There’s a mention of this on the Slurm mailing list, but don’t see that there was any resolution.

--
You received this message because you are subscribed to a topic in the Google Groups "mpi4py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mpi4py/7JZrDDfH6SI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mpi4py+un...@googlegroups.com.

To post to this group, send email to mpi...@googlegroups.com.
Visit this group at https://groups.google.com/group/mpi4py.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpi4py/CAEcYPwAkJf_2VyV7BZnxbbMUgmNKZnA1mOi7cR890_CVUnyK0w%40mail.gmail.com.

[Lisandro Dalcin]

On Sat, 27 Oct 2018 at 23:41, Geof Sawaya <geof....@gmail.com> wrote:

Hi Lisandro, thank you for your response.

I was able to build with pip from source.

My problem now is that I’m getting, at import, ‘auth_munge.so: missing symbol ‘slurm_debug’.

Mmm... looks like the MVAPICH shared library is not linked properly to the SLURM shared library.

What's the output of `mpicc -show` ? What's the output of running the `ldd` command on the MPI shared library?

You can try to workaround the issue on the Python side at runtime this way.

from mpi4py.dl import *

handle = dlopen('libslurm.so', RTLD_NOW|RTLD_GLOBAL)

if not handle: print(dlerror())

from mpi4py import MPI

PS: You may need to specify the full path to libslurm.so.

[Geof Sawaya]

Mm, thanks for thinking about this.  I am able to run C MPI through my MVAPICH.

I was looking at your mpi lib ldd, and it seemed to have the correct dependencies.

I may have some version conflicts mixed into the system 😜

Thanks for the lesson in selective importation as well, Lisandro

--

You received this message because you are subscribed to a topic in the Google Groups "mpi4py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mpi4py/7JZrDDfH6SI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mpi4py+un...@googlegroups.com.
To post to this group, send email to mpi...@googlegroups.com.
Visit this group at https://groups.google.com/group/mpi4py.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpi4py/CAEcYPwCJ4wssTkOgmO-P%2B3gFm9w_Sp%3Drzi2oEnSv%3Da8u79nrtQ%40mail.gmail.com.

[Geof Sawaya]

Hi Lisandro --

Man, no luck.  I've verified that I've got the right libraries.  

I believe that something you do in mpi4py uncovers some kind of a linkage bug in slurm.

Here's one reference: https://bugs.schedmd.com/show_bug.cgi?id=1508

I'm so sad that I can't use this right now!

Best -- Geof

--
You received this message because you are subscribed to a topic in the Google Groups "mpi4py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mpi4py/7JZrDDfH6SI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mpi4py+un...@googlegroups.com.
To post to this group, send email to mpi...@googlegroups.com.
Visit this group at https://groups.google.com/group/mpi4py.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpi4py/6e51fb5e-c37f-4acf-a2b3-dcc60b113a49%40googlegroups.com.

[jianguo si]

I also meet a similar error as following:

ImportError: /share/home/jgsi/anaconda3/envs/sscha/lib/python3.10/site-packages/mpi4py/MPI.cpython-310-x86_64-linux-gnu.so: undefined symbol: MPI_Aint_add

and I checked the dependens of  */MPI.cpython-310-x86_64-linux-gnu.so, the output is 

linux-vdso.so.1 =>  (0x00007ffdb07e9000)
libmpi.so.40 => /share/home/jgsi/anaconda3/envs/sscha/lib/libmpi.so.40 (0x00002ab99fb0c000)
libc.so.6 => /lib64/libc.so.6 (0x00002ab99fc2e000)
libopen-rte.so.40 => /share/home/jgsi/anaconda3/envs/sscha/lib/./libopen-rte.so.40 (0x00002ab99fffc000)
libopen-pal.so.40 => /share/home/jgsi/anaconda3/envs/sscha/lib/./libopen-pal.so.40 (0x00002ab9a00b6000)
librt.so.1 => /lib64/librt.so.1 (0x00002ab9a01b6000)
libm.so.6 => /lib64/libm.so.6 (0x00002ab9a03be000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ab9a06c0000)
/lib64/ld-linux-x86-64.so.2 (0x00002ab99f8e8000)
libz.so.1 => /share/home/jgsi/anaconda3/envs/sscha/lib/././libz.so.1 (0x00002ab99fa7d000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002ab9a08dc000)
libutil.so.1 => /lib64/libutil.so.1 (0x00002ab9a0ae0000)

and the output of ompi_info --version is 

Open MPI v4.1.5

http://www.open-mpi.org/community/help/

What should I do to deal with it ?

 I have removed mpi4py and re-install it, but nothing changed.

[Lisandro Dalcin]

You should provide us with the full details to reproduce the issue.

1) How did you exactly created the anaconda environment (that is, list of packages)? 

2) What's the output of "conda list"

3) How did you install mpi4py? Using conda or pip? 

4) If you installed via pip, did you try cleaning the pip cache first? Please read https://mpi4py.readthedocs.io/en/latest/install.html#using-pip

So far, all seems to indicate that you are somehow using mpi4py built with an old Open MPI version. That could happen if mpi4py comes from an old wheel artifact stored in pip's cache.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpi4py/58bcfd2b-a880-46f8-9869-fba1eaf7afe4n%40googlegroups.com.

--

Lisandro Dalcin
============
Senior Research Scientist