MPI version hard-coded into pbdR?

[Christopher Paciorek]

I just upgraded our system openMPI from version 1.8.4 to 1.8.5.  When I then tried to use pbdR, I got this message

paciorek@smeagol:~/staff/workshops/parallelR-biostat-2015 (master)> mpirun -np 12 Rscript pbd-apply.R
[smeagol:31371] [[8475,1],3] tcp_peer_recv_connect_ack: received different version from [[8475,0],0]: 1.8.5 instead of 1.8.4
[smeagol:31374] [[8475,1],5] tcp_peer_recv_connect_ack: received different version from [[8475,0],0]: 1.8.5 instead of 1.8.4
[smeagol:31417] [[8475,1],11] tcp_peer_recv_connect_ack: received different version from [[8475,0],0]: 1.8.5 instead of 1.8.4
[smeagol:31377] [[8475,1],6] tcp_peer_recv_connect_ack: received different version from [[8475,0],0]: 1.8.5 instead of 1.8.4
[smeagol:31402] [[8475,1],10] tcp_peer_recv_connect_ack: received different version from [[8475,0],0]: 1.8.5 instead of 1.8.4
....
etc.
....

Reinstalling the pbd R packages fixed the problem.

Presumably the openMPI version is hard-coded into the installed pbdR packages in some way (I didn't delve into it in any detail).

This is on a set of Ubuntu 14.04 workstations.

Just thought I would make a note of this and see if it were intentional. You do indicate on your install instructions that installing
pbdR should come after MPI, but I could easily see a person upgrading software on their system and not realizing they need
to re-install dependent R packages.

Chris

[Drew Schmidt]

Hi Chris,

That is the default behavior.  I will have to let Wei-Chen speak to the particulars, but I think the idea is to make it very easy to install for the end user, particularly on Windows.

You can use the (basically undocumented) configure option --enable-ld-library-path, the effect of which can be seen in R/zzz.r.in.  I've been meaning to expand on how this works and why someone might want it in the vignette, but haven't gotten around to it.

Thanks
-Drew

[Wei-Chen Chen]

Dear Chris,

I don't think I am able to do that check in run-time or hard-code the version for the check. The message is probably coming from OpenMPI itself started from earlier release at
http://www.open-mpi.org/community/lists/users/2015/04/26666.php

The ld-library-path will not solve this problem, because it only tells pbdMPI some locations/environments to load libmpi.so.

Sincerely,
Wei-Chen Chen

[Ostrouchov, George]

Hi Chris,

Could you be linking against a static OpenMPI library? The link that Wei-Chen sent, says the messages appear when an application is linked against the static (.a) rather than the dynamic (.so)  OpenMPI library. This may be user and locale dependent.

Thanks,

George

--
Programming with Big Data in R
Simplifying Scalability
http://r-pbd.org/
---
You received this message because you are subscribed to the Google Groups "RBigDataProgramming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rbigdataprogram...@googlegroups.com.
To post to this group, send email to rbigdatap...@googlegroups.com.
Visit this group at http://groups.google.com/group/rbigdataprogramming.
To view this discussion on the web visit https://groups.google.com/d/msgid/rbigdataprogramming/f406a746-2f63-46de-9a67-94bb3f4985d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Christopher Paciorek]

Hmm, I don't think so.  Assuming we are talking about libmpi.so in pbdMPI, here's what ldd shows:

paciorek@smeagol:/usr/local/linux/lib/R/Current/x86_64/site-library/pbdMPI/libs> ldd pbdMPI.so 

linux-vdso.so.1 =>  (0x00007ffd5c993000)

libmpi.so.1 => /usr/local/linux/openmpi/lib/libmpi.so.1 (0x00007fc793ca5000)

libR.so => /usr/lib/libR.so (0x00007fc7936f0000)

libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc79332a000)

libopen-rte.so.7 => /usr/local/linux/openmpi-1.8.5/lib/libopen-rte.so.7 (0x00007fc7930af000)

libopen-pal.so.6 => /usr/local/linux/openmpi-1.8.5/lib/libopen-pal.so.6 (0x00007fc792dd3000)

libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc792bb4000)

libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fc7913ab000)

libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc7910a5000)

libreadline.so.6 => /lib/x86_64-linux-gnu/libreadline.so.6 (0x00007fc790e5e000)

libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fc790c20000)

liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fc7909fe000)

libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fc7907ed000)

libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc7905d4000)

librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc7903cc000)

libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc7901c7000)

libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fc78ffb8000)

/lib64/ld-linux-x86-64.so.2 (0x00007fc7941b4000)

libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fc78fdb5000)

libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fc78fb8b000)

This works fine when /usr/local/linux/openmpi is a symlink to the 1.8.5 version of OpenMPI and pbdR packages were built under that configuration. (Note that we install openMPI in /usr/local/linux, which is NFS shared with all of our machines). But if I switch /usr/local/linux/openmpi to symlink to the 1.8.4 version of OpenMPI, then I get the error. 

I can do a simple mpirun invocation just fine, which makes me suspect something hard-coded into pbdR.

In any event, I'm not having problems once I installed pbdR packages after changing the OpenMPI version, so I'll just leave this thread as a note to the development team (and any users who might encounter this in the future) in the event that you are hard-coding the MPI version somehow into pbdR package(s) when they are built on a system.

Chris

To unsubscribe from this group and stop receiving emails from it, send an email to rbigdataprogramming+unsub...@googlegroups.com.