Link error with PLUMED 2.5.2 on NERSC's Cori

[Sean Marks]

Hi, everyone,

I'm running into some difficulties installing PLUMED 2.5.2 on NERSC's Cori (Haswell nodes). I get through compilation just fine but then encounter issues when linking. I think it has to do in part with the fact that the compiler wrappers default to static linkage rather than dynamic linkage. I've tried a bunch of things to no avail, so I decided to reach out.

Here is the essence of my installation script.

#!/bin/bash

### Set up directories ###

# Where to install PLUMED
export plumed_install_dir="$HOME/programs/haswell/plumed/2.5.2/custom/openmp-v3/mpi"

# Home directory of PLUMED tar ball/zip archive
plumed_dir="plumed-2.5.2"

### Compilers ###

# Building on/for Haswell
module unload craype-mic-knl
module load craype-haswell

# Use wrappers
export CC=cc
export CXX=CC
export MPICC=cc
export MPICXX=CC

# Pass important flags
export FLAGS="-fPIC -xCORE-AVX2 -g -qopenmp -qoverride-limits -DMPI_ENABLED"
export CFLAGS="${FLAGS}"
export CXXFLAGS="${FLAGS}"

# Pass OpenMP flag to linker
export LDFLAGS="-qopenmp"

### Configure ###

cd $plumed_dir

# Configure using Haswell module
module unload craype-mic-knl
module load craype-haswell

set -eE

# Configure from main PLUMED directory
module list
./configure --prefix=${plumed_install_dir} \
            --enable-modules=crystallization \
            --program-suffix=_mpi

### Compile and install ###

make -j 8

make install

When it comes time to link PLUMED, the linking fails as follows:

...
Building Plumed.cmake
Building Plumed.inc, static version
Building Plumed.cmake, static version
Building Plumed.cmake, shared version
Building Plumed.cmake, runtime-linking version
/usr/bin/ld: ../wrapper/Plumed.o: in function `plumed_attempt_dlopen':
/global/u1/s/smarks/source/haswell/plumed/2.5.2/custom/openmp-v3/mpi/shared/plumed-2.5.2/src/wrapper/Plumed.h:2046: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: ../tools/DLLoader.o: in function `PLMD::DLLoader::load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
/global/u1/s/smarks/source/haswell/plumed/2.5.2/custom/openmp-v3/mpi/shared/plumed-2.5.2/src/tools/DLLoader.cpp:42: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
CC -shared -o libplumed.so ../wrapper/PlumedStatic.o libplumedKernel.so -Wl,-soname,"libplumed.so"
CC -fPIC -xCORE-AVX2 -g -qopenmp -qoverride-limits -DMPI_ENABLED -fPIC -Wall -pedantic -std=c++11  ../main/main.o ../wrapper/PlumedStatic.o libplumedKernel.so -o plumed -ldl
/usr/bin/ld: attempted static link of dynamic object `libplumedKernel.so'

-----

I have also tried the suggestion in the manual to include export CRAYPE_LINK_TYPE=dynamic before configuring (since I am using a Cray Programming Environment, 'craype-haswell'), but this leads instead to a different error when making install/kernel.o:

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: BFD (GNU Binutils; SUSE Linux Enterprise 15) 2.31.1.20180828-6.3 assertion fail ../../bfd/elf.c:3583

-----

Any help is greatly appreciated. Thanks for your time!

Sean

[Giovanni Bussi]

If you configure with —disable-dlopen (I think) the warning should disappear

--
You received this message because you are subscribed to the Google Groups "PLUMED users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to plumed-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/465765c6-f7dc-4e78-ae02-d4eea3ebe252%40googlegroups.com.

--

Sent from Gmail mobile

[Sean Marks]

Hi, Giovanni,

Thanks for your reply. According to configure -h, seems to only disable searching for dlopen, which the compiler ends up finding anyway. I used the same module environment at runtime anyway, so there should be no problem finding the correct dlopen library.

The heart of the problem seems to be the following line, which corresponds to the target $(PLUMED_MAIN_SHARED) in src/lib/Makefile:

CC -fPIC -xCORE-AVX2 -g -qopenmp -qoverride-limits -DMPI_ENABLED -fPIC -Wall -pedantic -std=c++11  ../main/main.o ../wrapper/PlumedStatic.o  libplumedKernel.so -o plumed -ldl
/usr/bin/ld: attempted static link of dynamic object `libplumedKernel.so'

Since Cori's compilers implicitly do static linkage unless told otherwise, I believe that 'CC' in this context effectively means 'CC -static'. I managed to fix the problem by including

export CRAYPE_LINK_TYPE=dynamic

before confiuring or (or manually appending '-dynamic' to the recipe). I believe this tells the compilers to instead do dynamic linkage by default, which I believe is what the PLUMED installation framework assumes as the default.

----

Once I resolved the first issue, I also found that I had to configure with --disable-ld-r (or go into Makefile.conf and comment out LD_RO) since ld -r ... was still triggering the assertion failure in the linker for some unclear-to-me reason. Unfortunately, I think that means that I can't do a static patch of PLUMED because the static archive is not created. At least I only really wanted the shared library, anyway.

----

In the end, to get at least non-static PLUMED installed, my install script boiled down to:

# Building on/for Haswell
module unload craype-mic-knl
module load craype-haswell

# Use wrappers
export CC=cc
export CXX=CC
export MPICC=cc
export MPICXX=CC

# Pass important flags

export FLAGS="-g -fPIC -xCORE-AVX2 -qopenmp -qoverride-limits -DMPI_ENABLED"

export CFLAGS="${FLAGS}"
export CXXFLAGS="${FLAGS}"

# Pass OpenMP flag to linker
export LDFLAGS="-qopenmp"

# Do dynamic linkage for flexibility
export CRAYPE_LINK_TYPE=dynamic

# Configure
./configure --prefix=${plumed_install_dir} \
            --enable-modules=crystallization \
            --program-suffix=_mpi \
            --disable-ld-r

...

----

Best,

Sean

To unsubscribe from this group and stop receiving emails from it, send an email to plumed...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/465765c6-f7dc-4e78-ae02-d4eea3ebe252%40googlegroups.com.