CP2K compilation errors

Eric Shamay

unread,

May 23, 2007, 6:36:22 PM5/23/07

to cp2k

It's my hope that this group is also meant for posting problems
related to installation/compilation of cp2k. If not, please excuse the
post!

I'm in the midst of an installation of the popt for the Linux-ia64-
intel architecture. I've been able to compile without error previously
(about 6 months ago), but a newer version results in an error that
reads as follows:

<snippet>
mpif90 -FR -c -O2 -Vaxlib -tpp2 xc_pot_saop.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 constraint.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 qs_wf_history_methods.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 qs_environment_methods.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 harris_force.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 harris_functional.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 qs_electric_field_gradient.f90
mpif90 -FR -c -O2 -Vaxlib -tpp2 topology.f90
fortcom: Severe: **Internal compiler error: segmentation violation
signal raised** Please report this error along with the circumstances
in which it occurred in a Software Problem Report. Note: File and
line given may not be explicit cause of this error.

compilation aborted for mc_types.f90 (code 3)
make[2]: *** [mc_types.o] Error 3
make[2]: *** Waiting for unfinished jobs....
rm cp2k.f90
make[2]: Leaving directory `/mnt/netapp/home/users/eshamay/tmp/cp2k/
obj/Linux-ia64-intel/popt'
make[1]: *** [build] Error 2
make[1]: Leaving directory `/mnt/netapp/home/users/eshamay/tmp/cp2k/
makefiles'
make: *** [popt] Error 2
</snippet>

It seems to keep hanging on the compilation of mc_types.f90. Does
anyone have any advice for this particular build? Should I try to pull
an older version of that file from the CVS and recompile? Any help
would be greatly appreciated.

Thanks,
~Eric Shamay

Teodoro Laino

unread,

May 23, 2007, 6:50:30 PM5/23/07

to cp...@googlegroups.com

Ciao Eric,

>
> I'm in the midst of an installation of the popt for the Linux-ia64-
> intel architecture. I've been able to compile without error previously
> (about 6 months ago), but a newer version results in an error that
> reads as follows:
>
> <snippet>
> mpif90 -FR -c -O2 -Vaxlib -tpp2 xc_pot_saop.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 constraint.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 qs_wf_history_methods.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 qs_environment_methods.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 harris_force.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 harris_functional.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 qs_electric_field_gradient.f90
> mpif90 -FR -c -O2 -Vaxlib -tpp2 topology.f90
> fortcom: Severe: **Internal compiler error: segmentation violation
> signal raised** Please report this error along with the circumstances
> in which it occurred in a Software Problem Report. Note: File and
> line given may not be explicit cause of this error.
>

This to me looks clearly as a bug in the compiler..
could you please report the version of the compiler you're using?
I'm aware of people using the intel compiler (latest version) to
compile cp2k.
You may want to update to the latest available version...

Cheers
Teo

Eric Shamay

unread,

May 23, 2007, 6:57:02 PM5/23/07

to cp...@googlegroups.com

Thanks for the reply Teo,

I'm compiling with :
Intel(R) Fortran Itanium(R) Compiler for Itanium(R)-based applications
Version 9.1 Build 20060707 Package ID: l_fc_c_9.1.036

On 5/23/07, Teodoro Laino <teodor...@gmail.com> wrote:

Ciao Eric,

This to me looks clearly as a bug in the compiler..
could you please report the version of the compiler you're using?
I'm aware of people using the intel compiler (latest version) to
compile cp2k.
You may want to update to the latest available version...

Cheers
Teo

--
~E

Teodoro Laino

unread,

May 24, 2007, 1:54:12 AM5/24/07

to cp...@googlegroups.com

There's a more recent version available that you may want to try:

Package id is l_fc_c_9.1.043. This package contains the following files:

Full product (File l_fc_c_9.1.043.tar.gz): Gzipped tar file containing the Intel(r) Fortran Compiler for Linux*, Version 9.1.043, for users who develop on and build for IA-32, Intel 64, and IA-64 architectures, and Intel(r) Debugger Version 9.1-27. Compiler Build: 20070215. Debugger Build: 20070202.

Cheers

teo

Nichols A. Romero

unread,

May 24, 2007, 2:54:56 PM5/24/07

to cp...@googlegroups.com

9.1.041 also works for me.

--
Nichols A. Romero, Ph.D.
1613 Denise Dr. Apt. D
Forest Hill, MD 21050
443-567-8328 (C)
410-306-0709 (O)

Eric Shamay

unread,

May 28, 2007, 2:57:56 PM5/28/07

to cp2k

Thank you for the help with the compiler. I installed a newer version
and overcame the previous error. Now I'm stuck with an error involving
libmkl_scalapack:

IPO link: can not find -lmkl_scalapack
ifort: error: problem during multi-file optimization compilation (code
1)

Installing the latest version of Intel's mkl does not include
libmkl_scalapack, and so I'm left wondering where I can find that
particular library.

Thanks,
~Eric

On May 24, 11:54 am, "Nichols A. Romero" <narom...@gmail.com> wrote:
> 9.1.041 also works for me.
>

> On 5/24/07, Teodoro Laino <teodoro.la...@gmail.com> wrote:

akohlmey

unread,

May 28, 2007, 3:12:21 PM5/28/07

to cp2k

On May 28, 2:57 pm, Eric Shamay <eric.sha...@gmail.com> wrote:
> Thank you for the help with the compiler. I installed a newer version
> and overcame the previous error. Now I'm stuck with an error involving
> libmkl_scalapack:
>
> IPO link: can not find -lmkl_scalapack
> ifort: error: problem during multi-file optimization compilation (code
> 1)
>
> Installing the latest version of Intel's mkl does not include
> libmkl_scalapack, and so I'm left wondering where I can find that
> particular library.

eric,
you have to either use intel's cluster mkl or compile
BLACS/ScaLAPACK yourself.

you can see my notes on how i compiled both packages for our group at:
https://www.liniac.upenn.edu//wiki/tiki-index.php?page=scalapack+for+CMM

you will have to adapt the configuration files for your MPI
installation.
when compiling BLACS/ScaLAPACK it is essential to verify your
compilation
with the corresponding testers. BLACS should pass completely, in
ScaLAPACK
you may get away with a few tests failing (minor). the success of the
latter
depends also a lot on the optimizations in the BLAS/LAPACK that is
used
with ScaLAPACK. i had the best successes with mkl.
it is important not to overoptimize.

hope this helps,
axel.

Eric Shamay

unread,

May 29, 2007, 8:37:08 PM5/29/07

to cp2k

I've managed to get the entire program to compile. Here are the
pertinent stats of the system:

firstly, my arch file
---------------------------------------------
PERL = perl
CC = mpicc
CPP = cpp
FC = mpif90 -FR
FC_fixed = mpif90 -FI
LD = mpif90
AR = ar -r
MKLPATH = $(MKL)/lib/64
DFLAGS = -D__INTEL -D__FFTSG -D__FFTW3 -D__parallel -D__BLACS -
D__SCALAPACK -D__FFTMKL -I$(FFTW)/include -I$(MKL)/include
CPPFLAGS = -C $(DFLAGS) -P -traditional
FCFLAGS = -O2 -Vaxlib -tpp2
LDFLAGS = $(FCFLAGS) -L$(HOME)/lib -L$(FFTW)/lib -L$(MPI)/lib -L$
(MKL)/lib/64
#LIBS = -lmkl_scalapack -lmkl_blacsF77init -lmkl_blacs\
# -lmkl_lapack -lmkl -lguide -lmpi -lfftw3
LIBS = $(MKLPATH)/libmkl_blacs.a\
$(MKLPATH)/libmkl_scalapack.a\
$(MKLPATH)/libmkl_blacsF77init.a\
$(MKLPATH)/libmkl_lapack.a\
$(MKLPATH)/libmkl_blacs_intelmpi.a\
$(MKLPATH)/libguide.a\
-lpthread -lmpi -lmkl -lfftw3

OBJECTS_ARCHITECTURE = machine_intel.o
---------------------------------------------

intel/9.1 (for the compilers)

> mpif90 -V

Intel(R) Fortran Itanium(R) Compiler for Itanium(R)-based applications

Version 9.1 Build 20070320 Package ID: <installpackageid>

>mpicc -V
Intel(R) C Itanium(R) Compiler for Itanium(R)-based applications
Version 9.1 Build 20070320 Package ID: <installpackageid>

mkl/7.2cluster_64

fftw/3.1.1

mpi/openmpi-1.1.5_intel-9.1

This setup finishes through the entire compilation and exits without
error. I then attempt to run a test as per the INSTALL file but by
running the command:

> mpirun -np 8 $PATH_TO_CP2KEXE H2O.inp > H2O.out

and the program produces the following output:
--------------------------------------------------------------------------------------------------------------------------------------------
username@optix has created process number
0 14411
username@optix has created process number
1 14412
username@optix has created process number
2 14413
username@optix has created process number
3 14414
username@optix has created process number
4 14415
username@optix has created process number
5 14416
username@optix has created process number
6 14417
username@optix has created process number
7 14418

**** **** ****** ** PROGRAM STARTED AT Tue May 29
17:26:39 2007
***** ** *** *** ** PROGRAM STARTED
ON optix
** **** ****** PROGRAM STARTED
BY username
***** ** ** ** ** PROGRAM PROCESS
ID 14411
**** ** ******* ** PROGRAM STARTED IN /mnt/netapp/home/users/
eshamay/tmp/cp
2k/tests/QS

6 additional processes aborted (not shown)
--------------------------------------------------------------------------------------------------------------------------------------------

This is followed by a series of mpi segfault/crash error that look
like :

17:35:12 eshamay@optix (~/tmp/cp2k/tests/QS) Signal:11 info.si_errno:
0(Success) si_code:2(SEGV_ACCERR)
Failing at addr:0xf3
[0] func:/usr/local/packages/openmpi/1.1_intel-9.1/lib/libopal.so.0
[0x200000000044c800]
[1] func:[0xa0000000000107e0]
[2] func:/usr/local/packages/openmpi/1.1_intel-9.1/lib/libmpi.so.0
[0x2000000000182dd0]
[3] func:/usr/local/packages/openmpi/1.1_intel-9.1/lib/libmpi.so.
0(MPI_Comm_size-0x9d3c20) [0x2000000000182c20]
[4] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(Cblacs_pinfo+0x2000000001798240) [0x40000000022eea90]
[5] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(blacs_gridmap_+0x2000000001140bb0) [0x4000000001c97410]
[6] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(blacs_gridinit_+0x20000000011409d0) [0x4000000001c97240]

<snip>

*** End of error message ***
6] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(main+0x1fffffffff523f30) [0x4000000000076d40]
[17] func:/lib/tls/libc.so.6.1(__libc_start_main-0x2bccc0)
[0x2000000000899c50]
[18] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(_start+0x1fffffffff520160) [0x4000000000076a80]
*** End of error message ***
6] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(main+0x1fffffffff523f30) [0x4000000000076d40]
[17] func:/lib/tls/libc.so.6.1(__libc_start_main-0x2bccc0)
[0x2000000000899c50]
[18] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(_start+0x1fffffffff520160) [0x4000000000076a80]
*** End of error message ***
6] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(main+0x1fffffffff523f30) [0x4000000000076d40]
[17] func:/lib/tls/libc.so.6.1(__libc_start_main-0x2bccc0)
[0x2000000000899c50]
[18] func:/home/users/eshamay/tmp/cp2k/exe/Linux-ia64-intel/
cp2k.popt(_start+0x1fffffffff520160) [0x4000000000076a80]
*** End of error message ***
mpirun noticed that job rank 1 with PID 14565 on node "localhost"
exited on signal 11.

So far all the help was right on track and I've finally been able to
get it to compile, but now I'm stuck at this near-immediate crash
after trying to run the program in parallel. Any further help would be
greatly appreciated.

~Eric

Axel Kohlmeyer

unread,

May 29, 2007, 9:02:15 PM5/29/07

to cp...@googlegroups.com

eric,

did your BLACS compilation pass all the BLACS testers?

the fact, that it compiles successfully, does not mean that
it executes correctly. with a complex package like BLACS/ScaLAPACK
you need to make sure that this works correct independently.

i tried openmpi-1.1 and it did not work for me (x86_64, intel 9.1).
i upgraded to openmpi-1.2.1 and it worked.

cheers,
axel.

> So far all the help was right on track and I've finally been able to

> get it to compile, but now I'm stuck at this near-immediate crash
> after trying to run the program in parallel. Any further help would be
> greatly appreciated.
>
> ~Eric
>
>
>
>
>

--
=======================================================================
Axel Kohlmeyer akoh...@cmm.chem.upenn.edu http://www.cmm.upenn.edu
Center for Molecular Modeling -- University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582, fax: 1-215-573-6233, office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.

Eric Shamay

unread,

May 29, 2007, 9:22:45 PM5/29/07

to cp...@googlegroups.com

I'll install and run the testers, as I don't believe the sysadmin did indeed do that. thanks for the pointer, I'll post once I have a result (or working system!).

--
~E

Eric Shamay

unread,

Jun 1, 2007, 5:27:40 PM6/1/07

to cp2k

BLACS installed just fine, and the tester verified a 100% pass. I then
installed scalapack. Here's my current popt arch file:

PERL = perl
CC = mpicc
CPP = cpp
FC = mpif90 -FR
FC_fixed = mpif90 -FI
LD = mpif90
AR = ar -r

BLACS = $(HOME)/tmp/BLACS

DFLAGS = -D__INTEL -D__FFTSG -D__FFTW3 -D__parallel -D__BLACS -

D__SCALAPACK -I$(FFTW)/include -I$(MKL)/include

CPPFLAGS = -C $(DFLAGS) -P -traditional
FCFLAGS = -O2 -Vaxlib -tpp2

LDFLAGS = $(FCFLAGS) -L$(FFTW)/lib -L$(MPI)/lib -L$(MKL)/lib/64 -L$
(BLACS)/LIB
#LIBS = -lmkl_scalapack -lmkl_blacsF77init -lmkl_blacs -

lmkl_lapack -lmkl -lguide -lmpi -lfftw3

LIBS = -lmkl_scalapack -$(BLACS)/LIB/blacsF77init_MPI-LINUX-0.a -$
(BLACS)/LIB/blacs_MPI-LINUX-0.a -lmkl_lapack -lmkl - lguide -lmpi
-lfftw3

OBJECTS_ARCHITECTURE = machine_intel.o

I've installed openmpi 1.2, and I thought that by pointing to the new
blacs files (as above) that it would take care of the problem. This
doesn't seem to work. Using the commented LIBS line causes the
compiler to use the cluster mkl libraries, and the compilation works
just fine, the uncommented one does not compile.

Do I need to recompile BLACS and scalapack if using a newer version of
openmpi?
~Eric

akohlmey

unread,

Jun 1, 2007, 5:41:22 PM6/1/07

to cp2k

wait! did you compile BLACS against openmpi 1.2 or not?

when you have multiple copies of MPI and Libraries around,
you have to make sure that you are always using a consistent
set for everything. in general, you cannot expect that different
versions of a software are fully foward and backward compatible.

> doesn't seem to work. Using the commented LIBS line causes the
> compiler to use the cluster mkl libraries, and the compilation works
> just fine, the uncommented one does not compile.

> Do I need to recompile BLACS and scalapack if using a newer version of
> openmpi?

what are the compilation errors?

cheers,
axel.
> ~Eric

Eric Shamay

unread,

Jun 1, 2007, 6:17:06 PM6/1/07

to cp2k

Okay, I've got the openmpi 1.2 and recompiling BLACS, ATLAS, and
scalapack. So far BLACS and ATLAS have not complained or failed any
tests, and scalapack is recompiling using the updated libraries of the
former two.

The MPI errors are as follows:

#
# Error message from popt when running mpirun -np 8 cp2k.popt traj.inp
> traj.out &
#
#[optix:02952] [ 1] /usr/local/packages/openmpi/1.2_intel-9.1/lib/
libmpi.so.0 [0x200000000023dc10]
#[optix:02952] [ 2] /usr/local/packages/openmpi/1.2_intel-9.1/lib/
libmpi.so.0(MPI_Comm_size-0xc09e50) [0x200000000023da20]
#[optix:02952] [ 3] cp2k.popt(Cblacs_pinfo+0x200000000147ccd0)
[0x40000000022c4550]
#[optix:02952] [ 4] cp2k.popt(blacs_gridmap_+0x20000000014a5d00)
[0x40000000022ed590]
#[optix:02952] [ 5] cp2k.popt(blacs_gridinit_+0x20000000014a5b20)
[0x40000000022ed3c0]
#[optix:02952] [ 6] cp2k.popt(cp_blacs_calls_mp_cp_blacs_gridinit_
+0x1fffffffff40e6d0) [0x4000000000255f80]
#[optix:02952] [ 7] cp2k.popt(cp_blacs_env_mp_cp_blacs_env_create_
+0x1fffffffff40fa30) [0x40000000002572f0]
#[optix:02952] [ 8] cp2k.popt(qs_environment_types_mp_init_qs_env_
+0x20000000002b2320) [0x40000000010f9bf0]
#[optix:02952] [ 9] cp2k.popt(qs_environment_types_mp_qs_env_create_
+0x20000000002ab940) [0x40000000010f3220]
#[optix:02952] [10] cp2k.popt(qs_main_mp_quickstep_create_force_env_
+0x200000000059bcc0) [0x40000000013e35b0]
#[optix:02952] [11] cp2k.popt(f77_interface_mp_create_force_env_
+0x1fffffffff6e2d20) [0x400000000052a620]
#[optix:02952] [12] cp2k.popt(cp2k_runs_mp_cp2k_run_
+0x1fffffffff402ef0) [0x400000000024a800]
#[optix:02952] [13] cp2k.popt(cp2k_runs_mp_run_input_
+0x1fffffffff406370) [0x400000000024dc90]
#[optix:02952] [14] cp2k.popt(MAIN__+0x2000000000de76f0)
[0x4000000001c2f020]
#[optix:02952] [15] cp2k.popt(main+0x1fffffffff22a7e0)
[0x400000000006de80]
#[optix:02952] [16] /lib/tls/libc.so.6.1(__libc_start_main-0x4d9cf0)
[0x200000000096dc50]
#[optix:02952] [17] cp2k.popt(_start+0x1fffffffff226270)
[0x400000000006dbc0]
#[optix:02952] *** End of error message ***
#

The errors do point to BLACS, so I hope this will work. I didn't
realize that I'd have to go back in the recompiling-tree and redo each
component everytime I update a lower component (in this case, mpi).
Thanks for the heads up, I'll take note of what each was compiled
against for records from now on.

Why is it that cp2k is so touchy with regards to minor version changes
in the mpi routines? Do mpi libraries deprecate calls from earlier
versions?

akohlmey

unread,

Jun 1, 2007, 6:39:16 PM6/1/07

to cp2k

On Jun 1, 6:17 pm, Eric Shamay <eric.sha...@gmail.com> wrote:

> The errors do point to BLACS, so I hope this will work. I didn't
> realize that I'd have to go back in the recompiling-tree and redo each
> component everytime I update a lower component (in this case, mpi).

the question is always whether that are changes in the binary
interface, i.e. the ABI. also you have to make sure that your
configuration is correct.

> Thanks for the heads up, I'll take note of what each was compiled
> against for records from now on.
>
> Why is it that cp2k is so touchy with regards to minor version changes
> in the mpi routines? Do mpi libraries deprecate calls from earlier
> versions?

the problem is not a problem of cp2k directly (unless you want to
blame
cp2k for using a larger subset of the functionality of the underlying
libraries and language than other packages), but in the fact, that
openmpi
uses shared libraries and thus you have to always make sure, that you
use a set of libraries, that match the definitions of the header/
include files.

the best strategy is in fact, to always build everything from ground
up
until you have a consistent set. it helps to script the build process,
so that you know what set of flags you were using where.

just mix and matching different libraries can work, but you'd never
know why something was working (or not).

cp2k _is_ a large and complex package and that shows in the build
process. however, once you have a working setup, keeping up-to-date
is rather simple.

cheers,
axel.

Eric Shamay

unread,

Jun 1, 2007, 8:56:19 PM6/1/07

to cp2k

The final compilation of all the libraries went through just fine.
I've managed to get the linking process down in the arch (popt) file
to the point where I'm only getting reference errors from scalapack
(they look like the following):

/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pdlase2.o)(.text
+0x1b00): In function `pdlase2_':
: undefined reference to `dlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pzlarfb.o)(.text
+0x5440): In function `pzlarfb_':
: undefined reference to `zlacpy_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pzlarfb.o)(.text
+0x5460): In function `pzlarfb_':
: undefined reference to `zlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pzlarft.o)(.text
+0x3b80): In function `pzlarft_':
: undefined reference to `zlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pzlarft.o)(.text
+0x3ba0): In function `pzlarft_':
: undefined reference to `zlacgv_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pdlarfb.o)(.text
+0x5140): In function `pdlarfb_':
: undefined reference to `dlacpy_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pdlarfb.o)(.text
+0x5160): In function `pdlarfb_':
: undefined reference to `dlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pdlarft.o)(.text
+0x3600): In function `pdlarft_':
: undefined reference to `dlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pzlarf.o)(.text
+0x39c0): In function `pzlarf_':
: undefined reference to `zlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pzlarfc.o)(.text
+0x39c0): In function `pzlarfc_':
: undefined reference to `zlaset_'
/home/users/eshamay/tmp/SCALAPACK/libscalapack.a(pdlarf.o)(.text
+0x3780): In function `pdlarf_':
: undefined reference to `dlaset_'

I can't find any info on the dlaset_, zlaset_, etc. functions, or what
library I need to link to in addition. Also, I want to note for future
compilers the linking order I came up with, so below is my updated
popt file:

########################################################3
# using openmpi 1.2
# ATLAS 3.6.0
# mpiBLACS 1.1 with patch03
# scalapack 1.8.0
#

PERL = perl
CC = mpicc
CPP = cpp
FC = mpif90 -FR
FC_fixed = mpif90 -FI
LD = mpif90
AR = ar -r

###########
# Here are the locations of the newly built libraries
###########
SCALAPACK = $(HOME)/tmp/SCALAPACK
ATLASLIB = $(HOME)/tmp/ATLAS/lib/Linux_IA64Itan2_16
BLACSLIB = $(HOME)/tmp/BLACS/LIB
###########

DFLAGS = -D__INTEL -D__FFTSG -D__FFTW3 -D__parallel -D__BLACS -
D__SCALAPACK -I$(FFTW)/include

CPPFLAGS = -C $(DFLAGS) -P -traditional
FCFLAGS = -O2 -Vaxlib -tpp2

LDFLAGS = $(FCFLAGS) -L$(FFTW)/lib -L$(MPI)/lib -L$(BLACSLIB) -L$
(ATLASLIB) -L$(SCALAPACK)
###########
# These use the home-built libraries
LIBS = -lscalapack $(BLACSLIB)/blacs_MPI-LINUX-0.a $(BLACSLIB)/
blacsCinit_MPI-LINUX-0.a $(BLACSLIB)/blacsF77init_MPI-LINUX- 0.a $
(BLACSLIB)/blacs_MPI-LINUX-0.a -llapack -lcblas -lf77blas -latlas -
lblas -lg2c -lguide -lmpi -lfftw3
###########

OBJECTS_ARCHITECTURE = machine_intel.o

Note that the call to blacs_MPI-LINUX-0.a happens before and after the
other BLACS library files. Also, the order of the calls to lapack and
blas is important: -llapack -lcblas -lf77blas -latlas -lblas

Anyhow, any help with the undefined reference errors would be greatly
appreciated!
Thanks,
~Eric

Matthias Krack

unread,

Jun 2, 2007, 5:43:24 AM6/2/07

to cp2k

Eric,

does your ScaLAPACK build include the toolslib?
I always use the lib target

lib: toolslib pblaslib redistlib scalapacklib

in the ScaLAPACK makefile for the library build.

Matthias

Dr. Matthias Krack
Computational Science
Department of Chemistry and Applied Biosciences
ETH Zurich
USI-Campus, via G. Buffi 13
6900 Lugano
Switzerland

Phone: +41 (0)58 666 48 05 (direct)
+41 (0)58 666 48 00 (secretary)
Fax: +41 (0)58 666 48 17

Email: kr...@phys.chem.ethz.ch
URL: http://www.phys.chem.ethz.ch

Eric Shamay

unread,

Jun 2, 2007, 6:49:01 PM6/2/07

to cp...@googlegroups.com

Matthias,

The scalapack default 'make' includes all the lib targets, so they were built with my last compilation. Also, I ran all the test that come with the build in the TESTING directory and they passed just fine. I believe the problem is that when linking in the arch-file I'm not including a necessary library. Right now my arch file has

LIBS = -lscalapack $(BLACSLIB)/blacs_MPI-LINUX-0.a $(BLACSLIB)/blacsCinit_MPI-LINUX-0.a $(BLACSLIB)/blacsF77init_MPI-LINUX-0.a $(BLACSLIB)/blacs_MPI-LINUX-0.a -llapack -lcblas -lf77blas - latlas -lblas -lg2c -lguide -lmpi -lfftw3

I thought that adding -lg2c would clear up the problem, but scalapack is making reference to things like:

dlaset_
dlacpy_
zlacgv_
zlaset_
zlacpy_
dlamch_
dlapy2_
etc.

I can't seem to find the library that defines those, but including whichever library it is will probably clear up the problem. So it's either linking to one more library, or a more fundamental problem with the build of one of the components (blacs, atlas, scalapack?) that I can't diagnose on my own.

~Eric Shamay

Matthias Krack

unread,

Jun 3, 2007, 4:06:25 AM6/3/07

to cp...@googlegroups.com

Eric,

> The scalapack default 'make' includes all the lib targets, so they were
> built with my last compilation. Also, I ran all the test that come with the
> build in the TESTING directory and they passed just fine. I believe the
> problem is that when linking in the arch-file I'm not including a necessary
> library. Right now my arch file has
>
> LIBS = -lscalapack
> $(BLACSLIB)/blacs_MPI-LINUX-0.a$(BLACSLIB)/blacsCinit_MPI-
> LINUX-0.a $(BLACSLIB)/blacsF77init_MPI-LINUX-0.a $(BLACSLIB)/blacs_MPI-
> LINUX-0.a -llapack -lcblas -lf77blas - latlas -lblas -lg2c -lguide -lmpi
> -lfftw3
>
> I thought that adding -lg2c would clear up the problem, but scalapack is
> making reference to things like:
>
> dlaset_
> dlacpy_
> zlacgv_
> zlaset_
> zlacpy_
> dlamch_
> dlapy2_
> etc.

All these routines are part of the libscalapack. Just have a look in
SCALAPACK/TOOLS/LAPACK where you should find e.g. the file dlaset.f.
If you compiled also the toolslib as you claim, then the toolslib
should be linked into the libscalapack.a. It might be instructive
just to check for it:

ar -t libscalapack.a | grep dlaset

or for more details

nm libscalapack.a | grep dlaset

Sometime an additional or missing appended underscore troubles when
linking libs and objs generated with different compilers.

> I can't seem to find the library that defines those, but including whichever
> library it is will probably clear up the problem. So it's either linking to
> one more library, or a more fundamental problem with the build of one of the
> components (blacs, atlas, scalapack?) that I can't diagnose on my own.
>

Due to the facts above I don't think that a lib is missing. It's more
likely that you have either mismatched name or that the toolslib routines
are not linked into libscalapack. I am not sure if the tests in
SCALAPACK/TESTING include references to this routines.

Matthias

> On 6/2/07, Matthias Krack <kr...@phys.chem.ethz.ch> wrote:
> >
> >
> > Eric,
> >
> > does your ScaLAPACK build include the toolslib?
> > I always use the lib target
> >
> > lib: toolslib pblaslib redistlib scalapacklib
> >
> > in the ScaLAPACK makefile for the library build.
> >
> > Matthias
> >
> >
>
> >
>

Dr. Matthias Krack

Eric Shamay

unread,

Jun 5, 2007, 5:27:45 PM6/5/07

to cp2k

As a followup to this topic:

I managed to get the popt fully compiled without error and the tests
seem to pass just fine. Here is the final arch file:

# openmpi 1.2

# mpiBLACS 1.1 with patch03
# scalapack 1.8.0
#
PERL = perl
CC = mpicc
CPP = cpp
FC = mpif90 -FR
FC_fixed = mpif90 -FI
LD = mpif90
AR = ar -r

# Here are the locations of the newly built libraries
SCALAPACK = $(HOME)/tmp/SCALAPACK
BLACSLIB = $(HOME)/tmp/BLACS/LIB
MKLLIB = $(MKL)/lib/64

DFLAGS = -D__INTEL -D__FFTSG -D__FFTW3 -D__parallel -D__BLACS -
D__SCALAPACK -I$(FFTW)/include
CPPFLAGS = -C $(DFLAGS) -P -traditional
FCFLAGS = -O2 -Vaxlib -tpp2
LDFLAGS = $(FCFLAGS) -L$(FFTW)/lib -L$(MPI)/lib -L$(BLACSLIB) -L$

(SCALAPACK) -L$(MKLLIB)

LIBS = $(SCALAPACK)/libscalapack.a\
$(BLACSLIB)/blacs_MPI-LINUX-0.a\
$(BLACSLIB)/blacsCinit_MPI-LINUX-0.a\
$(BLACSLIB)/blacsF77init_MPI-LINUX-0.a\
$(BLACSLIB)/blacs_MPI-LINUX-0.a\
$(MKLLIB)/libmkl_lapack.a\
-lmkl -lguide -lmpi -lfftw3

OBJECTS_ARCHITECTURE = machine_intel.o

The difference from before is that the lapack library is the one from
the cluster MKL suite. After some discussion on the scalapack user
forum, it was made clear that the ATLAS lapack is not a complete
lapack library, and a specific vendor library (i.e. MKL) should be
used. Making that switch and eliminating ATLAS cleared up all the
undefined reference errors.

Thank you to everyone for the help!
~Eric Shamay

Reply all

Reply to author

Forward