Compilation errors

[Natalie Stuart]

Hello,

I'm trying to compile NWChem version 7.0.2 on a Linux64 system. I've been following the directions on https://github.com/nwchemgit/nwchem/wiki/Compiling-NWChem but still have errors in my make.log file.

My .bash_profile is set up as such:

export NWCHEM_TOP=/home/nwchem/nwchem-7.0.2-release

export NWCHEM_TARGET=LINUX64

export NWCHEM_MODULES=all

export USE_MPI=y

export USE_MPIF=y

export USE_MPIF4=y

export PATH=/usr/lib64/openmpi/bin:$PATH

export USE_NOFSCHECK=TRUE

export FC=gfortran

export BLAS_SIZE=4

export BLASOPT="-L/usr/lib64 -lblas"

export LAPACK_LIB="-L/usr/lib64 -lblas"

The PATH variable is the location of mpif90 and the BLASOPT variable is the location of my system's BLAS library. I'm not sure if BLAS_SIZE is set correctly, since I couldn't tell if my BLAS library uses 32- or 64-bit integers. 

The directions for 'How to deal with integer size of Linear Algebra libraries' were  unclear to me. If I understand correctly, the BLAS libraries in most systems are 32-bit, which are not (?) compatible with NWChem. The instructions tell us to execute 64_to_32, which would make the BLAS libraries 32-bit?

The errors I'm getting at the end of my make.log file are:

Warning: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/LOGICAL(4)).

Compiling util_dgeev.F...

Compiling util_nwsort.F...

Compiling util_mygabcast.c...

Compiling util_getppn.c...

Compiling util_units.F...

Compiling util_align.F...

Compiling util_intrsc2008.F...

Compiling util_fadvise.c...

Compiling util_scalapack_info.F...

Compiling ga_conjg.F...

Compiling nwmolden.F...

Compiling ao_1prdm_read.F...

Compiling ao_1prdm_write.F...

Compiling util_strips.F...

Compiling util_poltensor.F...

Compiling util_mygop.F...

util_mygop.F:14:17:

   14 |       bigint = (2**31)-1024

      |                 1

Error: Result of exponentiation at (1) exceeds the range of INTEGER(4)

util_mygop.F:55:17:

   55 |       bigint = (2**31)-1024

      |                 1

Error: Result of exponentiation at (1) exceeds the range of INTEGER(4)

make[1]: *** [../config/makefile.h:3040: /home/nwchem/nwchem-7.0.2-release/lib/LINUX64/libnwcutil.a(util_mygop.o)] Error 1

make[1]: Leaving directory '/home/nwchem/nwchem-7.0.2-release/src/util'

make: *** [GNUmakefile:105: libraries] Error 1

Any suggestions for what to change in my .bash_profile? 

Thanks much.

-Natalie Stuart

[Edoardo Aprà]

The directions for 'How to deal with integer size of Linear Algebra libraries' were  unclear to me. If I understand correctly, the BLAS libraries in most systems are 32-bit, which are not (?) compatible with NWChem. The instructions tell us to execute 64_to_32, which would make the BLAS libraries 32-bit?

If you are comping the source code from a 7.0.2 tar file from https://github.com/nwchemgit/nwchem/releases/tag/v7.0.2-release, then you do not need the make 64_to_32 step since the source code has already been processed with make 64_to_32

Blas system libraries are most often using 32-bit integers

Compilation error:

Could you  send the output generated with the following commands (V=1 argument supplied to make to get more verbose output)

cd $NWCHEM_TOP/srtc/util

make V=1

Instead of compiling the NWChem source code, you might consider installing NWChem using the NWChem package available in some of the Linux distributions as described at https://nwchemgit.github.io/Download.html#nwchem-availability-in-linux-distributions

[Edoardo Aprà]

Please post

$NWCHEM_TOP/src/tools/build/config.log

$NWCHEM_TOP/src/tools/build/comex/config.log

On Friday, September 10, 2021 at 12:07:18 PM UTC-7 Natalie Stuart wrote:

Hello,

This question may be redundant, but I can't find my previously-posted question.

I'm trying to compile NWChem version 7.0.2 on a Linux64 system using openmpi. I've been reading the instructions from https://github.com/nwchemgit/nwchem/wiki/Compiling-NWChem but am still getting errors.

The instructions on 'How to deal with integer size of Linear Algebra libraries' are rather confusing to me. Do the BLAS libraries need to be 64- or 32-bit to be compatible with NWChem? 

Here are the environment variables from my .bashrc:

export NWCHEM_TOP=/home/nwchem/nwchem-7.0.2-release

export NWCHEM_TARGET=LINUX64

export NWCHEM_MODULES="all"

export USE_MPI=y

export USE_MPIF=y

export USE_MPIF4=y

export PATH=/usr/lib64/openmpi/bin:$PATH

export USE_NOFSCHECK=TRUE

export FC=gfortran

export BLASOPT="-L/usr/lib64 -lblas -llapack"

export LAPACK_LIB="-L/usr/lib64 -lblas -llapack"

export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib:$LD_LIBRARY_PATH

Here are the errors I'm getting from my make.log file:

configure: error: in `/home/nwchem/nwchem-7.0.2-release/src/tools/build':

configure: error: could not compile simple C MPI program

See `config.log' for more details

make[1]: *** [GNUmakefile:695: build/config.status] Error 1

make[1]: Leaving directory '/home/nwchem/nwchem-7.0.2-release/src/tools'

make: *** [GNUmakefile:105: libraries] Error 1

I know that the 'could not compile simple C MPI program' is an MPI error. My PATH is set to the location of mpif90. Any suggestions?

Thanks much.

-Natalie Stuart

[Natalie Stuart]

I've included the config.log file (converted to .txt) from $NWCHEM_TOP/src/tools/build.

The output generated from % cd $NWCHEM_TOP/src/util % make V=1 is:

( cd /home/nwchem/nwchem-7.0.2-release/src/config; make /home/nwchem/nwchem-7.0.2-release/bin/LINUX64/depend.x ; )

make[1]: Entering directory '/home/nwchem/nwchem-7.0.2-release/src/config'

gcc  -g -o /home/nwchem/nwchem-7.0.2-release/bin/LINUX64/depend.x depend.c

/bin/rm -f depend.o

make[1]: Leaving directory '/home/nwchem/nwchem-7.0.2-release/src/config'

/home/nwchem/nwchem-7.0.2-release/bin/LINUX64/depend.x -I/usr/include/openmpi-x86_64 -I/usr/lib64/openmpi/lib -I/home/nwchem/nwchem-7.0.2-release/src/tools/install/include > dependencies

Making include_stamp in md5

make[1]: Entering directory '/home/nwchem/nwchem-7.0.2-release/src/util/md5'

/home/nwchem/nwchem-7.0.2-release/bin/LINUX64/depend.x  -I/home/nwchem/nwchem-7.0.2-release/src/tools/install/include > dependencies

cp -p checksum.h /home/nwchem/nwchem-7.0.2-release/src/include

touch include_stamp

make[1]: Leaving directory '/home/nwchem/nwchem-7.0.2-release/src/util/md5'

cp -p util.fh itri.fh msgids.fh  numerical_constants.fh stdio.fh printlevels.fh bitops.fh bitops_decls.fh bitops_funcs.fh bgj.fh bgj_common.fh nwc_const.fh errquit.fh util_sgroup.fh util_params.fh utilc_nwchem_srcdir.fh vtune.fh /home/nwchem/nwchem-7.0.2-release/src/include

touch include_stamp

gfortran -c -m64 -ffast-math  -Warray-bounds -std=legacy -fdefault-integer-8 -fno-tree-dominator-opts  -finline-functions -O2 -g  -fno-aggressive-loop-optimizations -fno-tree-dominator-opts  -g -O   output.f

make: *** No rule to make target '/home/nwchem/nwchem-7.0.2-release/src/include/mafdecls.fh', needed by '/home/nwchem/nwchem-7.0.2-release/lib/LINUX64/libnwcutil.a(errquit.o)'.  Stop.

[Edoardo Aprà]

Did you set MPI_INCLUDE and/or MPI_LIB or did you leave them unset (better choice as mentioned in https://nwchemgit.github.io/Compiling-NWChem.html#automatic-detection-of-mpi-variables-with-mpif90)?

What linux distribution are you using? Is it Centos, Redhat, ubuntu, debian?

What is the output of the command

mpif90 -show

[Natalie Stuart]

Hi Edoardo,

I left MPI_INCLUDE and MPI_LIB unset based on the Compiling-NWChem instructions. I'm using a Redhat distribution. As you suggested, I also tried installing NWChem using the NWChem package available in some of the Linux distributions as described at https://nwchemgit.github.io/Download.html#nwchem-availability-in-linux-distributions, but typing % module load openmpi gave me the following error: Lmod has detected the following error:  The following module(s) are unknown: "openmpi"

The command % mpif90 -show yields the following: -bash: mpif90: command not found

I do know that it's located in /usr/lib64/openmpi/bin though

Thanks for your help.

[Edoardo Aprà]

Please send output of the following commands

env | grep PATH

rpm -q --whatprovides /usr/lib64/openmpi/bin/mpif90

ls -l /usr/lib64/openmpi/bin/

rpm -q -a | grep openmpi

rpm -q -a | grep mpich

module avail

[Natalie Stuart]

Hello,

I am rather confident that my errors are with the BLASOPT environment variable, which is currently:

export BLASOPT="/usr/lib64 -lblas -llapack"

export LAPACK_LIB="/lib64 -lblas -llapack"

I'm unsure of what other libraries to link against besides -lblas and -llapaclk.

The outputs of the suggested commands are as follows:

% env | grep PATH

MANPATH=:

MODULEPATH_modshare=/usr/share/modulefiles:1:/usr/share/Modules/modulefiles:1:/etc/modulefiles:1

MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH

LD_LIBRARY_PATH=/lib64/openmpi/lib:

PATH=/lib64/openmpi/bin:/root/.local/bin:/root/bin:/home/gamess/gamess:/home/natalie/.local/bin:/home/natalie/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles

% rpm -q --whatprovides /usr/lib64/openmpi/bin/mpif90

openmpi-devel-4.0.5-1.fc32.x86_64

% ls -l /usr/lib64/openmpi/bin/

there was a large output for this command, but here is what I think we are looking for:

lrwxrwxrwx. 1 root root    12 Jan 24  2021 mpif90 -> opal_wrapper

% rpm -q -a | grep openmpi

nwchem-openmpi-7.0.2-4.fc32.x86_64

scalapack-openmpi-2.1.0-3.fc32.x86_64

openmpi-devel-4.0.5-1.fc32.x86_64

ga-openmpi-5.7.2-3.fc32.x86_64

openmpi-4.0.5-1.fc32.x86_64

% rpm -q -a | grep mpich

no output

% module avail

------------------------------------------------- /usr/share/Modules/modulefiles -------------------------------------------------

dot  module-git  module-info  modules  null  use.own  

----------------------------------------------------- /usr/share/modulefiles -----------------------------------------------------

mpi/openmpi-x86_64  

Thanks very much for your continued assistance!

[Edoardo Aprà]

On Monday, September 13, 2021 at 1:19:12 PM UTC-7 Natalie Stuart wrote:

Hello,

I am rather confident that my errors are with the BLASOPT environment variable, which is currently:

export BLASOPT="/usr/lib64 -lblas -llapack"

export LAPACK_LIB="/lib64 -lblas -llapack"

I'm unsure of what other libraries to link against besides -lblas and -llapaclk.

If this the case, why don't you unset BLASOPT and LAPACK_LIB and set USE_INTERNALBLAS=y to verify that all the rest of the settings are working?

It's better to deal with fewer problems at once.

By the way, the rpm -qa output shows that NWChem is installed on your system ... what's wrong about using it?

[Natalie Stuart]

Hi Edoardo,

You're right, NWChem is already installed on my system through openmpi. I followed the directions on this website that you directed me to earlier: https://nwchemgit.github.io/Download.html#nwchem-availability-in-linux-distributions

but  after the command:

% module load openmpi-x86_64

I get the following: 

ERROR: Unable to locate a modulefile for 'openmpi-x86_64'

Any advice? 

Thanks much!

[Edoardo Aprà]

If look at the module avail output, you can conclude that the correct command should be

module load mpi/openmpi-x86_64 

This is in agreement with what describe at https://nwchemgit.github.io/Download.html#example-of-nwchem-rpm-installation-under-centos-7-x86_64

[Natalie Stuart]

I ran 

% module load mpi/openmpi-x86_64 

and got no output. 

Then I tried to run a test water calculation by calling the directory where nwchem_openmpi resides

% /usr/lib64/openmpi/bin/nwchem_openmpi water >& water.log &

and got the following error in my water.log file:

/usr/lib64/openmpi/bin/nwchem_openmpi: error while loading shared libraries: libga.so.1: cannot open shared object file: No such file or directory

Which is the same error I got when previously trying to run the calculation after unsuccessfully commanding % module load openmpi-x86_64

In /lib64, i have a libg.a archive, but no libga.so.1. Any suggestions on how to get this library installed in the right place?

As always, thanks much for your prompt responses.

[Edoardo Aprà]

Can you report the output of these sequence of commands

module purge

module load mpi/openmpi-x86_64

module show mpi/openmpi-x86_64

env | grep PATH

which nwchem_openmpi

rpm -q -l ga-openmpi

rpm -q -l nwchem-openmpi

[Natalie Stuart]

Yes, 

% module purge 
no output

%module load mpi/openmpi-x86_64 

no output

%env | grep PATH

PKG_CONFIG_PATH=/usr/lib64/openmpi/lib/pkgconfig

PKG_CONFIG_PATH_modshare=/usr/lib64/openmpi/lib/pkgconfig:1

MANPATH_modshare=:2:/usr/share/man/openmpi-x86_64:1

MANPATH=/usr/share/man/openmpi-x86_64::

LD_LIBRARY_PATH_modshare=:1:/lib64/openmpi/lib:1:/usr/lib64/openmpi/lib:1

PATH_modshare=/usr/lib64/openmpi/bin:1:/usr/bin:1:/usr/local/bin:1:/usr/share/Modules/bin:1:/root/bin:1:/root/.local/bin:1:/home/natalie/bin:1:/home/natalie/.local/bin:1:/home/gamess/gamess:1:/lib64/openmpi/bin:1:/usr/sbin:1:/usr/local/sbin:1

MODULEPATH_modshare=/usr/share/modulefiles:1:/usr/share/Modules/modulefiles:1:/etc/modulefiles:1

MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH

LD_LIBRARY_PATH=/usr/lib64/openmpi/lib:/lib64/openmpi/lib:

PATH=/usr/lib64/openmpi/bin:/lib64/openmpi/bin:/lib64/openmpi/bin:/root/.local/bin:/root/bin:/lib64/openmpi/bin:/home/gamess/gamess:/home/natalie/.local/bin:/home/natalie/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles

%which nwchem_openmpi

/usr/lib64/openmpi/bin/nwchem_openmpi

%rpm -q -l ga-openmpi

/usr/lib/.build-id

/usr/lib/.build-id/14

/usr/lib/.build-id/14/c5a482d9a7b056588f3f0430580cbd51daa241

/usr/lib/.build-id/27

/usr/lib/.build-id/27/b873eb7cd4572da99a08141f5da888a145daf3

/usr/lib/.build-id/28

/usr/lib/.build-id/28/6f287993d992f463085a5d8eea0dd590d858b3

/usr/lib/.build-id/36

/usr/lib/.build-id/36/da8700a716ee8b15f744e409a6b0d31720fbe8

/usr/lib/.build-id/62/90e19c9324c61498e663ed6025c150c7fe8ab3

/usr/lib/.build-id/f5

/usr/lib/.build-id/f5/4af49207a5f5923d71a324b63503e7121f553a

/usr/lib64/openmpi/bin/adjust.x

/usr/lib64/openmpi/bin/collisions.x

/usr/lib64/openmpi/lib/libarmci.so.0

/usr/lib64/openmpi/lib/libarmci.so.0.0.0

/usr/lib64/openmpi/lib/libcomex.so.0

/usr/lib64/openmpi/lib/libcomex.so.0.0.0

/usr/lib64/openmpi/lib/libga++.so.0

/usr/lib64/openmpi/lib/libga++.so.0.0.0

/usr/lib64/openmpi/lib/libga.so.1

/usr/lib64/openmpi/lib/libga.so.1.1.1

/usr/share/doc/ga-openmpi

/usr/share/doc/ga-openmpi/COPYRIGHT

% rpm -q -l nwchem-openmpi

/usr/lib/.build-id

/usr/lib/.build-id/64

/usr/lib/.build-id/64/7be9060c03cee366ccbf6feabec55ccdf7426c

/usr/lib64/openmpi/bin/nwchem_openmpi

[Edoardo Aprà]

These command output results all look good (other than a strange :1 character showing up ... is this in your output or is it a result of the copy & paste operation?)

Can you send the output of

ls -l  /usr/lib64/openmpi/lib/libga.*

module load mpi/openmpi-x86_64

ldd nwchem-openmpi

ldd /usr/lib64/openmpi/bin/nwchem_openmpi

[Natalie Stuart]

Hi Edorado,

The :1 character in my PATH variables is not a result of my copy/paste operation. It's also not featured in the actual PATH variables in my .bashrc, only when I command 

% env | grep PATH

% ls -l  /usr/lib64/openmpi/lib/libga.*

lrwxrwxrwx. 1 root root      14 Mar 14  2020 /usr/lib64/openmpi/lib/libga.so.1 -> libga.so.1.1.1

-rwxr-xr-x. 1 root root 4982616 Mar 14  2020 /usr/lib64/openmpi/lib/libga.so.1.1.1

Are the permissions of libga.so.1.1.1 the possible problem? 

% module load mpi/openmpi-x86_64

no output

% ldd nwchem-openmpi

ldd: ./nwchem-openmpi: No such file or directory

% ldd /usr/lib64/openmpi/bin/nwchem_openmpi

linux-vdso.so.1 (0x00007ffdeafeb000)

libga.so.1 => /usr/lib64/openmpi/lib/libga.so.1 (0x00007f4a02762000)

libarmci.so.0 => /usr/lib64/openmpi/lib/libarmci.so.0 (0x00007f4a0274b000)

libopenblas.so.0 => /lib64/libopenblas.so.0 (0x00007f4a00300000)

libmpi_usempif08.so.40 => /usr/lib64/openmpi/lib/libmpi_usempif08.so.40 (0x00007f4a002c0000)

libmpi_usempi_ignore_tkr.so.40 => /usr/lib64/openmpi/lib/libmpi_usempi_ignore_tkr.so.40 (0x00007f4a002b2000)

libmpi_mpifh.so.40 => /usr/lib64/openmpi/lib/libmpi_mpifh.so.40 (0x00007f4a00249000)

libmpi.so.40 => /usr/lib64/openmpi/lib/libmpi.so.40 (0x00007f4a0011b000)

libcomex.so.0 => /usr/lib64/openmpi/lib/libcomex.so.0 (0x00007f4a00110000)

librt.so.1 => /lib64/librt.so.1 (0x00007f4a00105000)

libgfortran.so.5 => /lib64/libgfortran.so.5 (0x00007f49ffe3a000)

libm.so.6 => /lib64/libm.so.6 (0x00007f49ffcf5000)

libmvec.so.1 => /lib64/libmvec.so.1 (0x00007f49ffcc9000)

libscalapack.so.2 => /usr/lib64/openmpi/lib/libscalapack.so.2 (0x00007f49ff714000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f49ff6f2000)

libpython3.8.so.1.0 => /lib64/libpython3.8.so.1.0 (0x00007f49ff3b7000)

libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007f49ff37c000)

libdl.so.2 => /lib64/libdl.so.2 (0x00007f49ff375000)

libutil.so.1 => /lib64/libutil.so.1 (0x00007f49ff370000)

libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f49ff353000)

libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f49ff309000)

libc.so.6 => /lib64/libc.so.6 (0x00007f49ff13f000)

libopen-rte.so.40 => /usr/lib64/openmpi/lib/libopen-rte.so.40 (0x00007f49ff085000)

libopen-pal.so.40 => /usr/lib64/openmpi/lib/libopen-pal.so.40 (0x00007f49fefd7000)

libz.so.1 => /lib64/libz.so.1 (0x00007f49fefbd000)

libhwloc.so.15 => /lib64/libhwloc.so.15 (0x00007f49fef6e000)

libevent_core-2.1.so.6 => /lib64/libevent_core-2.1.so.6 (0x00007f49fef35000)

libevent_pthreads-2.1.so.6 => /lib64/libevent_pthreads-2.1.so.6 (0x00007f49fef30000)

/lib64/ld-linux-x86-64.so.2 (0x00007f4a029c5000)

libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f49fef24000)

libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f49fec30000)

[Edoardo Aprà]

Everything looks good to me.

Please stop trying to use the RedHat 6 instructions since it is likely you are on a later version.

You can check your Linux distribution version with the command (if present)

/usr/bin/lsb_release -a

or (if lsb_release is not installed)

cat /etc/*-release

You might want to get help from somebody with a working knowledge of linux system adminstration