program stuck when running DFT SIOSI3 input

[Duong Ngoc Yen]

Hi NWCHEM Team,

I compiled NWCHEM using the below setting. The compilation process is successful but when I run the simple siosi3 input, it gets stuck in the middle of the program for very long (>1 hour). There is nothing in the error log. I would really appreciate if you could help spot the cause of that.

Compilation: 

module load intel/20.4
module load mkl/2020.4.304
module load intelmpi/20.4-intel20.4

export NWCHEM_TOP=~
export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export TCGRSH=/usr/bin/ssh
export NWCHEM_TARGET=LINUX64
export USE_MPI=y
export IB_HOME=/usr
export IB_INCLUDE=/usr/include
export IB_LIB=/lib64
export IB_LIB_NAME="-libumad -libverbs -lpthread"
export ARMCI_NETWORK=OPENIB
#export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/application/intel/impi/lib64
export NWCHEM_MODULES=all
export FOPTIMIZE="-O2 -xSSE2,SSE3,SSSE3,SSE4.1,SSE4.2,AVX,AVX2,AVX512 -funroll-loops -unroll-aggressive"
export COPTIMIZE="-O2 -xSSE2,SSE3,SSSE3,SSE4.1,SSE4.2,AVX,AVX2,AVX512 -funroll-loops"

export USE_NOFSCHECK=True
export CC=icc
export FC=ifort
export USE_SCALAPACK=y
export SCALAPACK_SIZE=8
export BLAS_SIZE=8
export SCALAPACK=" ${MKLROOT}/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_ilp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -Wl,--end-group -lpthread -lm -ldl"
export BLASOPT=$SCALAPACK
export LAPACK_LIB=$SCALAPACK

cd $NWCHEM_TOP/src
echo $PWD
make nwchem_config NWCHEM_MODULES="all"
make V=1 FC=ifort CC=icc CXX=icc -j11

Run script

#SBATCH --nodes=1
#SBATCH --ntasks-per-node=32
#sbatch --cpus-per-tasks=2
#SBATCH --job-name=dft-simple-intelmpi-openib

module load intel/20.4
module load mkl/2020.4.304
module load intelmpi/20.4-intel20.4

mpirun -np 32 $NWCHEM_TOP/bin/LINUX64/nwchem $NWCHEM_TOP/web/benchmarks/dft/siosi3.nw

Thank you for your help.                                                                                                                                                                                                           

[Edoardo Aprà]

1. What version of NWChem are you using?
2. Redefining FOPTIMIZE and COPTIMIZE is not a good idea.
3. What about smaller tests on fewer processes? Do they work?