van der Waals regtests fail on Intel KNL, and build glitches

152 views
Skip to first unread message

Ronald Cohen

unread,
Jan 3, 2019, 3:59:33 PM1/3/19
to cp2k
regtest-dft-vdw-corr-2 and regtest-dft-vdw-corr-3 regtests fail badly with PSMP under the intelx arch file or intel.psmp . 

starting regression tests in /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-03_19-57-15/tests/QS/regtest-dft-vdw-corr-3 (1 of 3)
>>> /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-03_19-57-15/tests/QS/regtest-dft-vdw-corr-3
    argon05.inp                                               -85.02651698126142  WRONG RESULT TEST 1 
    argon06.inp                                               -85.18843047970840  WRONG RESULT TEST 1 
    argon07.inp                                               -85.05106863151636         RUNTIME FAIL 
    argon08.inp                                               -85.05196996972870         RUNTIME FAIL 
    argon09.inp                                               -85.05093416391124         RUNTIME FAIL 
    argon10.inp                                               -85.05042273661144         RUNTIME FAIL 
    argon11.inp                                               -84.69895559036574         RUNTIME FAIL 
    argon12.inp                                               -84.69905608268989         RUNTIME FAIL 
    argon13.inp                                               -84.81586971061908  WRONG RESULT TEST 1 
    argon14.inp                                               -84.69906586167036  WRONG RESULT TEST 1 
    argon-beef.inp                                            -42.46319102018110  WRONG RESULT TEST 1 
    dftd3bj_t1.inp                                             -0.00355123783846     OK (   2.04 sec) 
    dftd3bj_t2.inp                                             -0.05897356220363     OK (   3.56 sec) 
    dftd3bj_t3.inp                                             -0.00112424003807     OK (   4.34 sec) 
    dftd3bj_t4.inp                                                -84.2983390350     OK (   4.39 sec) 
<<< /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-03_19-57-15/tests/QS/regtest-dft-vdw-corr-3 (1 of 3) done in 1253.00 sec
Starting regression tests in /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-03_19-57-15/tests/QS/regtest-dft-vdw-corr-2 (2 of 3)
>>> /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-03_19-57-15/tests/QS/regtest-dft-vdw-corr-2
    dftd3_t4.inp                                               -0.02290735609940     OK (   2.99 sec) 
    dftd3_t5.inp                                               -0.00012095226817     OK (   2.60 sec) 
    dftd3_t6.inp                                                  -84.2986351038     OK (   4.46 sec) 
    dftd3_t7.inp                                                  -84.2986244998     OK (   4.21 sec) 
    dftd3_t8.inp                                               -0.00115782786021     OK (   4.42 sec) 
    dftd3_t9.inp                                               -0.00115772641844     OK (   2.57 sec) 
    dftd3_t10.inp                                              -0.00117296325816     OK (   2.65 sec) 
    dftd3_t11.inp                                                 -84.2986400085     OK (  14.46 sec) 
    dftd3_t12.inp                                              -0.00117296325816     OK (   6.65 sec) 
    dftd3_t13.inp                                              -0.00045037056834     OK (   1.77 sec) 
    dftd3_t14.inp                                              -0.00022349123009     OK (   1.83 sec) 
    argon01.inp                                               -85.04150374301580  WRONG RESULT TEST 1 
    argon02.inp                                               -85.20388952727301  WRONG RESULT TEST 1 
    argon03.inp                                               -84.83719734168366  WRONG RESULT TEST 1 
    argon04.inp                                               -84.50083943001719  WRONG RESULT TEST 1 

This being run with:

./tools/regtesting/do_regtest -arch Linux-x86-64-intel -version psmp -restrictdir QS/regtest-dft-vdw-corr-1/ -restrictdir QS/regtest-dft-vdw-corr-2/ -restrictdir QS/regtest-dft-vdw-corr-3/ -restrictdir QS/regtest-dft-vdw-corr-3/ -nobuild -mpiranks 4 -ompthreads 4 -maxtasks 16

and was compiled with 
make -j 8 ARCH=Linux-x86-64-intelx VERSION=psmp SYM=1 LIBINTROOT=/mnt/beegfs LIBXCROOT=/mnt/beegfs ELPAROOT=/mnt/beegfs AVX=3 MIC=0 CC=icc CXX=icpc OMP=1 IPO=0

There are also a few glitches in the build. In particular I had to
cp ./lib/Linux-x86-64-intelx/psmp/libxsmm/include/* ./obj/Linux-x86-64-intelx/psmp/
and
similarly for libxsmm mod files as the build process does not include them automatically even after building then via the arch file. Also
-j n does not work -- one gets: 
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule

Any help appreciated! 

Sincerely,

Ronald Cohen

Extreme Materials Initiative
Geophysical Laboratory
Carnegie Institution
5251 Broad Branch Rd., N.W.
Washington, D.C. 20015



Anton Kudelin

unread,
Jan 4, 2019, 1:33:37 AM1/4/19
to cp2k
Dear Ronald,

Could you attach your arch file, please? Did you try to build CP2K via the toolchain?

Anton K.

Ronald Cohen

unread,
Jan 4, 2019, 8:52:21 AM1/4/19
to cp...@googlegroups.com
Yes, I did use the tool chain. The arch file is here. It has only minor changes from the distributed one.
Thank you!

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3

Linux-x86-64-intelx.arch

Anton Kudelin

unread,
Jan 5, 2019, 6:43:50 AM1/5/19
to cp2k
What was the result of using the toolchain? Could you recompile the CP2K environment with a fresh GCC (7.4, 8.2, or any other that supports KNL extensions) by hand or by the toolchain? Also I'd like to look at output files which gave 'RUNTIME FAIL'.

Ronald Cohen

unread,
Jan 5, 2019, 6:11:40 PM1/5/19
to cp...@googlegroups.com
I don’t know what you mean by the toolchain. I followed the directions for compiling for Intel using the arch file. I  think that uses the toolchain in the arch file. I  tried using both the 2018 and 2019 Intel compilers. I don’t use gcc on my KNL system. If I type gcc -v it says 6.3.0 and is the most up to date on debian apparently. I think the problem is with reduction loops that are not marked in the OMP tags, and maybe variables that should be private not marked. Note it passes hundreds of the regtests but just fails on the vDW ones. 

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



--
You received this message because you are subscribed to a topic in the Google Groups "cp2k" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cp2k/gzmRqKNt62U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cp2k+uns...@googlegroups.com.
To post to this group, send email to cp...@googlegroups.com.
Visit this group at https://groups.google.com/group/cp2k.
For more options, visit https://groups.google.com/d/optout.

Ronald Cohen

unread,
Jan 5, 2019, 6:27:10 PM1/5/19
to cp...@googlegroups.com
I followed the directions at:


Here are the summary files. Thank you again for your help!

Ron
error_summary
summary.txt

Ronald Cohen

unread,
Jan 5, 2019, 6:28:02 PM1/5/19
to cp...@googlegroups.com
I don’t know what you mean by the toolchain. I followed the directions for compiling for Intel using the arch file. I  think that uses the toolchain in the arch file. I  tried using both the 2018 and 2019 Intel compilers. I don’t use gcc on my KNL system. If I type gcc -v it says 6.3.0 and is the most up to date on debian apparently. I think the problem is with reduction loops that are not marked in the OMP tags, and maybe variables that should be private not marked. Note it passes hundreds of the regtests but just fails on the vDW ones. 

Ron
On 5. Jan 2019, at 06:43, Anton Kudelin <archm...@gmail.com> wrote:

What was the result of using the toolchain? Could you recompile the CP2K environment with a fresh GCC (7.4, 8.2, or any other that supports KNL extensions) by hand or by the toolchain? Also I'd like to look at output files which gave 'RUNTIME FAIL'.

On Friday, January 4, 2019 at 4:52:21 PM UTC+3, Ronald Cohen wrote:
Yes, I did use the tool chain. The arch file is here. It has only minor changes from the distributed one.
Thank you!

Ron

-
You received this message because you are subscribed to a topic in the Google Groups "cp2k" group.

Ronald Cohen

unread,
Jan 5, 2019, 6:46:45 PM1/5/19
to cp...@googlegroups.com
I found this now—no I have not tried this: ./install_cp2k_toolchain.sh  . I doubt that would work on my intel KNL cluster. 

Ron

---
Ronald Cohen
Extreme Materials Initiative
Geophysical Laboratory
Carnegie Institution
5251 Broad Branch Rd., N.W.
Washington, D.C. 20015
rco...@carnegiescience.edu
office: 202-478-8937
skype: ronaldcohen
twitter: @recohen3

Ronald Cohen

unread,
Jan 5, 2019, 6:48:09 PM1/5/19
to cp...@googlegroups.com
Does that build the psmp version? The popt version without OMP works already.

Ronald Cohen

unread,
Jan 5, 2019, 7:59:50 PM1/5/19
to cp...@googlegroups.com
OK—sorry for all the noise. I am trying:
./install_cp2k_toolchain.sh --with-elpa=install --with-libint=install --with-gcc=install
I hate not being able to use my intel tools which work for me for everything else just fine.

Ron

Anton Kudelin

unread,
Jan 6, 2019, 9:52:58 AM1/6/19
to cp...@googlegroups.com
It's OK, CP2K is hard to compile for the first time. Don't worry, Intel tools can be used to build the program, but you should choose the right version: https://www.cp2k.org/dev:compiler_support
However, in my experience, Intel compilers are rather unstable and require careful using unlike GCC.
Anyway, please, report how successful your CP2K build process was, I guess it can become a guide for the KNL architecture that would help newcomers to get around its traps.

Robert Schade

unread,
Jan 7, 2019, 8:16:23 AM1/7/19
to cp...@googlegroups.com
Building cp2k on Intel Xeon Phi Knights Landing (KNL, not to be confused
with KNC!) is not different from building it on any other Intel CPU.
Hence, I think that the failing regtests point to an underlying issue.
Which exact version of the Intel Compiler and MKL have you tried?
Best Wishes
Robert
--
Robert Schade
Paderborn Center for Parallel Computing (PC2)
University of Paderborn
Warburger Str. 100
D-33098 Paderborn
Germany
robert...@uni-paderborn.de
+49/(0)5251/60-5393

Ronald Cohen

unread,
Jan 7, 2019, 8:52:55 AM1/7/19
to cp...@googlegroups.com
Yes, I agree. I have tried the 2018.05 and the 2019.1 intel compilers. The POPT version runs fine, but the PSMP version fails in the vDW routines. I find things like:
in qs_dispersion_nonloc.F

!$OMP PARALLEL DO DEFAULT(NONE)                      &
!$OMP             SHARED(ispin,i,n,lo,drho,drho_r)   &
!$OMP             PRIVATE(s)                         &
!$OMP             COLLAPSE(3)
            DO r = 0, n(3)-1
               DO q = 0, n(2)-1
                  DO p = 0, n(1)-1
                     s = r*n(2)*n(1)+q*n(1)+p+1
                     drho(s, i) = drho(s, i)+drho_r(i, ispin)%pw%cr3d(p+lo(1), q
+lo(2), r+lo(3))
                  END DO
               END DO
            END DO
!$OMP END PARALLEL DO
         END DO
      END DO

Doesn’t this have to be marked as a reduction? And shouldn’t r, q, p be labeled private? Perhaps this is automatic, but I 
do not see that said anywhere. Does gnu treat such differently than intel? Just ideas.

I am currently trying the toolchain, but it is building everything from scratch, including blas, lapack, scalapack etc etc, so will take days. 

Thank you for your help,

Sincerely,

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



--
signature.asc

Robert Schade

unread,
Jan 7, 2019, 11:39:59 AM1/7/19
to cp...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

r is automatically private because it is the first iteration variable.
Every drho(s, i) is only read and written in exactly one loop
iteration. The statement "COLLAPSE(3)" collapses the three perfectly
nested loops into one loop. So, IMHO, this code looks ok.
Best Wishes
Robert


On 07.01.19 14:52, Ronald Cohen wrote:
> Yes, I agree. I have tried the 2018.05 and the 2019.1 intel
> compilers. The POPT version runs fine, but the PSMP version fails
> in the vDW routines. I find things like: in qs_dispersion_nonloc.F
>
> !$OMP PARALLEL DO DEFAULT(NONE) & !$OMP
> SHARED(ispin,i,n,lo,drho,drho_r) & !$OMP PRIVATE(s)
> & !$OMP COLLAPSE(3) DO r = 0, n(3)-1 DO q = 0, n(2)-1
> DO p = 0, n(1)-1 s = r*n(2)*n(1)+q*n(1)+p+1 drho(s, i) = drho(s,
> i)+drho_r(i, ispin)%pw%cr3d(p+lo(1), q +lo(2), r+lo(3)) END DO END
> DO END DO !$OMP END PARALLEL DO END DO END DO
>
> Doesn’t this have to be marked as a reduction? And shouldn’t r, q,
> p be labeled private? Perhaps this is automatic, but I do not see
> that said anywhere. Does gnu treat such differently than intel?
> Just ideas.
>
> I am currently trying the toolchain, but it is building everything
> from scratch, including blas, lapack, scalapack etc etc, so will
> take days.
>
> Thank you for your help,
>
> Sincerely,
>
> Ron
>
> --- Ron Cohen reco...@gmail.com <mailto:reco...@gmail.com>
> skypename: ronaldcohen twitter: @recohen3
>
>
>
>
>> On Jan 7, 2019, at 2:16 PM, Robert Schade
>> <robert...@uni-paderborn.de
>> <mailto:robert...@uni-paderborn.de>> wrote:
>>
>> Building cp2k on Intel Xeon Phi Knights Landing (KNL, not to be
>> confused with KNC!) is not different from building it on any
>> other Intel CPU. Hence, I think that the failing regtests point
>> to an underlying issue. Which exact version of the Intel Compiler
>> and MKL have you tried? Best Wishes Robert
>>
>> On 06.01.19 01:59, Ronald Cohen wrote:
>>> OK—sorry for all the noise. I am trying:
>>> ./install_cp2k_toolchain.sh --with-elpa=install
>>> --with-libint=install --with-gcc=install I hate not being able
>>> to use my intel tools which work for me for everything else
>>> just fine.
>>>
>>> Ron
>>>
>>
>> -- Robert Schade Paderborn Center for Parallel Computing (PC2)
>> University of Paderborn Warburger Str. 100 D-33098 Paderborn
>> Germany robert...@uni-paderborn.de
>> <mailto:robert...@uni-paderborn.de> +49/(0)5251/60-5393
>>
>> -- You received this message because you are subscribed to a
>> topic in the Google Groups "cp2k" group. To unsubscribe from this
>> topic, visit
>> https://groups.google.com/d/topic/cp2k/gzmRqKNt62U/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email
>> to cp2k+uns...@googlegroups.com. To post to this group, send
>> email to cp...@googlegroups.com. Visit this group at
>> https://groups.google.com/group/cp2k. For more options, visit
>> https://groups.google.com/d/optout.
>
> -- You received this message because you are subscribed to the
> Google Groups "cp2k" group. To unsubscribe from this group and stop
> receiving emails from it, send an email to
> cp2k+uns...@googlegroups.com
> <mailto:cp2k+uns...@googlegroups.com>. To post to this group,
> send email to cp...@googlegroups.com
> <mailto:cp...@googlegroups.com>. Visit this group at
- --
Robert Schade
Paderborn Center for Parallel Computing (PC2)
University of Paderborn
Warburger Str. 100
D-33098 Paderborn
Germany
robert...@uni-paderborn.de
+49/(0)5251/60-5393
-----BEGIN PGP SIGNATURE-----

iHUEAREIAB0WIQTxLkqaR9fh1LxjbrgesT2Bq8ihsAUCXDOA0wAKCRAesT2Bq8ih
sDXnAP9xoqXUWF04AJfMb07qE6pTl5hP2KCBVK0C6Ht/+I7qXwD9EKSG0T4awN0z
kDvGdq5a+CJhRkxTXz8GCwME0f3TjKg=
=yyb4
-----END PGP SIGNATURE-----

Ronald Cohen

unread,
Jan 7, 2019, 11:46:47 AM1/7/19
to cp...@googlegroups.com
In any event, the OMP version fails. Maybe I chose a bad example. 
The toolchain crashed and burned trying to build elpa. It seems to be using the wrong libraries. I attach the make.log file.
I guess I will now try to build without elpa.
Thanks again,
Ron
make.log
signature.asc

Ronald Cohen

unread,
Jan 7, 2019, 12:00:17 PM1/7/19
to cp...@googlegroups.com
BTW, in case it was not clear. My Intel builds of POPT and PSMP versions were error free.
The problems were all run time.

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3




On Jan 7, 2019, at 5:39 PM, Robert Schade <robert...@uni-paderborn.de> wrote:

Signed PGP part
signature.asc

Robert Schade

unread,
Jan 7, 2019, 12:12:46 PM1/7/19
to cp...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Could you try setting KMP_STACKSIZE to something large in the terminal
session with "export KMP_STACKSIZE=512m" before you rerun the regtests
with your intel-psmp-binary that failed before?
Please also make sure that the general stack size is not the problem
by running "ulimt -s unlimited" in the same terminal where you want to
execute the regtests.
Best Wishes
Robert

On 07.01.19 18:00, Ronald Cohen wrote:
> BTW, in case it was not clear. My Intel builds of POPT and PSMP
> versions were error free. The problems were all run time.
>
> Ron
>
> --- Ron Cohen reco...@gmail.com <mailto:reco...@gmail.com>
> skypename: ronaldcohen twitter: @recohen3
>
>
>
>
>> On Jan 7, 2019, at 5:39 PM, Robert Schade
>> <robert...@uni-paderborn.de
>>>> to cp2k+uns...@googlegroups.com
>> <mailto:cp2k+uns...@googlegroups.com>. To post to this
>> group, send
>>>> email to cp...@googlegroups.com
>>>> <mailto:cp...@googlegroups.com>.
>> Visit this group at
>>>> https://groups.google.com/group/cp2k. For more options,
>>>> visit https://groups.google.com/d/optout.
>>>
>>> -- You received this message because you are subscribed to the
>>> Google Groups "cp2k" group. To unsubscribe from this group and
>>> stop receiving emails from it, send an email to
>>> cp2k+uns...@googlegroups.com
>> <mailto:cp2k+uns...@googlegroups.com>
>>> <mailto:cp2k+uns...@googlegroups.com>. To post to this
>>> group, send email to cp...@googlegroups.com
>>> <mailto:cp...@googlegroups.com> <mailto:cp...@googlegroups.com>.
>>> Visit this group at https://groups.google.com/group/cp2k. For
>>> more options, visit https://groups.google.com/d/optout.
>>
>> -- Robert Schade Paderborn Center for Parallel Computing (PC2)
>> University of Paderborn Warburger Str. 100 D-33098 Paderborn
>> Germany robert...@uni-paderborn.de
>> <mailto:robert...@uni-paderborn.de> +49/(0)5251/60-5393
>>
>
> -- You received this message because you are subscribed to the
> Google Groups "cp2k" group. To unsubscribe from this group and stop
> receiving emails from it, send an email to
> cp2k+uns...@googlegroups.com
> <mailto:cp2k+uns...@googlegroups.com>. To post to this group,
> send email to cp...@googlegroups.com
> <mailto:cp...@googlegroups.com>. Visit this group at
> https://groups.google.com/group/cp2k. For more options, visit
> https://groups.google.com/d/optout.

- --
Robert Schade
Paderborn Center for Parallel Computing (PC2)
University of Paderborn
Warburger Str. 100
D-33098 Paderborn
Germany
robert...@uni-paderborn.de
+49/(0)5251/60-5393
-----BEGIN PGP SIGNATURE-----

iHUEAREIAB0WIQTxLkqaR9fh1LxjbrgesT2Bq8ihsAUCXDOIggAKCRAesT2Bq8ih
sK8nAPkBwCp9jWlAeXe51okHPNDMT4dY/0XDWIGRhdL2CzQMmwD/RmthpKOMFj3n
VSOSWZ+Fqv/bHk2yuwXYFOp48WJ6Tfc=
=VQB6
-----END PGP SIGNATURE-----

Ronald Cohen

unread,
Jan 7, 2019, 1:06:28 PM1/7/19
to cp...@googlegroups.com
So I tried:

export KMP_STACKSIZE=512M
rcohen@tomcat3:~/CP2K/cp2k$ ./tools/regtesting/do_regtest -arch Linux-x86-64-intel -version psmp -restrictdir QS/regtest-dft-vdw-corr-1/ -restrictdir QS/regtest-dft-vdw-corr-2/ -restrictdir QS/regtest-dft-vdw-corr-3/ -restrictdir QS/regtest-dft-vdw-corr-3/ -nobuild -mpiranks 4 -ompthreads 4 -maxtasks 16 |& tee testwith512MKMP_STACKSIZE.out &
and I still get:

< /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-07_18-24-16/tests/QS/regtest-dft-vdw-corr-3 (1 of 3) done in 775.00 sec
>>> /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-07_18-24-16/tests/QS/regtest-dft-vdw-corr-3
    argon05.inp                                               -85.02462435591488  WRONG RESULT TEST 1 
    argon06.inp                                               -85.18989253445228  WRONG RESULT TEST 1 
    argon07.inp                                               -85.05087192159809         RUNTIME FAIL 
    argon08.inp                                               -85.05201740647929         RUNTIME FAIL 
    argon09.inp                                               -85.05086520280044         RUNTIME FAIL 
    argon10.inp                                               -85.05070440200512         RUNTIME FAIL 
    argon11.inp                                               -84.69892988333885         RUNTIME FAIL 
    argon12.inp                                               -84.69900817368848         RUNTIME FAIL 
    argon13.inp                                               -84.81306482759408  WRONG RESULT TEST 1 
    argon14.inp                                               -84.69889654472566  WRONG RESULT TEST 1 
    argon-beef.inp                                            -42.46311172518392  WRONG RESULT TEST 1 
    dftd3bj_t1.inp                                             -0.00355123783846     OK (   1.19 sec) 
    dftd3bj_t2.inp                                             -0.05897356220363     OK (   2.20 sec) 
    dftd3bj_t3.inp                                             -0.00112424003807     OK (   3.75 sec) 
    dftd3bj_t4.inp                                                -84.2983390350     OK (   3.86 sec) 
<<< /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-07_18-24-16/tests/QS/regtest-dft-vdw-corr-3 (1 of 3) done in 775.00 sec
Starting regression tests in /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-07_18-24-16/tests/QS/regtest-dft-vdw-corr-2 (2 of 3)
Starting regression tests in /home/rcohen/CP2K/cp2k/TEST-Linux-x86-64-intel-psmp-2019-01-07_18-24-16/tests/QS/regtest-dft-vdw-corr-2 (2 of 3)


Almost all of the non vdw routines pass.

Sincerely,

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3




Signed PGP part
signature.asc

Anton Kudelin

unread,
Jan 7, 2019, 2:13:16 PM1/7/19
to cp2k
Could you add "-fp-model precise" to CFLAGS and FCFLAGS? It won't fix 'RUNTIME FAIL', but could help with 'WRONG RESULT'.
So I tried:
Signed PGP part

> skypename: ronaldcohen twitter: @recohen3
>
>
>
>
>> On Jan 7, 2019, at 5:39 PM, Robert Schade
>> <robert...@uni-paderborn.de

>>> skypename: ronaldcohen twitter: @recohen3
>>>
>>>
>>>
>>>
>>>> On Jan 7, 2019, at 2:16 PM, Robert Schade
>>>> <robert...@uni-paderborn.de

>>>> <mailto:rob...@uni-paderborn.de>> wrote:
>>>>
>>>> Building cp2k on Intel Xeon Phi Knights Landing (KNL, not to
>>>> be confused with KNC!) is not different from building it on
>>>> any other Intel CPU. Hence, I think that the failing regtests
>>>> point to an underlying issue. Which exact version of the
>>>> Intel Compiler and MKL have you tried? Best Wishes Robert
>>>>
>>>> On 06.01.19 01:59, Ronald Cohen wrote:
>>>>> OK—sorry for all the noise. I am trying:
>>>>> ./install_cp2k_toolchain.sh --with-elpa=install
>>>>> --with-libint=install --with-gcc=install I hate not being
>>>>> able to use my intel tools which work for me for everything
>>>>> else just fine.
>>>>>
>>>>> Ron
>>>>>
>>>>
>>>> -- Robert Schade Paderborn Center for Parallel Computing
>>>> (PC2) University of Paderborn Warburger Str. 100 D-33098
>>>> Paderborn Germany robert...@uni-paderborn.de

>>>>
>>>> -- You received this message because you are subscribed to a
>>>> topic in the Google Groups "cp2k" group. To unsubscribe from
>>>> this topic, visit
>>>> https://groups.google.com/d/topic/cp2k/gzmRqKNt62U/unsubscribe.
>>
>>>>
>> To unsubscribe from this group and all its topics, send an email
>>>> to cp2k+uns...@googlegroups.com
>> <mailto:cp2k...@googlegroups.com>. To post to this

>> group, send
>>>> email to cp...@googlegroups.com

>> Visit this group at
>>>> https://groups.google.com/group/cp2k. For more options,
>>>> visit https://groups.google.com/d/optout.
>>>
>>> -- You received this message because you are subscribed to the
>>> Google Groups "cp2k" group. To unsubscribe from this group and
>>> stop receiving emails from it, send an email to
>>> cp2k+uns...@googlegroups.com

>>> group, send email to cp...@googlegroups.com

>>> Visit this group at https://groups.google.com/group/cp2k. For
>>> more options, visit https://groups.google.com/d/optout.
>>
>> -- Robert Schade Paderborn Center for Parallel Computing (PC2)
>> University of Paderborn Warburger Str. 100 D-33098 Paderborn
>> Germany robert...@uni-paderborn.de

>>
>
> -- You received this message because you are subscribed to the
> Google Groups "cp2k" group. To unsubscribe from this group and stop
> receiving emails from it, send an email to
> cp2k+uns...@googlegroups.com
> <mailto:cp2k...@googlegroups.com>. To post to this group,
> send email to cp...@googlegroups.com
> <mailt...@googlegroups.com>. Visit this group at

Ron Cohen

unread,
Jan 7, 2019, 2:34:23 PM1/7/19
to cp...@googlegroups.com
I did build also with precise but did not help. The values are very wrong , not slightly. Ron

Sent from my iPhone
To unsubscribe from this group and all its topics, send an email to cp2k+uns...@googlegroups.com.
To post to this group, send email to cp...@googlegroups.com.

Alfio Lazzaro

unread,
Jan 8, 2019, 3:13:41 AM1/8/19
to cp2k
Hi Ron,
Could you share one of the FAILED logs? For instance the log of argon07.inp. The regtests script prints the last part of the log at the end of its execution... My suspicious is that these failing tests are dying because of a numerical assert in CP2K, so they can be included in the WRONG category. Now, you are saying that the problem comes from PSMP build, so my first try (very conservative) would be to use a single thread and see if it works. Note that CP2K tests with 2 threads (while you are using 4). Another possibility would be to avoid AVX512 vectorization (CP2K doesn't test it yet). Also, I have just realized that CP2K doesn't test 18.0.5 for PSMP and 19.x at all (see CP2K tests at https://dashboard.cp2k.org/ ). So, my suggestion is to reproduce what it is already tested by CP2K. A good starting point is this test 
 

Alfio

Ronald Cohen

unread,
Jan 8, 2019, 4:12:58 AM1/8/19
to cp...@googlegroups.com
Thank you so much. I don’t have 18.03 installed. I was also having problem with earlier versions, but did not document so carefully
and did not run regtests. When I try my own job (not the regtest) with non-local vdW PSMP just never converges and it segfaults on the stress calculation.



Here is the end of the argon07 test:

 Leaving inner SCF loop after reaching     2 steps.


  Electronic density on regular grids:        -32.0000000000        0.0000000000
  Core density on regular grids:               31.9999999977       -0.0000000023
  Total charge density on r-space grids:       -0.0000000023
  Total charge density g-space grids:          -0.0000000023

  Overlap energy of the core charge distribution:               0.00000000000000
  Self energy of the core charge distribution:               -180.54066673528200
  Core Hamiltonian energy:                                     42.11893140752033
  Hartree energy:                                              68.47313966072379
  Exchange-correlation energy:                                -15.35702018709375
  Dispersion energy:                                            0.25474393253353

  Total energy:                                               -85.05087192159809

 *** WARNING in qs_scf.F:542 :: SCF run NOT converged ***

forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
forrtl: severe (174): SIGSEGV, segmentation fault occurred
EXIT CODE:  174  MEANING:  RUNTIME FAIL
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx



---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Alfio Lazzaro

unread,
Jan 8, 2019, 5:07:19 AM1/8/19
to cp2k
OK, let's focus on this test then.
The message is not really useful. Could you try a single thread? I think 18.0.5 should be fine, but I would suggest to start with -O0 run. Somehow it should run. Then we can use the output as a reference....Alfio

Alfio

Ronald Cohen

unread,
Jan 8, 2019, 12:58:19 PM1/8/19
to cp...@googlegroups.com
OK, the results of the argon06 are rather surprising. There is no crash in the batch, as opposed to regtest, environment. However, the psmp answers are wrong,
even with OMP1. The timings are quite strange too!

MPI: 64

Intel POPT gives
Total Energy               =       -85.1949606240 time 30.98
regtest -85.19494678034609

Intel PSMP OMP=1 gives
85.1906685165 time 434.5
Intel PSMP OMP=2 gives
-85.1907567561 time 436.9
Intel PSMP OMP=4 gives
-85.1915871949  time 8.2

Intel PSMP MPI 4 OMP=4 gives
-85.18890965 4.5 seconds 18:33:12.002 18:33:04.127
 
---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Alfio Lazzaro

unread,
Jan 8, 2019, 1:05:28 PM1/8/19
to cp2k
OK, let's restrict more: could you try with a single MPI rank and a single thread? Have you tried -O0?
Timings are related to the computation done by CP2K. I would assume that the number of SCF steps and/or the density matrices are different in all cases (this is a consequence of some numerical problems). In particular, there are much more calls for the PSMP and that's why the execution is much slower. Could you confirm that? 

We really need to have a common baseline and then understand what's wrong with PSMP...

Ronald Cohen

unread,
Jan 8, 2019, 1:18:30 PM1/8/19
to cp...@googlegroups.com
I mpi and 1 thread  also gives wrong answers.
It takes days to compile, so no I have not tried O0. 

PSMP MPI 1 OMP1
-85.189665204271 6.8 seconds

I do not see any difference in number of steps in the different outputs. Comparing MPI1OMP1 with MPI64OMP4
diff argon06_intelpsmpMPI1OMP1.out argon06_intelpsmpOMP4.out


<   **** **** ******  **  PROGRAM STARTED AT               2019-01-08 19:08:52.037
<  ***** ** ***  *** **   PROGRAM STARTED ON                            tomcat3-05
---
>   **** **** ******  **  PROGRAM STARTED AT               2019-01-08 18:19:26.450
>  ***** ** ***  *** **   PROGRAM STARTED ON                            tomcat3-02
18c20
<  ***** **    ** ** **   PROGRAM PROCESS ID                                 18449
---
>  ***** **    ** ** **   PROGRAM PROCESS ID                                 41162
46c48
<  GLOBAL| Total number of message passing processes                             1
---
>  GLOBAL| Total number of message passing processes                            64
41,142c143,144
<      1 P_Mix/Diag. 0.40E+00    0.4     0.15211079       -85.2350037910 -8.52E+01
<      2 P_Mix/Diag. 0.40E+00    0.4     0.08795282       -85.2158845405  1.91E-02
---
>      1 P_Mix/Diag. 0.40E+00    0.6     0.14637716       -85.2350037910 -8.52E+01
>      2 P_Mix/Diag. 0.40E+00    0.5     0.08521017       -85.2177777050  1.72E-02
154,157c156,159
<   Core Hamiltonian energy:                                     42.08030491078642
<   Hartree energy:                                              68.50424209653565
<   Exchange-correlation energy:                                -15.48646644500879
<   Dispersion energy:                                            0.22670163247355
---
>   Core Hamiltonian energy:                                     42.19622650003371
>   Hartree energy:                                              68.41080078690479
>   Exchange-correlation energy:                                -15.51158810982745
>   Dispersion energy:                                            0.22744985321490
159c161
<   Total energy:                                               -85.21588454049517
---
>   Total energy:                                               -85.21777770495605
164c166
<  ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.):              -85.187309335887107
---
>  ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.):              -85.191969739260287
169,170c171,172
<   Total Energy               =       -85.1873093359
<   Used time                  =                1.739
---
>   Total Energy               =       -85.1919697393
>   Used time                  =                2.061
191,192c193,194

  number of processed stacks                    36       0.0%    100.0%      0.0%
<  average stack size                                     0.0       3.5       0.0
---
>  number of processed stacks                   126       0.0%    100.0%      0.0%
>  average stack size                                     0.0       1.0       0.0
334c336
<  max memory usage/rank             146.509824E+06
---
>  max memory usage/rank             164.413440E+06
337c339
<  # MPI messages exchanged                       0
---
>  # MPI messages exchanged                   10752


---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Ronald Cohen

unread,
Jan 8, 2019, 1:24:34 PM1/8/19
to cp...@googlegroups.com
Comparing with POPT (first evaluation is always correct):
MPI=64

diff  argon06_intelpopt.out argon06_intelpsmpMPI1OMP1.out
<      1 P_Mix/Diag. 0.40E+00    0.3     0.13604108       -85.2350037910 -8.52E+01
<      2 P_Mix/Diag. 0.40E+00    0.3     0.08202063       -85.2189452160  1.61E-02
---
>      1 P_Mix/Diag. 0.40E+00    0.6     0.14637716       -85.2350037910 -8.52E+01
>      2 P_Mix/Diag. 0.40E+00    0.5     0.08521017       -85.2177777050  1.72E-02
154,157c156,159
<   Core Hamiltonian energy:                                     42.38882870025685
<   Hartree energy:                                              68.25714165278610
<   Exchange-correlation energy:                                -15.55292898251896
<   Dispersion energy:                                            0.22868014879907
---
>   Core Hamiltonian energy:                                     42.19622650003371
>   Hartree energy:                                              68.41080078690479
>   Exchange-correlation energy:                                -15.51158810982745
>   Dispersion energy:                                            0.22744985321490
159c161
<   Total energy:                                               -85.21894521595894
---
>   Total energy:                                               -85.21777770495605
164c166
<  ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.):              -85.194770111545679
---
>  ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.):              -85.191969739260287
169,170c171,172
:
<  matmuls total                                126     100.0%      0.0%      0.0%
<  number of processed stacks                   126     100.0%      0.0%      0.0%
<  average stack size                                     1.0       0.0       0.0
---
>  matmuls total                                126       0.0%    100.0%      0.0%
>  number of processed stacks                   126       0.0%    100.0%      0.0%
>  average stack size                                     0.0       1.0       0.0
334c336
<  max memory usage/rank             114.085888E+06
---
>  max memory usage/rank             164.413440E+06
363,364c365,366
<  MP_ISend              384                    175.
<  MP_IRecv              384                     79.
---
>  MP_ISend              384                    179.
>  MP_IRecv              384                     83.
369c371
<  MEMORY| Estimated peak process memory [MiB]                                 109
---
>  MEMORY| Estimated peak process memory [MiB]                                 157
382c384



---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Alfio Lazzaro

unread,
Jan 8, 2019, 1:41:53 PM1/8/19
to cp2k
Sorry, I'm confused... In your previous table, I see that you have

Intel PSMP OMP=1 gives
85.1906685165 time 434.5

In your last table, MPI1OMP1 takes 

Used time                  =                1.739

Could you compare these two cases? I don't see any reason for such a large difference besides a numerical problem...

OK, I see that CP2K people will soon have a test based on 18.0.5 and PSMP (see https://github.com/cp2k/cp2k/pull/158/commits/11ad8588ee2cf6bbdbb9dcd78e62a30da96d8c1c ). Let's see if there is something wrong for the compiler, at least on the Xeon. Sorry to repeat myself, the only strategy I see is to have a common baseline... 

Ronald Cohen

unread,
Jan 8, 2019, 1:58:15 PM1/8/19
to cp...@googlegroups.com
The Intel PSMP OMP=1 was with 64 MPI processes, and MPI1 was with one process.

The times I quoted before were under:

------------------------------------------------------------------------------
 -                                                                             -
 -                                T I M I N G                                  -
 -                                                                             -
 -------------------------------------------------------------------------------
 SUBROUTINE                       CALLS  ASD         SELF TIME        TOTAL TIME
                                MAXIMUM       AVERAGE  MAXIMUM  AVERAGE  MAXIMUM
 CP2K                                 1  1.0    0.040    0.040    6.885    6.885

Thank you!

Ron



---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Ronald Cohen

unread,
Jan 8, 2019, 2:14:40 PM1/8/19
to cp...@googlegroups.com
So I understand ( and have from the beginning) that the PSMP does not give correct answers with the Intel compilers for vdW.
I run Quantum Espresso and ABINIT and other codes using the same compilers and have no problems. I am not sure
I understand what is special about cp2k that makes it sensitive to the compiler.
Anyway, I am still doing the toolchain pure gnu build . It takes days and is still building. I will let you know how that goes.

BTW, the water benchmark (and many hundreds of regtests) work OK with my intel PSMP build. It seems to be mainly a problem with vdW.
But the first cycle works so I don’t understand that either.

Thanks again!

---
Ronald Cohen
Extreme Materials Initiative
Geophysical Laboratory
Carnegie Institution
5251 Broad Branch Rd., N.W.
Washington, D.C. 20015
rco...@carnegiescience.edu
office: 202-478-8937
skype: ronaldcohen
twitter: @recohen3
signature.asc

Anton Kudelin

unread,
Jan 8, 2019, 3:08:25 PM1/8/19
to cp2k
Why is the toolchain taking so much time? Did you set -j option? Usually it runs several hours on a multicore workstation.

Ronald Cohen

unread,
Jan 8, 2019, 3:09:47 PM1/8/19
to cp...@googlegroups.com
Yes, it is set with -j 64! 

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Ronald Cohen

unread,
Jan 8, 2019, 3:11:37 PM1/8/19
to cp...@googlegroups.com
I guess it has failed. It just printed:

 USE mpi  ! compiler *errors* mean mpi installation and fortran compiler mismatch: see INSTALL (-D__HAS_NO_MPI_MOD)

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



On Jan 8, 2019, at 9:08 PM, Anton Kudelin <archm...@gmail.com> wrote:

signature.asc

Ronald Cohen

unread,
Jan 9, 2019, 2:48:54 AM1/9/19
to cp...@googlegroups.com
Unfortunately the toolchain does not work on my system, even though I had it build and install everything including gcc. 
For example:
source /home/rcohen/CP2K/cp2k/tools/toolchain/install/setup
make ARCH=local VERSION=ssmp
iscovering programs ...
Removing stale archives for ssmp ... 
Resolving dependencies for ssmp ... 
make -C /home/rcohen/CP2K/cp2k/exts/dbcsr \
   INCLUDEMAKE=/home/rcohen/CP2K/cp2k/arch/local.ssmp \
   LIBDIR=/home/rcohen/CP2K/cp2k/lib/local/ssmp/exts/dbcsr \
   OBJDIR=/home/rcohen/CP2K/cp2k/obj/local/ssmp/exts/dbcsr \
   FYPPEXE=/home/rcohen/CP2K/cp2k/tools/build_utils/fypp
Removing stale archives ... 
Resolving dependencies ... 
/home/rcohen/CP2K/cp2k/tools/build_utils/fypp -n --line-marker-format=gfortran5 /home/rcohen/CP2K/cp2k/src/qs_outer_scf.F qs_outer_scf.F90
/home/rcohen/CP2K/cp2k/tools/toolchain/install/gcc-8.2.0/bin/gfortran -c -march=native -fno-omit-frame-pointer -g  -O3 -funroll-loops -ffast-math  -fopenmp   -m64 -I/mnt/beegfs/intel/compilers_and_libraries_2018.5.274/linux/mkl/include -I/mnt/beegfs/intel/compilers_and_libraries_2018.5.274/linux/mkl/include/fftw -I'/home/rcohen/CP2K/cp2k/tools/toolchain/install/fftw-3.3.7/include' -I'/home/rcohen/CP2K/cp2k/tools/toolchain/install/libint-1.1.6/include' -I'/home/rcohen/CP2K/cp2k/tools/toolchain/install/libxc-4.0.3/include' -I'/home/rcohen/CP2K/cp2k/tools/toolchain/install/libxsmm-1.10.0/include' -I'/home/rcohen/CP2K/cp2k/tools/toolchain/install/gsl-2.5/include' -I/home/rcohen/CP2K/cp2k/tools/toolchain/install/spglib-1.10.4/include -I/home/rcohen/CP2K/cp2k/tools/toolchain/install/hdf5-1.10.4/include -I'/home/rcohen/CP2K/cp2k/tools/toolchain/install/json_fortran-6.9.0/include' -ffree-form -std=f2003 -fimplicit-none  -Werror=aliasing -Werror=ampersand -Werror=c-binding-type -Werror=intrinsic-shadow -Werror=intrinsics-std -Werror=line-truncation -Werror=tabs -Werror=target-lifetime -Werror=underflow -Werror=unused-but-set-variable -Werror=unused-variable -Werror=unused-dummy-argument -Werror=conversion -Werror=zerotrip -Werror=uninitialized -Wno-maybe-uninitialized -Wuse-without-only  -D__LIBXSMM   -D__FFTW3  -D__LIBINT -D__LIBINT_MAX_AM=6 -D__LIBDERIV_MAX_AM1=5 -D__LIBXC   -D__SPGLIB  -D__JSON   -D__COMPILE_ARCH="\"local\"" -D__COMPILE_DATE="\"Wed Jan  9 08:37:47 CET 2019\"" -D__COMPILE_HOST="\"tomcat3\"" -D__COMPILE_REVISION="\"git:89b110c40\"" -D__DATA_DIR="\"/home/rcohen/CP2K/cp2k/data\"" -D__SHORT_FILE__="\"qs_outer_scf.F\"" -I'/home/rcohen/CP2K/cp2k/src/' -I'/home/rcohen/CP2K/cp2k/obj/local/ssmp/exts/dbcsr' qs_outer_scf.F90 
/home/rcohen/CP2K/cp2k/src/qs_outer_scf.F:27:7:

    USE qs_basis_gradient,               ONLY: qs_basis_center_gradient,&
       1
Fatal Error: Can't open module file \u2018qs_basis_gradient.mod\u2019 for reading at (1): No such file or directory
compilation terminated.
/home/rcohen/CP2K/cp2k/Makefile:447: recipe for target 'qs_outer_scf.o' failed
make[3]: *** [qs_outer_scf.o] Error 1


find . -name qs_basis_gradient.mod
./obj/Linux-x86-64-intelx/psmp/qs_basis_gradient.mod
./obj/Linux-x86-64-intelx/popt/qs_basis_gradient.mod
./obj/local/sdbg/qs_basis_gradient.mod
./obj/local/sopt/qs_basis_gradient.mod

It doesn’t even try to build qs_basis_gradient.mod for the parallel versions.
signature.asc

Ronald Cohen

unread,
Jan 9, 2019, 3:07:56 AM1/9/19
to cp...@googlegroups.com
It seems that the gcc -E doesn’t work:

/home/rcohen/CP2K/cp2k/tools/toolchain/install/gcc-8.2.0/bin/gcc -E -D__COMPILE_ARCH="\"local\"" -D__COMPILE_DATE="\"Wed Jan  9 08:49:48 CET 2019\"" -D__COMPILE_HOST="\"tomcat3\"" -D__COMPILE_REVISION="\"git:89b110c40\"" -D__DATA_DIR="\"/home/rcohen/CP2K/cp2k/data\"" -I/home/rcohen/CP2K/cp2k/src -D__SHORT_FILE__="\"qs_basis_gradient.F\"" -I'/home/rcohen/CP2K/cp2k/src/' qs_basis_gradient.fypped > qs_basis_gradient.f90

It outputs a zero length file:

-rw-r--r--  1 rcohen tomcat   15467 Jan  9 08:49 qs_basis_gradient.fypped
-rw-r--r--  1 rcohen tomcat       0 Jan  9 08:55 qs_basis_gradient.f90

It must work sometimes because some things get built!

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Ronald Cohen

unread,
Jan 9, 2019, 3:17:10 AM1/9/19
to cp2k
It seems that the gcc -E doesn’t work:

/home/rcohen/CP2K/cp2k/tools/toolchain/install/gcc-8.2.0/bin/gcc -E -D__COMPILE_ARCH="\"local\"" -D__COMPILE_DATE="\"Wed Jan  9 08:49:48 CET 2019\"" -D__COMPILE_HOST="\"tomcat3\"" -D__COMPILE_REVISION="\"git:89b110c40\"" -D__DATA_DIR="\"/home/rcohen/CP2K/cp2k/data\"" -I/home/rcohen/CP2K/cp2k/src -D__SHORT_FILE__="\"qs_basis_gradient.F\"" -I'/home/rcohen/CP2K/cp2k/src/' qs_basis_gradient.fypped > qs_basis_gradient.f90
gcc: warning: qs_basis_gradient.fypped: linker input file unused because linking not done


It outputs a zero length file:

-rw-r--r--  1 rcohen tomcat   15467 Jan  9 08:49 qs_basis_gradient.fypped
-rw-r--r--  1 rcohen tomcat       0 Jan  9 08:55 qs_basis_gradient.f90

It must work sometimes because some things get built! 

If I use /usr/bin/gcc (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)) the same thing happens. 

I do not see anything wrong with qs_basis_gradient.fypped . Why does gcc -E output nothing?

Ron

Ronald Cohen

unread,
Jan 9, 2019, 3:30:27 AM1/9/19
to cp...@googlegroups.com
It seems that gcc -E will not work on a file called x.fypped . If I call this file x.f90 and do gcc -E -cpp on it preprocesses. So it seems the toolchain doesn’t not work properly.

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Ronald Cohen

unread,
Jan 9, 2019, 3:39:41 AM1/9/19
to cp...@googlegroups.com
This shows clearly that the file cannot be called fypped:

cp qs_basis_gradient.f90 t.fypped
rcohen@tomcat3:~/CP2K/cp2k/src$ gcc -E -cpp t.fypped
gcc: warning: t.fypped: linker input file unused because linking not done
rcohen@tomcat3:~/CP2K/cp2k/src$ cp t.fypped t.f
rcohen@tomcat3:~/CP2K/cp2k/src$ gcc -E -cpp t.f | head
# 1 "t.f"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "t.f"
!--------------------------------------------------------------------------------------------------!
!   CP2K: A general program to perform molecular dynamics simulations                              !
!   Copyright (C) 2000 - 2018  CP2K developers group                                               !
!--------------------------------------------------------------------------------------------------!

! *****************************************************************************

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Alfio Lazzaro

unread,
Jan 9, 2019, 6:08:44 AM1/9/19
to cp2k
OK, for some reason you set

CPP

in the arch file. Could you check under arch directory what is the value of CPP?
An easy solution would be to remove it.

Ronald Cohen

unread,
Jan 9, 2019, 6:18:44 AM1/9/19
to cp...@googlegroups.com
Sorry I am so much trouble, but No, I didn’t set CPP or edit the arch file:

cohen@tomcat3:~/CP2K/cp2k/arch$ grep CPP local*
local.pdbg:CPP         =
local.popt:CPP         =
local.psmp:CPP         =
local.psmp~:CPP         =
local.sdbg:CPP         =
local.sopt:CPP         =
local.ssmp:CPP         =
local.ssmp~:CPP         =
local_warn.psmp:CPP         =

I also checked env and CPP is not defined. Somehow CPP is getting defined anyway during the make.
I got the build to start working by adding:

undefine CPP

to the Makefile!

Thank you again,

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Alfio Lazzaro

unread,
Jan 9, 2019, 6:21:49 AM1/9/19
to cp2k
To be more precise, probably you have in your arch/local.ssmp

CPP = gcc
CPPFLAGS   = -E

Although this is strange (I've never had these flags set in my toolchain), I get a similar problem if I do set in my arch file.
This is a bug in the CP2K makefile:


I will open a PR for that. Thanks for reporting.

Ronald Cohen

unread,
Jan 9, 2019, 6:31:05 AM1/9/19
to cp...@googlegroups.com
No No! I did not edit the arch file:

grep CPP arch/local.ssmp
CPP         =


---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3



signature.asc

Alfio Lazzaro

unread,
Jan 9, 2019, 6:36:35 AM1/9/19
to cp2k
Well, in any case I'm glad that you found the solution ;) I hope it will solve the other problems.

Ronald Cohen

unread,
Jan 9, 2019, 8:50:33 AM1/9/19
to cp...@googlegroups.com
OK! Good news. The cp2k.ssmp finished building and the argon06 test gives exactly the correct number with OMP=64
and takes only 4.871 seconds. I will see how it works with my real job. I will also try building the mpi versions. I did notice a problem with the make dependencies.
I had to do a make clean before cp2k.ssmp would build. Otherwise it complains about missing modules and doesn’t try to make them.

I included the flags -mavx512f -mavx512er -mavx512cd -mavx512pf so that the KNL features would be used.

Thanks for all of your help.

Sincerely,

Ron

---
Ron Cohen
reco...@gmail.com
skypename: ronaldcohen
twitter: @recohen3




signature.asc
Reply all
Reply to author
Forward
0 new messages