The Kohn Sham matrix is not 100% occupied

[Claudio]

Dear all, when I try to run a periodic HF calculation or a mixed HF
DFT run (b3lyp) for example. I always get

*** 06:49:15 WARNING in
hfx_energy_potential:integrate_four_center ***
*** err=-300 The Kohn Sham matrix is not 100% occupied. This may
result ***
*** in uncorrect Hartree-Fock results. Try to decrease EPS_PGF_ORB in
the ***
*** QS
section.
***

Same system converges without problem without HF using for example
PBE. I tried decreasing EPS_PGF_ORB and EPS_DEFAULT without much
success. I also tried decreasing basis set size. I am not using a
truncated potential.

Is this warning relevant? I am having a very hard time converging the
b3lyp calculation.

Thanks,
Claudio

[mguidon]

Hi Claudio.

There are several issues that could cause troubles in a periodic HFX
calculation:

1. Always use the truncated Coulomb operator, otherwise your results
might converge to an unphysical wavefunction (i.e. total energies a
far lower than what you would expect)
2. The sparsity-patterns of the Fock- and Kohn-Sham matrice are not
the same. This issue can be solved by putting EPS_PGF_ORB to 1.0E-32
or similar. (But usually, a 90 % occupied Kohn-Sham matrix is good
enough, especially when you apply the truncated potential)
3. By putting a threshold on all the integrals (EPS_SCHWARZ) you
introduce some random errors of the same order. As a rule of thumb,
eps_schwarz should be roughly equal to the inverse square of the
condition number of the overlap matrix. For example: If the condition
number of the overlap matrix is 1.0E4, then you should put eps_schwarz
to 1.0E-8 (but this is a worst case estimate!) Keep in mind, that the
amount of work roughly doubles by decreasing eps_schwarz by one order
of magnitude. It might be better to decrease the condition number of
the overlap matrix by choosing a less diffuse basis set.
4. Always add the keyword EPS_STORAGE_SCALING 1.0E-1 in order to avoid
suppress noise in the compression/decompression routines.

For more details see

J. Chem. Theory Comput., 2009, 5 (11), pp 3010-3021

Hope that helps

Manuel

[Claudio]

Thanks Manuel, I just read the paper.
I will try some of this and will report back.

Cheers,
Claudio

[Claudio]

Dear Manuel,
thanks again for providing some guidance on this calculation. I did
what you suggested, and the HF run appears to converge

1 OT CG 0.15E+00 1241.8 0.00004886 -6103.3087142795
-6.10E+03

however cube files for orbitals are full on NaN, not the density cube
though. Furthermore I get at the end of the output something that I
hadn't seen before on any dft calculations. Part of the output
follows:




-------------------------------------------------------------------------------

-
-
- T I M I N
G -

-
-

-------------------------------------------------------------------------------
SUBROUTINE CALLS ASD SELF TIME
TOTAL TIME
AVERAGE MAXIMUM
AVERAGE MAXIMUM
CP2K 1 1.0 0.223 0.325
1305.554 1305.638
qs_energies_scf 1 2.0 0.000 0.001
1301.807 1301.899
scf_env_do_scf 1 3.0 0.000 0.000
1243.079 1243.128
init_scf_loop 1 4.0 0.000 0.000
1241.378 1241.400
qs_ks_update_qs_env 2 5.0 0.000 0.000
1240.934 1240.956
qs_ks_build_kohn_sham_matrix 1 6.0 0.004 0.017
1240.932 1240.954
integrate_four_center 1 7.0 143.888 246.392
1238.092 1238.114
integrate_four_center_main 1 8.0 0.000 0.001
1068.609 1211.585
integrate_four_center_bin 64 9.0 1068.608 1211.585
1068.608 1211.585
scf_post_calculation_gpw 1 3.0 0.047 0.049
55.556 55.600
ot_eigensolver 1 4.0 0.010 0.012
43.961 43.963
dbcsr_sm_fm_multiply 304 5.0 0.031 0.036
43.093 43.389
dbcsr_sm_sm_multiply 304 6.0 0.038 0.045
41.428 41.557
dbcsr_mult_e_ec_m_rma 304 7.0 38.174 38.293
38.197 38.306

-------------------------------------------------------------------------------

**** **** ****** ** PROGRAM ENDED AT 2009-11-17
05:26:51.494
***** ** *** *** ** PROGRAM RAN
ON node014
** **** ****** PROGRAM RAN
BY claudiom
***** ** ** ** ** PROGRAM PROCESS
ID 11756
**** ** ******* ** PROGRAM STOPPED IN /home/claudiom/snapshots/
cp2k/mmim-cl
/b3lyp/neutral
Remaining memory: 80 bytes at 00000000375ad338 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at Remaining memory: 80 bytes at
0000000035dd0d98 allocated at line 1806 of /raid/home/claudiom/src/
cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000359e0eb8 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
0000000034c0f9a8Remaining memory: 80 bytes at 0000000034853078
allocated at line 1808 of /raid/home/claudiom/src/cp2k/makefiles/../
src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000347705e8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 000000003466da98 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000036afd98 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000344fc578 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 000000000472d9c8 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000047211d8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 000000000469c168 allocated at line
allocated at line 1808 of /raid/home/claudiom/src/cp2k/makefiles/../
src/dbcsr_util.F1806 of /raid/home/claudiom/src/cp2k/makefiles/../src/
dbcs\
r_util.F
Remaining memory: 80 bytes at 00000000046ab708 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000048116f8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80
bytes at 00000000047d4938 allocated at line 1806 of /raid/home/
claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: Remaining memory: 80 bytes at 00000000047d4308
allocated at line 1808 of /raid/home/claudiom/src/cp2k/makefiles/../
src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000047d0158 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
80Remaining memory: 80 bytes at 0000000004812368 allocated at line
1808 of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 0000000004813718 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: bytes at 80 bytes at 000000000484e8d8 allocated at
line 1806 of /raid/home/claudiom/src/cp2k/makefiles/../src/
dbcsr_util.F
Remaining memory: 80 bytes at 0000000004819838 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/
dbcsr_util.F00000000346ce238
Remaining memory: 80 bytes at 00000000048184a8 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000048146e8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000344fc278 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
allocated at line
Remaining memory: 80 bytes at 000000003455bbd8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000345270e8 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 0000000034527f18 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
... More segments remain
1806 of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 000000003469e8b8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 000000000490fea8 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 0000000004897f28 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 0000000004890c18 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 000000000473b218 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 00000000031fe748 allocated at line 1808
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 0000000004712328 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F
Remaining memory: 80 bytes at 0000000004719fd8 allocated at line 1806
of /raid/home/claudiom/src/cp2k/makefiles/../src/dbcsr_util.F


and many more lines of memory complaints... Any ideas on how to
overcome this are really welcome.

Thanks,
Claudio

[mguidon]

Hi Claudio.

All the "remaining memory" messages are due to a memory leak in the
code that has been fixed quite a while ago. What version of the code
are you using? Just update CP2K to the latest version and these
messages should disappear.


Manuel

> ...
>
> read more »

[Claudio]

Manuel, the code version is 2.1.56

I got it no more than a month ago. Was this fixed before or after this
revision?

Also, what about the NaNs in the cube files.

Thanks!
Claudio

[mguidon]

Hi Claudio

> I got it no more than a month ago. Was this fixed before or after this
> revision?
>

Probably afterwards. The code is currently undergoing some major
changes and I strongly recommend you to always use the latest version.

> Also, what about the NaNs in the cube files.

This could be related. But as usual, without input file, there is no
way to track down your problem.

Manuel

[Claudio]

OK, so I compiled today's CVS version of cp2k. The input file follows
at the end of this message. The memory leak errors are gone, but my
cubes are still full of NaNs.

Disregard the fact that this system has an unpaired electron, I get
the same result with a closed shell system. The one below is just an
example. The size of the box is large because I am just testing the
effect of the cutoff. This is just a small test system, what I am
really trying to run is a snapshot of a liquid made of these
molecules.

&FORCE_EVAL
METHOD Quickstep
&DFT
LSD
CHARGE -1
MULTIPLICITY 2
BASIS_SET_FILE_NAME /home/claudiom/src/cp2k/tests/QS/
EMSL_BASIS_SETS
POTENTIAL_FILE_NAME /home/claudiom/src/cp2k/tests/QS/POTENTIAL
&MGRID
! CUTOFF 200
! REL_CUTOFF 50
&END MGRID
&QS
METHOD GAPW
EPS_PGF_ORB 1.0E-16
&END QS
&PRINT
&MO_CUBES
NHOMO 5
NLUMO 5
&END
&E_DENSITY_CUBE
&END
&END PRINT
&SCF
EPS_SCF 1.0E-4
SCF_GUESS RESTART
MAX_SCF 60
&OT
&END OT
&OUTER_SCF
MAX_SCF 1
&END OUTER_SCF
&END SCF
#These are the coefficients used for B3LYP using VWN3, this is the
default Gaussian definition, but not recommended
&XC
&XC_FUNCTIONAL
&LYP
SCALE_C 0.81
&END LYP
&BECKE88
SCALE_X 0.72
&END BECKE88
&VWN
FUNCTIONAL_TYPE VWN3
SCALE_C 0.19
&END VWN
&XALPHA
SCALE_X 0.08
&END XALPHA
&END XC_FUNCTIONAL
&HF
&SCREENING
EPS_SCHWARZ 1.0E-10
SCREEN_ON_INITIAL_P FALSE
&END SCREENING
&INTERACTION_POTENTIAL
POTENTIAL_TYPE TRUNCATED
CUTOFF_RADIUS 15.000
T_C_G_DATA /home/claudiom/src/cp2k/tests/QS/t_c_g.dat
&END INTERACTION_POTENTIAL
&MEMORY
MAX_MEMORY 768
EPS_STORAGE_SCALING 1.0E-1
&END MEMORY
FRACTION 0.20
&END HF
&END XC
&END DFT
&SUBSYS
&CELL
ABC 31.000 31.000 31.000
PERIODIC XYZ
&END CELL
&COORD
C 2.724300 -0.210525 1.679657
N 3.378813 0.142337 0.367635
C 2.628981 -0.547029 -0.747413
C 3.308675 1.641065 0.186405
C 4.824483 -0.335730 0.427386
C 5.707747 0.076171 -0.750114
C 7.039619 -0.686925 -0.723046
N -2.115674 0.098149 0.566927
S -1.065879 1.188462 -0.004405
O -0.096986 1.688012 0.992480
O -0.543935 0.938005 -1.360293
S -1.679594 -1.425080 0.861473
O -0.223810 -1.659926 0.878041
O -2.517197 -1.975152 1.928867
C -2.318886 2.581971 -0.206516
C -2.273741 -2.356150 -0.672280
F -3.277763 2.255810 -1.085288
F -1.666785 3.662915 -0.676494
F -2.893950 2.911808 0.957784
F -2.046785 -3.675730 -0.512894
F -1.628775 -1.949980 -1.775445
F -3.592764 -2.180301 -0.862609
H 3.269846 0.294602 2.481570
H 2.778648 -1.293048 1.815246
H 1.683606 0.110359 1.652541
H 1.595470 -0.201469 -0.749459
H 2.666731 -1.624615 -0.575470
H 3.104631 -0.308923 -1.698754
H 3.927011 2.115369 0.954315
H 2.265541 1.948040 0.284571
H 3.693613 1.904240 -0.800118
H 4.768472 -1.425574 0.513280
H 5.238050 0.053710 1.365517
H 5.910151 1.153814 -0.709845
H 5.210145 -0.119752 -1.706544
H 7.575766 -0.520996 0.219045
H 6.886641 -1.766428 -0.838317
H 7.681022 -0.342145 -1.545307
&END COORD
&KIND H
BASIS_SET 6-31Gxx
POTENTIAL ALL
&END KIND
&KIND F
BASIS_SET 6-31Gxx
POTENTIAL ALL
&END KIND
&KIND C
BASIS_SET 6-31Gxx
POTENTIAL ALL
&END KIND
&KIND N
BASIS_SET 6-31Gxx
POTENTIAL ALL
&END KIND
&KIND S
BASIS_SET 6-31Gxx
POTENTIAL ALL
&END KIND
&KIND O
BASIS_SET 6-31Gxx
POTENTIAL ALL
&END KIND
&END SUBSYS
&END FORCE_EVAL
&GLOBAL
PROJECT HF-ion-pair
PRINT_LEVEL MEDIUM
RUN_TYPE ENERGY
ECHO_INPUT
&END GLOBAL