Wave function optimization don't converge

[Jiani]

Dear QMCPACK developers,

Thank you for your long-term contribution to the development of QMCPACK! I am Jiani Hu from the School of Physics, Peking University, China. I have been employing Quantum Espresso and QMCPACK for QMC calculations, specifically studying a system where CO2 is adsorbed on graphene-confining single-atom Co–N3. Its structure file can be found in the attached POSCAR file. Unfortunately, I encountered a series of problems when calculating this system. Unfortunately, I encountered a series of problems when calculating this system.

1. When performing calculations with B-spline orbitals in QMCPACK, memory problems are often reported when using B-spline orbits. The process is usually terminated after outputting "MEMORY 6973 MB allocated for the coefficients in 3D spline orbital representation" and reporting insufficient memory. My task is run on a supercomputer, usually 4 nodes with 256 cores, and each node has 256 GB of memory, but the output memory obviously does not reach the node's memory limit. The termination occurs at around 4 GB of memory usage, suggesting the process might be running on a single core. I am uncertain if this reflects a problem with the QMCPACK installation or my parallel configuration.
   
2. In order to avoid memory problems, a workaround is to set meshfactor to 0.5, which will significantly reduce the memory requirements during the calculation. Will this parameter selection cause risks to subsequent calculations? For such a calculation of about 150 electrons system, is it necessary to build a supercell calculation?
   
3. During the wavefunction optimization process, when I was optimizing 2-body and 3-body, it was difficult for me to reduce the variance/Local energy ratio to the ideal situation (the reference value given in the manual was about 0.02, and I could only optimize to Around 0.2). I have encountered this problem before when calculating the adsorption energy of water molecules on the graphene surface, I found that the degauss parameter setting in the QE calculation was unreasonable. However, for this system, I adjusted the degauss parameter and it did not seem to improve the quality of the wavefunction.
   
4. I have completed some successful wave function optimization calculations before (a system composed of water molecules and graphene), but I am not sure how to choose the best Jastrow parameters. According to the description in the manual, the variance energy ratio is generally about It is safer when 0.02. This is the energy variance sequence I obtained during optimization(as shown at the bottom), but after I select the s036 sequence, subsequent calculations will encounter the Problem 5. I don't know if there is a connection between these. Is there any specific criterion for selecting the best Jastrow parameters, such as the lowest energy or the smallest variance?
   
5. When calculating DMC, it is often encountered that the total number of walkers exceeds the upper and lower limits allowed, causing the energy output by DMC to drop abnormally and rapidly. This problem is more likely to occur when the timestep value is larger. Is there any good way to avoid this problem?

                            LocalEnergy               Variance           ratio
optJ3  series 0  -300.923634 +/- 0.015431   8.282798 +/- 0.223775   0.0275
optJ3  series 1  -300.966691 +/- 0.019415   8.190784 +/- 0.093041   0.0272
optJ3  series 2  -300.938715 +/- 0.012795   8.608224 +/- 0.374875   0.0286
optJ3  series 3  -300.920760 +/- 0.014264   8.160477 +/- 0.141171   0.0271
optJ3  series 4  -300.917912 +/- 0.010327   8.071611 +/- 0.076651   0.0268
optJ3  series 5  -300.937036 +/- 0.022230   8.229880 +/- 0.105500   0.0273
optJ3  series 6  -300.910398 +/- 0.009955   8.121194 +/- 0.090392   0.0270
optJ3  series 7  -300.938982 +/- 0.014229   8.427071 +/- 0.144109   0.0280
optJ3  series 8  -300.929331 +/- 0.013866   8.799157 +/- 0.411269   0.0292
optJ3  series 9  -300.943123 +/- 0.014685   8.354525 +/- 0.086467   0.0278
optJ3  series 10  -300.913091 +/- 0.014537   8.308522 +/- 0.137808   0.0276
optJ3  series 11  -300.925558 +/- 0.012458   8.039715 +/- 0.122318   0.0267
optJ3  series 12  -300.945276 +/- 0.015904   8.182601 +/- 0.076964   0.0272
optJ3  series 13  -300.924240 +/- 0.014549   8.337045 +/- 0.084842   0.0277
optJ3  series 14  -300.931582 +/- 0.009878   8.059256 +/- 0.083733   0.0268
optJ3  series 15  -300.913289 +/- 0.015258   8.233391 +/- 0.105577   0.0274
optJ3  series 16  -300.925756 +/- 0.012152   8.309005 +/- 0.122414   0.0276
optJ3  series 17  -300.892713 +/- 0.024585   8.116152 +/- 0.258089   0.0270
optJ3  series 18  -300.924853 +/- 0.011775   8.434078 +/- 0.161878   0.0280
optJ3  series 19  -300.932000 +/- 0.011592   8.222952 +/- 0.131098   0.0273
optJ3  series 20  -300.926577 +/- 0.012667   8.236947 +/- 0.106318   0.0274
optJ3  series 21  -300.885018 +/- 0.031024   8.163382 +/- 0.179160   0.0271
optJ3  series 22  -300.989170 +/- 0.040127   7.764708 +/- 0.176598   0.0258
optJ3  series 23  -300.916619 +/- 0.009837   8.234577 +/- 0.127153   0.0274
optJ3  series 24  -300.925775 +/- 0.014836   8.529961 +/- 0.166703   0.0283
optJ3  series 25  -300.907376 +/- 0.013416   8.124708 +/- 0.102086   0.0270
optJ3  series 26  -301.032323 +/- 0.051770   8.314366 +/- 0.325574   0.0276
optJ3  series 27  -300.938231 +/- 0.022824   8.283174 +/- 0.137807   0.0275
optJ3  series 28  -300.920497 +/- 0.012269   8.318313 +/- 0.112553   0.0276
optJ3  series 29  -300.938134 +/- 0.010626   8.400825 +/- 0.145536   0.0279
optJ3  series 30  -300.942336 +/- 0.014501   8.131714 +/- 0.142547   0.0270
optJ3  series 31  -300.917506 +/- 0.014949   8.363806 +/- 0.213613   0.0278
optJ3  series 32  -300.957620 +/- 0.012761   8.202248 +/- 0.121595   0.0273
optJ3  series 33  -300.944914 +/- 0.010337   8.447501 +/- 0.131620   0.0281
optJ3  series 34  -300.919365 +/- 0.011349   8.262931 +/- 0.126944   0.0275
optJ3  series 35  -300.929277 +/- 0.012285   8.325243 +/- 0.149728   0.0277
optJ3  series 36  -300.945183 +/- 0.014451   8.288131 +/- 0.145089   0.0275
optJ3  series 37  -300.917734 +/- 0.013946   8.109379 +/- 0.100138   0.0269
optJ3  series 38  -300.904192 +/- 0.016594   8.247370 +/- 0.198002   0.0274
optJ3  series 39  -300.931691 +/- 0.013334   9.692077 +/- 1.094072   0.0322

Pseudopotential in CCECP form, from https://pseudopotentiallibrary.org/

Best wishes,

Jiani Hu



 

[Paul R. C. Kent]

Dear Jiani Hu

We should tackle these problems in sequence. Regarding the memory usage, since you have 256 core nodes with 256GB memory, you will need to use some combination of MPI and OpenMP on the node. Since the splines are taking 6973MB, using 256 MPI processes won't work since the usage would be 6973*256 MB, even before spending any memory on the walkers in the Monte Carlo. Ideally you use 1 MPI per socket or NUMA region, and then OpenMP threads to use all the cores. e.g. 4 MPI processes with 64 threads each (OMP_NUM_THREADS=64). Your supercomputer center and its administrators should have some instructions on this. Please check with them. Once this is sorted out you will be able to use the normal meshfactor of ~1.0. Using a small meshfactor can damage the wavefunction.

-- Paul

[Jiani]

Dear Paul

Thank you very much for your reply~  With your help, I solved the memory issue by employing a combination of MPI and OpenMP, enabling the task to proceed with a meshfactor of 1. Unfortunately, Problem 3 still exists. I am not sure whether it is a problem with the DFT input, which prevents me from generating a better trial wave function. 

Best wishes,

Jiani

[Paul R. C. Kent]

Have you done some tests running e.g. a small supercell of carbon diamond? It is good to build up to a potentially difficult problem so that the source of any errors can be easily isolated. I find it also helps to get a feel for the statistics that are going to required. If you have not already, I would check VMC, VMC optimization, then DMC is OK for a simple system using wavefunctions completely made by yourself.

[Jiani]

Dear Paul

Nice to receive your reply! I once tried QMC calculation of the adsorption energy of water molecules on graphene, and obtained a relatively ideal variance/energy ratio when optimizing the wave function. But it doesn't work when calculating CO2 adsorbed on graphene-confined single atoms Co-N3 😿

Best wishes,

Jiani