Deallocate memory used by Hamiltonian-related subroutines

58 views
Skip to first unread message

Cindy Pham

unread,
Oct 3, 2023, 4:17:14 PM10/3/23
to cp...@googlegroups.com
Hi CP2K forum,

I am running a lengthy SCF calculation (over 10k iterations) and noticed a gradual increase in the allocated memory (I used TRACE keyword to print current allocated memory). It appears that the step-by-step increase in memory allocation happens when the Kohn-Sham Hamiltonian is re-calculated, specifically within the qs_ks_did_change function (in qs_ks_types.F).The SCF routine is my own code that relies on the CP2K built-in Hamiltonian subroutines. It functions properly, but its only problem is the ever increasing memory consumption.Since I do not need to keep previous Hamiltonians (for any kind of DIIS extrapolation), is there any way to deallocate all memory used by the Hamiltonian-related subroutines (at least once in a while, say, after 1000 SCF iterations)?Alternatively, are there any input keywords that can ensure that the Hamiltonian structures are reset once in a while?

Thank you in advance for your time and your suggestions.

Best regards,
Cindy Pham.

Jürg Hutter

unread,
Oct 4, 2023, 4:15:43 AM10/4/23
to cp...@googlegroups.com
Hi

without knowing the details of your program it is impossible to point to
a easy solution. As there are no memory leaks in CP2K, you must miss some
routines that clean up at the end of the SCF loop.
I would suggest you compile the code with memory leak detection in order
to find the problematic structure and then write a routine to deallocate them.
See the sdbg arch files for the gfortran options needed for leak detection.

regards
JH

________________________________________
From: cp...@googlegroups.com <cp...@googlegroups.com> on behalf of Cindy Pham <cindyp...@gmail.com>
Sent: Tuesday, October 3, 2023 10:16 PM
To: cp...@googlegroups.com
Subject: [CP2K:19284] Deallocate memory used by Hamiltonian-related subroutines
--
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k+uns...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/CAN4Jpm3f3BUuJ%2B3EDEuqpp0TbBEkaHfOjQm%2Bn9%3D6vYquYQNQvQ%40mail.gmail.com<https://groups.google.com/d/msgid/cp2k/CAN4Jpm3f3BUuJ%2B3EDEuqpp0TbBEkaHfOjQm%2Bn9%3D6vYquYQNQvQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Cindy Pham

unread,
Oct 4, 2023, 8:57:01 PM10/4/23
to cp...@googlegroups.com
Hi Prof. Hutter,
 
Thank you for your suggestion!

Best regards,
Cindy.



To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/ZR0P278MB07591FF04A66D61DBAA83F3A9FCBA%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM.

Cindy Pham

unread,
Oct 30, 2023, 1:53:06 PM10/30/23
to cp...@googlegroups.com
Hi Prof.Hutter,

The memory used CP2K (as read by TRACE from /proc/self/statm file) keeps increasing even if I run a standard OT job (the input file is attached). The TRACE helped us to establish that the memory increase happens within qs_rho_update_rho_low() subroutine (the increase is shown in the attached plot). A closer look into the TRACE printout of subroutines called by qs_rho_update_rho_low() produces confusing data. TRACE tells that memory allocation can increase not only in complex subroutines, but even in simple subroutines such as pw_zero(). How is it possible that pw_zero() increases TRACE printout? Is it because allocated memory cannot be measured precisely in Unix (via /proc/self/statm)?If so, it still does not explain the SYSTEMATIC memory increase in more complex subroutines called by qs_rho_update_rho_low(). I have noticed two arrays, grids_c and npts_local, within the grid_collocate_task_list subroutine that are not explicitly deallocated. Do you know if it is by design? I know that valgrind tells that the memory is properly deallocated (somehow) in the END of the job, but the minor increases DURING a long run eventually lead to "out of memory" problem.

Best regards,
Cindy.
He_pw_zero_OT_mem_increase_cell888.txt
He_crystal_OT.inp
trace_He_crystal_OT_cell888.pdf

Jürg Hutter

unread,
Nov 1, 2023, 4:34:02 AM11/1/23
to cp...@googlegroups.com
Hi

I run your input for 15000 steps and don't see any memory increase.
The run was on 9 CPU's using the Trunk version, compiled with
gcc-13.2.0 and openmpi-4.1.5.

Can you verify your setup? Especially some mpi versions are known to
have internal memory leaks.

regards
JH

________________________________________
From: cp...@googlegroups.com <cp...@googlegroups.com> on behalf of Cindy Pham <cindyp...@gmail.com>
Sent: Monday, October 30, 2023 6:52 PM
To: cp...@googlegroups.com
Subject: Re: [CP2K:19445] Deallocate memory used by Hamiltonian-related subroutines

Hi Prof.Hutter,

The memory used CP2K (as read by TRACE from /proc/self/statm file) keeps increasing even if I run a standard OT job (the input file is attached). The TRACE helped us to establish that the memory increase happens within qs_rho_update_rho_low() subroutine (the increase is shown in the attached plot). A closer look into the TRACE printout of subroutines called by qs_rho_update_rho_low() produces confusing data. TRACE tells that memory allocation can increase not only in complex subroutines, but even in simple subroutines such as pw_zero(). How is it possible that pw_zero() increases TRACE printout? Is it because allocated memory cannot be measured precisely in Unix (via /proc/self/statm)?

If so, it still does not explain the SYSTEMATIC memory increase in more complex subroutines called by qs_rho_update_rho_low(). I have noticed two arrays, grids_c and npts_local, within the grid_collocate_task_list subroutine that are not explicitly deallocated. Do you know if it is by design? I know that valgrind tells that the memory is properly deallocated (somehow) in the END of the job, but the minor increases DURING a long run eventually lead to "out of memory" problem.

Best regards,
Cindy.

On Wed, Oct 4, 2023 at 8:56 PM Cindy Pham <cindyp...@gmail.com<mailto:cindyp...@gmail.com>> wrote:
Hi Prof. Hutter,

Thank you for your suggestion!

Best regards,
Cindy.



On Wed, Oct 4, 2023 at 4:15 AM Jürg Hutter <hut...@chem.uzh.ch<mailto:hut...@chem.uzh.ch>> wrote:
Hi

without knowing the details of your program it is impossible to point to
a easy solution. As there are no memory leaks in CP2K, you must miss some
routines that clean up at the end of the SCF loop.
I would suggest you compile the code with memory leak detection in order
to find the problematic structure and then write a routine to deallocate them.
See the sdbg arch files for the gfortran options needed for leak detection.

regards
JH

________________________________________
From: cp...@googlegroups.com<mailto:cp...@googlegroups.com> <cp...@googlegroups.com<mailto:cp...@googlegroups.com>> on behalf of Cindy Pham <cindyp...@gmail.com<mailto:cindyp...@gmail.com>>
Sent: Tuesday, October 3, 2023 10:16 PM
To: cp...@googlegroups.com<mailto:cp...@googlegroups.com>
Subject: [CP2K:19284] Deallocate memory used by Hamiltonian-related subroutines

Hi CP2K forum,

I am running a lengthy SCF calculation (over 10k iterations) and noticed a gradual increase in the allocated memory (I used TRACE keyword to print current allocated memory). It appears that the step-by-step increase in memory allocation happens when the Kohn-Sham Hamiltonian is re-calculated, specifically within the qs_ks_did_change function (in qs_ks_types.F).

The SCF routine is my own code that relies on the CP2K built-in Hamiltonian subroutines. It functions properly, but its only problem is the ever increasing memory consumption.

Since I do not need to keep previous Hamiltonians (for any kind of DIIS extrapolation), is there any way to deallocate all memory used by the Hamiltonian-related subroutines (at least once in a while, say, after 1000 SCF iterations)?

Alternatively, are there any input keywords that can ensure that the Hamiltonian structures are reset once in a while?

Thank you in advance for your time and your suggestions.

Best regards,
Cindy Pham.

--
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com><mailto:cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com>>.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/ZR0P278MB07591FF04A66D61DBAA83F3A9FCBA%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM.

--
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k+uns...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/CAN4Jpm0fQ2eh-B5VbkYRoMcphU9eYDL5f6Jki7FkvY%3Duh9h3GQ%40mail.gmail.com<https://groups.google.com/d/msgid/cp2k/CAN4Jpm0fQ2eh-B5VbkYRoMcphU9eYDL5f6Jki7FkvY%3Duh9h3GQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Cindy Pham

unread,
Nov 14, 2023, 9:08:01 PM11/14/23
to cp...@googlegroups.com
Hi Prof.Hutter,

My setup was with gcc-7.3.0 and openmpi-3.1.2. I tried to use gcc-12.3.0 and openmpi-4.1.5 as you suggested but I could not compile using these versions with my makefile. May I ask if you also used TRACE as the memory check tool or others? 

Best,
Cindy.

To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/ZR0P278MB075927A9473369B2C0BD1F419FA7A%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM.

Jürg Hutter

unread,
Nov 15, 2023, 11:08:02 AM11/15/23
to cp...@googlegroups.com
Yes, I was using TRACE to get detailed information and the Linux "top"
command for an external measure.

regards
JH

________________________________________
From: cp...@googlegroups.com <cp...@googlegroups.com> on behalf of Cindy Pham <cindyp...@gmail.com>
Sent: Wednesday, November 15, 2023 3:07 AM
To: cp...@googlegroups.com
Subject: Re: [CP2K:19502] Deallocate memory used by Hamiltonian-related subroutines

Hi Prof.Hutter,

My setup was with gcc-7.3.0 and openmpi-3.1.2. I tried to use gcc-12.3.0 and openmpi-4.1.5 as you suggested but I could not compile using these versions with my makefile. May I ask if you also used TRACE as the memory check tool or others?

Best,
Cindy.

On Wed, Nov 1, 2023 at 4:34 AM Jürg Hutter <hut...@chem.uzh.ch<mailto:hut...@chem.uzh.ch>> wrote:
Hi

I run your input for 15000 steps and don't see any memory increase.
The run was on 9 CPU's using the Trunk version, compiled with
gcc-13.2.0 and openmpi-4.1.5.

Can you verify your setup? Especially some mpi versions are known to
have internal memory leaks.

regards
JH

________________________________________
Sent: Monday, October 30, 2023 6:52 PM
To: cp...@googlegroups.com<mailto:cp...@googlegroups.com>
Subject: Re: [CP2K:19445] Deallocate memory used by Hamiltonian-related subroutines

Hi Prof.Hutter,

The memory used CP2K (as read by TRACE from /proc/self/statm file) keeps increasing even if I run a standard OT job (the input file is attached). The TRACE helped us to establish that the memory increase happens within qs_rho_update_rho_low() subroutine (the increase is shown in the attached plot). A closer look into the TRACE printout of subroutines called by qs_rho_update_rho_low() produces confusing data. TRACE tells that memory allocation can increase not only in complex subroutines, but even in simple subroutines such as pw_zero(). How is it possible that pw_zero() increases TRACE printout? Is it because allocated memory cannot be measured precisely in Unix (via /proc/self/statm)?

If so, it still does not explain the SYSTEMATIC memory increase in more complex subroutines called by qs_rho_update_rho_low(). I have noticed two arrays, grids_c and npts_local, within the grid_collocate_task_list subroutine that are not explicitly deallocated. Do you know if it is by design? I know that valgrind tells that the memory is properly deallocated (somehow) in the END of the job, but the minor increases DURING a long run eventually lead to "out of memory" problem.

Best regards,
Cindy.

On Wed, Oct 4, 2023 at 8:56 PM Cindy Pham <cindyp...@gmail.com<mailto:cindyp...@gmail.com><mailto:cindyp...@gmail.com<mailto:cindyp...@gmail.com>>> wrote:
Hi Prof. Hutter,

Thank you for your suggestion!

Best regards,
Cindy.



On Wed, Oct 4, 2023 at 4:15 AM Jürg Hutter <hut...@chem.uzh.ch<mailto:hut...@chem.uzh.ch><mailto:hut...@chem.uzh.ch<mailto:hut...@chem.uzh.ch>>> wrote:
Hi

without knowing the details of your program it is impossible to point to
a easy solution. As there are no memory leaks in CP2K, you must miss some
routines that clean up at the end of the SCF loop.
I would suggest you compile the code with memory leak detection in order
to find the problematic structure and then write a routine to deallocate them.
See the sdbg arch files for the gfortran options needed for leak detection.

regards
JH

________________________________________
From: cp...@googlegroups.com<mailto:cp...@googlegroups.com><mailto:cp...@googlegroups.com<mailto:cp...@googlegroups.com>> <cp...@googlegroups.com<mailto:cp...@googlegroups.com><mailto:cp...@googlegroups.com<mailto:cp...@googlegroups.com>>> on behalf of Cindy Pham <cindyp...@gmail.com<mailto:cindyp...@gmail.com><mailto:cindyp...@gmail.com<mailto:cindyp...@gmail.com>>>
Sent: Tuesday, October 3, 2023 10:16 PM
To: cp...@googlegroups.com<mailto:cp...@googlegroups.com><mailto:cp...@googlegroups.com<mailto:cp...@googlegroups.com>>
Subject: [CP2K:19284] Deallocate memory used by Hamiltonian-related subroutines

Hi CP2K forum,

I am running a lengthy SCF calculation (over 10k iterations) and noticed a gradual increase in the allocated memory (I used TRACE keyword to print current allocated memory). It appears that the step-by-step increase in memory allocation happens when the Kohn-Sham Hamiltonian is re-calculated, specifically within the qs_ks_did_change function (in qs_ks_types.F).

The SCF routine is my own code that relies on the CP2K built-in Hamiltonian subroutines. It functions properly, but its only problem is the ever increasing memory consumption.

Since I do not need to keep previous Hamiltonians (for any kind of DIIS extrapolation), is there any way to deallocate all memory used by the Hamiltonian-related subroutines (at least once in a while, say, after 1000 SCF iterations)?

Alternatively, are there any input keywords that can ensure that the Hamiltonian structures are reset once in a while?

Thank you in advance for your time and your suggestions.

Best regards,
Cindy Pham.

--
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com><mailto:cp2k%2Bunsu...@googlegroups.com<mailto:cp2k%252Buns...@googlegroups.com>><mailto:cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com><mailto:cp2k%2Bunsu...@googlegroups.com<mailto:cp2k%252Buns...@googlegroups.com>>>.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com><mailto:cp2k%2Bunsu...@googlegroups.com<mailto:cp2k%252Buns...@googlegroups.com>>.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/ZR0P278MB07591FF04A66D61DBAA83F3A9FCBA%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM.

--
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com><mailto:cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com>>.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k%2Bunsu...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/ZR0P278MB075927A9473369B2C0BD1F419FA7A%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM.

--
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com<mailto:cp2k+uns...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/CAN4Jpm0bV%2BsPKUX6-hwEFhk4pfyTrVAXka27tBGvO4zcYq8%2Bow%40mail.gmail.com<https://groups.google.com/d/msgid/cp2k/CAN4Jpm0bV%2BsPKUX6-hwEFhk4pfyTrVAXka27tBGvO4zcYq8%2Bow%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages