CC calculations hangs up

31 views
Skip to first unread message

jstysz

unread,
Dec 1, 2021, 6:01:37 AM12/1/21
to dirac-users

Dear Dirac Users,

my CC calculations crashed due to a not sufficient memory.

After increasing memory per node, it progresses farther

but now hangs for hours in the last step of CC, i.e. when perturbative treatment of triple excitations starts (corresponding output attached).

What can I do about it? Should I increase a memory farther or the 64-bit version of Dirac

is necessary?

Best regards, Jacek

OgTs4_cc_OgTs4.out

Ilias Miroslav, doc. RNDr., PhD.

unread,
Dec 1, 2021, 8:00:49 AM12/1/21
to dirac-users
Hello Jacek,

looking into your output I see

Type VVVV :      2 643 582 485 integrals, what nears to max. 32-bit integer value,  232 − 1 = 4,294,967,295 ( https://en.wikipedia.org/wiki/Integer_overflow#Origin )

I assume that perturbative triples do exceed 32bit range, because of their scaling of  O(N^7).


And concerning the memory, please check the count, http://diracprogram.org/doc/master/tutorials/cc_memory_count/count_cc_memory.html .

Miro






From: dirac...@googlegroups.com <dirac...@googlegroups.com> on behalf of jstysz <jst...@gmail.com>
Sent: Wednesday, December 1, 2021 12:01
To: dirac-users <dirac...@googlegroups.com>
Subject: [dirac-users] CC calculations hangs up
 
--
You received this message because you are subscribed to the Google Groups "dirac-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dirac-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dirac-users/4a80e5cd-e002-4c16-9248-d24f8f200739n%40googlegroups.com.

jstysz

unread,
Dec 1, 2021, 2:13:18 PM12/1/21
to dirac-users
Hi Miro,
thanks a lot. I will try to install 64-bit version of Dirac21.
Regards, Jacek

Juan Jose Aucar

unread,
Dec 1, 2021, 2:26:04 PM12/1/21
to dirac...@googlegroups.com
Dear Jacek,


You can try a compilation with OpenMP or ThreadedMKL. In the manual you can also find something about a mixed MPI and threaded MKL compilation. That way you can save some RAM (and time). You will notice its advantages on the moltra module.


Juan José

Johann Pototschnig

unread,
Dec 2, 2021, 3:17:59 AM12/2/21
to dirac-users
How do you know it hangs? Are the processors empty?
For large systems the triples take more time than the index transformation or CC.
I see in your output that the sorting of the integrals takes about 45 min, so I think the triples should take more than 270 min.

There are improvements you should first consider:

1) As already mentioned in earlier answers you should link your code to BLAS and LAPACK libraries, otherwise it uses a rather inefficient internal implementation, which is not recommended for any production runs.

2) Also you can play around with the distribution of MPI parallelization to OPENMP threads.
OPENMP threads uses shared memory, but has difficulties for parallelization over different nodes.

3) You could switch on printing (increase print level) to get an idea of the progress of triples. 

These things you can test for smaller systems to get an idea how much time you can expect the triples to take.

best,
Johann

Johann Pototschnig

unread,
Dec 2, 2021, 3:35:33 AM12/2/21
to dirac-users
Regarding the Crash and memory problem:
You are using 64 MPI threads, which means means that you have 64 copies of the active arrays.
This can use up a lot of memory, so more OpenMP threads and less MPI threads might help.

best, Johann

jstysz

unread,
Dec 2, 2021, 7:19:43 AM12/2/21
to dirac-users

Hello Johan and Juan José,

my job did not write over the night (more than 6 hrs) to any file in a scratch directory, so I assumed that it hangs.

Thank you very much for your comments and suggestions.

I will install Dirac21 with 64-bit integers and see if it helps.

Best regards, Jacek

Reply all
Reply to author
Forward
0 new messages