oq run stuck on classical 44% but not resource is being used

[Nima Dolatabadi]

i am runing a hazard calculation and the log is: 

[2024-06-01 00:31:48 #7 INFO] classical  11% [9 submitted, 0 queued]
[2024-06-01 00:35:49 #7 INFO] classical  22% [9 submitted, 0 queued]
[2024-06-01 00:40:33 #7 INFO] classical  33% [9 submitted, 0 queued]
[2024-06-01 00:44:48 #7 INFO] classical  44% [9 submitted, 0 queued]

but no cpu or ram usage is being shown. using top command on linux

what is the problem here?

i have attached the image of both issus.

Regards

Nima

[Peter Pažák]

Hi,

for us this quite often happens - but on Windows. Most of the time it means some of the subjobs run out of memory

but the main process did not get information about it - still waiting to get result from those subjobs which died.

You should see how many concurrent tasks the engine used in the listing above and if you did not set it, try to increase (maybe double?)

the number of concurrent tasks in the ini file for example:

concurrent_tasks = 500

That may help avoid this hanging behaviour.

https://docs.openquake.org/oq-engine/3.16/advanced/general.html#concurrent-tasks-parameter

Please also always note the OQ version you are using in your post

so that it is clear you are not using a version that had some bugs which were eventually later corrected.

Peter

so 1. 6. 2024 o 21:01 Nima Dolatabadi <610590...@gmail.com> napísal(a):

--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/d4a7c8c2-9eaf-4db8-90c1-f629491f3c92n%40googlegroups.com.

[Nima Dolatabadi]

i am using openquake.engine==3.19

is there any possibilty to break down the proccess or using gpu? I have defined a very dense grid and i need it for the analysis. I cannot lower the resoloution.

regards

Nima

[Peter Pažák]

Exactly this parameter concurrent_tasks breaks the task into smaller pieces...

Peter

Dátum: nedeľa 2. júna 2024, čas: 11:35:01 UTC+2, odosielateľ: Nima Dolatabadi

[Yves Robert]

I have the same problem when I am running a disaggregation calculation for western Canada. I have attached a copy of the calculation log and of the .ini file. I tried to use concurrent_tasks but only the classical calculation was impacted. I tried to add memory (I now have 64 Gb) but the number of tasks was reduced and the calculation stopped at the same place. I tried to use a smaller number of cores but it didn't work. I am using OpenQuake 3.19 but I tried OpenQuake 3.21 and I got the same result. I would like to point out that doing a disaggregation for Mag_Dist works. I only have that problem for Mag_Dist_Eps.

[Michele Simionato]

The log has a revealing line

Using over 93% of the memory in YVES01!
so it is a memory issue. Try to reduce the parameters affecting the disaggregation to reduce the calculation.

      Michele

[Sujan Raj Adhikari]

mag_bin_width = 0.1
distance_bin_width = 20.0
coordinate_bin_width = 90.

oh this is way too much for canada...  memory issiue

try

mag_bin_width = 1.0
distance_bin_width = 100.0
coordinate_bin_width = 3.5

if run perfectly then lower the value one by one

[Tyler Southam]

You're going to be better off running 1-2 IM instead of the whole suite if you don't have a computer with a lot of memory available. Typically I've been running 3 or maybe 4 IM but having 11 is likely too much for your computer to handle with the deaggregation. If you copy the file and run the different IM in different runs ie, run one for PGA, 0.05, 0.1, another file for 0.2,0.3,0.5, and another for 2,5,10 you'll likely have more luck. Depending on the computer you may need to go down to one IM.