Error in OQ run: Potential number of logic tree

[shivam huriya]

Dear all,

I'm trying to run OQ model and it fails 

What I obtain is this error message:

2020-12-07T15:57:15.27,INFO,MainProcess/8556,Potential number of logic tree paths = 49_918_749_901_659_832_320

2020-12-07T17:10:05.18,CRITICAL,MainProcess/8556,

2020-12-07T17:10:05.34,CRITICAL,MainProcess/8556,Traceback (most recent call last): File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\engine\engine.py", line 346, in run_calc calc.run(exports=exports, **kw) File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\calculators\base.py", line 251, in run self.pre_execute() File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\calculators\base.py", line 594, in pre_execute self.read_inputs() File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\calculators\base.py", line 485, in read_inputs oq, self.datastore.hdf5) File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\commonlib\readinput.py", line 744, in get_composite_source_model full_lt = get_full_lt(oqparam) File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\commonlib\readinput.py", line 706, in get_full_lt full_lt = logictree.FullLogicTree(source_model_lt, gsim_lt) File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\commonlib\logictree.py", line 1187, in __init__ self.init() # set .sm_rlzs and .trt_by_grp File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\commonlib\logictree.py", line 1192, in init self.sm_rlzs = get_effective_rlzs(self.source_model_lt) File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\commonlib\logictree.py", line 144, in get_effective_rlzs for group in groupby(rlzs, operator.attrgetter('pid')).values(): File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\baselib\general.py", line 874, in groupby kgroups = itertools.groupby(sorted(objects, key=key), key) File "c:\program files\openquake engine\python3.6\lib\site-packages\openquake\commonlib\logictree.py", line 436, in __iter__ tuple(smlt_branch_ids), 1) MemoryError

What can be the reason for this error and what changes I can make to run? 

Also,Potential number of logic tree paths = 49_918_749_901_659_832_320,  what does it represents? Anyone has the same problem? Any suggestion?

Thanks

 shivam

[Sujan Raj Adhikari]

can you add the job file here

[Homayoun Tashak]

put some samples for logic tree , for example 200 , following section without ""

"[logic_tree]

number_of_logic_tree_samples = 200"

it might help

[Sujan Raj Adhikari]

or number_of_logic_tree_samples = 0

[shivam huriya]

Thanks homayoung and Sujan for your kind response, can anyone please explain  in ",Potential number of logic tree paths = 49_918_749_901_659_832_320 "  what this parameter and value define?

And as you mentioned I am attaching my job file here:-

    [general]

 

description = OQ MODEL
calculation_mode = classical
random_seed = 1022

 

[geometry]

 

sites_csv = sites.csv

 

[logic_tree]

 

number_of_logic_tree_samples = 0

 

[erf]

 

rupture_mesh_spacing = 5.0
width_of_mfd_bin = 0.1
area_source_discretization = 10.0

 

[site_params]

 

reference_vs30_type = inferred
reference_vs30_value = 760.0
reference_depth_to_2pt5km_per_sec = 0
reference_depth_to_1pt0km_per_sec = 0

 

[calculation]

 

source_model_logic_tree_file = LT1.xml
gsim_logic_tree_file = LT2.xml
investigation_time = 50.0
intensity_measure_types_and_levels = {​​​​"PGA": [0.0025, 0.0045, 0.0075, 0.0113, 0.0169, 0.0253, 0.038, 0.057, 0.0854, 0.128, 0.192, 0.288, 0.432, 0.649, 0.973, 1.46, 2.19, 3.28, 4.92, 7.38], "SA(0.3)": [0.0025, 0.0045, 0.0075, 0.0113, 0.0169, 0.0253, 0.038, 0.057, 0.0854, 0.128, 0.192, 0.288, 0.432, 0.649, 0.973, 1.46, 2.19, 3.28, 4.92, 7.38], "SA(1.0)": [0.0025, 0.0045, 0.0075, 0.0113, 0.0169, 0.0253, 0.038, 0.057, 0.0854, 0.128, 0.192, 0.288, 0.432, 0.649, 0.973, 1.46, 2.19, 3.28, 4.92, 7.38], "SA(3.0)": [0.0025, 0.0045, 0.0075, 0.0113, 0.0169, 0.0253, 0.038, 0.057, 0.0854, 0.128, 0.192, 0.288, 0.432, 0.649, 0.973, 1.46, 2.19, 3.28, 4.92, 7.38]}​​​​
truncation_level = 3
maximum_distance = 400.00

 

[event_based_params]

 

ses_per_logic_tree_path = 1
ground_motion_correlation_model = 
ground_motion_correlation_params = 

 

[output]

 

export_dir = /tmp
save_ruptures = true
ground_motion_fields = true
hazard_curves_from_gmfs = true
mean_hazard_curves = true
quantile_hazard_curves =
hazard_maps = true
poes = 0.1 0.05 0.02                                 

--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/3d7a7b76-6f29-4c98-8f4e-f0aab9da3910n%40googlegroups.com.

[Graeme Weatherill]

Dear Shivam,

In a way OpenQuake is telling you what the problem is. From the inputs it is reading in the logic tree files (source model and ground motion model) the resulting logic tree will contain 49,918,479,901,659,832,320 end branches! When trying to build this logic tree, OpenQuake is running out of memory and that is why you are getting the MemoryError and the job is failing.

An earlier reply to this question by another user was  partly incorrect, as setting "number_of_logic_tree_samples=0" means that you are telling OpenQuake to run all of the branches (i.e. you are not sampling). You could try setting the number_of_logic_tree_samples to, say, 200 and set it running. However, if that is still producing the same problem then what is happening is that OpenQuake is effectively trying to build the whole logic tree before sampling the branches - so you will still run out of memory with a logic tree that size regardless of whether or not you sample. [OpenQuake developers can correct me if that is no longer the case or if there are new sampling setting that can avoid this problem].

The problem then is in your logic tree itself and how to construct it. Without know what you are trying to do it is difficult to help you, but problems of this nature can emerge if you are trying to apply large numbers of uncertainties to individual sources within large-scale regional models. Such configurations can be set and even validated by OpenQuake, but the logic tree can easily blow up to the sizes such as those you are seeing here and won't run.

I'm not sure if that helps you to get your calculation working, but it does explain the error.

Thanks,

Graeme

To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/CAM4hf7GbRP%2B4q2n9LH4JrvJJ0-AVpdYAzn0sFAXgzD1wy-4XKA%40mail.gmail.com.

[shivam huriya]

Thanks Graeme, thank you so much for your explanation. Just curious to know how much no. of logic tree sample should be sufficient for a model? is there any thumb rule or way to calculate that?

To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/7765f58a-35e9-31e2-09c1-488578eb6cf7%40openquake.org.

[Graeme Weatherill]

Dear Shivam,

I'm not sure there is a fixed rule, and it depends on the application and on the computational resources that you have available, as well as the structure of the logic tree in general. For non-critical facilities where you are mostly interested in the mean of the logic tree, then you may get a stable estimate with only a few hundred samples. To constrain the mean and 16th to 84th percentiles to an acceptable level of accuracy you would really need at least a couple of thousand samples, and to get to the more extreme percentiles (say 5 - 95 %) then I would recommend at least 10,000 - 20,000 or more. For a site-specific PSHA this may be achievable on an ordinary computer within a reasonable length of time. If you are running PSHA for a large region (e.g. many thousands of sites), or if you are running event-based calculations for risk, then even a few thousand samples would be challenging and you would need a relatively good server to perform the calculation. If you are willing to sacrifice accuracy for speed then you could probably get away with halving the numbers I'm quoting above and still get a reasonable estimate of the percentiles, but I would not recommend this if you are working with critical facilities.

As I said, in your particular case sampling may not solve your problem because OpenQuake is usually building the logic tree before sampling, and here it would need to index nearly 50 quintillion end branches before it will start taking samples [Again, OQ devs please correct me if I'm misrepresenting how the current engine is working or if there is some new setting that can avoid this]. Even with all of the current optimisations in place that's not really feasible. It's worth trying to run the calculation with sampling, but don't be surprised if you get the same error.

Graeme

To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/CAM4hf7FULSTyoiZJb0qaEDbrBjka%3DVijYrv25nq36FfyesG--g%40mail.gmail.com.