Error when use a very big source model logic tree

[Yongfei Wang]

Hi,

When I implement GNS NSHM OQ source model, the source model (although I already split into three trunks based on source types) is large enough (324 branches). Then I encountered an error (It stopped at the source model reading section). Output is as follows. Attached is the source model logic tree. Any hints for this error? Thanks a lot.

Yongfei

fiona/GDAL is not installed properly! <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/gsim/base.py:296: ExperimentalWarning: KuehnEtAl2020SInter is experimental and may change in future versions - the user is liable for their application
  warnings.warn(msg, ExperimentalWarning)
/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/gsim/base.py:296: ExperimentalWarning: KuehnEtAl2020SSlab is experimental and may change in future versions - the user is liable for their application
  warnings.warn(msg, ExperimentalWarning)
[2023-09-12 01:28:27 #27 INFO] yongf...@QuakeSim.local running /Users/yongfeiwang/Projects/New_Zealand_model/NSHM2022/Extract_GNS_NSHM_model_20230911/job_eventbased.ini [--hc=None]
[2023-09-12 01:28:27 #27 INFO] Using engine version 3.18.0-git79d1c37237
[2023-09-12 01:28:28 #27 WARNING] Using 16 cores on QuakeSim.local
[2023-09-12 01:28:28 #27 INFO] Running EventBasedCalculator with concurrent_tasks = 32
[2023-09-12 01:28:28 #27 INFO] Checksum of the inputs: 3097495649 (total size 476.68 MB)
[2023-09-12 01:28:28 #27 WARNING] Entering SLOW MODE. You have 6.7 GB available, but the engine would like at least 0.5 GB per core, i.e. 8.0 GB: https://github.com/gem/oq-engine/blob/master/doc/faq.md
[2023-09-12 01:28:28 #27 INFO] Inferring the hazard grid
/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/gsim/base.py:296: ExperimentalWarning: KuehnEtAl2020SInter is experimental and may change in future versions - the user is liable for their application
  warnings.warn(msg, ExperimentalWarning)
/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/gsim/base.py:296: ExperimentalWarning: KuehnEtAl2020SSlab is experimental and may change in future versions - the user is liable for their application
  warnings.warn(msg, ExperimentalWarning)
[2023-09-12 01:28:28 #27 INFO] Read N=47 hazard sites and L=80 hazard levels
[2023-09-12 01:28:28 #27 INFO] Reading /Users/yongfeiwang/Projects/New_Zealand_model/NSHM2022/Extract_GNS_NSHM_model_20230911/Active_Shallow_Crust_source_v9.0.1_lt.xml
Traceback (most recent call last):
  File "/Users/yongfeiwang/openquake/bin/oq", line 33, in <module>
    sys.exit(load_entry_point('openquake.engine', 'console_scripts', 'oq')())
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/commands/__main__.py", line 48, in oq
    sap.run(commands, prog='oq')
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/baselib/sap.py", line 212, in run
    return _run(parser(funcdict, **parserkw), argv)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/baselib/sap.py", line 203, in _run
    return func(**dic)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/commands/engine.py", line 175, in main
    run_jobs(jobs)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/engine/engine.py", line 413, in run_jobs
    run_calc(jobctx)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/engine/engine.py", line 285, in run_calc
    calc.run(shutdown=True)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/calculators/base.py", line 253, in run
    raise exc from None
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/calculators/base.py", line 241, in run
    self.pre_execute()
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/calculators/base.py", line 640, in pre_execute
    self.read_inputs()
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/calculators/base.py", line 506, in read_inputs
    self.csm = csm = readinput.get_composite_source_model(
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/commonlib/readinput.py", line 868, in get_composite_source_model
    full_lt = get_full_lt(oqparam, branchID)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/commonlib/readinput.py", line 735, in get_full_lt
    source_model_lt = get_source_model_lt(oqparam, branchID)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/commonlib/readinput.py", line 714, in get_source_model_lt
    smlt = get_smlt(vars(oqparam), branchID)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/__init__.py", line 165, in get_smlt
    smlt = logictree.SourceModelLogicTree(*args)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/logictree.py", line 400, in __init__
    self.parse_tree(tree)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/logictree.py", line 456, in parse_tree
    self.parse_branchset(bsnode, bsno)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/logictree.py", line 496, in parse_branchset
    self.parse_branches(branchset_node, branchset)
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/logictree.py", line 598, in parse_branches
    self.shortener[branch_id] = keyno(
  File "/Users/yongfeiwang/software/oq-engine-3.17/openquake/hazardlib/gsim_lt.py", line 147, in keyno
    return chars[brno] + str(bsno)
IndexError: string index out of range

[Michele Simionato]

Yes, you cannot have more than 185 branches per branchset. You can probably rewrite your logic tree to have more branchsets, for instance two branchsets producing 81 x 4 = 324 total branches.

You may want to read this section of the advanced manual: https://docs.openquake.org/oq-engine/advanced/latest/logic_trees.html
HTH,

               Michele Simionato

[Yongfei Wang]

Hi Michele,

Thanks a lot for your quick response. That is the problem. I have another question. If we divide my original logic tree (324 branches) into multiple branch sets. But given requirement, the weights should sum up to 1 with each branch set. Thus 4 divisions of original branches (varying weights for each branch) will modulate the relative weights when sampling the ruptures. I wonder if there is any other way to resolve the large tree problem while maintaining the relative weights of those individual tress. 

Thanks a lot.

Yongfei

-- 
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/d075fc65-1eb0-415a-b282-a5fe10e4562fn%40googlegroups.com.

[Yuri Wang]

Hi Michele,

I also have the same problem. There are too many branches (larger than 184) in the source model logic tree. When dividing the logic tree into several branches, can we apply a weight to each branchSet, so that the sum of all weights is still 1? In Fig. 2.10 (Generic Logic Tree structure as described in terms of Branch sets, and individual branches) of the OpenQuake Manual, it seems that adding weights to branchSets is achievable. Could you please give me some examples?

There is one more question when dividing the original logic tree. The uncertaintyType for all branchSets should be "sourceModel", but when I try the divided logic tree, there is an error saying "uncertainty of type "sourceModel" can be defined on first branchset only". Which uncertaintyType should I choose for other branchSets?

Thanks and regards

Yuri

[Hasib Abdulrazaq]

hi 

i have the same problem. I have more than 183 branches. the newer versions of openquake do not allow more branches. if i divide logic tree so that i have multiple branch sets then it says "sourceModel" can be defined on first branchset only. I think multiple branch sets only works if you have different uncertaintyType for each branchset. Is there a solution?  

[Marco Pagani]

Apologies, but your problem is not clear.

To view this discussion visit https://groups.google.com/d/msgid/openquake-users/7094f628-a9fe-45a8-adaf-7a2c2c931033n%40googlegroups.com.