Traditional disaggregation

[Yves Robert]

I am trying to get a traditional disaggregation using the following commands:

oq export disagg-rlzs-traditional <calc_id>

oq export disagg-stats-traditional <calc_id>

The first command gives me results but the second command gives me these messages:

C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine>oq export disagg-rlzs-traditional 29
INFO:root:Building 48 realizations
INFO:root:Building 48 realizations
Exported 118.87 MB in ['.\\Mag_Dist-traditional-0_29.csv', '.\\Mag_Dist_Eps-traditional-0_29.csv']
<Monitor export[rymtl], duration=16.93170475959778s, memory=9.84 MB>

C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine>oq export disagg-stats-traditional 29
Traceback (most recent call last):
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\commonlib\datastore.py", line 512, in __getitem__
    val = self.hdf5[key]
          ~~~~~~~~~^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\hdf5.py", line 498, in __getitem__
    h5obj = super().__getitem__(path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\h5py\_hl\group.py", line 357, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5o.pyx", line 189, in h5py.h5o.open
KeyError: 'Unable to synchronously open object (component not found)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Scripts\oq.exe\__main__.py", line 7, in <module>
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\commands\__main__.py", line 48, in oq
    sap.run(commands, prog='oq')
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\sap.py", line 212, in run
    return _run(parser(funcdict, **parserkw), argv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\sap.py", line 203, in _run
    return func(**dic)
           ^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\commands\export.py", line 38, in main
    fnames = export_((datastore_key, fmt), dstore)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\general.py", line 673, in __call__
    return self[key](obj, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\calculators\export\hazard.py", line 579, in export_disagg_csv
    aw = extract(dstore, ex % (k, s, spec))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\calculators\extract.py", line 204, in __call__
    data = self[k](dstore, v)
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\calculators\extract.py", line 1177, in extract_disagg
    fullmatrix = dstore['disagg-%s/%s' % (spec, label)][sid]
                 ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\commonlib\datastore.py", line 522, in __getitem__
    raise KeyError('No %r found in %s' % (key, self))
KeyError: "No 'disagg-stat/Mag_Dist' found in <DataStore C:\\Users\\rymtl\\oqdata\\calc_29.hdf5 open>"

I tried openquake 3.20 and 3.21 with the same results. I have included the job.ini file to help answer my question.

Can anybody tell me how to make it work?

[Michele Simionato]

Unfortunately there is a typo. The exporter is looking for `disagg-stat/Mag_Dist` instead of the correct `disagg-stats/Mag_Dist`. I will fix the missing "s" in master. Thanks for the report,

          Michele Simionato

[Yves Robert]

Thank you for fixing the typo. I used the job.ini file again and I got the following messages when I tried to get the traditional disaggregation.

C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine>oq export disagg-rlzs-traditional 30
Exported 118.87 MB in ['.\\Mag_Dist-traditional-0_30.csv', '.\\Mag_Dist_Eps-traditional-0_30.csv']
<Monitor export[rymtl], duration=16.831286430358887s, memory=15.54 MB>

C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine>oq export disagg-stats-traditional 30

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Scripts\oq.exe\__main__.py", line 7, in <module>
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\commands\__main__.py", line 48, in oq
    sap.run(commands, prog='oq')
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\sap.py", line 212, in run
    return _run(parser(funcdict, **parserkw), argv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\sap.py", line 203, in _run
    return func(**dic)
           ^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\commands\export.py", line 38, in main
    fnames = export_((datastore_key, fmt), dstore)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\general.py", line 673, in __call__
    return self[key](obj, *args, **kw)

           ~~~~^^^^^
  File "C:\Users\rymtl\AppData\Local\Programs\OpenQuake Engine\python3\Lib\site-packages\openquake\baselib\general.py", line 678, in __missing__
    raise KeyError(key)
KeyError: ('disagg-stats-traditional', 'csv')

[Michele Simionato]

On Thursday, June 20, 2024 at 9:01:29 PM UTC+2 yr...@videotron.ca wrote:

Thank you for fixing the typo. I used the job.ini file again and I got the following messages when I tried to get the traditional disaggregation.

The reason is that in https://github.com/gem/oq-engine/pull/9789 I removed the exporter for the statistics since I realized it was buggy.

We have still to decide what to do with it. The exporter for the individual realizations instead is correct and will stay.

      Michele