Help on testing a new developped GMPE

[francesco....@ingv.it]

Dear developpers,
I am testing a GMPE we coded.
The test results with nosetests were not as expected,
and I'd like to get the values calculated by OQ for each single test,
so that I can better understand the issue.

First question:
Is it possible?


In any case, as second question,
I wanted to make a test for pga only,
so that I wrote the test case XXX_MeanPgaOnly_test.py:
======================================================================
from openquake.hazardlib.gsim.XXX import XXX
from openquake.hazardlib.tests.gsim.utils import BaseGSIMTestCase

class XXXTestCase(BaseGSIMTestCase):
    GSIM_CLASS = Bindietal2011

    def test_std_solo_pga(self):
        self.check(XXX/XXX_MEAN_SOLOPGA.csv',
                    max_discrep_percentage=0.1)   
======================================================================


I wrote the csv file, whose first lines were:
======================================================================
rup_mag,rup_rake,dist_rjb,site_vs30,result_type,damping,pga
4,0,0,800,MEAN,5,0.0266237706
4.5,0,0,800,MEAN,5,0.0479038204
======================================================================


And I run "nosetests XXX_MeanPgaOnly_test.py" getting the following result:
======================================================================
FAIL: test_std_solo_pga (openquake.hazardlib.tests.gsim.XXX_MeanPgaOnly_test.XXXTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/openquake/GEM/openquake/oq-hazardlib/openquake/hazardlib/tests/gsim/XXX_MeanPgaOnly_test.py", line 29, in test_std_solo_pga
    max_discrep_percentage=0.1)
  File "/home/openquake/GEM/openquake/oq-hazardlib/openquake/hazardlib/tests/gsim/utils.py", line 49, in check
    raise AssertionError(stats)
AssertionError: total of 672 checks done, 0 were successful and 672 failed.
0.1 seconds spent, avg rate is 7250.1 checks per seconds.
success rate = 0.0%
average discrepancy = 10.1252%
maximum discrepancy = 10.6912%
standard deviation = 0.3993%

----------------------------------------------------------------------
Ran 1 test in 0.094s

FAILED (failures=1)
======================================================================


What is wrong here please?

Thank you,
Francesco

[Graeme Weatherill]

Dear Francesco,

The error that you sent shows that the tests are running but the results computer by your implementation are not in agreement with those in the test files.

There are a couple of things that you could do in order to understand the cause of the discrepancy.  One option is to change the "debug" option from False to True in the openquake/hazardlib/tests/gsim/check_gsim.py file (https://github.com/gem/oq-hazardlib/blob/master/openquake/hazardlib/tests/gsim/check_gsim.py#L36). This will give you more information to point to the line in the file where the tests are failing.

Another option I sometimes do is to inset the following statement below line 91 in the check_gsim.py file (https://github.com/gem/oq-hazardlib/blob/master/openquake/hazardlib/tests/gsim/check_gsim.py#L91):

print imt, rctx.mag
print numpy.column_stack([result, expected_result])

This will mean that when the test fails you will see the full vector of expected and calculated results. Just be sure to remove the lines once the tests are passing.

I also note that it looks like the GMPE you are coding is Bindi et al. (2011). One thing to be careful of is the fact that the logarithm of the mean of this GMPE is in base 10, whereas OpenQuake will expect the values to be in terms of the natural logarithm. If you haven't done the conversion already then that could be a possible cause of discrepancy between expected and calculated values.

Please do not hesitate to contact me directly if you need further assistance with the implementation.

Thanks,

Graeme Weatherill

--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.