I shared this with a couple of folks earlier today, but I wanted to spread it more broadly:
Something that I've been thinking more and more is that for any
results presented at a conference or in a paper, there should be a
corresponding input file and test(s) in the mainline Zapdos repository. I
would want the following:
- One syntax check
test with input file corresponding to the exact input file used to
generate the conference results. A syntax check test just has
`check_input = true` in the `tests` spec file, and does not actually run
the simulation; it just checks to make sure that all your syntax is
valid. This prevents developers from making parameter changes that would
break those inputs
- One Exodiff-type test that has a coarser
mesh and/or reduced number of timesteps. Ideally this mesh would just
be a uniformly coarsened version of the original mesh. If that Exodiff
test exists, then we can be reasonably sure that any changes to the
Zapdos code base that would break the smaller test would also break the
larger input that produced conference-worthy results, and we could
investigate further.
I really want to both:
- Bump up our test coverage
-
Make it very easy and straightforward for a new Zapdos user to
reproduce our results and hence have a strong starting point for their
own research.
Alex