For those who might be interested. Note that while this is a US action it
is potentially a precedent for other Governments, so it deserves some
international attention.
In particular, at the end I begin to discuss the actual open science
procedures needed for implementation. Below is that section, which
assumes some familiarity with the EPA proposal. This will not be my last
submission so I am wide open to ideas.
The proposal and other stuff are here:
https://www.regulations.gov/docket?D=EPA-HQ-OA-2018-0259
David
http://insidepublicaccess.com/
EPA usability and use procedures
Not using a given research result looks to be procedurally easy. EPA
simply does not cite the research in its justification documents or
statements. They might even create a new support document that
specifically identifies the research used to justify the
regulation.
Determining which research is not usable is the difficult part. This
looks very much like what is called a "certification"
procedure. That is, EPA will need to certify that each piece of research
it wants to use is properly available for attempted replication. Note
that neither attempted or successful replication is required under this
procedural transparency rule. All that is required for EPA use is that
the research be properly available for possible attempted
replication.
The best way, perhaps the only way, to do this is for EPA to contact the
researchers and determine that the research is properly available.
Another possibility is to establish a practice of self certification
among those researchers who desire that their research be usable by EPA.
Or certification of EPA usability might be done by third parties, such as
scholarly societies. Such certification might even become a condition of
funding with some funders.
In any case, what EPA needs to do is specify the requirements of proper
availability as needed for usability. This might be a simple checklist of
questions, or perhaps something more complex. There is already
considerable discussion and action within the open science community, as
to just what availability for the purpose of replication requires. EPA
should draw on this existing work and expertise.
The basic requirements are not that mysterious. The data and codes need
to be available and properly documented. The procedures followed and
decisions made along the way need to be properly explained. (There is
already a relevant literature on this topic in some fields, especially
biomedical.) Research that cannot be certified is then excluded.