Discussion Paper on Sampling Uncertainty for Editorial Boards

[Will Hopkins]

I have just published an article with this title at Sportscience.  I wrote it for consideration by an editorial board of which I am a member, that of the International Journal of Sports Physiology and Performance, but the article should be useful for any editorial board. Here is the abstract:

The compatibility (or confidence) interval is probably the best measure of uncertainty in the value of an effect derived from a sample. The interval, or the sampling distribution from which it is derived, is the basis of the following methods for assessing acceptable sampling uncertainty: informal assessment of the interval as precision of estimation; Bayesian assessment of the interval or sampling distribution with minimally or other informative priors; the nil-hypothesis significance test; and tests of substantial and non-substantial hypotheses. Editorial boards should decide which of these methods they would prefer to see in submitted manuscripts. For theoretical and practical reasons, I recommend Bayesian assessment and tests of substantial and non-substantial hypotheses; I also recommend magnitude-based decisions, which is consistent with both these methods.

Will