Misleading Conclusions with Statistical Significance

[Will Hopkins]

I have now published a full article at Sportscience with the title "Misleading Conclusions Based on Statistical Significance and Non-significance at a Recent International Conference". The conference was last year's virtual annual meeting of the European College of Sport Science, and the article represents an analysis of the first 33 presentations in my report on the conference in last year's issue of Sportscience.

The article features four figures that show quite nicely how hypothesis tests of non-substantial and substantial magnitudes work, and how magnitude-based decisions (MBD) correspond to such tests. MBD also has a purely Bayesian basis and interpretation, which I prefer to the hypothesis tests.

The conclusion? "There was an unacceptably high prevalence of misleading assessments of magnitude based on statistical significance and non-significance at this conference. Researchers should account for sampling variation by replacing the nil-hypothesis test with tests of substantial and non-substantial magnitudes, including magnitude-based decisions."

If you can't see the new homepage with a link to the article, Shift-refresh or click on this link: https://sportsci.org/2021.

Will