MBI Rehabilitation: Lessons from Engaging the Statistical Establishment

[Will Hopkins]

Over the past few months, I've been engaged in an email exchange with a group of prominent statisticians including Sander Greenland, Andrew Vickers, Kristin Sainani, and others, attempting to get MBI rehabilitated by addressing their technical criticisms directly. The exchange has been instructive, though not in the way I'd hoped.

Despite presenting evidence that (1) MBI's mathematics are sound, (2) the high Type-I error claim was based on misinterpretation, and (3) MBI outperforms NHST on magnitude-based error rates, the response was largely ad hominem attacks rather than engagement with the evidence. Vickers called me a narcissistic conspiracy theorist. Greenland, who was the only one to engage substantively and who explained how to position MBI properly within Bayesian inference frameworks, ultimately suggested I was experiencing "AI-induced psychosis" for seeking correction of published false claims. The rest remained silent.

The one positive outcome: Greenland confirmed that "the problem is not the math of MBD" but rather its presentation outside established statistical frameworks. He outlined how to position MBI as Bayesian inference for interval hypotheses, which I will pursue in a future article.

Below is the final message I sent to the group, to which I have received no reply. I think it's important for our community to understand what we're dealing with when the statistical establishment closes ranks against methods they don't like, regardless of the evidence.

Will

From: Will Hopkins <wi...@sportsci.org>
Sent: Monday, December 15, 2025 4:21 PM
Subject: Moving forward with MBI positioning

Dear colleagues,

I want to thank Sander for taking the time to outline how best to position MBI within established statistical frameworks. His guidance has been invaluable in clarifying that MBI maps to Bayesian inference for interval hypotheses (computing posterior probabilities for regions of practical importance) and that utility MBD maps to Bayesian decision theory (integrating posteriors with utility functions).

I will write and submit a manuscript making this positioning explicit, connecting MBI to the established literature on Bayesian inference for interval hypotheses and to Marschner's confidence distribution framework. This should resolve any remaining questions about statistical legitimacy.

However, I must address what this resolution reveals about the original criticisms. As the positioning paper will demonstrate, the mathematics of MBI were never wrong - they simply weren't connected to established statistical lineages in the way the statistics community expected. This raises an important question about how the criticisms were framed:

Sainani et al. (2019) proclaimed that "Magnitude-based inference is not Bayesian and is not a valid method of inference." Andrew characterized MBI as "a math trick that bears no relation to the real world." These weren't constructive critiques suggesting better positioning - they were wholesale dismissals that led journals to ban the method and damaged the careers of researchers using it.

If the actual problem was inadequate positioning within established frameworks, why wasn't that the critique? Applied researchers developing practical tools cannot be expected to use the precise language of theoretical statistics, but they can expect charitable interpretation and constructive feedback from statistical experts. That didn't happen here.

The claim about MBI's high Type-I error rate (Lohse et al., 2020) was based on a fundamental misinterpretation - treating "possibly beneficial" as "definitely beneficial," an interpretation users of MBI did not make. Our simulation study (Hopkins & Batterham, 2016) showed that with correct interpretation, MBI outperformed NHST on the error rates that actually matter: incorrect decisions about magnitudes, not NHST's point-null Type-I error rate. These results have been ignored in this discussion.

I developed MBI from existing publications in good faith, building on work by previous authors who interpreted confidence intervals in a Bayesian manner. The subsequent evolution - incorporating Sander's suggestions, developing decision frameworks, adding utility-based analysis - has been scientifically responsible. The treatment this work received was not.

I'm grateful to Sander for the technical guidance that will allow proper positioning of MBI. I will move forward with writing that paper. But I want it on the record that the original criticisms went far beyond what was justified, and the harm they caused was real and unnecessary.

I wish you all well in your work.

Will Hopkins