ASA President’s Task Force Statement on Statistical Significance and Replicability
62 views
Skip to first unread message
Marc Schwartz
Aug 11, 2021, 8:27:08 AM8/11/21
to MedStats
Hi All,
Just thought that I would pass this on:
https://magazine.amstat.org/blog/2021/08/01/task-force-statement-p-value/
Regards,
Marc Schwartz
Just thought that I would pass this on:
https://magazine.amstat.org/blog/2021/08/01/task-force-statement-p-value/
Regards,
Marc Schwartz
Paul Thompson
Aug 11, 2021, 9:25:02 AM8/11/21
to meds...@googlegroups.com
There is a lot of sensible stuff in that article. I point out specifically this:
"Thresholds are helpful when actions are required. Comparing p-values to a significance level can be useful, though p-values themselves provide valuable information. P-values and statistical significance should be understood as assessments of observations or effects relative to sampling variation, and not necessarily as measures of practical significance. If thresholds are deemed necessary as part of decision-making, they should be explicitly defined based on study goals, considering the consequences of incorrect decisions. Conventions vary by discipline and purpose of analyses."
This is the key point which is not understood by the opponents of "statistically significant". To make a decision, a threshold is required. A threshold is not magic or certain - there is uncertainty involved with the conclusion that "statistical significant" results have been found. But we need the threshold.
"Thresholds are helpful when actions are required. Comparing p-values to a significance level can be useful, though p-values themselves provide valuable information. P-values and statistical significance should be understood as assessments of observations or effects relative to sampling variation, and not necessarily as measures of practical significance. If thresholds are deemed necessary as part of decision-making, they should be explicitly defined based on study goals, considering the consequences of incorrect decisions. Conventions vary by discipline and purpose of analyses."
This is the key point which is not understood by the opponents of "statistically significant". To make a decision, a threshold is required. A threshold is not magic or certain - there is uncertainty involved with the conclusion that "statistical significant" results have been found. But we need the threshold.
Paul A. Thompson, PhD, PSTAT(R)
--
--
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules
---
You received this message because you are subscribed to the Google Groups "MedStats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to medstats+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/medstats/8b440ff1-feeb-d913-f9fa-dbeda8c3faa1%40me.com.
Abhaya Indrayan
Aug 11, 2021, 11:34:53 AM8/11/21
to meds...@googlegroups.com
Nice to see that the matter pertaining to P-values and statistical significance has come up again. We need to balance the statistical significance with medical significance. Please see
DOI: 10.4103/ijmr.IJMR_980_19
--Abhaya
Dr Abhaya Indrayan
Personal website: http://indrayan.weebly.com
To view this discussion on the web, visit https://groups.google.com/d/msgid/medstats/CANLSjUu3UiXH4XceUYm6kAt-gSV1Xy52VaPLU%3DcrfCa%2B5xFWfA%40mail.gmail.com.
Stan Alekman
Aug 11, 2021, 1:11:45 PM8/11/21
to meds...@googlegroups.com
Hi,
Marc, thanks for the link.
I completely agree with the content but I wish that it included the statement that the p-value is a random variable hence replication of results is needed. It states this in an indirect way but it could be clearer.
Also, confidence intervals of effect sizes and of the p-value itself are informative.
Regards,
Stanley Alekman
--
Marc Schwartz
Aug 11, 2021, 1:51:40 PM8/11/21
to 'Stan Alekman' via MedStats
Hi
Stanley,
That all may depend upon the intended audience, and given the background rationale for this new article, and the attention of the original ASA publications in 2016 and 2019 by Wasserstein et al, I am not clear that it is purely for consumption by statisticians. Hence some of the more technical aspects were not discussed and the scope limited. Albeit, even within that specific audience there is a wide spectrum of opinions.
The notion of p values as random variables is not new. It was covered, to my quick recollection, by Duncan Murdoch back in 2008 in TAS:
Duncan J Murdoch, Yu-Ling Tsai & James Adcock
P-Values are Random Variables,
The American Statistician, 62:3, 242-245
https://www.tandfonline.com/doi/abs/10.1198/000313008X332421
I know Duncan, as he is a former member of the R Core group, and have interacted with him in that role over the years.
I think that the rather narrow scope of this new article was to largely refute the perception that has evolved in some circles, that the ASA's formal opinion/position, given the prior work, is the need to rapidly move to a post p value world, and completely reject their use, which is not the case. So, I do believe that there is important value in this new article.
A number of the related issues that you raise on effect sizes and such, as alternatives/supplements to p values, were covered in the prior work, and I am not sure that they needed to be repeated in this new article, but opinions can differ.
Regards,
Marc
'Stan Alekman' via MedStats wrote on 8/11/21 1:11 PM:
That all may depend upon the intended audience, and given the background rationale for this new article, and the attention of the original ASA publications in 2016 and 2019 by Wasserstein et al, I am not clear that it is purely for consumption by statisticians. Hence some of the more technical aspects were not discussed and the scope limited. Albeit, even within that specific audience there is a wide spectrum of opinions.
The notion of p values as random variables is not new. It was covered, to my quick recollection, by Duncan Murdoch back in 2008 in TAS:
Duncan J Murdoch, Yu-Ling Tsai & James Adcock
P-Values are Random Variables,
The American Statistician, 62:3, 242-245
https://www.tandfonline.com/doi/abs/10.1198/000313008X332421
I know Duncan, as he is a former member of the R Core group, and have interacted with him in that role over the years.
I think that the rather narrow scope of this new article was to largely refute the perception that has evolved in some circles, that the ASA's formal opinion/position, given the prior work, is the need to rapidly move to a post p value world, and completely reject their use, which is not the case. So, I do believe that there is important value in this new article.
A number of the related issues that you raise on effect sizes and such, as alternatives/supplements to p values, were covered in the prior work, and I am not sure that they needed to be repeated in this new article, but opinions can differ.
Regards,
Marc
'Stan Alekman' via MedStats wrote on 8/11/21 1:11 PM:
SteveDrD
Aug 12, 2021, 7:19:57 AM8/12/21
to MedStats
Sounds almost like a case for tolerance intervals and bounds. Or in the mixed model sense, broad inference space estimates of variability.
Steve Denham
Senior Biostatistics Scientist
Charles River Laboratories
Reply all
Reply to author
Forward
0 new messages