info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Q multi-comparison and deLong test
14 views
Skip to first unread message
Cosine
Aug 8, 2023, 10:56:06 AM8/8/23
to
Hi:
When doing multi-comparison we need to do the correction for the potential increase of the type I error, e.g., the Bonferroni correction. Could we avoid doing this correction by using the deLong test since this method is somehow conservative for comparing multiple pairs of the area under the curve?
When doing multi-comparison we need to do the correction for the potential increase of the type I error, e.g., the Bonferroni correction. Could we avoid doing this correction by using the deLong test since this method is somehow conservative for comparing multiple pairs of the area under the curve?
Rich Ulrich
Aug 8, 2023, 7:51:34 PM8/8/23
to
On Tue, 8 Aug 2023 07:56:02 -0700 (PDT), Cosine <ase...@gmail.com>
wrote:
The deLong test is a test for ROC curves, which is NOT a
circumstance where Bonferroni correction would be
appropriate.
Looking for the deLong test, I found this article by Frank
Harrell. On a quick read, I take it as a bit of a tough read. It
recommends 'Other' -- The link has the full article.
(Harrell is reliable. Years ago, I promoted his comments
on the hazards of using stepwise regression, and they have
been much cited since then.)
https://www.fharrell.com/post/addvalue/
After two paragraphs of introduction, it says,
Statisticians have no better sense of history than other
scientists. In the quest for publishing new ideas, measures of added
value are constantly being invented by statisticians, without asking
whether older methods already solve the problem at hand. Some of the
examples of measures that are commonly used but are not needed in
this setting are the -index (F. E. Harrell et al. (1982); area under
the ROC curve if the outcome is binary), and IDI and NRI. They are
not needed because measures based on standard regression methods are
not only adequate to the task, but are more powerful and more
flexible and insightful,
--
Rich Ulrich
wrote:
circumstance where Bonferroni correction would be
appropriate.
Looking for the deLong test, I found this article by Frank
Harrell. On a quick read, I take it as a bit of a tough read. It
recommends 'Other' -- The link has the full article.
(Harrell is reliable. Years ago, I promoted his comments
on the hazards of using stepwise regression, and they have
been much cited since then.)
https://www.fharrell.com/post/addvalue/
After two paragraphs of introduction, it says,
Statisticians have no better sense of history than other
scientists. In the quest for publishing new ideas, measures of added
value are constantly being invented by statisticians, without asking
whether older methods already solve the problem at hand. Some of the
examples of measures that are commonly used but are not needed in
this setting are the -index (F. E. Harrell et al. (1982); area under
the ROC curve if the outcome is binary), and IDI and NRI. They are
not needed because measures based on standard regression methods are
not only adequate to the task, but are more powerful and more
flexible and insightful,
--
Rich Ulrich
0 new messages