On Tue, 11 Jan 2022 19:10:23 -0800 (PST), Cosine <
ase...@gmail.com>
wrote:
>Cosine ? 2022?1?12? ?????2:22:04 [UTC+8] ??????
>> Hi:
>>
>> When doing a statistical test, we often compute the p-value and confidence interval (CI) at a given significance level of alpha.
>>
>> Questions arise: would it be better to have a lower p-value? Likewise, would it be better to have a narrower CI? Why and why not?
>>
>
>For p-value, suppose we have two new diagnostic methods, A and B. We want to know:
>1) are they both better than the standard method?
>2) is method A better than B?
I think you want to mull the idea that TESTing is separate
from ESTIMATION. Testing starts by designating a cutoff.
Estimation reports "effect sizes" -- usually, in natural units of the
experiment, rather than by comparing p-values.
Yeah, I know that if tests are entirely commensurate (same Ns,
SDs,), then I know that a p= 0.001 reflects a t-test mean-difference
which is about twice that for p= 0.05. I would only ever mention
the comparisosn if I were already engaged in explaining "effect
sizes" in a more thorough way, such as "Why we should ignore
all the 'tiny' effects where p is not < 0.001."
And this statement of yours is not the way statisticians ever discuss
either -- "For p-value, suppose we have ....".
I assume that you intended to say something like, "For two new
diagnostic methods, we have tests (with p-values) comparing each
to a standard and also to each other."
In a testing environment, or when one is TALKing about testing,
we never would ASSERT that A is better than B unless the test
for A vs B has a p-value meets the cutoff that was designated.
>
> We desing studies and use the accuracy (Acc) to check the performances.
>
> By comparing the methods A and the standard one, we have: Acc_Asmp, p-value_A, CI_A
> B Acc_Bsmp, p-value_B, CI_B
>
> If p-value_A < p-value_B, could we say that Acc_Asmp is more significant than Acc_Bsmp?
>
> Similarly, we define the width of CI as WCI. We have WCI_A and WCI_B.
>
> If WCI_A < WCI_B, could we say that Acc_A is more significant or more reliable, since
> we could be sure that the true value of Acc_A would fall in a narrower CI (smaller width)?
--
Rich Ulrich