Statistical significance
Tim Frink
I've a question about on statistical hypothesis testing. Let's say I've
two approaches A and B and I would like to see if one approach is better
than the other.
These are my results (p-values) for the statistical testing:
A B
A - 0.002
B 0.998 -
using the alternative hypothesis that the approaches in the first column
are better than the approaches in the 2. and 3. column. The significance
level was 5%.
Is my interpretation correct that statically significant differences are
observed in the second row, i.e. it can be inferred that A outperforms B
(since 0.002 < 5%)?
If on the other hand, the table would be a result of significance testing
using the null hypothesis that approaches in the first column perform as
well as approaches in the 2. and 3. column. Could it be then inferred
that
a) the null hypothesis can be rejected for 0.002 (since smaller than
significance level), i.e. A and B do not perform equally
b) the null hypothesis is valid for 0.998, i.e. B and A perform equally?
So, is the interpretation of the p-value w.r.t. to a given significance
level oppositional for the null and alternative hypothesis? Can it be
said in general that if the null hypothesis is rejected then the
alternative hypothesis can be accepted?
Regards,
Tim
Chip Eastham
Hi, Tim:
Since you don't appear to have received any
response so far, I'll reluctantly give my
meager 2 cents...
I don't understand the "results" you've
presented, a two-by-two table with two
entries omitted:
> These are my results (p-values) for the
> statistical testing:
> A B
> A - 0.002
> B 0.998 -
A and B are described as "two approaches".
There seems to be a question about whether
"A outperforms B", corresponding to some
null hypothesis that they "perform equally".
A p-value is ordinarily the probability
of obtaining results at least as favorable
to the alternative hypothesis as observed
results, assuming the null hypothesis holds.
Intuitively one rejects the null hypothesis
if the chance of that happening is small by
some criterion chosen prior to experiment.
You've not given any detail that allows an
informed opinion about whether this was done
correctly, much less interpreted correctly.
regards, chip