[R] Interpreting the results of Friedman test

[Doerte]

Hello,

I have problems interpreting the results of a Friedman test. It seems
to me that the p-value resulting from a Friedman test and with it the
"significance" has to be interpreted in another way than the p-value
resulting from e.g. ANOVA?

Let me describe the problem with some detail: I'm testing a lot of
different hypotheses in my observer study and only for some the
premises for performing an ANOVA are fulfilled (tested with Shapiro
Wilk and Bartlett). For the others I perform a Friedman test.

To my surprise, the p-value of the Friedman test is < 0.05 for all my
tested hypotheses. Thus, I tried to "compare" the results with the
results of an ANOVA by performing both test methods (Friedman, ANOVA)
to a given set of data.
While ANOVA results in p = 0.34445 (--> no significant difference
between the groups), the Friedman test results in p = 1.913e-06 (-->
significant difference between the groups?).

How can this be?

Or am I doing something wrong? I have three measured values for each
condition. For ANOVA I use them all, for the Friedman test I
calculated the geometric mean of the three values, since this test
does not work with replicated values. Is this a crude mistake?

Thanks in advance for any help.

Doerte

______________________________________________
R-h...@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Jim Lemon]

Doerte wrote:
> Hello,
>
> I have problems interpreting the results of a Friedman test. It seems
> to me that the p-value resulting from a Friedman test and with it the
> "significance" has to be interpreted in another way than the p-value
> resulting from e.g. ANOVA?
>
> Let me describe the problem with some detail: I'm testing a lot of
> different hypotheses in my observer study and only for some the
> premises for performing an ANOVA are fulfilled (tested with Shapiro
> Wilk and Bartlett). For the others I perform a Friedman test.
>
> To my surprise, the p-value of the Friedman test is < 0.05 for all my
> tested hypotheses. Thus, I tried to "compare" the results with the
> results of an ANOVA by performing both test methods (Friedman, ANOVA)
> to a given set of data.
> While ANOVA results in p = 0.34445 (--> no significant difference
> between the groups), the Friedman test results in p = 1.913e-06 (-->
> significant difference between the groups?).
>
> How can this be?
>
> Or am I doing something wrong? I have three measured values for each
> condition. For ANOVA I use them all, for the Friedman test I
> calculated the geometric mean of the three values, since this test
> does not work with replicated values. Is this a crude mistake?
>
>

Hi Doerte,
There is a non-parametric repeated measures analog to ANOVA developed by
Edgar Brunner available at:

http://www.ams.med.uni-goettingen.de/de/sof/ld/makros.html

Unfortunately, the test that you (and I) would like to have does not
appear to have been translated to R code. I intend to contact Professor
Brunner and try to complete this, or at least contribute to the effort,
but have not had the time to do so. There are several other methods,
notably that of Joe McKean and Tom Hettmansperger, but I don't have the
URL for their code at hand. I'll try to forward this from work next week.

Jim

[doerte...@gmx.de]

Hello,

Peter Dalgaard wrote:
> Not necessarily, but the first suspicion one gets is that what you're
> doing with Friedman is not equivalent to what you're doing with ANOVA,
> could you show us the code and data (or an outline of it)?

Please find attached the R script and the data input files.
The values in dataForFriedmanTest.dat represent the geometric mean of
the values of three repetitions for one observer and one condition.

Am I doing something wrong there?

Thanks in advance

Doerte

PS: I'm not sure if it is possible to send the dat-files by mail.
Therefore I renamed them to *.txt. Please rename back to *.dat.

-------- Original-Nachricht --------
> Datum: Fri, 24 Apr 2009 14:52:44 +0200
> Von: Peter Dalgaard <P.Dal...@biostat.ku.dk>
> An: Doerte <doerte...@gmx.de>
> Betreff: Re: [R] Interpreting the results of Friedman test

> Doerte wrote:
> > Hello,
> >
> > I have problems interpreting the results of a Friedman test. It seems
> > to me that the p-value resulting from a Friedman test and with it the
> > "significance" has to be interpreted in another way than the p-value
> > resulting from e.g. ANOVA?
> >
> > Let me describe the problem with some detail: I'm testing a lot of
> > different hypotheses in my observer study and only for some the
> > premises for performing an ANOVA are fulfilled (tested with Shapiro
> > Wilk and Bartlett). For the others I perform a Friedman test.
> >
> > To my surprise, the p-value of the Friedman test is < 0.05 for all my
> > tested hypotheses. Thus, I tried to "compare" the results with the
> > results of an ANOVA by performing both test methods (Friedman, ANOVA)
> > to a given set of data.
> > While ANOVA results in p = 0.34445 (--> no significant difference
> > between the groups), the Friedman test results in p = 1.913e-06 (-->
> > significant difference between the groups?).
> >
> > How can this be?
> >
> > Or am I doing something wrong? I have three measured values for each
> > condition. For ANOVA I use them all, for the Friedman test I
> > calculated the geometric mean of the three values, since this test
> > does not work with replicated values. Is this a crude mistake?
>

> Not necessarily, but the first suspicion one gets is that what you're
> doing with Friedman is not equivalent to what you're doing with ANOVA,
> could you show us the code and data (or an outline of it)?
>
> --
> O__ ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B
> c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K
> (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918
> ~~~~~~~~~~ - (p.dal...@biostat.ku.dk) FAX: (+45) 35327907

--

[Peter Dalgaard]

doerte...@gmx.de wrote:
> Hello,
>
> Peter Dalgaard wrote:
>> Not necessarily, but the first suspicion one gets is that what you're
>> doing with Friedman is not equivalent to what you're doing with ANOVA,
>> could you show us the code and data (or an outline of it)?
>
> Please find attached the R script and the data input files.
> The values in dataForFriedmanTest.dat represent the geometric mean of
> the values of three repetitions for one observer and one condition.
>
> Am I doing something wrong there?

The .R file came as BIN file to me, so r-help may be missing it.

Anyways, Friedman's test is a replacement for a two-way ANOVA and you
are comparing it to a one-way analysis, and the latter is likely just wrong.

Try

anova(lm(AUC~as.factor(Condition)+as.factor(Observer),data=dataForANOVA))

or, maybe better

summary(aov(AUC ~ as.factor(Condition) + Error(as.factor(Observer) /
as.factor(Condition)), data=dataForANOVA))

> Thanks in advance
>
> Doerte
>
> PS: I'm not sure if it is possible to send the dat-files by mail.
> Therefore I renamed them to *.txt. Please rename back to *.dat.

--

O__ ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B
c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K
(*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918
~~~~~~~~~~ - (p.dal...@biostat.ku.dk) FAX: (+45) 35327907

______________________________________________

[Doerte]

Hello again,

it seems, the R script has been filtered from the previous posting.
Thus, I include is as text here:

dataForANOVA <- read.csv("C:/R/dataForANOVA.dat",
sep="|",
as.is=T,
na.strings=".",
header=TRUE)
dataForFriedman <- read.csv("C:/R/dataForFriedmanTest.dat",
sep="|",
as.is=T,
na.strings=".",
header=FALSE)

anova1 <- lm(AUC~as.factor(Condition),data=dataForANOVA)
result_shapiro <- shapiro.test(anova1$residuals)
result_bartlett <- bartlett.test(anova1$residuals~as.factor
(Condition),data=dataForANOVA)
summaryANOVA1 <- summary(anova1)

anova1Results <- anova(lm(AUC~as.factor(Condition),data=dataForANOVA))
print(anova1Results)

friedman.test(as.matrix(dataForFriedman))

[Doerte]

> Anyways, Friedman's test is a replacement for a two-way ANOVA and you
> are comparing it to a one-way analysis, and the latter is likely just wrong.

Okay. Thanks for the hint.

> Try
>
> anova(lm(AUC~as.factor(Condition)+as.factor(Observer),data=dataForANOVA))

This results in p-value = 0.37969. This value is still quite different
from p-value = 1.913e-06, which is the result of friedman.test
(as.matrix(dataForFriedman)).

Is the method friedman.test (version R 2.9.0) working correctly for
certain types of data and hypotheses? Which limitations are known?

@ Jim: Thanks for the link.
Unfortunately, I'm a newbie in statistics, and I'm not sure, which
method can be used instead of the Friedman Test. Do you have an eye on
a certain R-program from this given website?

Doerte

[Peter Dalgaard]

Doerte wrote:
>> Anyways, Friedman's test is a replacement for a two-way ANOVA and you
>> are comparing it to a one-way analysis, and the latter is likely just wrong.
>
> Okay. Thanks for the hint.
>
>> Try
>>
>> anova(lm(AUC~as.factor(Condition)+as.factor(Observer),data=dataForANOVA))
>
> This results in p-value = 0.37969. This value is still quite different
> from p-value = 1.913e-06, which is the result of friedman.test
> (as.matrix(dataForFriedman)).

If you had read ALL that I wrote, then you would have seen this:

> summary(aov(AUC ~ as.factor(Condition) + Error(as.factor(Observer) /

+ as.factor(Condition)), data=dataForANOVA))

Error: as.factor(Observer)
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 6 0.0066907 0.0011151

Error: as.factor(Observer):as.factor(Condition)
Df Sum Sq Mean Sq F value Pr(>F)
as.factor(Condition) 3 0.275825 0.091942 405.23 < 2.2e-16 ***
Residuals 18 0.004084 0.000227
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Error: Within
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 56 6.5337 0.1167

I.e. the thing that is going on is that the Observer:Condition
interaction is incredibly small (0.000227) compared to the replication
variation (0.1167), or put differently that you seem to have strong
negative correlation between replicates. This can happen if you have
things like plants growing in the same pot, but you need to consider
what could be happening in your case.

Looking at the table of means

> with(dataForANOVA,tapply(AUC,list(Observer,Condition),mean))
1 2 3 4
1 11.55841 11.51786 11.43727 11.59314
2 11.55448 11.51013 11.42099 11.56479
3 11.55994 11.49924 11.44884 11.57226
4 11.55410 11.51704 11.42321 11.58310
6 11.52043 11.48902 11.40812 11.56951
7 11.55013 11.51438 11.42279 11.57906
8 11.53763 11.50768 11.41197 11.57795

this is essentially the same as your dataForFriedman and it is pretty
clear that the 3rd column is substantially below the others, which is
what the Friedman test and also the F value of 405.23 above is picking
up on.

>
> Is the method friedman.test (version R 2.9.0) working correctly for
> certain types of data and hypotheses? Which limitations are known?
>
> @ Jim: Thanks for the link.
> Unfortunately, I'm a newbie in statistics, and I'm not sure, which
> method can be used instead of the Friedman Test. Do you have an eye on
> a certain R-program from this given website?
>
> Doerte
>
> ______________________________________________
> R-h...@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
O__ ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B
c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K
(*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918
~~~~~~~~~~ - (p.dal...@biostat.ku.dk) FAX: (+45) 35327907

______________________________________________