issues with ezANOVA - hope someone can help!

[David Braithwaite]

I do a lot of mixed ANOVA (both within and between subjects factors) and have found the ez package very helpful for this. However, lately, it seems to have stopped working for me half the time. I'm hoping someone can tell me what, if anything, I'm doing wrong and how to fix it.

Here's a typical analysis:

> D = droplevels( subset( lf.data, section == 'Pretest' ) )
> ezANOVA( data=D, dv=.(accuracy), wid=.(mturkcode), between=.(traintype), within=.(questype) )
Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().
Using accuracy as value column: use value.var to override.
$ANOVA
              Effect DFn DFd         F          p p<.05         ges
2          traintype   2 157 1.0412880 0.35542459       0.003783887
3           questype   4 628 3.1120174 0.01495624     * 0.013948667
4 traintype:questype   8 628 0.7066735 0.68582197       0.006383498

$`Mauchly's Test for Sphericity`
              Effect        W            p p<.05
3           questype 0.793061 3.913992e-05     *
4 traintype:questype 0.793061 3.913992e-05     *

$`Sphericity Corrections`
              Effect      GGe      p[GG] p[GG]<.05       HFe      p[HF] p[HF]<.05
3           questype 0.893964 0.01891644         * 0.9173675 0.01795816         *
4 traintype:questype 0.893964 0.66953425           0.9173675 0.67327393         

So, first of all, why is it having to guess which variable is the dv? I told it dv=.(accuracy). Its guess was correct, but still, seems strange.

Now here's the more serious issue. If I remove the within-subjects variable from the above analysis, it no longer works:

> ezANOVA( data=D, dv=.(accuracy), wid=.(mturkcode), between=.(traintype) )
Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().
Error in `colnames<-`(`*tmp*`, value = c("Effect", "DFn", "DFd", "SSn",  :
  'names' attribute [8] must be the same length as the vector [7]
Error in ezANOVA_main(data = data, dv = dv, wid = wid, within = within,  :
  The car::Anova() function used to compute results and assumption tests seems to have failed. Most commonly this is because you have too few subjects relative to the number of cells in the within-Ss design. It is possible that trying the ANOVA again with "type=1" may yield results (but definitely no assumption tests).
>

Maybe there's a problem because I have multiple rows of data for each subject, but there's no within subjects factor? But no, if I use a compressed file with only one row per subject, I get the same problem:

> D = dcast( lf.data, mturkcode+traintype~section, fun.aggregate=mean, value.var="accuracy" )
> ezANOVA( data=D, dv=.(Pretest), wid=.(mturkcode), between=.(traintype) )
Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().
Error in `colnames<-`(`*tmp*`, value = c("Effect", "DFn", "DFd", "SSn",  :
  'names' attribute [8] must be the same length as the vector [7]
Error in ezANOVA_main(data = data, dv = dv, wid = wid, within = within,  :
  The car::Anova() function used to compute results and assumption tests seems to have failed. Most commonly this is because you have too few subjects relative to the number of cells in the within-Ss design. It is possible that trying the ANOVA again with "type=1" may yield results (but definitely no assumption tests).
>

I have no idea what is going on here. accuracy takes values 0.0, 0.5, and 1.0.  Pretest takes values from 0 to 1 in increments of 0.1. mturkcode is just a participant ID. traintype is a 3-level factor with 40 participants for each of 2 levels and 80 for the other level. I don't think the issue is the unbalanced design because that's never caused a problem before. Anyone have any idea what my problem might be?

Thanks much in advance!!

[Mike Lawrence]

What version of ez are you using? Load ez then run sessionInfo() and reply back with the result.

--
Cheers,

Mike

--
Mike Lawrence
Graduate Student
Department of Psychology
Dalhousie University

Looking to arrange a meeting? Check my public calendar: http://goo.gl/BYH99

~ Certainty is folly... I think. ~

[David Braithwaite]

Version 3.0-1. I'm posting the full output in case a conflict with something else, such as reshape, might be causing an issue. (I know ez relies on reshape2, but I also have plain old reshape loaded. I load reshape first, then reshape2, then ez, if that makes any difference.)

> sessionInfo()
R version 2.15.0 (2012-03-30)
Platform: x86_64-pc-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252 LC_NUMERIC=C                         
[5] LC_TIME=English_United States.1252   

attached base packages:
[1] stats4    splines   stats     graphics  grDevices utils     datasets  methods   base    

other attached packages:
 [1] sciplot_1.0-9     coin_1.0-21       modeltools_0.2-19 mvtnorm_0.9-9992  survival_2.36-12  ez_3.0-1          stringr_0.6       scales_0.2.1      RCurl_1.91-1.1  
[10] bitops_1.0-4.1    memoise_0.1       mgcv_1.7-13       lme4_0.999375-42  Matrix_1.0-6      lattice_0.20-6    ggplot2_0.9.1     car_2.0-12        nnet_7.3-1      
[19] MASS_7.3-17       reshape2_1.2.1    reshape_0.8.4     plyr_1.7.1      

loaded via a namespace (and not attached):
 [1] colorspace_1.1-1   dichromat_1.2-4    digest_0.5.2       grid_2.15.0        labeling_0.1       munsell_0.3        nlme_3.1-103       proto_0.3-9.2    
 [9] RColorBrewer_1.0-5 tools_2.15.0

[Denise]

New to the ez package and just reinstalled new version of R and all
associated packages. Great package by the way, it's helping me to run
unbalanced mixed ANOVAs before delving more deeply into the theory
behind linear mixed effect models. However, I'm getting the exact same
error when I remove my within-subject factor and analyze only the 2
between-subject factors (where there is lack of balance). Any
solutions?

> sessionInfo()
R version 2.15.0 (2012-03-30)

Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United
States.1252

[3] LC_MONETARY=English_United States.1252

LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:

[1] stats graphics grDevices utils datasets methods
base

other attached packages:
[1] ez_3.0-1 stringr_0.6 scales_0.2.1
reshape2_1.2.1 RCurl_1.91-1.1
[6] bitops_1.0-4.1 plyr_1.7.1 memoise_0.1
mgcv_1.7-17 lme4_0.999375-42
[11] Matrix_1.0-6 lattice_0.20-6 ggplot2_0.9.1
car_2.0-12 nnet_7.3-1
[16] MASS_7.3-18 nlme_3.1-103

loaded via a namespace (and not attached):
[1] colorspace_1.1-1 dichromat_1.2-4 digest_0.5.2
grid_2.15.0

[5] labeling_0.1 munsell_0.3 proto_0.3-9.2
RColorBrewer_1.0-5
[9] stats4_2.15.0 tools_2.15.0

Thanks,
Denise

[David Braithwaite]

I haven't figured this out yet - I'm still having the same problem. I'm reduced to adding extraneous within-subjects factors so that my analysis will work. Anyone who could tell us how to fix this would be a lifesaver!

[Matt Weber]

Just ran into the same issue (the value.var thing, I haven't tried removing the within-subjects factors). I don't think the issue has to do with a conflict with another package -- when I load up R, only attach ez, and then run the ezANOVA examples from the ez manual, I get the value.var message for the first two. The (balanced) between-only example from the ez manual runs without an error or a warning, but it crashes if I remove a subject, regardless of the type argument I supply. So it looks like the problem is specific to an unbalanced design. (Copious output in monospace below.)

I just switched versions of ez; I wasn't getting this before. I also recently upgraded from R 2.11 something to the newest version, which is what occasioned the update of ez.

R version 2.15.0 (2012-03-30)

Copyright (C) 2012 The R Foundation for Statistical Computing

ISBN 3-900051-07-0

Platform: i386-apple-darwin9.8.0/i386 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.

You are welcome to redistribute it under certain conditions.

Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.

Type 'contributors()' for more information and

'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or

'help.start()' for an HTML browser interface to help.

Type 'q()' to quit R.

[R.app GUI 1.51 (6148) i386-apple-darwin9.8.0]

[Workspace restored from /Users/mweb/.RData]

[History restored from /Users/mweb/.Rhistory]

> 

> 

> 

> library(ez)

Loading required package: car

Loading required package: MASS

Loading required package: nnet

Loading required package: ggplot2

Loading required package: lme4

Loading required package: Matrix

Loading required package: lattice

Attaching package: ‘lme4’

The following object(s) are masked from ‘package:stats’:

    AIC, BIC

Loading required package: mgcv

This is mgcv 1.7-13. For overview type 'help("mgcv-package")'.

Loading required package: memoise

Loading required package: plyr

Loading required package: RCurl

Loading required package: bitops

Loading required package: reshape2

Loading required package: scales

Loading required package: stringr

> #Read in the ANT data (see ?ANT).

> data(ANT)

> head(ANT)

  subnum     group block trial     cue     flank location direction       rt error

1      1 Treatment     1     1    None   Neutral       up      left 398.6773     0

2      1 Treatment     1     2  Center   Neutral       up      left 389.1822     0

3      1 Treatment     1     3  Double   Neutral       up      left 333.2186     0

4      1 Treatment     1     4 Spatial   Neutral       up      left 419.7640     0

5      1 Treatment     1     5    None Congruent       up      left 446.4754     0

6      1 Treatment     1     6  Center Congruent       up      left 338.9766     0

> ezPrecis(ANT)

Data frame dimensions: 5760 rows, 10 columns

             type missing values      min         max

subnum     factor       0     20        1          20

group      factor       0      2  Control   Treatment

block     numeric       0      6        1           6

trial     numeric       0     48        1          48

cue        factor       0      4     None     Spatial

flank      factor       0      3  Neutral Incongruent

location   factor       0      2     down          up

direction  factor       0      2     left       right

rt        numeric       0   5760 179.5972    657.6986

error     numeric       0      2        0           1

> #Run an ANOVA on the mean correct RT data.

> rt_anova = ezANOVA(

+ data = ANT[ANT$error==0,]

+ , dv = .(rt)

+ , wid = .(subnum)

+ , within = .(cue,flank)

+ , between = .(group)

+ )

Using rt as value column: use value.var to override.

> #Show the ANOVA & assumption tests.

> print(rt_anova)

$ANOVA

           Effect DFn DFd           F            p p<.05        ges

2           group   1  18   18.430592 4.377562e-04     * 0.07633358

3             cue   3  54  516.605213 1.005518e-39     * 0.89662286

4       group:cue   3  54    2.553236 6.497492e-02       0.04110445

5           flank   2  36 1350.598810 1.386546e-34     * 0.92710583

6     group:flank   2  36    8.768499 7.900829e-04     * 0.07627434

7       cue:flank   6 108    5.193357 9.938494e-05     * 0.11436699

8 group:cue:flank   6 108    6.377225 9.012515e-06     * 0.13686958

$`Mauchly's Test for Sphericity`

           Effect         W         p p<.05

3             cue 0.7828347 0.5366835      

4       group:cue 0.7828347 0.5366835      

5           flank 0.8812738 0.3415406      

6     group:flank 0.8812738 0.3415406      

7       cue:flank 0.1737053 0.1254796      

8 group:cue:flank 0.1737053 0.1254796      

$`Sphericity Corrections`

           Effect       GGe        p[GG] p[GG]<.05       HFe        p[HF] p[HF]<.05

3             cue 0.8652559 1.115029e-34         * 1.0239520 1.005518e-39         *

4       group:cue 0.8652559 7.472046e-02           1.0239520 6.497492e-02          

5           flank 0.8938738 3.763312e-31         * 0.9858964 3.964046e-34         *

6     group:flank 0.8938738 1.297752e-03         * 0.9858964 8.438369e-04         *

7       cue:flank 0.6022111 1.546166e-03         * 0.7721473 4.745714e-04         *

8 group:cue:flank 0.6022111 3.424499e-04         * 0.7721473 7.170939e-05         *

> #Run an ANOVA on the mean_rt data, ignoring group.

> rt_anova2 = ezANOVA(

+ data = ANT[ANT$error==0,]

+ , dv = .(rt)

+ , wid = .(subnum)

+ , within = .(cue,flank)

+ )

Using rt as value column: use value.var to override.

Note: model has only an intercept; equivalent type-III tests substituted.

> #Show the ANOVA & assumption tests.

> print(rt_anova2)

$ANOVA

     Effect DFn DFd          F            p p<.05        ges

2       cue   3  57 477.564650 2.435084e-40     * 0.86387868

3     flank   2  38 958.640865 3.040261e-33     * 0.90297213

4 cue:flank   6 114   4.047785 1.026734e-03     * 0.08633287

$`Mauchly's Test for Sphericity`

     Effect         W          p p<.05

2       cue 0.8670854 0.77271988      

3     flank 0.9088146 0.42293876      

4 cue:flank 0.1506008 0.04917243     *

$`Sphericity Corrections`

     Effect       GGe        p[GG] p[GG]<.05      HFe        p[HF] p[HF]<.05

2       cue 0.9165014 3.647676e-37         * 1.086943 2.435084e-40         *

3     flank 0.9164345 1.182224e-30         * 1.009411 3.040261e-33         *

4 cue:flank 0.6261487 6.059761e-03         * 0.799682 2.641207e-03         *

> #Run a purely between-Ss ANOVA on the mean_rt data.

> rt_anova3 = ezANOVA(

+ data = ANT[ANT$error==0,]

+ , dv = .(rt)

+ , wid = .(subnum)

+ , between = .(group)

+ )

> #Show the ANOVA & assumption tests.

> print(rt_anova3)

$ANOVA

  Effect DFn DFd        F            p p<.05       ges

1  group   1  18 18.50661 0.0004292561     * 0.5069385

$`Levene's Test for Homogeneity of Variance`

  DFn DFd       SSn     SSd         F         p p<.05

1   1  18 0.5531777 42.6997 0.2331913 0.6349885      

> sessionInfo()

R version 2.15.0 (2012-03-30)

Platform: i386-apple-darwin9.8.0/i386 (32-bit)

locale:

[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:

 [1] ez_3.0-1         stringr_0.6      scales_0.2.1     reshape2_1.2.1   RCurl_1.91-1     bitops_1.0-4.1  

 [7] plyr_1.7.1       memoise_0.1      mgcv_1.7-13      lme4_0.999375-42 Matrix_1.0-6     lattice_0.20-6  

[13] ggplot2_0.9.1    car_2.0-12       nnet_7.3-1       MASS_7.3-17     

loaded via a namespace (and not attached):

 [1] colorspace_1.1-1   dichromat_1.2-4    digest_0.5.2       grid_2.15.0        labeling_0.1       munsell_0.3       

 [7] nlme_3.1-103       proto_0.3-9.2      RColorBrewer_1.0-5 stats4_2.15.0      tools_2.15.0 

> unbData = ANT[(ANT$error==0 & ANT$subnum!=20),]

> ezANOVA(data=unbData,dv=.(rt),wid=.(subnum),between=.(group))

Warning: You have removed one or more Ss from the analysis. Refactoring "subnum" for ANOVA.

Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().

Error in `colnames<-`(`*tmp*`, value = c("Effect", "DFn", "DFd", "SSn",  : 

  'names' attribute [8] must be the same length as the vector [7]

Error in ezANOVA_main(data = data, dv = dv, wid = wid, within = within,  : 

  The car::Anova() function used to compute results and assumption tests seems to have failed. Most commonly this is because you have too few subjects relative to the number of cells in the within-Ss design. It is possible that trying the ANOVA again with "type=1" may yield results (but definitely no assumption tests).

> ezANOVA(data=unbData,dv=.(rt),wid=.(subnum),between=.(group),type=3)

Warning: You have removed one or more Ss from the analysis. Refactoring "subnum" for ANOVA.

Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().

Error in `colnames<-`(`*tmp*`, value = c("Effect", "DFn", "DFd", "SSn",  : 

  'names' attribute [8] must be the same length as the vector [7]

Error in ezANOVA_main(data = data, dv = dv, wid = wid, within = within,  : 

  The car::Anova() function used to compute results and assumption tests seems to have failed. Most commonly this is because you have too few subjects relative to the number of cells in the within-Ss design. It is possible that trying the ANOVA again with "type=1" may yield results (but definitely no assumption tests).

> ezANOVA(data=unbData,dv=.(rt),wid=.(subnum),between=.(group),type=2)

Warning: You have removed one or more Ss from the analysis. Refactoring "subnum" for ANOVA.

Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().

Error in `colnames<-`(`*tmp*`, value = c("Effect", "DFn", "DFd", "SSn",  : 

  'names' attribute [8] must be the same length as the vector [7]

Error in ezANOVA_main(data = data, dv = dv, wid = wid, within = within,  : 

  The car::Anova() function used to compute results and assumption tests seems to have failed. Most commonly this is because you have too few subjects relative to the number of cells in the within-Ss design. It is possible that trying the ANOVA again with "type=1" may yield results (but definitely no assumption tests).

> ezANOVA(data=unbData,dv=.(rt),wid=.(subnum),between=.(group),type=1)

Warning: You have removed one or more Ss from the analysis. Refactoring "subnum" for ANOVA.

Warning: Data is unbalanced (unequal N per group). Make sure you specified a well-considered value for the type argument to ezANOVA().

Warning: Data are unbalanced, switching to "type=2" (see documentation in ?ezAnova)

Error in `colnames<-`(`*tmp*`, value = c("Effect", "DFn", "DFd", "SSn",  : 

  'names' attribute [8] must be the same length as the vector [7]

Error in ezANOVA_main(data = data, dv = dv, wid = wid, within = within,  : 

  The car::Anova() function used to compute results and assumption tests seems to have failed. Most commonly this is because you have too few subjects relative to the number of cells in the within-Ss design. It is possible that trying the ANOVA again with "type=1" may yield results (but definitely no assumption tests).

[David Braithwaite]

I also just reinstalled R and ez, which may have resulted in switching to a newer version from an older one, which I don't think ever produced this problem.

[Mike Lawrence]

Hi folks,

Sorry for taking so long to reply to this thread, I've been away at a
conference then at a workshop.

The error encountered when you try a completely between-Ss design with
unbalanced groups is a bug that was reported to the github bug tracker
(https://github.com/mike-lawrence/ez/issues/1) and is fixed in the dev
version. To get the dev version, see the instructions here:
https://github.com/mike-lawrence/ez#readme


Mike

[Matt Weber]

I installed the dev version of ez and ran the example code from the ezANOVA manual, as well as the code from the error report that Mike linked. Everything worked. Thanks!

[Katharina Schwarz]

Hey guys,

 I just (re)installed the newest R and R Studio versions, got ez (also version 3.0-1), and ran into the same problem. I followed the bug removal instruction point by point and did everythig as instructed (R-Tools installation etc.), but for some reason it always produces this error 

  

Installing ez
* checking for file xxx ... OK
* preparing 'ez':
* checking DESCRIPTION meta-information ... OK
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* looking to see if a 'data/datalist' file should be added
* building 'ez_4.0-0.tar.gz'
 ERROR
packaging into .tar.gz failed
Fehler: Command failed (1)

and subsequently the anova problem still remains. 

I tried the solution on another pc some time ago and it worked there - do you have any idea what could be the reason for the error? I'm a bit out of my depth here...

Thanks!

Kat