Planned contrasts with ezANOVA

[David Braithwaite]

Can ezANOVA do planned contrasts? If so, how? If not, is there some way to do the contrasts using the output of ezANOVA?

Here's my model for reference:

options(contrasts=c("contr.sum","contr.poly"))

ezANOVA( data=D, dv=.(change), wid=.(subjid), within=.(schema), between=.(training,prevexp), type=3 )

change ranges from -1.0 to 1.0. schema has 5 levels. prevexp has two levels - high or low. training has 3 levels, let's say A, B, and C. I get a significant interaction of training with prevexp. I have a priori reasons to group training B and C together, and if I recode training with two levels, i.e. A vs B/C, I still get the above interaction with prevexp. What I'd like to do is put all of this into a single analysis that will say

1. whether there's an interaction of A vs. B/C, or A vs. B vs. C, with prevexp? (I am fine posing this question in either of these two ways, whichever works better.)

2. whether A differs from B/C at each level of prevexp?

3. whether B differs from C at each level of prevexp?

Hope this is clear. If anyone wants to give guidance on the stats aspect of this it would be very helpful, but I'm mainly asking about how to do the above with ez. Thanks in advance!

[Thom]

I would reccommend using multcomp (or maybe gmodels) for planned contrasts in R as you have a lot of flexibility there. The

free sample chapter for my book gives examples of how to set these up for independent designs (examples for repeated measures

designs are in a later chapter).

See

http://seriousstats.wordpress.com/2012/03/23/serious-stats-companion-web-site-now-live/

That said I'd be interested to know about options for planned contrasts in ezANOVA.

Thom

[Mike Lawrence]

Unfortunately there's no way to do planned contrasts in ez; my research typically has only either dichotomous or continuous predictors, so I never got around to coding something for contrasts in ez. 

[David Braithwaite]

Thanks for the replies!

If there's no way to do the contrasts in ez as such, how about the next best thing - any way to use the output of ez to run contrasts with some other package?

I'm hesitant about starting from scratch in another package for this particular analysis, for the reason that most of my other analyses are in ez, and I'm worried that different packages make different assumptions or implement little things in different ways that would make their results mutually inconsistent in ways that I wouldn't know about or would find hard to fix. That's one reason I don't just go do it in SPSS, where I know how to do contrasts! I've frequently run into the issue that analyses I think are identical in SPSS and R give different results on the same data. Anyway, I'd like to not have to deal with that kind of thing if I don't have to.

[Henrik Singmann]

Hi guys,

let me add my two cents on the issue of multiple comparisons with two solutions (neither of which involve ez).

1. If you have zero or only one within-subjects factors, you can use lme from the nlme package to fit the standard ANOVA (using a compound symmetric correlation structure) and then use multcomp on the returned model. The solution is laid out in my accepted answer to this stats.stackexchange question: http://stats.stackexchange.com/q/15526/442

The advantage of this solution is that one can use the whole range of functionality available in the multcomp package (for which a book exists, Bretz, et al., 2011). The clear downside is that you can only have up to one within-subjects factors.

2. If you have more than one within-subjects factor, you need to resort on my afex package (which I have mentioned here a few times and is, identical to ez, a wrapper for the car package) for fitting the ANOVA and the phia package for the multiple comparisons. Contrary to multcomp, phia uses multivariate tests for the contrasts. Here is a worked out exampke using the OBrien & Kaiser (1985) dataset which comes with afex and car.

#### R code follows ###

require(afex)

data(obk.long)

str(obk.long)
## 'data.frame':   240 obs. of  7 variables:
##  $ id       : Factor w/ 16 levels "1","2","3","4",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ treatment: Factor w/ 3 levels "control","A",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ gender   : Factor w/ 2 levels "F","M": 2 2 2 2 2 2 2 2 2 2 ...
##  $ age      : num  -4.75 -4.75 -4.75 -4.75 -4.75 -4.75 -4.75 -4.75 -4.75 -4.75 ...
##  $ phase    : Factor w/ 3 levels "fup","post","pre": 3 3 3 3 3 2 2 2 2 2 ...
##  $ hour     : Factor w/ 5 levels "1","2","3","4",..: 1 2 3 4 5 1 2 3 4 5 ...
##  $ value    : num  1 2 4 2 1 3 2 5 3 2 ...

# ez.glm is one convenience function for fitting an ANOVA
# nice.anova returns a nice anova table
# you need to set return to NOT "Anova", to run contrasts!
nice.anova(m1 <- ez.glm("id", "value", obk.long, between = c("treatment", "gender"), within = c("phase", "hour"), return = "list")[[1]])
##                         Effect          df   MSE         F     p
## 1                    treatment       2, 10 22.81    3.94 +   .05
## 2                       gender       1, 10 22.81    3.66 +   .08
## 3             treatment:gender       2, 10 22.81      2.86   .10
## 4                        phase  1.6, 15.99  5.02 16.13 *** <.001
## 5              treatment:phase  3.2, 15.99  5.02    4.85 *   .01
## 6                 gender:phase  1.6, 15.99  5.02      0.28   .71
## 7       treatment:gender:phase  3.2, 15.99  5.02      0.64   .61
## 8                         hour 1.84, 18.41  3.39 16.69 *** <.001
## 9               treatment:hour 3.68, 18.41  3.39      0.09   .98
## 10                 gender:hour 1.84, 18.41  3.39      0.45   .63
## 11       treatment:gender:hour 3.68, 18.41  3.39      0.62   .64
## 12                  phase:hour  3.6, 35.96  2.67      1.18   .33
## 13        treatment:phase:hour 7.19, 35.96  2.67      0.35   .93
## 14           gender:phase:hour  3.6, 35.96  2.67      0.93   .45
## 15 treatment:gender:phase:hour 7.19, 35.96  2.67      0.74   .65


# use phia for contrasts
require(phia)

# on within-subjects factor phase:
testFactors(m1[["lm"]], list("phase" = c(post = -1, pre = 1)), idata = m1[["idata"]])
## [output trunacted]
## Multivariate Tests:
##                  Df test stat approx F num Df den Df  Pr(>F) 
## Pillai            1 0.4969992 9.880682      1     10 0.01045 *
## Wilks             1 0.5030008 9.880682      1     10 0.01045 *
## Hotelling-Lawley  1 0.9880682 9.880682      1     10 0.01045 *
## Roy               1 0.9880682 9.880682      1     10 0.01045 *
## ---
## Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
## ------

# on between-subjects factor treatment:
testInteractions(m1[["lm"]], pairwise = "treatment", idata = m1[["idata"]], adjustment = "none")
## Multivariate Test: Pillai test statistic
## P-value adjustment method: none
##              Value Df test stat approx F num Df den Df  Pr(>F) 
## control-A -2.02778  1   0.37110   5.9008      1     10 0.03550 *
## control-B -1.80556  1   0.37711   6.0543      1     10 0.03364 *
##       A-B  0.22222  1   0.00814   0.0821      1     10 0.78038 
## ---
## Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

## R code ends here ##

The clear advantage of this approach is that you can get contrasts for *any* factors in any type of standard ANOVA model. The downside is that I am not sure how kosher these tests are (in terms of alpha and better errors). As far as I have an overview of the literature on repeated-measures ANOVA (mainly the work by Keselman & Algina), the multivariate contrasts are not discussed at all. Unfortunately, these tests are also not discussed in the book by Thom (which I full-heartedly recommend to anyone).

So, if we could get some agreement that the tests in phia are reasonable (which I suspect as John Fox endorses phia), this is probably the easiest way to obtain contrasts. I would even be inclined to write an even simpler interface than the current one if I ever have the time and there is some demand.

Oh, let me add one final caveata. Although I think ez and afex are pretty similar in terms of the underlying computation as they both wrap car::Anova, there are a two main differences. First, afex uses type 3 sums of squares and sets the default contrasts to effects coding (i.e., sum to zero contrasts) replicating SPSS and SAS, ez uses type 2 and uses the current contrasts.  Second, afex does (currently) not report any type of effects sizes.

I am really hoping for some discussion on this and happy to provide more details.

Cheers,
Henrik

** References **

Bretz, F., Hothorn, T., & Westfall, P. H. (2011). Multiple comparisons using R. Boca Raton, FL: CRC Press.

O’Brien, R. G., & Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive primer. Psychological Bulletin, 97(2), 316–333. doi:10.1037/0033-2909.97.2.316

2012/11/30 David Braithwaite <baix...@gmail.com>

[David Braithwaite]

I find myself once again facing the question I posed at the beginning of this thread and, since a few years have passed, I thought I'd check in to see whether anything new has emerged.

I have data involving a 3-level within-subjects factor as well as several between-subjects factors. I'd like to conduct my main analysis with ezANOVA since that's what I'm used to and is what I use for my other analyses in the same project. Is there any way I can use ezANOVA to run orthogonal planned contrasts on the levels of the within-subjects factor (e.g., A vs. B+C and B vs. C)? If this is still not possible to do within ezANOVA, is there any way to do it using another package but still using the output of ezANOVA?

@Henrik, I believe this fits into the first case mentioned in your reply above. However, if I correctly understood your reply, it would involve doing the entire analysis (not just the contrasts) using nlme, correct? I'd prefer still to do the main analysis using ezANOVA if possible.