Tests of fixed effect vs tests of parameter estimates in MIXED
Andrew
understanding the output of what should be a straightforward mixed
model. This started off looking easy - possible random slopes and
intercept, fixed effect of gender, linear time trend and interaction.
The syntax is:
MIXED
y BYgender WITH time
/FIXED time gender time*gender | SSTYPE(3)
/PRINT = G R SOLUTION TESTCOV
/RANDOM Intercept time | SUBJECT (ID) COVTYPE (UN).
Then the colleague I'm working with asked an insightful question
regarding the output:
Type III Tests of Fixed Effects
Source Numerator df Denominator df F Sig.
Intercept 1 99.819 1791.236 .000
time 1 73.075 26.887 .
000
gender 1 99.819 1.041 .
310
gender * time 1 73.075 6.057 .016
Estimates of Fixed Effects
Parameter Estimate Std. Error df
t Sig.
Intercept 23.819457 .805803 100.524 29.560 .000
time .008031 .004448 70.761 1.805 .
075
[gender=1] 1.176728 1.153407 99.819 1.020 .310
[gender=2] 0(a) 0 . . . . .
[gender=1] * time .014512 .005896 73.075 2.461 .016
[gender=2] * time 0(a) 0 . . . . .
They noticed that the test for time in the first table is highly
significant but the test associated with the parameter estimate is not
- the difference is quite dramatic. I had always thought that for
tests with one df in the numerator, the denominator df of the test of
parameter should be the same as for the omnibus F-test, and the t
value squared should equal F test. Sometimes it does, like the gender
test above, or if the interaction term is omitted but, obviously, not
always.
I've believed that you could use these interchangeably and you should
be able to - you can hardly claim that the effect of time is highly
significant (F(1,73.1)=26.9, p<<0.0001), but that the parameter is not
significantly different from zero (t(70.8)=1.8, p=0.075).
Now I've had a look at this in other analyses and found instances of
similar differences. It doesn't seem to be due to the type of sums of
squares - Type I gives different results but the same issue as Type
III. I can't pin it down on a design either. There aren't any
obvious data problems - empty cells, zero variance etc.
Now I feel I have a gaping hole in my understanding of ANOVA and mixed
models. Any light you can shed on this would be helpful.
Andrew
Bruce Weaver
> I was wondering whether someone could clarify I problem I'having
> understanding the output of what should be a straightforward mixed
> model. This started off looking easy - possible random slopes and
> intercept, fixed effect of gender, linear time trend and interaction.
> The syntax is:
>
> MIXED
> y BYgender WITH time
> /FIXED time gender time*gender | SSTYPE(3)
> /PRINT = G R SOLUTION TESTCOV
> /RANDOM Intercept time | SUBJECT (ID) COVTYPE (UN).
>
> Then the colleague I'm working with asked an insightful question
> regarding the output:
--- snip ---
That table didn't display very cleanly in my newsreader. Maybe
this will work better.
Type III Tests of Fixed Effects
Source Num df Denom df F Sig.
Intercept 1 99.819 1791.236 .000
time 1 73.075 26.887 .000
gender 1 99.819 1.041 .310
gender * time 1 73.075 6.057 .016
Estimates of Fixed Effects
Parameter Est. SE df t Sig.
Intercept 23.819457 .805803 100.524 29.560 .000
time .008031 .004448 70.761 1.805 .075
[gender=1] 1.176728 1.153407 99.819 1.020 .310
[gender=2] 0(a) 0 . . . . .
[gender=1]*time .014512 .005896 73.075 2.461 .016
[gender=2]*time 0(a) 0 . . . . .
>
>
> They noticed that the test for time in the first table is highly
> significant but the test associated with the parameter estimate is not
> - the difference is quite dramatic. I had always thought that for
> tests with one df in the numerator, the denominator df of the test of
> parameter should be the same as for the omnibus F-test, and the t
> value squared should equal F test. Sometimes it does, like the gender
> test above, or if the interaction term is omitted but, obviously, not
> always.
>
> I've believed that you could use these interchangeably and you should
> be able to - you can hardly claim that the effect of time is highly
> significant (F(1,73.1)=26.9, p<<0.0001), but that the parameter is not
> significantly different from zero (t(70.8)=1.8, p=0.075).
>
> Now I've had a look at this in other analyses and found instances of
> similar differences. It doesn't seem to be due to the type of sums of
> squares - Type I gives different results but the same issue as Type
> III. I can't pin it down on a design either. There aren't any
> obvious data problems - empty cells, zero variance etc.
>
> Now I feel I have a gaping hole in my understanding of ANOVA and mixed
> models. Any light you can shed on this would be helpful.
>
> Andrew
I don't have an answer for you. I just wanted to say that I had
the same expectations as you. Out of curiosity, do you see the
same thing if you code Gender as 0/1, and include it as a
continuous variable (i.e., MIXED y WITH gender time etc)?
Also, you might try posting your question to the multilevel
mailing list (http://www.jiscmail.ac.uk/lists/multilevel.html). I
expect that most of the regulars there use something other than
SPSS MIXED (e.g., MLwiN), but they might be able to help.
Good luck. And if you do find the answer elsewhere, please post
it back here.
--
Bruce Weaver
bwe...@lakeheadu.ca
www.angelfire.com/wv/bwhomedir
"When all else fails, RTFM."
Andrew
Thanks for two excellent suggestions. The post to the multilevel list
yielded the response below from Paul Swank. I believe his
interpretation of the parameters is correct, but I don’t think this
helps me understand the difference between the tests of the parameters
and the fixed effects.
Treating the appropriately recoded gender factor as a 0/1 covariate (y
WITH gender time) was revealing. With this model the tests of the
fixed effects was identical with those of the parameters.
I also printed out the L matrix to see what parameters were involved
in the Fixed Effects Tests. This was also revealing. Treating gender
as a factor results in a test of the time effect equivalent to:
/TEST = 'Fixed Effect of time' time 1 time*gender .5 .5
With gender as a covariate the test of the fixed effect is equivalent
to
/TEST = 'Fixed Effect of time' time 1
so in the first case, the fixed effect test includes the ‘average’ of
the time effect of each level of gender. The value of the contrast
underlying the test of the fixed effect is thus:
0.0153=0.0080+0.5*0.0145+0.05*0
As far as I understand it, with gender treated as a covariate, the
test of the effect of time is evaluated at gender=0 so the value of
the contrast underlying the test of the fixed effect is just 0.0080.
I think this is the solution to my problem, but I’m not 100% sure, so
I’d be grateful for any comments or corrections.
Andrew
---------------------------
Paul Swank’s reply to my post on the Multilevel mailing list:
> The parameter estimate for time is saying that the slope for gender = 2
> is not quite significant but the slope for gender = 1 is. Standard
> indicator coding zeros parameters for the last level of the categorical
> variable so that the intercept and slope represent the values for that
> level and the gender and slope by gender represent the differences in
> intercept and slope between the levels of gender.
dan.was...@gmail.com
Hope this helps.
Why do the "Solution for Fixed Effects" and the "Tests of Fixed Effects" sometimes differ?
The F-tests are by default Type III tests, so that if there is an F-test for a main effect involved in an interaction, it is an approximate test for the main effect valid only if the interaction is zero (Murray, 1998, p. 293, note 23). If you look at the t-test in the solution table in the same analysis, the t-test for that same main effect is a test for the simple main effect of that variable at the zero level of the other term involved in the interaction. In other words, the t-tests in the solution table are always interpretable just as any test of a regression coefficient would be from a linear regression analysis, but the F-tests aren't.
Consider male (1=male, 0=female) and cond (condition; 1=intervention, 0=control). If we have an interaction model that includes male, cond and male × cond, the F-test for male is an approximate test of whether males are different from females in general, and is valid only if the interaction effect is zero. In the same analysis, the t-test for male in the solution table is a test of whether males are different from females when we restrict ourselves to students in the control condition. So the t-test and F-test are testing different null hypotheses in that circumstance and their p-values may be different, especially if the interaction effect is meaningful. In a purely main effects model, the t-test and F-test are testing the same null hypothesis and their p-values will be identical. (Answer by David Murray, 04-28-00 e-mail, edited.)
On Murray's (1998) note 23 on page 293, he comments that some controversy lies around Type III F-tests and that people have often misinterpreted them. In an analysis of condition and time, the F-test for condition tests the null hypothesis that condition does not vary at an average level of time. The Type III F-test of time tests for a time effect at the average level of condition. With an interaction, Type III F-tests reduce the denominator degrees of freedom and inflate the standard errors of tests of main effects for the included interaction.