Post Hoc for Repeated Measures
neila...@msn.com
I need to obtain either LSD or Duncan PostHoc results
for a Repeated Measures factor. SPSS doesn't provide
this as an option. Anybody have a solid correct solution?
Am I stuck with running all possible paired t-tests and
following up with a bonferroni correction? Client is currently
running the analysis as a OneWay (I have informed them
this is not a correct analysis). What are the consequences
of this incorrect analysis -error term is wrong of course-.
WIll the results be invariably conservative or are there situations
where the results will be overly liberal?
I know one can set something up using the SUBJECT as
a factor and then using the WITHIN*SUBJECT interaction as
an error term (or something cery close to this).
Clearly, if there were a simple solution, I am sure it would already
be implimented in SPSS. I studied LOTS of advanced Statistics
YEARS ago and can't seem to find my copies of Bock or Kirk at the
moment.
Neila
-
Bruce Weaver
Fisher's LSD entails carrying out all pairwise contrasts, but only if
the omnibus F-test is significant. In the between-Ss designs it was
designed for, those pairwise contrasts are t-tests that use MS_error
from the ANOVA as the pooled variance term in the denominator. And
there is no adjustment to the per contrast alpha.
So, I would say that if you want an analog to Fisher's LSD for repeated
measures, just perform all pairwise contrasts using garden variety
paired t-tests (so the error term involves only the scores in the pair
of conditions being compared). Using the Factor*Subjects interaction
term from the overall ANOVA is not recommended because of concerns about
sphericity, etc.
But bear in mind that Fisher's LSD is generally viewed with suspicion
these days, because it does not control familywise alpha very well.
However, as Dave Howell shows in his textbook (Statistical Methods for
Psychology), it DOES control familywise alpha when there are only 3
groups/conditions.
Speaking of Howell, he has a nice discussion of multiple comparisons for
repeated measures designs on his webstie. The link is at the bottom of
this page:
http://www.uvm.edu/~dhowell/StatPages/More_Stuff/Additional.html
I don't think he addresses Duncan's method, but the discussion might
give you some ideas. (I think Duncan's method has also fallen out of
favour, has it not?)
Cheers,
Bruce
--
Bruce Weaver
bwe...@lakeheadu.ca
www.angelfire.com/wv/bwhomedir
Art Kendall
Even if effects are statistically significant are the differences
between levels or differences-of-differences large enoughto be policy /
theoretically / clinically meaningful?
Do you have planned contrasts (specific hypotheses) or are these
post-hoc comparisons?
How many repeated factors?
How many levels in each repeated factor?
Are the repeats crossed? Are the repeats strictly nominal, roughly
ordered, or equal appearing intervals? (if equal appearing or roughly
ordered simply showing the polynomial contrasts significance level and
plots may suffice. The lack of parallel effects may be obvious.)
Are there between subject factors? How many? How many levels?
Are there interactions involving any of the between or within subject
factors? (you may want to use different error terms if you are doing
simple effects.)
Did you try plotting the results? Can the eyeball be all the followup
you need?
If you do put together software to do specific specific approaches you
find in the literature, see if the conclusions are very different from
doing matrix input to ONEWAY.
I came across an interesting presentation on some website, but cannot
locate it at the moment.
Hope this helps.
Art
Are there
Richard Ulrich
On 22 Nov 2005 19:17:48 -0800, neila...@msn.com wrote:
> Hello,
> I need to obtain either LSD or Duncan PostHoc results
> for a Repeated Measures factor. SPSS doesn't provide
> this as an option. Anybody have a solid correct solution?
> Am I stuck with running all possible paired t-tests and
> following up with a bonferroni correction?
- probably.
> Client is currently
> running the analysis as a OneWay (I have informed them
> this is not a correct analysis). What are the consequences
> of this incorrect analysis -error term is wrong of course-.
> WIll the results be invariably conservative or are there situations
> where the results will be overly liberal?
The overall test in repeated measures makes use of a
pooled term which "takes advantage of" the correlation
within a subject, so that test that ignores the correlation
will be conservative if the correlations are positive.
- Repeated measures of one outcome will typically be
positively correlated. Multiple-measured outcomes at one
time point can be negatively correlated. These may be, for
instance, mutually exclusive choices (time spent at A
versus time spent at B ... ) -- so that the oneway test will be
liberal, given those wrong circumstances.
The separate tests are affected by both the correlation and
the variance. Negative correlation will have the same
bad effect. Using total pooled variance instead of using the
separate variances and separate correlations can give errors
in either direction. - The test for 'sphericity' is not very
sensitive in warning of hazards in testing contrasts, unless
you use it (say) at a nominal 0.50 (not 0.05) level; that's the
last thing I read about that.
> I know one can set something up using the SUBJECT as
> a factor and then using the WITHIN*SUBJECT interaction as
> an error term (or something cery close to this).
That gets the overall test. Not so good for contrasts (see above).
> Clearly, if there were a simple solution, I am sure it would already
> be implimented in SPSS. I studied LOTS of advanced Statistics
> YEARS ago and can't seem to find my copies of Bock or Kirk at the
> moment.
There's not a *good* simple solution.
--
Rich Ulrich, wpi...@pitt.edu
http://www.pitt.edu/~wpilib/index.html
Thom
You can get Sidak, Bonferroni or LSD tests via SPSS GLM Repeated
measures commands. Using the menus just click on Options from the
Repeated Measures box ... select the factor and dispaly means for that
factor - and check the compare means box. Once checked you can select
Sidak, Bonferroni or LSD from the menu. [This is from memory ... but it
is pretty easy to work out once you've clicked on the Options button
...]
In general the sources I've seen reccommend a Bonferroni correction or
modified Bonferroni correction (e.g., Holm procedure).
As Howell notes (see Bruce's link) the Bonferroni correction is
conservative. LSD and Duncan are very liberal. I've not seen Duncan's
procedure used in published work. LSD controls Type I error for the
omnibus null hypothesis and therefore offers no protection for the
number of Type I errors that might arise if the omnibus null is not
true. Three means is a special case (see Bruce's comments!).
Thom