What is wid?
Dan
[Just starting to use ez here...]
I'm trying to use ez to do a repeated ANOVA of a setup where I have
three "performance" measures (beginning, middle, end) for each
student.
For example, Student A might have performance scores 4, 5, and 6;
student B might have performance scores 12, 14, 15, etc.
Looking at the ?ezANOVA documentation, I see that I have to pass a
data frame with my columns, then the name of the column with my
"performance" data (dependent variable)... but then I am lost with
this next one:
":
wid .() object specifying the column in data that contains the
variable specifying the case/Ss identifier.
:"
Could you please help explain what this means?
In the ANT data, 'subnum' is used here for 'wid', which is a factor
with 20 levels. But even using that example, I can't seem to
understand why there are 20 levels and what they mean.
Thanks,
Dan
Mike Lawrence
"wid" is supposed to be whatever variable in your data that labels the
units of observation and assignment. That is, when you have a
"between" variable, you are assigning some set of units randomly to
groups, and when you have a "within" variable, you measure each of
multiple units multiple times.
I study psychology, so "person" is a typical unit of
observation/assignment in my work. I often assign people to groups,
which then experience different levels of the variables in the
"between" part of my experimental design. I also often observe the
same individuals repeatedly, in each of the levels of the variables in
the "within" part of my experimental design. Thus, my data usually
includes a column that labels the data according to which person it
came from, and this is the column I'd specify as the value of "wid".
In naming this argument, I went with "wid" instead of "person" (or
"subject", or "participant") because I wanted to take into account the
fact that other domains of science have different units of
observation/assignment; an economist may use households in this way, a
biologist petri disses, etc. The common feature is that you need
unique identifiers for each unit when you have any variables that are
manipulated "within" unit, hence the name "wid" for "within unit
identifier". Having written this out now, I imagine its debatable
whether "uoa" (for "unit of observation and assignment") might be
clearer, but I've already broken a lot of code switching from "sid"
("subject identifier") to the more general (and politically correct)
"wid", so I'm hesitant to switch things up again.
Does that make sense?
Mike
--
Mike Lawrence
Graduate Student
Department of Psychology
Dalhousie University
Looking to arrange a meeting? Check my public calendar:
http://tr.im/mikes_public_calendar
~ Certainty is folly... I think. ~
Dan
Thanks so much. Makes perfect sense. I now understand that the ANT
data has 20 "subjects", each of which results in many many
measurements of the dependent variable.
Now I can run my within-subjects ANOVA.
I have one further question. I'm wondering now if there's a way I can
run the TukeyHST on the aov object that I get from ezANOVA. The aov
variable is the wrong type:
> res = ezANOVA (data=ANT [ANT$error == 0,], dv=.(rt), wid=.(subnum), between=.(group), return_aov=TRUE)
> TukeyHSD (res$aov)
Error in UseMethod("TukeyHSD") :
no applicable method for 'TukeyHSD' applied to an object of class
"c('aovlist'
, 'listof')"
Thanks again,
Dan
Mike Lawrence
Least Significant Difference for post-hoc stuff (see ezStats &
ezPlot). However, it seems that TukeyHSD has trouble when it
encounters an aov object fit with an Error() term in the formula,
which is always the case when ezANOVA does it. For example:
utils::data(npk, package="MASS")
TukeyHSD( aov(yield ~ block + N*P*K, npk) )
npk$sub = factor(1:nrow(npk))
TukeyHSD( aov(yield ~ block + N*P*K + Error(sub), npk) )
The last call to TukeyHSD replicates the error you encountered. The
only way I can think of to fix this is to change ezANOVA so that if
there are no within variables it will omit the Error() term in the aov
formula, thereby yielding an aov object that can be submitted to
TukeyHSD.
On Wed, Feb 16, 2011 at 12:52 PM, Dan <daniel....@gmail.com> wrote:
> Hi Mike
Dan
to interpret that LSD? I have three levels of a within-group variable,
each of which ez shows as having the same LSD. (I suppose this is the
case because the number of observations for each level are equal.)
However, I can't seem to find an explanation of using this single LSD
number to decide which pairs of means are different...
Incidentally [ot], there's a paper you recommended in the ez help
pages:
Bakeman, Recommended effect size statistics for repeated measures
designs
I've searched and searched online, but all links go to a Springer page
from where I can't find this article. Do you have a working link?
Mike Lawrence
If the means differ by more than the FLSD, they may be considered
significantly different. This decision mechanism contstrains the alpha
to .05 if applied when the omnibus ANOVA suggests that you should find
a difference (eg. the effect was significant at .05).
If you use ezPlot, the FLSD is plotted around each point (half an FLSD
on each side of the mean), which permits visualizing all the
comparisons; if a given pair of two points have error bars that fail
to overlap, then the difference between them is greater than the FLSD
and may (according to the rule above) be considered different.
Here's the link to the Bakeman paper:
http://www.springerlink.com/content/210757612263v7h1/
Cheers,
Mike
Daniel Zingaro
Hmm. So if I understand correctly, ezANOVA gives me an exact p-value...
but if I use the LSD returned by ezstats, looking at differences between
means, I am finding significant differences only at alpha = 0.05? Is
there a way to use LSD with a different significance level... or have I
completely missed the boat here :)
Thanks again,
Dan
On 25/02/2011 2:43 PM, Mike Lawrence wrote:
> Hi Dan,
>
Mike Lawrence
computation of the FLSD in ezStats & ezPlot. I'll add this to the next
update, but in the meantime attached is a version of ezStats that lets
you specify the alpha value for the FLSD.
Mike
Thom
On Feb 25, 7:43 pm, Mike Lawrence <Mike.Lawre...@dal.ca> wrote:
> Hi Dan,
>
> If the means differ by more than the FLSD, they may be considered
> significantly different. This decision mechanism contstrains the alpha
> to .05 if applied when the omnibus ANOVA suggests that you should find
> a difference (eg. the effect was significant at .05).
an uncorrected test for a difference (which is probably what most
people want in a plot). However, with more than three means it only
protects against the complete null hypothesis (that all means are
unequal). So it offers very weak Type I error protection - if that is
the goal. Tukey's HSD offers strong Type I error correction
(protection against all partial null hypotheses e.g., if all means
apart from one are equal).
> If you use ezPlot, the FLSD is plotted around each point (half an FLSD
> on each side of the mean), which permits visualizing all the
> comparisons; if a given pair of two points have error bars that fail
> to overlap, then the difference between them is greater than the FLSD
> and may (according to the rule above) be considered different.
>
> Here's the link to the Bakeman paper:http://www.springerlink.com/content/210757612263v7h1/
- it also covers ANCOVA designs and generalized omega squared.
Thom
Mike Lawrence
>
> On Feb 25, 7:43 pm, Mike Lawrence <Mike.Lawre...@dal.ca> wrote:
>> Hi Dan,
>>
>> If the means differ by more than the FLSD, they may be considered
>> significantly different. This decision mechanism contstrains the alpha
>> to .05 if applied when the omnibus ANOVA suggests that you should find
>> a difference (eg. the effect was significant at .05).
>
> I think it is a reasonable choice for plotting - as it is essentially
> an uncorrected test for a difference (which is probably what most
> people want in a plot). However, with more than three means it only
> protects against the complete null hypothesis (that all means are
> unequal). So it offers very weak Type I error protection - if that is
> the goal. Tukey's HSD offers strong Type I error correction
> (protection against all partial null hypotheses e.g., if all means
> apart from one are equal).
I'll look into putting an option to choose FLSD vs THSD in the next
update, which I hope to release in the coming days.
> I like the original Olejnik & Algina (2003) Psych Methods paper better
> - it also covers ANCOVA designs and generalized omega squared.
I'll have to take a look at that again, particularly as I was
considering adding ANCOVA to the next release.
Baguley, Thomas
> From: Mike Lawrence <Mike.L...@dal.ca>
> Date: Sun, 20 Mar 2011 21:46:03 -0300
>
>>> If the means differ by more than the FLSD, they may be considered
>>> significantly different. This decision mechanism contstrains the alpha
>>> to .05 if applied when the omnibus ANOVA suggests that you should find
>>> a difference (eg. the effect was significant at .05).
>>
>> I think it is a reasonable choice for plotting - as it is essentially
>> an uncorrected test for a difference (which is probably what most
>> people want in a plot). However, with more than three means it only
>> protects against the complete null hypothesis (that all means are
>> unequal). So it offers very weak Type I error protection - if that is
>> the goal. Tukey's HSD offers strong Type I error correction
>> (protection against all partial null hypotheses e.g., if all means
>> apart from one are equal).
>
> I'll look into putting an option to choose FLSD vs THSD in the next
> update, which I hope to release in the coming days.
I'm working on a paper about plotting CIs for ANOVA and I think FLSD is a
reasonable option. THSD is OK for plotting differences - but for strong
control methods is quite conservative. The multcomp package implements much
more powerful comparison methods that maintain strong control (notably the
Westfall procedure). The down side is that these can't usefully be plotted -
as they are sequential tests.
>> I like the original Olejnik & Algina (2003) Psych Methods paper better
>> - it also covers ANCOVA designs and generalized omega squared.
>
> I'll have to take a look at that again, particularly as I was
> considering adding ANCOVA to the next release.
The problem with generalized eta-squared and partial eta-squared is that you
need input about what are measured and what are manipulated factors. This
has to be user-supplied and isn't necessarily straight-forward (e.g.,
sometimes experiments manipulate measured variables that are normally
considered measured).
Thom
This email is intended solely for the addressee. It may contain private and confidential information. If you are not the intended addressee, please take no action based on it nor show a copy to anyone. In this case, please reply to this email to highlight the error. Opinions and information in this email that do not relate to the official business of Nottingham Trent University shall be understood as neither given nor endorsed by the University.
Nottingham Trent University has taken steps to ensure that this email and any attachments are virus-free, but we do advise that the recipient should check that the email and its attachments are actually virus free. This is in keeping with good computing practice.
Daniel Zingaro
This post-test discussion has been quite interesting... but due to my
lack of knowledge about these tests, I'm now thoroughly confused as to
which one I might use :)
For each student, I have four measurements of the DV. (I am not
manipulating the DV, just measuring it four times... so I think this is
an example of an observed/measured rather than manipulated variable.)
I do a pure within-ANOVA with ez, and find p < 0.05, so null is
rejected. If I use FLSD at this point, taking each pair of means and
comparing their difference to the FLSD, I find that observation 4 is
different from the other three. (As I was doing this, I wondered if I
was falling into the trap of doing multiple independent post-hoc tests,
which I remember reading can increase the chance of finding something
significant.)
But from this discussion, I've learned that FLSD may not be the best
choice for avoiding type I errors. I assume this means that I could have
found that observation 4 is "different" when that is not actually the
case (even though my null was rejected in the overall ANOVA test?).
I'm not going to be plotting the CIs, so from what I understand I can
decrease type I error by using THSD or Westfall. I can't seem to find an
accessible description of Westfall online, though.
I'd be grateful if someone could 'debug' me here :)
Thanks,
Dan
On 21/03/2011 3:33 AM, Baguley, Thomas wrote:
>
>> From: Mike Lawrence<Mike.L...@dal.ca>
>> Date: Sun, 20 Mar 2011 21:46:03 -0300
>>
>>>> If the means differ by more than the FLSD, they may be considered
>>>> significantly different. This decision mechanism contstrains the alpha
>>>> to .05 if applied when the omnibus ANOVA suggests that you should find
>>>> a difference (eg. the effect was significant at .05).
>>>
>>> I think it is a reasonable choice for plotting - as it is essentially
>>> an uncorrected test for a difference (which is probably what most
>>> people want in a plot). However, with more than three means it only
>>> protects against the complete null hypothesis (that all means are
>>> unequal). So it offers very weak Type I error protection - if that is
>>> the goal. Tukey's HSD offers strong Type I error correction
>>> (protection against all partial null hypotheses e.g., if all means
>>> apart from one are equal).
>>
>> I'll look into putting an option to choose FLSD vs THSD in the next
>> update, which I hope to release in the coming days.
>
> I'm working on a paper about plotting CIs for ANOVA and I think FLSD is a
> reasonable option. THSD is OK for plotting differences - but for strong
> control methods is quite conservative. The multcomp package implements much
> more powerful comparison methods that maintain strong control (notably the
> Westfall procedure). The down side is that these can't usefully be plotted -
> as they are sequential tests.
>
>>> I like the original Olejnik& Algina (2003) Psych Methods paper better
Mike Lawrence
I'd argue that you've encountered a fortunately straight-forward
situation; the omnibus suggests that there is at least one difference
amongst your 4 conditions, and the FLSD has shown you one difference .
I'd go ahead and conclude that your data suggests condition 4 is
different from the other three. A stickler might point out that the
FLSD has *actually* shown you 3 differences (4vs1, 4vs2, 4vs3), and
that FLSD's failure to hold type 1 errors at alpha might mean that one
of these differences might be spurious, in which case you'd maybe want
to soften your conclusion to say that 4 is different from at least one
of the other three groups.
Alternatively, unless you're a politician assigning funding or a
clinician deciding treatment, you should really re-consider employing
statistics that aim for a dichotomous decision with a known controlled
rate of one type of decision error. The alternative, more
scientifically useful approach is to simply represent the relative
strength of evidence for different models of the data. A likelihood
ratio of contrasting a null model versus a model containing an effect
of condition would represent the omnibus evidence for an effect of
condition, and subsequent likelihood ratios could be computed for each
pair of conditions to represent evidence for a difference amongst each
pair. Present the quantification of evidence thus computed, present
your conclusions, and let your readers agree or disagree using their
own subjective interpretations.
Cheers,
Mike
Thom
Daniel, the issue of post hoc testing is a huge minefield.
The traditional approach in psychology is to use multiple comparison
procedures to control Type I errors, but these lead to a number of
well known problems. Mike alludes to one set - which relate to the
problem of interpreting individual tests that you may not be
interested in rather than than directly comparing the models that test
your hypothesis or hypotheses. They can also be inconsistent in
various ways. A further practical difficulty is that in many fields
people rely on test procedures with weak Type I error control or with
low power. So for traditional tests that are more powerful you'd need
something like the multcomp package in R (this comes with references
to detailed examples but there is no easy description of how to run it
that I'm aware of though I cover it in a book I'm writing).
An alternative approach is to switch to using contrasts to test and
compare specific hypotheses (this shifts the approach more towards the
model comparison approach that Mike favours). Even more along those
lines is to specifically compare patterns of means using information
criteria - Dayton's PCIC approach - in essence using AIC, AICc or BIC
to compare competing models with all means equal, all unequal and all
steps in between.
Another - deceptively simple approach - is to report unadjusted mean
comparisons by plotting CIs and interpret the pattern informally (more-
or-less as advocated by Loftus and Masson). This is rather the like
the FLSD approach - but focussing on the pattern and less on formal
tests.
What is appropriate for you will depend on what your hypotheses are,
what discipline you work in and where you are trying to publish. If
you are trying to publish in an area where people favour the
traditional approach you may well have an uphill battle doing anything
other than traditional post hoc type tests. Having said that, the PCIC
approach is known to have good power and Type I error protection
relative to many standard procedures (e.g., Tukey's HSD) and can be
presented that way.
If this is hard to get to grips with that's because (in my opinion)
the whole area is a bit of a mess!
Thom