[edstat] Testing differences between non-independent correlations
David C. Howell
I was asked a question this morning that I can't answer (hardly a new experience). It may be that the answer is obvious and that I just can't see it, or that it is a question that does not have an easy answer.
It is well known how to test the difference between correlations in two independent samples. It is also well known how to test the difference between r(xy) and r(xz) by taking into account the correlation between y and z. But what about the correlation between xy and time 1 and xy at time 2? Presumably we need to take into account the lack of independence between Time 1 and Time 2, which means the correlation between x1 and x2 and the correlation between y1 and y2.
Can anyone tell me how to go about this? Am I just being blind?
Thanks,
Dave Howell
**************************************************************************
Professor Emeritus
University of Vermont
David C. Howell
Phone:
(970) 871-4556
P.O. Box
770059
Steamboat Springs, CO
80477 email:
David....@uvm.edu
http://www.uvm.edu/~dhowell/StatPages/StatHomePage.html
Donald Burrill
Dunno as I have an answer for you, but maybe thinking aloud (so to
speak!) will stimulate an idea or two. Thinking of this in a slightly
more general sense, and assuming that the persons or objects being
correlated are the same at both times (because if they aren't you'd
treat the coefficients as unrelated and wouldn't have a question), we
have a correlation matrix of at least four rows & columns. To avoid
messy subscripts, I'll use t = x at time 1, u = y at time 1, v = x at
time 2, w = y at time 2. The question your querent wants to address
involves comparing r(tu) with r(vw).
(Side comment: formally equivalent, so far as I can see, to asking
whether the pre-post correlation in x, r(tv), differs from the pre-post
correlation in y, r(uw). In any case there are 6 different off-diagonal
values, one pair of which is of particular interest here.)
My first question to your querent would be, "Why do you want to know?"
What difference does it make if the correlations are indistinguishable,
or if the correlation at time 1 exceeds that at time 2, or vice versa?
And if you want to know only so's you can predict r(xy) at time 3, why
not just wait and see (let the universe unfold as it will)?
One interesting possibility occurs to mind: you could orthogonalize v
and w with respect to t and u, yielding variables v* and w*; then
r(tv*) = 0 and r(uw*) = 0, and with a certain amount of hand-waving one
might be able to convince oneself that comparing r(tu) with r(v*w*)
using the independent-samples method may be both (1) legitimate and (2)
useful.
If, as may be the case, the underlying concern is whether r(xy) remains
approximately constant over time, I think of generating r(xy) at a
number of different times. Then I might think of plotting r(xy) as a
function of time, maybe surrounding each value with a suitable
confidence interval, and looking for a trend that might be in danger of
moving r(xy) at a sufficiently later time out of the CI for r(xy) at
time 1. This too is hand-waving, since the CI presumably would
incorporate the assumption that the several times did not entail
troublesome effects from previous times.
And I cannot resist pointing out that if the variables x and y are
positively correlated across time (as one would tend to expect), a test
that assumes independence across times will be less sensitive to real
differences than a test (if one could be found or invented) that took
those temporal correlations into account: that is, it would be
"conservative".
Anyway -- maybe that's worth 2 cents, but I wouldn't gamble on it.
Good luck! -- Don.
On Thu, 25 Mar 2004, David C. Howell wrote:
> I was asked a question this morning that I can't answer (hardly a new
> experience). It may be that the answer is obvious and that I just can't see
> it, or that it is a question that does not have an easy answer.
>
> It is well known how to test the difference between correlations in two
> independent samples. It is also well known how to test the difference
> between r(xy) and r(xz) by taking into account the correlation between y
> and z. But what about the correlation between xy and time 1 and xy at time
> 2? Presumably we need to take into account the lack of independence between
> Time 1 and Time 2, which means the correlation between x1 and x2 and the
> correlation between y1 and y2.
>
> Can anyone tell me how to go about this? Am I just being blind?
------------------------------------------------------------
Donald F. Burrill d...@mv.mv.com
56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
Bruce Weaver
> (Reply sent to DCH and to the list. -- DFB.)
>
> Dunno as I have an answer for you, but maybe thinking aloud (so to
> speak!) will stimulate an idea or two. Thinking of this in a slightly
> more general sense, and assuming that the persons or objects being
> correlated are the same at both times (because if they aren't you'd
> treat the coefficients as unrelated and wouldn't have a question), we
> have a correlation matrix of at least four rows & columns. To avoid
> messy subscripts, I'll use t = x at time 1, u = y at time 1, v = x at
> time 2, w = y at time 2. The question your querent wants to address
> involves comparing r(tu) with r(vw).
> (Side comment: formally equivalent, so far as I can see, to asking
> whether the pre-post correlation in x, r(tv), differs from the pre-post
> correlation in y, r(uw). In any case there are 6 different off-diagonal
> values, one pair of which is of particular interest here.)
---- snip -----
And with the data in that format, I suspect Jim Steiger's
MULTICORR program could probably be cajoled into making the
desired comparison. Here's Steiger's website:
http://www.interchg.ubc.ca/steiger/homepage.htm
I first read about Steiger and correlations, by the way, in
Dave Howell's excellent textbook (Statistical Methods for
Psychology).
Cheers,
Bruce
--
Bruce Weaver
wea...@mcmaster.ca
www.angelfire.com/wv/bwhomedir/
Ray Koopman
news:<5.2.1.1.0.200403...@pop.uvm.edu>...
> To Edstat-L list:
>
> I was asked a question this morning that I can't answer (hardly a new
> experience). It may be that the answer is obvious and that I just can't see
> it, or that it is a question that does not have an easy answer.
>
> It is well known how to test the difference between correlations in two
> independent samples. It is also well known how to test the difference
> between r(xy) and r(xz) by taking into account the correlation between y
> and z. But what about the correlation between xy and time 1 and xy at time
> 2? Presumably we need to take into account the lack of independence between
> Time 1 and Time 2, which means the correlation between x1 and x2 and the
> correlation between y1 and y2.
>
> Can anyone tell me how to go about this? Am I just being blind?
You want the Pearson-Filon test. See Jim Steiger's 1980 Psych.Bull paper.
David Kenny's 1987 book also covers it, but there is a mistake in the formula
on p.280: the denominator should have Q/(1-r^2)^2, not Q(1-r^2)^2.
Peter Chen
Karl L. Wuensch
Karl L. Wuensch, Department of Psychology,
East Carolina University, Greenville NC 27858-4353
Voice: 252-328-4102 Fax: 252-328-6283
Wuen...@mail.ecu.edu
http://core.ecu.edu/psyc/wuenschk/klw.htm
Ray Koopman
news:<004101c41396$d28acb30$80b186ce@DHJC4K01>...
> Raghunathan, Rosenthal, and Rubin (1996, Psychological Methods,
> 1, 178-183) demonstrated that a modified Pearson-Filon statistic,
> the ZPF statistic, is superior to the PF statistic. [...]
I should have said the SPF test -- Steiger's Zbar_2^* (eqn 15)
modification of the Pearson-Filon test -- which improves on
the ZPF test by asserting the null. See Steiger, pp 247 & 250.
IMO, the referees of the RRR paper blew this one.