Re: Comparing Spearman Correlation Coefficients

1842 views
Skip to first unread message

Graham Ashe

unread,
Nov 7, 2008, 2:26:00 AM11/7/08
to
I'm tempted to use the Fisher r-to-z transformation here:

http://faculty.vassar.edu/lowry/rdiff.html

However, I'm not sure this is the right test.

David Jones

unread,
Nov 7, 2008, 5:19:04 AM11/7/08
to

I suggest you search for an old thread in this newsgroup titled "comparing two Spearman's rho coefficients", starting 30 Nov 2006. This is possibly accesible at

http://groups.google.com/group/sci.stat.math/browse_frm/thread/9ce8c39e74b9ab06/07aff45c6714ef85?hl=en aff45c6714ef85

David Jones

Graham Ashe

unread,
Nov 7, 2008, 6:42:30 AM11/7/08
to
> Graham Ashe said:
>
>
> *** Date: Nov 2, 2008 2:42 AM
> Author: Graham Ashe
> Subject: Comparing Spearman Correlation Coefficients
>
> I have two Spearman correlation coefficients I'd like
> to compare (to see if the difference is statistically
> significant). The same data sets were used in both
> cases. Only the evaluation criteria changed. What's
> the best method of comparing these correlations?
> Thanks. ***
>
>
> My response
>
> ******
>
> doi.wiley.com/10.1002/0471667196.ess5050 See (though
> you didn’t yet).
>
> Abstract
> Testing the equality of two population correlation
> coefficients when the data are bivariate normal and
> Pearson correlation coefficients are used as
> estimates of the population parameters is a
> straightforward procedure covered in many
> introductory statistics courses. The coefficients are
> converted using Fisher's z-transformation with
> standard errors (N – 3)–1/2. The two transformed
> values are then compared using a standard normal
> procedure. When data are not bivariate normal,
> Spearman's correlation coefficient rho is often used
> as the index of correlation. Comparison of two
> Spearman rhos is not as well documented. Three
> approaches were investigated using Monte Carlo
> simulations. Treating the Spearman coefficients as
> though they were Pearson coefficients and using the
> standard Fisher's z-transformation and subsequent
> comparison was more robust with respect to Type I
> error than either ignoring the nonnormality and
> computing Pearson coefficients or converting the
> Spearman coefficients to Pearson equivalents prior to
> transformation. ***
> Is this useful?
>
> Luis A. Afonso

If I understood that last paragraph correctly, I think it translates to:

"Yes, Fisher's z-transformation is probably the right test".

Thanks. ;)

John Uebersax

unread,
Nov 7, 2008, 8:06:53 AM11/7/08
to
On Nov 7, 12:42 pm, Graham Ashe <knight_arm...@yahoo.com> wrote:

> If I understood that last paragraph correctly, I think it translates to:
>
> "Yes, Fisher's z-transformation is probably the right test".

That would seem consistent with the fact that the Spearman correlation
coefficient is the same as the Pearson correlation coefficient
calculated on the ranks of two variables. That is, in a sense, the
Spearman correlation *is* a Pearson correlation of ranks.

John Uebersax PhD

Peter

unread,
Nov 7, 2008, 8:44:34 AM11/7/08
to
Not too fast... Look at what is written on that page you gave the link for: http://faculty.vassar.edu/lowry/rdiff.html
"Using the Fisher r-to-z transformation, this page will calculate a value of z that can be applied to assess the significance of the difference between two correlation coefficients, ra and rb, found in two independent samples."
Rewind: "found in two independent samples"... In your original post you wrote that your spearman rho's come from one and the same sample in which some criterion was changed or so. I might be wrong but... I don't think the same rules apply. Maybe you can give some more info about the kind of data you have? Typical sample size? Kind of variables (metric, ordinal,...)? In what way did the evaluation criterion change?

P.

Graham Ashe

unread,
Nov 7, 2008, 9:39:43 AM11/7/08
to

I have:

1) 80 human-rated aesthetic values of a set of items.

2) 80 computer-rated aesthetic values of the same set of items using a particular set of evaluation criteria.

3) 80 computer-rated aesthetic values of the same set of items using a modified set of evaluation criteria.

I am trying to see if the difference in Spearman correlation coefficients for 1 and 2, and 1 and 3 is statistically significant. The aesthetic values are precise to one decimal point.

The Fisher's z-transformation appears a little too straightforward. If one of the r values isn't "significantly" higher than the other, the difference isn't statistically significant (i.e. P>0.05).

I am curious to know if a small difference in the Spearman correlation coefficients could actually be significant. I suppose it can't.

Art Kendall

unread,
Nov 7, 2008, 10:24:13 AM11/7/08
to
What is your response scale on the aesthetic values?

Why do you think the intervals are severely discrepant from equal so
that you need nonparametric?

Art Kendall
Social Research Consultants

Graham Ashe

unread,
Nov 7, 2008, 10:46:45 AM11/7/08
to
> What is your response scale on the aesthetic values?
>

1 to 10 for the human ratings. 0 to 7 for the computer ratings. There is actually no upper limit for the computer's assessment, but none have exceeded 7.

> Why do you think the intervals are severely
> discrepant from equal so
> that you need nonparametric?

I didn't want to assume that the distributions were normal. Spearman seemed like a stronger test of correlation, in any case.

Bruce Weaver

unread,
Nov 7, 2008, 12:03:13 PM11/7/08
to


Art didn't ask if the distributions were (approximately) normal. He
asked if it is reasonable to treat the data as if there are equal
intervals all along the scale (i.e., in terms of the thing you are
measuring, is the difference between 1 and 2 approximately the same as
the difference between 2 and 3, 3 and 4, etc all the way up the
scale).

--
Bruce Weaver
bwe...@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/
"When all else fails, RTFM."

Graham Ashe

unread,
Nov 7, 2008, 12:37:56 PM11/7/08
to


Yes, I'm treating the aesthetic scores as interval data for both the human and computer ratings. In other words, the difference between 1 and 2 is the same as between 3 and 4 etc.

I don't see how this matters, though. Should it affect my initial choice of correlation test (i.e. Pearson or Spearman)?

Or does it perhaps influence what I'm trying to do now i.e. determine if the difference in Spearman correlations is statistically significant using the Fisher r-to-z transformation?

Bruce Weaver

unread,
Nov 7, 2008, 2:04:08 PM11/7/08
to


See the Multicorr program on Jim Steiger's website.

http://www.statpower.net/

RichUlrich

unread,
Nov 7, 2008, 2:52:50 PM11/7/08
to


An important fact is introduced above. The correlations are
*not* independent. The test for independent correlations
does not apply, unless you want to throw away a *lot* of power.

There are a couple of tests for "correlated correlations", which
Googling < group:sci.stat.* "correlated correlations" > will find.

If the question is whether (2) adds to (3) or vice-versa, it
could be more direct to see if the either one adds to the
prediction achieved by the other -- That is, when you
put them both in a regression predicting (1), do you see one
or two significant "partial regression coefficients", or none?

Before you jump to that, consider one more thing --
If these are two computer models with a tiny modification,
then you may have a lot of individual scores with no-change.
This effectively reduces the d.f. for *whatever* is taking place.
The *proper* test looks only at the changed predictions, however
you elect to look.

In any case, you probably want to compute the difference
between the scores for (2) and (3). How is *this* distributed?
This is what underlies any test that you will.


--
Rich Ulrich

Reply all
Reply to author
Forward
0 new messages