P value for corspearman

[Diego Javier Zea]

Hi!
How can I get a P value for a Spearman's rho (like in R's cor.test) in Julia?
I'm using StatsBase's corspearman and RCall isn't an option since I'll run the test in parallel.
Best,

[Andreas Noack]

Unfortunately, I don't think this has been implemented yet so either you'd have to save all the correlations and use RCall afterwards or you'd have to implement the test.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Diego Javier Zea]

Can I write a Julia version for HypothesisTests or StatsBase based on  /R-3.3.0/src/library/stats/src/prho.c ? The license of the R's file is:

/*
 *  R : A Computer Language for Statistical Data Analysis
 *  Copyright (C) 2000-2016     The R Core Team
 *  Copyright (C) 2003 The R Foundation
 *  based on AS 89 (C) 1975 Royal Statistical Society
 *
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; either version 2 of the License, or
 *  (at your option) any later version.
 *
 *  This program is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *  GNU General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License
 *  along with this program; if not, a copy is available at
 *  https://www.R-project.org/Licenses/
 *
 */

I was trying to do my own version following: Vettering; Teukolsky; Flannery (1992). Numerical Recipes in C: The Art of Scientific Computing (2nd ed.). p. 640.

But I don't get the same values than R (I get the same number in some situation). My code is:

n = length(x)
ρ = corspearman(x, y)
S = (ρ-1)*(n^3 - n)/(-6)
rank_x,f_x = HypothesisTests.tiedrank_adj(x)
rank_y,f_y = HypothesisTests.tiedrank_adj(y)
meanS = ((n^3 - n)/6) - (f_x/12) - (f_y/12)
sdS = sqrt((((n-1)*(n^2)*((n+1)^2))/36) * (1-(f_x/(n^3 - n))) * (1-(f_y/(n^3 - n))))
cdf(Normal(), (S-meanS)/sdS)

Thanks!

[Andreas Noack]

No. Unfortunately, neither of the sources can be used for an implementation here because of their licenses. You could try to write an implementation based on "The Upper Tail Probabilities of Spearman's rho" without using the actual printed implementation in the paper.

[Diego Javier Zea]

Thanks! 

"For n > 7 a power series expansion in n-1 based on the Edgeworth series approximation (David et al., 1951) is used." 

I'm reading the David's papers, but I don't understand exactly how to perform the approximation... 

I'm going for the R/RCall solution right now.

Best,

[Michael Krabbe Borregaard]

Can you elaborate on why you think an implementation based on the printed formulas in a paper could not be used? That does not sound correct to me (I'm a scientific journal editor). R itself relies heavily on algorithms from published papers, properly cited in the documentation for each function.

[Stefan Karpinski]

An implementation based on formulas in a paper is fine, but an implementation derived from GPL code must be GPL. R is GPL so has no issue with using GPL code.

[Andreas Noack]

The actual Fortran implementation of the algorithm is printed in the paper. That is the part I was talking about. It's not a problem to write an implementation based on the formulas.

On Thu, May 19, 2016 at 4:26 PM, Michael Krabbe Borregaard <mkborr...@gmail.com> wrote:

[Michael Krabbe Borregaard]

Cool, thanks for clearing that up. I am home without my uni VPN so could only read the first page of the paper...

[Diego Javier Zea]

I sent my implementation of Spearman test to HypothesisTests. It's based on the formulas (but not in the code) of Numerical Recipes and other references:

https://github.com/JuliaStats/HypothesisTests.jl/pull/53

Best,