Gmail Calendar Documents Reader Web more »
Recently Visited Groups | Help | Sign in
Google Groups Home
independent regression for false dicovary rate
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  7 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
oslo  
View profile  
 More options Nov 11, 1:05 pm
Newsgroups: comp.soft-sys.sas
From: hok...@yahoo.com (oslo)
Date: Wed, 11 Nov 2009 10:05:26 -0800
Local: Wed, Nov 11 2009 1:05 pm
Subject: independent regression for false dicovary rate
Dear SAS-L users;=0A=A0=0AI have about 36000 independent variables (x1-x360=
00) and one dependent variable (y). I need to run 36000 independent single =
regression models to use their p values in MULTTEST. So;=0A=A0=0Aa) Is ther=
e any pratical way to run 36000 independent single regression at the same t=
ime=0Ab) Is there any way to pick only p values from 36000 independent regr=
ession model?=0A=A0=0AHelps would be greatly appriciated,=0A=A0=0AOslo=0A=
=0A=0A

    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Sigurd Hermansen  
View profile  
 More options Nov 11, 4:23 pm
Newsgroups: comp.soft-sys.sas
From: HERMA...@WESTAT.COM (Sigurd Hermansen)
Date: Wed, 11 Nov 2009 16:23:06 -0500
Local: Wed, Nov 11 2009 4:23 pm
Subject: Re: independent regression for false dicovary rate
Oslo:
Let me ask a naïve question if you will ...

How do you know that any and all linear combinations of 36000 variables are independent? I've seen observed linear dependence with significant p-values in much smaller sets of randomly selected variables.
S


    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
oslo  
View profile  
 More options Nov 11, 7:20 pm
Newsgroups: comp.soft-sys.sas
From: hok...@yahoo.com (oslo)
Date: Wed, 11 Nov 2009 16:20:19 -0800
Local: Wed, Nov 11 2009 7:20 pm
Subject: Re: independent regression for false dicovary rate
=0A=0ASigurd;=0A=0AYes I am expecting linear dependence as well. However I =
am 36000 independent variable and only 350 observations. So I have no chanc=
e to run mutliple regression with stepwise or else. I can run LASSO or LAR1=
. However I need select independent variables=A0for construct genetic algor=
itm=0Amodel =0A=0AThanks,=0A=0AOslo=0A=0A________________________________=
=0AFrom: Sigurd Hermansen <HERMA...@WESTAT.COM>=0ATo: SA...@LISTSERV.UGA.ED=
U=0ASent: Wed, November 11, 2009 3:23:06 PM=0ASubject: Re: independent regr=
ession for false dicovary rate=0A=0AOslo:=0ALet me ask a na=EFve question i=
f you will ...=0A=0AHow do you know that any and all linear combinations of=
 36000 variables are independent? I've seen observed linear dependence with=
 significant p-values in much smaller sets of randomly selected variables.=


    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Arthur Tabachneck  
View profile  
 More options Nov 11, 8:44 pm
Newsgroups: comp.soft-sys.sas
From: art...@NETSCAPE.NET (Arthur Tabachneck)
Date: Wed, 11 Nov 2009 20:44:33 -0500
Local: Wed, Nov 11 2009 8:44 pm
Subject: Re: independent regression for false dicovary rate
Oslo,

Can you get what you want by running simple pearson correlation
coefficients?  If so, then you could just use something like:

ods output FisherPearsonCorr=corr (keep=var pvalue);
proc corr data=sashelp.class fisher ( biasadj=no )
   nocorr noprint noprob nosimple;
  var height--weight;
  with age;
run;

and get what you want from the resulting work.corr file.
HTH,
Art
--------

On Wed, 11 Nov 2009 16:20:19 -0800, oslo <hok...@YAHOO.COM> wrote:

Sigurd;

Yes I am expecting linear dependence as well. However I am 36000 independent
variable and only 350 observations. So I have no chance to run mutliple
regression with stepwise or else. I can run LASSO or LAR1. However I need
select independent variables for construct genetic algoritm
model

Thanks,

Oslo

________________________________
From: Sigurd Hermansen <HERMA...@WESTAT.COM>
To: SA...@LISTSERV.UGA.EDU
Sent: Wed, November 11, 2009 3:23:06 PM
Subject: Re: independent regression for false dicovary rate

Oslo:
Let me ask a na ve question if you will ...

How do you know that any and all linear combinations of 36000 variables are
independent? I've seen observed linear dependence with significant p-values
in much smaller sets of randomly selected variables.
S


    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
oslo  
View profile  
 More options Nov 11, 11:44 pm
Newsgroups: comp.soft-sys.sas
From: hok...@yahoo.com (oslo)
Date: Wed, 11 Nov 2009 20:44:30 -0800
Local: Wed, Nov 11 2009 11:44 pm
Subject: Re: independent regression for false dicovary rate
Dear Arthur;=0A=0AThanks so much. I appreciate for your help. I have checke=
d. It worked perfectly. For example the pvalue from proc corr is 0.1299 whi=
le pvalue from proc reg is 0.1298.=A0=0A=A0=0ARegards,=0A=0AOslo=0A=0A=0A=
=0A________________________________=0AFrom: Arthur Tabachneck <art297@NETSC=
APE.NET>=0ATo: SA...@LISTSERV.UGA.EDU; hok...@YAHOO.COM=0ASent: Wed, Novemb=
er 11, 2009 7:44:33 PM=0ASubject: Re: independent regression for false dico=
vary rate=0A=0AOslo,=0A=0ACan you get what you want by running simple pears=
on correlation=0Acoefficients?=A0 If so, then you could just use something =
like:=0A=0Aods output FisherPearsonCorr=3Dcorr (keep=3Dvar pvalue);=0Aproc =
corr data=3Dsashelp.class fisher ( biasadj=3Dno )=0A=A0 nocorr noprint nopr=
ob nosimple;=0A=A0 var height--weight;=0A=A0 with age;=0Arun;=0A=0Aand get =
what you want from the resulting work.corr file.=0AHTH,=0AArt=0A--------=0A=
On Wed, 11 Nov 2009 16:20:19 -0800, oslo <hok...@YAHOO.COM> wrote:=0A=0A>=

=0A=0ASigurd;=0A=0AYes I am expecting linear dependence as well. However I =
have=A036000 independent=0Avariable and only 350 observations. So I have no=
 chance to run mutliple=0Aregression with stepwise or else. I can run LASSO=
 or LAR1. However I need=0Aselect independent variables=EF=BF=BDfor constru=
ct genetic algoritm=0Amodel=0A=0AThanks,=0A=0AOslo=0A=0A___________________=
_____________=0AFrom: Sigurd Hermansen <HERMA...@WESTAT.COM>=0ATo: SAS-L@LI=
STSERV.UGA.EDU=0ASent: Wed, November 11, 2009 3:23:06 PM=0ASubject: Re: ind=
ependent regression for false dicovary rate=0A=0AOslo:=0ALet me ask a na=EF=
=BF=BDve question if you will ...=0A=0AHow do you know that any and all lin=
ear combinations of 36000 variables are=0Aindependent? I've seen observed l=
inear dependence with significant p-values=0Ain much smaller sets of random=


    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Sigurd Hermansen  
View profile  
 More options Nov 12, 3:03 pm
Newsgroups: comp.soft-sys.sas
From: HERMA...@WESTAT.COM (Sigurd Hermansen)
Date: Thu, 12 Nov 2009 15:03:57 -0500
Local: Thurs, Nov 12 2009 3:03 pm
Subject: Re: independent regression for false dicovary rate
Oslo:
Without diminishing Art's artful solution, I'll suggest that the reason SAS doesn't have a more direct method of passing large numbers of p-values to MULTTEST likely is that combining information from such a large number of two-way correlations 1) doesn't encompass linear combinations of more than two variables; and 2) won't adjust estimates sufficiently to account for risks of false discoveries.

Far less naïve questions than mine and a better guide to correlations of many column variables appears at
http://www-stat.stanford.edu/~ckirby/brad/papers/2008ASetofMicroarray... .

Efron's recent articles on control of false discovery rates take us close to the forefront of that field. It may give you more defensible results if you consider alternatives to MULTTEST before spending much time trying to implement that procedure.
S


    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
oslo  
View profile  
 More options Nov 12, 5:25 pm
Newsgroups: comp.soft-sys.sas
From: hok...@yahoo.com (oslo)
Date: Thu, 12 Nov 2009 14:25:58 -0800
Local: Thurs, Nov 12 2009 5:25 pm
Subject: Re: independent regression for false dicovary rate
Dear Sigurd;=0A=0AThanks for ideas and paper. Both are very helpful.=0A=0AR=
egards,=0A=0AOslo=0A=0A=A0=0A=0A=0A=0A________________________________=0AFr =
om: Sigurd Hermansen <HERMA...@WESTAT.COM>=0ATo: SA...@LISTSERV.UGA.EDU=0AS=
ent: Thu, November 12, 2009 2:03:57 PM=0ASubject: Re: independent regressio=
n for false dicovary rate=0A=0AOslo:=0AWithout diminishing Art's artful sol=
ution, I'll suggest that the reason SAS doesn't have a more direct method o=
f passing large numbers of p-values to MULTTEST likely is that combining in=
formation from such a large number of two-way correlations 1) doesn't encom=
pass linear combinations of more than two variables; and 2) won't adjust es=
timates sufficiently to account for risks of false discoveries.=0A=0AFar le=
ss na=EFve questions than mine and a better guide to correlations of many c=
olumn variables appears at=0Ahttp://www-stat.stanford.edu/~ckirby/brad/pape=
rs/2008ASetofMicroarrays.pdf .=0A=0AEfron's recent articles on control of f=
alse discovery rates take us close to the forefront of that field. It may g=
ive you more defensible results if you consider alternatives to MULTTEST be=
fore spending much time trying to implement that procedure.=0AS=0A=0A=0A=0A=


    Reply    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2009 Google