Multiple testing correction

[Seungjin]

Hello,
I am not familiar with statistics and involved in human genetics study.
We did candidate gene capture sequencing followed by SKAT analysis.
I am wondering how people do multiple correction after SKAT analysis.
It seems Bonferroni correction is too strict for my SKAT analysis result with limitation of sample numbers.
No gene survived after Bonferroni correction.
Would you suggest any other way to prove that the result is reliable?
I appreciate your help.

Best,
Seungjin

[Seunggeun (Shawn) Lee]

Hi Seungjin,

People usually use a Bonferroni correction to identify the significant genes. If you feel it is too stringent, 

you can use resamplings (i.e. permutation) to control family-wise error rates. You can find an example code in the package vignettes (page 9).

Thanks,
Shawn

[Seungjin]

Dear Shawn,
Thank you for your response.

Best,
Seungjin

[Seungjin]

Dear Shawn,
When I tried the script in page 9, that was not working because it's for SSD file format.
Last time, I tried obj<-SKAT_Null_Model(y ~ 1, out_type="D", n.Resampling=1000, type.Resampling="bootstrap") for every gene.
As a result, they got all "FALSE" for  "is_smaller" function.
I think they all didn't pass the threshold.
But, I am wondering how I can set the FWER as Resampling_FWER(out,FWER=0.5) in case I don't use SSD file.
Please let me know.
Thank you.

Best,
Seungjin

[Seunggeun (Shawn) Lee]

Hi Seungjin,

I've attached a R-code file that has a function to convert SKAT outputs to SKAT_SSD_ALL output object. You can find the example code below. Please let me know if you have any questions.

######################################

#

library(SKAT)

data(SKAT.example)

attach(SKAT.example)

# This R file has Mk_SSD_ALL_Output function.

source("./MK_SSD_All_Output.R")     

         

# continuous trait

obj<-SKAT_Null_Model(y.c ~ 1, out_type="C", n.Resampling=1000)

out.list<-list()

# make list of SKAT output

# SKAT.example only has one set, so these ten outputs are the exactly same. 

 

for(i in 1:10){

out.list[[i]]<-SKAT(Z, obj)

} 

#

# MK_SSD_ALL_Output: input is a list object of SKAT outputs. 

#

#

out.ssd<-Mk_SSD_ALL_Output(out.list)

Resampling_FWER(out.ssd)

[Nicholas Bowker]

Hi Shaun,

I am also wondering about Multiple testing corrections. I would like to implement FDR correction instead of the FWER correction used in the package. I have done this using other packages, however, I was wondering what the effect of resampling the p.value was in the construction of both the null model (MomentAdjust - because my sample size is 20) and in the SkatBinary test where it resamples 2E06 times? Does this not influence the subsequent multiple testing correction by being too stringent?

Sorry if I'm on the wrong track here, but I'm not a statistician by trade and trying to get my head around this is making me more confused?
I guess what I really want to know is if resampling is conducted during the test, is multiple testing correction really needed?

Thanks for your help,

Nicholas

[Seunggeun (Shawn) Lee]

Hi Nicholas, 

Both MomentAdjust and SKATBinary use resampling to get better (individual test) p-values. They don't do multiple test correction. You still need multiple test correction if you are testing more than one test (i.e more than one gene or region). 

Thanks,

Shawn

[Nicholas Bowker]

Hi Shaun,

Thanks for the help, I have since had a look at my data set and through using FWER correction through resampling one of the genes that I tested is significant. I was wondering if there was a method to use FDR correction to correct for multiple testing that would be less strict? For example an R package I could plug my raw p-values into and get adjusted p-values out as a result?

Thanks for your help,

Nick

[Seunggeun (Shawn) Lee]

Hi Nick,

You can use p.adjust function in R if you want to use Benjamini-Hochberg (BH) procedure. There exist methods to control FDR using resampling, but I don't aware any R-package to do this. 

Thanks,

Shawn