FDR q value interpretation

[Ai Zhang]

Hi,

I'm using GSEA Preranked tool with different numbers of permutation. I first tried 1000 permutation 2 weeks ago, and got FDR q value of 0.056 for one gene set. However, I just tried with the same gene ranked list and gene set database, and only changed to 10,000 permutation, then I got FDR q of 0.104 for the same gene set. And, after I changed to 1000 permutation again, FDR q was not 0.056 but 0.152.

I know all of the above results are < 0.25, but under same condition at different time, FDR q values changed a lot. Is this normal? How can we interpret these results?

Thank you,

Ai

[Anthony Castanza]

Hi Ai,

 

GSEA uses a random number seed to initialize the permutation matrix. This seed is printed on the index page under the comments section. It can be supplied to the Seed for Permutation parameter under the Advanced fields section of GSEA which will cause GSEA to reproduce the previous results exactly. What is happening here is that when GSEA is computing permutations on subsequent runs slightly different genes are getting selected to construct the permuted sets resulting in slightly different scores and thus slightly different pValues/qValues when the true score is tested against those permuted scores. Considering the range between 0.056 and 0.152 with a smaller number of permutations (less sampling from the distribution) and with the 10,000 permutation test returning a score of 0.104 (more sampling from the distribution) the true FDR is probably closer to 0.1, which in Gene Set permutation mode we would probably consider to be only marginally significant. The FDR<0.25 recommendation is generally considered for the more stringent phenotype permutation mode and gene set permutation mode (which is what is used in GSEA Preranked) should generally use the more tradition 0.05 threshold.

 

-Anthony

 

Anthony S. Castanza, PhD

Curator, Molecular Signatures Database

Mesirov Lab, Department of Medicine

University of California, San Diego

--
You received this message because you are subscribed to the Google Groups "gsea-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gsea-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gsea-help/2d9ac907-32de-4840-9187-6dd88a1eacbdn%40googlegroups.com.