Selection of cut-offs and thresholds

[Arabidopsis thaliana]

Dear GSEA Help Team / Community,
I have a question regarding cut-offs and thresholds in GSEA analysis. I recently discussed this with my supervisor, and while I understand that the cut-off should be significance based on FDR q-values or nominal p-values, my supervisor insists on additionally using a strict NES cut-off of >1.8.
From my understanding, the NES value is strongly influenced by dataset size and gene set size itself? Is it correct to say that applying a fixed NES cut-off is largely arbitrary?
Additionally, I am curious about the comparability of NES values across different gene set collections. For example, would it be expected that smaller collections like Hallmark generally produce lower NES values compared to larger collections such as C2 for a "true" enrichment?

Thanks in advance for clarification!

[Anthony Castanza]

Hello,

Setting an additional NES cutoff would be largely arbitrary, yes. A cutoff of 1.8 is pretty high, but not unreasonable to picking the strongest enrichment signals in the dataset, and as long as it is done in combination with consideration for standard NOM pValue and FDR cutoffs, it's not inherently scientifically objectionable (although you may be missing weaker signals in the data).
There is a small misconception about the NES here though. The NES of a set is not affected by any other set in the calculation - only the FDR is a global statistic like that. The NES is only affected by a set's own null distribution (either the scoring of random permutations of the samples using the same set in phenotype permutation mode, or random permutations of the genes to construct a null set of identical size in gene set permutation mode). In neither mode does the NES have any correlation with collection size, additionally, because the null is always generated from a set of the same size, the normalization there in fact removes the effect of the gene set size on the enrichment score. What can be adversely impacted by collection size is the FDR statistic, which is why we recommend running GSEA with the lowest-level subcollection applicable to the analysis you're running.

Do let us know if you have any additional questions

-Anthony

Anthony S. Castanza, PhD
Curator, Molecular Signatures Database
Mesirov Lab, Department of Medicine
University of California, San Diego

[Arabidopsis thaliana]

Thanks for the clarification!