Hi Martin,
My apologies for missing your question it somehow slipped through the cracks. Unfortunately I can’t really give hard recommendations for proteomics data as this isn’t an area we have explicit support for, but I hope these answers help somewhat.
The key question is, for gene set permutation for a given expression dataset, and a given gene set of a particular size, can I sample from the distribution unique random sets of that size enough times to construct a valid null distribution. For expression datasets from microarrays or RNA-seq where you have 10’s of thousands of genes, that answer is always yes. For small datasets like proteomics, your chances to construct materially different sets to produce a valid null are much lower. That said, if this is failing, I would actually expect the significance statistics to be worse since the null scores would be more similar to the true scores.
As a general case, gene set permutation will (almost) always produce more “significant” pValues/FDRs than phenotype permutation. The phenotype permutation test is just substantially more stringent and does a better job of preserving gene-gene correlation effects that can impact the scores (and should be accounted for), whereas gene set permutation breaks these correlations. We generally recommend the phenotype permutation test if there are enough samples to perform it as any significant results from it are much more likely to be robust findings. Its at least partially for this reason that we recommend a FDR<=0.25 cutoff for data run with phenotype permutation, but the standard FDR<=0.05 cutoff for data run with gene set permutation.
Sorry I couldn’t be of more help, let me know if you have follow-up questions and I’ll answer as best I can.
-Anthony
Anthony S. Castanza, PhD
Curator, Molecular Signatures Database
Mesirov Lab, Department of Medicine
University of California, San Diego
--
You received this message because you are subscribed to the Google Groups "gsea-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gsea-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gsea-help/17ffe58e-5d50-4e96-a981-37e3b9508e1fn%40googlegroups.com.