Multiple comparisons clarification

[Audrey Hilk]

Hi,

I am currently using methylKit to analyze a dataset from two cell lines, each with three treatment groups. I read through the thread describing comparing multiple treatment groups and did have success subsetting with reorganize() and running the pairwise comparisons I am interested in. However, I am a little worried about the statistics here.

To clarify, is it ok to use the q-values given by methylKit? I am doing quite a few different comparisons here and am not really sure how this is handled by methylKit. Or is it better to combine the output data, run Bonferroni, and then split things out again?

Thank you very much for your help!

[alex....@gmail.com]

Hi Audrey,

Both approaches are valid, but your choice should depend on your plans for downstream processing. 

a) If you are following up on each comparison separately, you can use the methylKit q-values, which are corrected for multiple tests across the CpGs using the SLIM method (https://academic.oup.com/bioinformatics/article/27/2/225/286449). 

b) If you plan to combine the comparison results for a joint interpretation, you could proceed with the combine-correct-split method, probably using the less conservative BH correction (FDR). You could also simply lower your significance threshold to apply Bonferroni correction (https://www.graphpad.com/support/faq/point-of-confusion-alpha-with-bonferroni-correction/), though this would be more stringent the more comparisons you perform (3-6 comparisons should be fine).

Alternatively, based on your hypothesis and your setup, this article may provide some guidance on your choice (https://link.springer.com/article/10.1007/s11229-021-03276-4).

Best,

Alex