Enquiries about multiple comparisons

[Heather Chen]

Dear developers,

Thank you for your work in promoting estimation statistics and in developing the web application and code to make estimation statistics so accessible for all. Our experimental design includes several groups: Injury + treatment; Sham + treatment: Injury + vehicle; Sham + vehicle. While analysing our data with estimation stats, we have found that it is difficult to perform multiple comparisons with estimation stats for data with both sham and vehicle controls. This resulted in running the same analysis each time with a different group as the assigned control. We were wondering if you have any recommendations or advises for such situations and whether there is a way to control for the multiple comparisons?

Thank you.

 Kind regards,

Heather

[Heather Chen]

Hi there,

Regarding multiple comparisons, are corrections such as Bonferroni or Holms adjustments needed for the confidence intervals gained from estimations statistics? If corrections are recommended, how can the adjusted confidence intervals be reflected in the effect size graphs?

Kind regards,

Heather

[Adam Claridge-Chang]

Hi Heather,

We don't recommend any kind of adjustment, since this isn't significance testing.

You are able to increase the confidence interval to 99.5% or some other value, which will lengthen the error bars on the ∆ curve.

Adam

PS Somehow my earlier reply to your first question got lost by Google Groups; the two options were:

• Plot the four groups using the Shared-Control option, choosing one group as the shared control.
  
• Use the dev version of the python code to plot a ∆∆ graph to isolate the injury x treatment effect.
  

[Heather Chen]

Thank you so much for answering! :)