bootstrap and significance

[CSM]

Dear all,

I have a basic question regarding bootstrapping and significance: in the analysis of a mediator model, the indirect effect provided a 95% bootstrap CI = [-.006 , .855], and a bootstrap based p-value =.032. Just as additional information, using se="robust" instead of se="boot" provides p-value = .041, 95% robust CI = [.022, 1.016].

The significance level of my general analysis is set at 5%. As far as I understand, p-values are not important to assess the significance in this bootstrapping case, correct?

So, please, confirm me if it is better to report this effect as non-significant or as marginally significant (since 0 is approaching one of the 95% CI limit). How do you recommend me to report it?

Any help will be much appreciated!

[Terrence Jorgensen]

As far as I understand, p-values are not important to assess the significance in this bootstrapping case, correct?

The p value is calculated using the bootstrap SE, and assumes that the sampling distribution is normal.  To the degree to which that assumption is false, the 95% CI will disagree with the p value.  The 95% CI makes no assumption about the shape of the sampling distribution, so it should be more robust.

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

[CSM]

Dear Professor Terrence Jorgensen,

Thank you very much for this clarification. I understand that I must report only 95% CI. Just as a curiosity, do you know why the 95% bootstrap CI is even more robust (or at least, more conservative) than the 95% robust CI ("SE=robust")?

[Terrence Jorgensen]

do you know why the 95% bootstrap CI is even more robust (or at least, more conservative) than the 95% robust CI ("SE=robust")?

I'm not sure it generally is.  But I think a reason for preferring the CI over a p value is that the CI does not assume a normal sampling distribution, so it should capture the true parameter 95% of the time even when the distribution is asymmetric or kurtotic.