Path analysis: Discrepancy between bootstrapped CI and p-value

[atrs2]

Dear Lavaan group,

I am using a path modeling approach to estimate indirect effects in the model (mediation), using 95% bootstrapped CI

Code: 

model=lavaan::sem(model1, data=dat, meanstructure=TRUE,  missing="fiml",se="boot",bootstrap=10000)

> parameterestimates(model, boot.ci.type = "bca.simple",standardized=TRUE)

              lhs op                rhs     label    est    se      z                    pvalue ci.lower ci.upper std.lv std.all std.nox

24      indirect0 :=          a1*b1 indirect0   0.027 0.046  0.576        0.565   -0.053    0.135   0.027   0.032   0.005

25      indirect1 :=      a1*d21*b2 indirect1   0.073 0.036  2.063     0.039    0.026    0.172   0.073   0.088   0.013

26      indirect2 :=          a2*b2 indirect2   0.057 0.037  1.567         0.117    0.003    0.155   0.057   0.068   0.010

   

Across multiple models, I get a p-value that is >.10 but the CI does not include a 0. 

I realize this may not be a Lavaan question but would appreciate any help with interpretation/reporting--if specific references could be suggested along with some feedback, that would be very helpful.

Thank you.

[atrs2]

Dear Terrence et al.

While looking a bit further into my question, I located your response to a same question here:

https://groups.google.com/forum/#!searchin/lavaan/ci$20and$20p-value%7Csort:date/lavaan/1zeHeDCpqtY/U7LBUTp6BgAJ

I'm following up as I've also this conversation in the Mplus forum:

https://groups.google.com/forum/#!searchin/lavaan/ci$20and$20p-value%7Csort:date/lavaan/1zeHeDCpqtY/U7LBUTp6BgAJ

where they note:

1) in the case of p-value and CI disagreement, asymmetric CI should be used--> I believe Lavaan bca CI does provide asymmetric CI?

2) a recent recommendation against using bca CI for testing indirect effect (e.g., based in part on Biesanz et al., 2010) (with the "regular" CI method being more conservative)

My follow-up question:

Though not familiar with the recent recommendations or literature on this topic, my results appear to suggest that the p-values are more conservative estimates that align with my predictions and that the bca CI's indicate significant indirect effect for far more models than I would expect (i.e., less conservative). 

If I wish to heed the recent recommendation (against bca CI) and try a different CI, what option would be suitable within Lavaan? I see that se=robust was part of the discussion in another thread but wanted to get your opinion.

[Terrence Jorgensen]

1) in the case of p-value and CI disagreement, asymmetric CI should be used--> I believe Lavaan bca CI does provide asymmetric CI?

It provides percentile CIs by default (which can be asymmetric).  You can request bias-corrected CIs in the parameterEstimates() function (see help page)

2) a recent recommendation against using bca CI for testing indirect effect (e.g., based in part on Biesanz et al., 2010) (with the "regular" CI method being more conservative)

By "regular" do you mean the default CI in lavaan when bootstrapping is not requested?  That is the delta method, and yes it works well in larger samples.  Bootstrapping also tends to work better in larger N, but Preacher and friends tend to keep recommending it over the delta method in smaller samples.  If your N < 150, I would instead recommend the Monte Carlo CI (parametric bootstrap), which Preacher and colleagues also recommend in complex scenarios that prohibit easy bootstrapping (e.g., multilevel data, but also if you have multiple imputed data, or models that simply take a long time to converge).  You can find a Monte Carlo CI computer in the semTools package (?monteCarloMed), but if N > 150 all these methods tend to give the same result.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[David Adedia]

Dear lavaan community,

I am using simsem package for simulation, is it possible to have the individual fit indices for the various data sets simulated and not the averages alone?

Thank you for your responses.

Regards,

DMA.

Virus-free. www.avg.com

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

--

David Adedia

Department of Basic Sciences

University of Health and Allied Sciences

Private Mail Bag 31

Ho, Ghana

Tel: +233244845294

[Alex Schoemann]

There sure is. The inspect() function allows you to pull information from a simResult object (created while running the sim function). If your object is call out then inspect(out, "fit") will get you fit information from each replication.