Path analysis: Discrepancy between bootstrapped CI and p-value

442 views
Skip to first unread message

atrs2

unread,
Mar 12, 2019, 6:41:10 PM3/12/19
to lavaan
Dear Lavaan group,

I am using a path modeling approach to estimate indirect effects in the model (mediation), using 95% bootstrapped CI

Code: 
model=lavaan::sem(model1, data=dat, meanstructure=TRUE,  missing="fiml",se="boot",bootstrap=10000)

> parameterestimates(model, boot.ci.type = "bca.simple",standardized=TRUE)
              lhs op                rhs     label    est    se      z                    pvalue ci.lower ci.upper std.lv std.all std.nox
24      indirect0 :=          a1*b1 indirect0   0.027 0.046  0.576        0.565   -0.053    0.135   0.027   0.032   0.005
25      indirect1 :=      a1*d21*b2 indirect1   0.073 0.036  2.063     0.039    0.026    0.172   0.073   0.088   0.013
26      indirect2 :=          a2*b2 indirect2   0.057 0.037  1.567         0.117    0.003    0.155   0.057   0.068   0.010
   
Across multiple models, I get a p-value that is >.10 but the CI does not include a 0. 

I realize this may not be a Lavaan question but would appreciate any help with interpretation/reporting--if specific references could be suggested along with some feedback, that would be very helpful.

Thank you.

atrs2

unread,
Mar 13, 2019, 10:05:20 AM3/13/19
to lavaan

Dear Terrence et al.

While looking a bit further into my question, I located your response to a same question here:


I'm following up as I've also this conversation in the Mplus forum:

where they note:
1) in the case of p-value and CI disagreement, asymmetric CI should be used--> I believe Lavaan bca CI does provide asymmetric CI?

2) a recent recommendation against using bca CI for testing indirect effect (e.g., based in part on Biesanz et al., 2010) (with the "regular" CI method being more conservative)

My follow-up question:

Though not familiar with the recent recommendations or literature on this topic, my results appear to suggest that the p-values are more conservative estimates that align with my predictions and that the bca CI's indicate significant indirect effect for far more models than I would expect (i.e., less conservative). 

If I wish to heed the recent recommendation (against bca CI) and try a different CI, what option would be suitable within Lavaan? I see that se=robust was part of the discussion in another thread but wanted to get your opinion.

Terrence Jorgensen

unread,
Mar 21, 2019, 6:15:13 AM3/21/19
to lavaan
1) in the case of p-value and CI disagreement, asymmetric CI should be used--> I believe Lavaan bca CI does provide asymmetric CI?

It provides percentile CIs by default (which can be asymmetric).  You can request bias-corrected CIs in the parameterEstimates() function (see help page)

2) a recent recommendation against using bca CI for testing indirect effect (e.g., based in part on Biesanz et al., 2010) (with the "regular" CI method being more conservative)

By "regular" do you mean the default CI in lavaan when bootstrapping is not requested?  That is the delta method, and yes it works well in larger samples.  Bootstrapping also tends to work better in larger N, but Preacher and friends tend to keep recommending it over the delta method in smaller samples.  If your N < 150, I would instead recommend the Monte Carlo CI (parametric bootstrap), which Preacher and colleagues also recommend in complex scenarios that prohibit easy bootstrapping (e.g., multilevel data, but also if you have multiple imputed data, or models that simply take a long time to converge).  You can find a Monte Carlo CI computer in the semTools package (?monteCarloMed), but if N > 150 all these methods tend to give the same result.

Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

David Adedia

unread,
Mar 24, 2019, 9:34:15 AM3/24/19
to lav...@googlegroups.com
Dear lavaan community,

I am using simsem package for simulation, is it possible to have the individual fit indices for the various data sets simulated and not the averages alone?

Thank you for your responses.

Regards,

DMA.


Virus-free. www.avg.com

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.


--
David Adedia
Department of Basic Sciences
University of Health and Allied Sciences
Private Mail Bag 31
Ho, Ghana

Alex Schoemann

unread,
Mar 25, 2019, 10:56:30 AM3/25/19
to lavaan
There sure is. The inspect() function allows you to pull information from a simResult object (created while running the sim function). If your object is call out then inspect(out, "fit") will get you fit information from each replication.
Reply all
Reply to author
Forward
0 new messages