Hello all,
We are running a model to test indirect effects (mediation) with latent change score as mediator variable and latent growth as outcome variable. When we run it without bootstrapping, the estimates and standard error values seemed to be in the range we would expect. However, when we run the model with 10,000 bootstrap samples, the parameter estimate and standard error values are irregularly large for some values.
All observed variables that make up the latent variables in the model are standardized, so we would expect values around the range of -2 to 2. Prior to constructing the mediation model, we ran CFAs and separate models for the latent change score and latent growth curve, and everything looked normal (i.e., normal estimate values, no negative variances, good model fit, etc.). We also extracted the predicted values for dRAR, and they seemed to be normal relative to what we would expect. We also checked the data file to ensure that all values of the observed variables were correctly inputted. When we then constructed the full model to test mediation, the dRAR change parameter resulted in a negative variance. Finally, we tried bounding the dRAR variance to be zero (since it was non-significantly negative), which did not fix the issue of the abnormal parameter estimates. We were going to run it with the robust estimator (MLR) but this does not work with bootstrapping. We are not sure what else to try.
Below, please find the code for the two models in question, along with key portions of the model output and parameter estimates. Thanks so much for any help!
model2A <- '
## RAR Latent Change Score
RAR_T1=~1*IS_m0_z3+rp1*IV_m0r_z3+rp2*RA_m0_z3
RAR_T2=~1*IS_m6_z3+rp1*IV_m6r_z3+rp2*RA_m6_z3
# Estimates
RAR_T2 ~ 1*RAR_T1
dRAR =~ 1*RAR_T2
dRAR~RAR_T1
# Intercepts
dRAR ~ 1
RAR_T1 ~ 1
RAR_T2 ~ 0*1
IS_m0_z3~0*1
IV_m0r_z3~0*1
RA_m0_z3~0*1
IS_m6_z3~0*1
IV_m6r_z3~0*1
RA_m6_z3~0*1
# Covariances
IS_m0_z3~~rcv*RA_m0_z3
IS_m6_z3~~rcv*RA_m6_z3
# Covariances within indicators
#IS_m0_z3 ~~ IS_m6_z3
#IV_m0r_z3 ~~ IV_m6r_z3
RA_m0_z3 ~~ RA_m6_z3
# Variances
IS_m0_z3~~rv1*IS_m0_z3
IV_m0r_z3~~0*IV_m0r_z3 ## non-significant negative variance ##
RA_m0_z3~~rv3*RA_m0_z3
IS_m6_z3~~rv1*IS_m6_z3
IV_m6r_z3~~0*IV_m6r_z3 ## non-significant negative variance ##
RA_m6_z3~~rv3*RA_m6_z3
dRAR ~~ dRAR
RAR_T1 ~~ RAR_T1
RAR_T2 ~~ RAR_T2
## Cognitive Latent Growth Curve
Cog_T1=~1*immc_m0_z3+delmc_m0_z3+ccc_m0_z3
Cog_T2=~1*immc_m6_z3+delmc_m6_z3+ccc_m6_z3
Cog_T3=~1*immc_m18_z3+delmc_m18_z3+ccc_m18_z3
# Estimates
intercept=~1*Cog_T1+1*Cog_T2+1*Cog_T3
slope=~-3*Cog_T1+-2*Cog_T2+0*Cog_T3
# Intercepts
intercept ~1
slope ~1
Cog_T1 ~ 0*1
Cog_T2 ~ 0*1
Cog_T3 ~ 0*1
immc_m0_z3 ~ 0*1
immc_m6_z3 ~ 0*1
immc_m18_z3 ~ 0*1
delmc_m0_z3 ~ 0*1
delmc_m6_z3 ~ 0*1
delmc_m18_z3 ~ 0*1
ccc_m0_z3 ~ 0*1
ccc_m6_z3 ~ 0*1
ccc_m18_z3 ~ 0*1
# Covariances (removed equality constraint)
immc_m0_z3~~ccv*delmc_m0_z3
immc_m6_z3~~ccv*delmc_m6_z3
immc_m18_z3~~ccv*delmc_m18_z3
# Covariances Within Indicators
ccc_m0_z3~~ccc_m6_z3
ccc_m0_z3~~ccc_m18_z3
ccc_m6_z3~~ccc_m18_z3
delmc_m0_z3~~delmc_m6_z3
delmc_m0_z3~~delmc_m18_z3
delmc_m6_z3~~delmc_m18_z3
# immc_m0_z3~~immc_m6_z3
# immc_m0_z3~~immc_m18_z3
# immc_m6_z3~~immc_m18_z3
# Variances (removed certain equality constraints if variances across time are not as similar originally)
immc_m0_z3~~cv1*immc_m0_z3
delmc_m0_z3~~delmc_m0_z3
ccc_m0_z3~~ccc_m0_z3
immc_m6_z3~~immc_m6_z3
delmc_m6_z3~~cv2*delmc_m6_z3
ccc_m6_z3~~cv3*ccc_m6_z3
immc_m18_z3~~cv1*immc_m18_z3
delmc_m18_z3~~cv2*delmc_m18_z3
ccc_m18_z3~~cv3*ccc_m18_z3
con > 0
intercept ~~ intercept
slope ~~ slope
intercept ~~ slope
Cog_T1 ~~ con*Cog_T1
Cog_T2 ~~ Cog_T2
Cog_T3 ~~ con*Cog_T3
## Mediation Model
# Specify the regression paths from mediator to outcomes
intercept ~ b1 * dRAR
slope ~ b2 * dRAR
#specify path from predictor to mediator
dRAR ~ a * Exercise. + age + gender + site
# Specify the direct effects from predictor to outcomes (will need site covariate)
slope ~ c * Exercise. + age + gender + site
# Specify the indirect effects
ind := a * b2
#Specify the total effects
total := a*b2 + c
'
system.time(model2A_med <- sem(model2A, data=df, se = "boot", bootstrap = 10000, parallel = "multicore", ncpus = cores-1, fixed.x = FALSE, estimator='ML',missing='fiml'))
summary(model2A_med, fit.measures=TRUE, standardized=TRUE, rsquare=TRUE)
parameterestimates(model2A_med)
## user system elapsed
## 154928.170 1093.087 10557.209
## lavaan 0.6-19 ended normally after 583 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 73
## Number of inequality constraints 2
## Row rank of the constraints matrix 12
##
## Number of observations 290
## Number of missing patterns 12
##
## Model Test User Model:
##
## Test statistic 423.139
## Degrees of freedom 147
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 3013.995
## Degrees of freedom 165
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.903
## Tucker-Lewis Index (TLI) 0.891
##
## Robust Comparative Fit Index (CFI) 0.908
## Robust Tucker-Lewis Index (TLI) 0.897
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -5561.249
## Loglikelihood unrestricted model (H1) -5349.679
##
## Akaike (AIC) 11246.498
## Bayesian (BIC) 11474.031
## Sample-size adjusted Bayesian (SABIC) 11277.418
##
## Root Mean Square Error of Approximation:
##
10
## RMSEA 0.080
## 90 Percent confidence interval - lower 0.072
## 90 Percent confidence interval - upper 0.090
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.545
##
## Robust RMSEA 0.083
## 90 Percent confidence interval - lower 0.073
## 90 Percent confidence interval - upper 0.093
## P-value H_0: Robust RMSEA <= 0.050 0.000
## P-value H_0: Robust RMSEA >= 0.080 0.682
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.111
##
## Parameter Estimates:
##
## Standard errors Bootstrap
## Number of requested bootstrap draws 10000
## Number of successful bootstrap draws 10000
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## RAR_T1 =~
## IS_m0_z3 1.000 0.124 0.125
## IV_m0r_3 (rp1) 4.606 0.963 4.784 0.000 0.570 1.000
## RA_m0_z3 (rp2) 1.058 0.147 7.189 0.000 0.131 0.133
## RAR_T2 =~
## IS_m6_z3 1.000 0.141 0.142
## IV_m6r_3 (rp1) 4.606 0.963 4.784 0.000 0.648 1.000
## RA_m6_z3 (rp2) 1.058 0.147 7.189 0.000 0.149 0.151
## dRAR =~
## RAR_T2 1.000 NA NA
## Cog_T1 =~
## immc_0_3 1.000 0.656 0.859
## dlmc_0_3 1.002 0.040 25.343 0.000 0.657 0.788
## ccc_m0_3 0.723 0.081 8.974 0.000 0.474 0.462
## Cog_T2 =~
## immc_6_3 1.000 0.866 0.860
## dlmc_6_3 0.901 0.043 21.049 0.000 0.781 0.793
## ccc_m6_3 0.540 0.068 7.974 0.000 0.467 0.491
## Cog_T3 =~
## imm_18_3 1.000 1.001 0.931
## dlm_18_3 0.867 0.038 22.969 0.000 0.868 0.823
## ccc_18_3 0.441 0.052 8.510 0.000 0.441 0.470
## intercept =~
## Cog_T1 1.000 1.526 1.526
## Cog_T2 1.000 1.155 1.155
## Cog_T3 1.000 1.000 1.000
## slope =~
## Cog_T1 -3.000 -0.652 -0.652
## Cog_T2 -2.000 -0.329 -0.329
## Cog_T3 0.000 0.000 0.000
11
##
## Regressions:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## RAR_T2 ~
## RAR_T1 1.000 0.879 0.879
## dRAR ~
## RAR_T1 -0.698 0.177 -3.940 0.000 NA NA
## intercept ~
## dRAR (b1) -0.032 104.715 -0.000 1.000 NA NA
## slope ~
## dRAR (b2) 0.008 32.919 0.000 1.000 NA NA
## dRAR ~
## Exercise. (a) 0.008 0.016 0.474 0.636 NA NA
## age -0.001 0.004 -0.376 0.707 NA NA
## gender -0.007 0.026 -0.267 0.789 NA NA
## site -0.013 0.019 -0.682 0.495 NA NA
## slope ~
## Exercise. (c) -0.020 0.082 -0.243 0.808 -0.140 -0.070
## age -0.001 0.037 -0.040 0.968 -0.010 -0.048
## gender -0.052 0.352 -0.148 0.882 -0.366 -0.163
## site -0.017 0.171 -0.099 0.921 -0.120 -0.060
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .IS_m0_z3 ~~
## .RA_m0_z3 (rcv) 0.717 0.063 11.421 0.000 0.717 0.753
## .IS_m6_z3 ~~
## .RA_m6_z3 (rcv) 0.717 0.063 11.421 0.000 0.717 0.753
## .RA_m0_z3 ~~
## .RA_m6_z3 0.105 0.028 3.772 0.000 0.105 0.111
## .immc_m0_z3 ~~
## .dlmc_0_3 (ccv) 0.077 0.022 3.592 0.000 0.077 0.385
## .immc_m6_z3 ~~
## .dlmc_6_3 (ccv) 0.077 0.022 3.592 0.000 0.077 0.251
## .immc_m18_z3 ~~
## .dlm_18_3 (ccv) 0.077 0.022 3.592 0.000 0.077 0.330
## .ccc_m0_z3 ~~
## .ccc_m6_3 0.549 0.055 10.033 0.000 0.549 0.727
## .ccc_18_3 0.510 0.054 9.514 0.000 0.510 0.676
## .ccc_m6_z3 ~~
## .ccc_18_3 0.496 0.057 8.631 0.000 0.496 0.720
## .delmc_m0_z3 ~~
## .dlmc_6_3 0.137 0.021 6.604 0.000 0.137 0.446
## .dlm_18_3 0.115 0.021 5.457 0.000 0.115 0.373
## .delmc_m6_z3 ~~
## .dlm_18_3 0.176 0.032 5.598 0.000 0.176 0.491
## .intercept ~~
## .slope 0.126 0.247 0.509 0.610 0.898 0.898
## Exercise. ~~
## age 0.009 0.137 0.066 0.948 0.009 0.004
## gender 0.019 0.013 1.503 0.133 0.019 0.087
## site -0.014 0.015 -0.933 0.351 -0.014 -0.055
## age ~~
## gender -0.142 0.119 -1.194 0.233 -0.142 -0.067
12
## site -0.318 0.138 -2.306 0.021 -0.318 -0.135
## gender ~~
## site -0.016 0.013 -1.183 0.237 -0.016 -0.070
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .dRAR 0.333 0.245 1.358 0.174 NA NA
## RAR_T1 -0.167 0.031 -5.464 0.000 -1.350 -1.350
## .RAR_T2 0.000 0.000 0.000
## .IS_m0_z3 0.000 0.000 0.000
## .IV_m0r_z3 0.000 0.000 0.000
## .RA_m0_z3 0.000 0.000 0.000
## .IS_m6_z3 0.000 0.000 0.000
## .IV_m6r_z3 0.000 0.000 0.000
## .RA_m6_z3 0.000 0.000 0.000
## .intercept 0.234 40.072 0.006 0.995 0.234 0.234
## .slope 0.326 14.258 0.023 0.982 2.292 2.292
## .Cog_T1 0.000 0.000 0.000
## .Cog_T2 0.000 0.000 0.000
## .Cog_T3 0.000 0.000 0.000
## .immc_m0_z3 0.000 0.000 0.000
## .immc_m6_z3 0.000 0.000 0.000
## .immc_m18_z3 0.000 0.000 0.000
## .delmc_m0_z3 0.000 0.000 0.000
## .delmc_m6_z3 0.000 0.000 0.000
## .delmc_m18_z3 0.000 0.000 0.000
## .ccc_m0_z3 0.000 0.000 0.000
## .ccc_m6_z3 0.000 0.000 0.000
## .ccc_m18_z3 0.000 0.000 0.000
## Exercise. 0.476 0.029 16.465 0.000 0.476 0.953
## age 71.122 0.279 255.360 0.000 71.122 15.081
## gender 0.728 0.026 27.809 0.000 0.728 1.634
## site 0.500 0.029 17.244 0.000 0.500 1.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .IS_m0_z3 (rv1) 0.958 0.065 14.771 0.000 0.958 0.984
## .IV_m0r_3 0.000 0.000 0.000
## .RA_m0_z3 (rv3) 0.948 0.064 14.924 0.000 0.948 0.982
## .IS_m6_z3 (rv1) 0.958 0.065 14.771 0.000 0.958 0.980
## .IV_m6r_3 0.000 0.000 0.000
## .RA_m6_z3 (rv3) 0.948 0.064 14.924 0.000 0.948 0.977
## .dRAR -0.130 285.133 -0.000 1.000 NA NA
## RAR_T1 0.015 0.006 2.646 0.008 1.000 1.000
## .RAR_T2 0.149 285.133 0.001 1.000 7.512 7.512
## .immc_0_3 (cv1) 0.153 0.024 6.333 0.000 0.153 0.263
## .dlmc_0_3 0.264 0.031 8.424 0.000 0.264 0.379
## .ccc_m0_3 0.829 0.080 10.360 0.000 0.829 0.787
## .immc_6_3 0.263 0.038 6.886 0.000 0.263 0.260
## .dlmc_6_3 (cv2) 0.360 0.040 9.005 0.000 0.360 0.371
## .ccc_m6_3 (cv3) 0.689 0.054 12.640 0.000 0.689 0.759
## .imm_18_3 (cv1) 0.153 0.024 6.333 0.000 0.153 0.133
## .dlm_18_3 (cv2) 0.360 0.040 9.005 0.000 0.360 0.323
## .ccc_18_3 (cv3) 0.689 0.054 12.640 0.000 0.689 0.780
13
## .intercpt 1.001 2.586 0.387 0.699 1.000 1.000
## .slope 0.020 0.048 0.405 0.686 0.965 0.965
## .Cog_T1 (con) 0.000 0.003 0.000 1.000 0.000 0.000
## .Cog_T2 0.171 0.048 3.567 0.000 0.227 0.227
## .Cog_T3 (con) 0.000 0.003 0.000 1.000 0.000 0.000
## Exercis. 0.249 0.002 134.678 0.000 0.249 1.000
## age 22.241 1.507 14.763 0.000 22.241 1.000
## gender 0.198 0.012 16.599 0.000 0.198 1.000
## site 0.250 0.001 208.012 0.000 0.250 1.000
##
## R-Square:
## Estimate
## IS_m0_z3 0.016
## IV_m0r_z3 1.000
## RA_m0_z3 0.018
## IS_m6_z3 0.020
## IV_m6r_z3 1.000
## RA_m6_z3 0.023
## dRAR NA
## RAR_T2 -6.512
## immc_m0_z3 0.737
## delmc_m0_z3 0.621
## ccc_m0_z3 0.213
## immc_m6_z3 0.740
## delmc_m6_z3 0.629
## ccc_m6_z3 0.241
## immc_m18_z3 0.867
## delmc_m18_z3 0.677
## ccc_m18_z3 0.220
## intercept -0.000
## slope 0.035
## Cog_T1 1.000
## Cog_T2 0.773
## Cog_T3 1.000
##
## Defined Parameters:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## ind 0.000 0.080 0.001 0.999 Inf Inf
## total -0.020 0.019 -1.027 0.304 Inf Inf
## lhs op rhs label est se z pvalue ci.lower
## 84 intercept ~ dRAR b1 -0.032 104.715 0.000 1.000 -17.269
## 85 slope ~ dRAR b2 0.008 32.919 0.000 1.000 -1.649
## 86 dRAR ~ Exercise. a 0.008 0.016 0.474 0.636 -0.023
## 87 dRAR ~ age -0.001 0.004 -0.376 0.707 -0.007
## 88 dRAR ~ gender -0.007 0.026 -0.267 0.789 -0.063
## 89 dRAR ~ site -0.013 0.019 -0.682 0.495 -0.051
## 90 slope ~ Exercise. c -0.020 0.082 -0.243 0.808 -0.057