RMSEA issues

hal...@gmail.com

unread,

Nov 15, 2020, 9:26:23 AM11/15/20

to blavaan

Hi,

I am unable to obtain rmsea values for a latent change score model using ppmc(). I can obtain chi-square just fine. The model converges and there are no other warnings/errors in model estimation.

Here's my code for getting RMSEA and chi-square:

rmsea <- ppmc(fit, thin = 10, fit.measures = "rmsea")

chisq <- ppmc(fit, thin = 10, fit.measures = "chisq")

The object "rmsea" is just filled with NAs, and I do not receive any error messages. The object "chisq" has the PPP value, values for the observed and simulated distributions, and I can plot the values. I tried getting the CFI for the same model, and I couldn't obtain that either. I can obtain RMSEA, CFI, and chi-square for a simple linear growth model estimated on the exact same data (no convergence/estimation issues with the linear model either). All the models are estimated with blavaan, not bgrowth.

Thanks,

Garret

Mauricio Garnier-Villarreal

unread,

Nov 16, 2020, 5:23:11 AM11/16/20

to blavaan

Garret

Can you be more specific about which model gives you this error? When I try it wit then HolzingerSwineford1939 data it works well for me.

Also, a couple pointers, make sure you have the latest blavaan version that was recently updated on CRAN. Second, if you want fit indices in the metric similar to frequentist SEM you should use the blavFitIndices() function

Garret Hall

unread,

Nov 16, 2020, 10:02:13 PM11/16/20

to Mauricio Garnier-Villarreal, blavaan

Hi Mauricio,

I am unable to obtain RMSEA and CFI values using ppmc when I try to get these values for a latent change score model. However, I can obtain chi-square values for the same model using ppmc. I'm using the most recent version of blavaan.

I tried using blavFitIndices and encountered a similar problem trying to obtain the BRMSEA for the latent change score model. I received the following error message:

> fitindices <- blavFitIndices(fit2)
Warning messages:
1: 850 (100.0%) p_waic estimates greater than 0.4. We recommend trying loo instead.
2: Some Pareto k diagnostic values are too high. See help('pareto-k-diagnostic') for details.
3: In sqrt(nonc/(dif.ppD * N)) : NaNs produced

The NaNs in the object fitindices are all in the column of BRMSEA values (values are either NaN or 0). The other indices (e.g., BGammaHat) are all 1 or very close to 1.

Thank you,

Garret

--
You received this message because you are subscribed to a topic in the Google Groups "blavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/blavaan/wZFKzPWpbaY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blavaan+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blavaan/6da23c28-12f3-4288-b8ba-7c3f387411a9n%40googlegroups.com.

Ed Merkle

unread,

Nov 16, 2020, 11:20:07 PM11/16/20

to Garret Hall, Mauricio Garnier-Villarreal, blavaan

Garret,

I wonder whether the model looks ok in terms of convergence diagnostics and sensible parameter estimates. It might be helpful to see the model syntax and blavaan command, along with data if you are able to share it.

Ed

Mauricio Garnier-Villarreal

unread,

Nov 17, 2020, 7:30:46 AM11/17/20

to blavaan

Garret

As Ed said, would be useful to see the model and call for blavaan.

A couple of thoughts, to get the CFI/TLI from the blavFitIndices you need to provide the estimated null/baseline model to the fnction, see the example in the ?blavFitIndices page

Change score models usually have very few df, or can be saturated. If this is the case, the adjustment based on the effective estimated parameters pD will lead to a negative df in the RMSEA formula which causes the NaNs. Anyway, with small models RMSEA is not recommended measure. In our paper we ind that Gamma-Hat and CFI are the most recommended meaures to use

Garnier-Villarreal, M., & Jorgensen, T. D. (2020). Adapting Fit Indices for Bayesian Structural Equation Modeling: Comparison to Maximum Likelihood. Psychological Methods, 25(1), 46–70. https://doi.org/dx.doi.org/10.1037/met0000224

Garret Hall

unread,

Nov 17, 2020, 8:52:52 AM11/17/20

to Mauricio Garnier-Villarreal, blavaan

I'm unable to share any raw data but here is the call to blavaan, model results. The convergence looks fine and I don't receive any warnings about convergence issues. The variables mcf, mcw, mcs are each of the repeated measures. I also provided the call to blavFitIndices and the fit index results. My guess is the issue has to do with the small df, but I'm not certain so I will look into this more. I'll do a close read of the paper too. Thank you!

dcm2 <-
'

y2~1*y1
y3~1*y2

dy2=~1*y2
dy3=~1*y3

dy2 ~ C*y1 + prior("normal(-.125, 3.16)")*y1
dy3 ~ D*y2 + prior("normal(-.31, 3.16)")*y2

dy2 ~~ 0*dy2
dy3 ~~ 0*dy3

y1~~0*y1
y2~~0*y2
y3~~0*y3

y1~0*1
y2~0*1
y3~0*1

y1=~1*mcf
y2=~1*mcw
y3=~1*mcs

mcf~0*1
mcw~0*1
mcs~0*1

mcf~~mcf
mcw~~mcw
mcs~~mcs

ys =~ 1*dy2 + 1*dy3
y0 =~1*y1
ys ~ B*1 + prior("normal(20, 3.16)")*1
y0 ~ A*1 + prior("normal(16, 3.16)")*1
ys ~~ ys
y0 ~~ y0
ys ~~ y0

'
fit2 <- blavaan(dcm2, data = imp2, adapt = 500, burnin = 1000, sample = 2000, seed = 123, save.lvs = T)

blavaan (0.3-12) results of 2000 samples after 1000 adapt/burnin iterations

Number of observations 850

Number of missing patterns 1

Statistic MargLogLik PPP
Value -9308.179 0.479

Latent Variables:
Estimate Post.SD pi.lower pi.upper Rhat Prior
dy2 =~
y2 1.000 NA
dy3 =~
y3 1.000 NA
y1 =~
mcf 1.000 NA
y2 =~
mcw 1.000 NA
y3 =~
mcs 1.000 NA
ys =~
dy2 1.000 NA
dy3 1.000 NA
y0 =~
y1 1.000 NA

Regressions:
Estimate Post.SD pi.lower pi.upper Rhat Prior
y2 ~
y1 1.000 NA
y3 ~
y2 1.000 NA
dy2 ~
y1 (C) -0.302 0.201 -0.635 0.123 1.007 normal(-.125,3.16)
dy3 ~
y2 (D) -0.417 0.116 -0.612 -0.174 1.007 normal(-.31,3.16)

Covariances:
Estimate Post.SD pi.lower pi.upper Rhat Prior
ys ~~
y0 13.503 7.701 -0.909 28.256 1.004 beta(1,1)

Intercepts:
Estimate Post.SD pi.lower pi.upper Rhat Prior
.y1 0.000 NA
.y2 0.000 NA
.y3 0.000 NA
.mcf 0.000 NA
.mcw 0.000 NA
.mcs 0.000 NA
ys (B) 18.903 3.692 11.123 25.102 1.007 normal(20,3.16)
y0 (A) 18.515 0.372 17.781 19.24 1.000 normal(16,3.16)
.dy2 0.000 NA
.dy3 0.000 NA

Variances:
Estimate Post.SD pi.lower pi.upper Rhat Prior
.dy2 0.000 NA
.dy3 0.000 NA
.y1 0.000 NA
.y2 0.000 NA
.y3 0.000 NA
.mcf 30.491 15.745 0.229 55.194 1.008 gamma(1,.5)[sd]
.mcw 36.302 4.927 26.762 45.384 1.006 gamma(1,.5)[sd]
.mcs 44.129 6.360 32.843 57.886 1.007 gamma(1,.5)[sd]
ys 15.900 8.158 0.753 31.516 1.008 gamma(1,.5)[sd]
y0 80.532 15.848 56.719 113.664 1.007 gamma(1,.5)[sd]

fit_indices <- blavFitIndices(fit2)

Posterior summary statistics and highest posterior density (HPD) 90% credible intervals for devm-based fit indices:

EAP Median MAP SD lower upper
BRMSEA 0.000 0 -0.001 0.000 0.000 0.000
BGammaHat 0.999 1 1.000 0.002 0.995 1.000
adjBGammaHat 1.016 1 1.000 0.026 1.000 1.055
BMc 0.999 1 1.000 0.002 0.996 1.000

--
You received this message because you are subscribed to a topic in the Google Groups "blavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/blavaan/wZFKzPWpbaY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blavaan+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/blavaan/abc03733-5f64-479a-a90d-05673f54246an%40googlegroups.com.

Mauricio Garnier-Villarreal

unread,

Nov 17, 2020, 10:27:37 AM11/17/20

to blavaan

Garret

Yes, in general the issue is for the low df. With a small model like this RMSEA, would not be recommended.

Now, more generally the problem is that you are estimating a model with negative df (in frequentist). This becuase I think you are missing some constraints in the model.

I am attaching code and data set (links) I used for a course with these models, from which I follow the Grimm et.al boook. Please see the Dual change model on line 270, which was ran with 4 time points but can be adjusted for 3 time points

You can download the R code and data set from these links

https://www.dropbox.com/s/e6io0az08tqdl0g/latent_change_scores.R?dl=0

https://www.dropbox.com/s/mk9qzmogm3pgzo1/panas_4wave.csv?dl=0

Grimm, K. J., Ram, N., & Estabrook, R. (2016). Growth Modeling: Structural Equation and Multilevel Modeling Approaches.

I think yours would work like this

dcm2 <-'

y1=~1*mcf
y2=~1*mcw
y3=~1*mcs

y1~~0*y1
y2~~0*y2
y3~~0*y3

y1~0*1
y2~0*1
y3~0*1

mcf~0*1
mcw~0*1
mcs~0*1

mcf~~ uv*mcf
mcw~~ uv*mcw
mcs~~ uv*mcs

y2~1*y1
y3~1*y2

dy2=~1*y2
dy3=~1*y3

dy2 ~~ v1*dy2
dy3 ~~ v2*dy3

dy2 ~0*1
dy3 ~0*1

dy2 ~ C*y1 #+ prior("normal(-.125, 3.16)")*y1
dy3 ~ C*y2 #+ prior("normal(-.31, 3.16)")*y2

ys =~ 1*dy2 + 1*dy3
y0 =~1*y1

ys ~ B*1 #+ prior("normal(20, 3.16)")*1
y0 ~ A*1 #+ prior("normal(16, 3.16)")*1

Garret Hall

unread,

Nov 17, 2020, 1:36:19 PM11/17/20

to Mauricio Garnier-Villarreal, blavaan

Thank you for pointing out the negative df -- I was testing with the models with and without the freed residuals, and I wasn't paying attention to the df! I ran the model fixing the residuals to equality, and I was able to obtain the indices I was trying to get. Also thank you for passing along the additional resources -- this is very helpful.

Garret

--
You received this message because you are subscribed to the Google Groups "blavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blavaan+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blavaan/659fba83-c4cb-4a48-b515-140080e55bc4n%40googlegroups.com.

Reply all

Reply to author

Forward