Measurement invariance results-minimal fit index changes but significant chi-square difference

28 views
Skip to first unread message

Blair Burnette

unread,
Feb 26, 2019, 3:56:37 PM2/26/19
to lavaan

Hi,


I am still working on a measurement invariance in lavaan in R. I am comparing a measure of sociocultural appearance pressures across White and Black women. Because this is a 5-point scale, I am treating the items as ordinal. So, I'm presenting/analyzing the robust estimations.


I have large, but uneven samples. There are 553 White and 273 Black women. I'm evaluating a combo of change in x2 and fit indices. The fit indices are generally adequate for the initial cfa, though the RMSEA is a bit high: RMSEA=.085, 95% CI [.081, .090], TLI=.983; SRMR=.054, χ2(199)=1346.800, p<.001.


The configural model is somewhat similar: RMSEA=.088, 95% CI [.083, .092]; TLI=.982; SRMR=.061, χ2(398)=1612.698, p<.001.


The metric model had the following changes: ΔRMSEA=-.009 (i.e., .79), ΔTLI=.001 (i.e. .983), and ΔSRMR=.002 (i.e., 063). Only the SRMR suggested a decrement in model fit, but the change (.002) was below the recommended cut-off of .025 by Chen (2007). The adjusted chi-square difference test was significant, Δχ2(22)=38.62308, p=.016 (I used the scaling correction factors, etc. to calculate the change in the robust x2 estimates).


This is a tricky one for me. I believe I have seen on this forum that the change in robust chi-square estimates IS notable and something to explore. However, my co-authors are advising that I ignore the change in chi-square because of the large sample size and the minimal changes of the other fit indices. Given research and theory in the area, I would have hypothesized factor loadings would differ by group, so I feel uncomfortable not probing the significant change in chi-square.


I start to really question my data when I run the scalar model. In the scalar model, I constrained loadings and thresholds to equality. With this model, I was surprised to see that chi-square goes down, from 1620.556 (420 df) in the metric model to 1462.593 (481 df) in the scalar model. When I run the x2 difference test, I get a p value of 1 because the chi-square went down instead of up. Again, the fit indices changes are minimal, compared to the metric model: ΔRMSEA=-.002 (.077), ΔTLI=.003 (.986), and ΔSRMR=-.001 (.062). I question this also because latent factor means are clearly non-equivalent across groups. Would that not affect scalar invariance?


So, I guess my questions are: do you all agree that I should ignore the significant change in x2 from the configural to metric model? And is it OK for the chi-square to decrease from the metric model to the scalar model?


Thank you!

Terrence Jorgensen

unread,
Feb 28, 2019, 10:15:37 PM2/28/19
to lavaan

I question this also because latent factor means are clearly non-equivalent across groups. Would that not affect scalar invariance?


So, I guess my questions are: do you all agree that I should ignore the significant change in x2 from the configural to metric model? And is it OK for the chi-square to decrease from the metric model to the scalar model?


I don't see any questions about the lavaan software.  If you only have general SEM questions, please post them to SEMNET.


Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Reply all
Reply to author
Forward
0 new messages