Comparing nested CFA models...options to compare robust chi-square?

[Blair Burnette]

Hello,

I am working on a measurement invariance paper. The scale I'm using is ordinal, with a 5-point scale. So, I have run the models with the ordered command, rather than treating the data as continuous. 

To evaluate invariance, I am triangulating the change in fit indices and examining the change in chi-square. I know that chi-square is sensitive to sample size. But, I have unequal group sizes, and know from Chen (2007) that unequal group sizes or invariance patterns can bias the fit indices. 

Because my data are ordinal, I am interpreting the robust estimates rather than DWLS. When I run lavTestLRT, it compares the DWLS chi-square values rather than the robust estimations.

Does anyone know of a method to compare the robust test statistics? If it can't be done with lavaan, does anyone know of a workaround or know how to compute this? 

[şule selçuk]

Hi Blair,

I think you can add estimator="MLM" to your fit for both models. Then use anova (fit1, fit2) function . This will provide the output for Satorra-Benler scaled chi square difference test.

Şule

[Terrence Jorgensen]

To evaluate invariance, I am triangulating the change in fit indices and examining the change in chi-square. I know that chi-square is sensitive to sample size.

You mean that larger samples increase power?  Yes, that is how statistics should work.

 

But, I have unequal group sizes, and know from Chen (2007) that unequal group sizes or invariance patterns can bias the fit indices. 

Indeed, many other qualities of the data and model can also affect results.  You can use permutation to take such features into account:

https://doi.org/10.1080/10705511.2017.1421467

http://dx.doi.org/10.1037/met0000152

Because my data are ordinal, I am interpreting the robust estimates rather than DWLS. When I run lavTestLRT, it compares the DWLS chi-square values rather than the robust estimations.

That is how it works.  A chi-squared statistic is always a model comparison.  A single model's chi-squared fit statistic is merely comparing that model to a saturated model.  When you compare 2 nested (but not saturated) models, you can subtract their 2 chi-squared fit statistics to obtain the chi-squared for that model comparison.  That is the value that must be "robustified".  It is not valid to simply subtract the robustified chi-squared statistics from 2 nested models.

https://www.statmodel.com/chidiff.shtml

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen