measurementinvariancecat chisq diff scores

Skip to first unread message

Jan Brederecke

Oct 26, 2018, 6:03:09 AM10/26/18
to lavaan
Hello everyone,

I am using semTools' measurementinvariancecat function to look for measurement invariance across groups that filled out a questionnaire.
The only thing that I cannot answer myself concerning this is: Why are my qhisquare-differences not correct?
I understand that there could a) be a mistake in my code or b) something regarding the distribution of chisq when it comes to testing with categorical data. 
It would be great if someone could explain this to me or recommend literature on that topic because my own extensive search has not yet shed light on that for me.

Thank you all for taking the time

Here are my model, syntax and Output

MIModellSelbst <- '

Selbst =~ q4301 + q4302 + q4304u + q4305 + q4307 + q4309u
measurementInvarianceCat(model = MIModellSelbst,
                         data = Subset_SIS,
                         group = "s2")

Scaled Chi Square Difference Test (method = "satorra.2000")

               Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
fit.configural 18         509.03                                  
fit.loadings   23         512.20      3.699       5    0.59353    
fit.thresholds 40         558.36     53.338      17  1.263e-05 ***
fit.means      41         596.05      7.621       1    0.00577 ** 
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Fit measures:

               cfi.scaled rmsea.scaled
fit.configural      0.899        0.283               NA                 NA
fit.loadings        0.916        0.227            0.018              0.056
fit.thresholds      0.917        0.171            0.001              0.056
fit.means           0.932        0.154            0.015              0.018

Terrence Jorgensen

Oct 26, 2018, 6:25:00 AM10/26/18
to lavaan
Why are my qhisquare-differences not correct?

They are correct, they are simply not the differences between the naïve test statistics in the column on the left.  A robust chi-sq difference test is not the difference between 2 robustified chi-squared values.  You calculate the naïve chi-squared difference, then robustify that value.

FYI, if this function is too restrictive, there is a more general syntax-writing function you might find useful:


It is already available on CRAN, but you can still install the development version as described at the link above, which fixed some bugs.

Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Jan Brederecke

Oct 26, 2018, 6:31:04 AM10/26/18
to lavaan

Dear Terrence

thank you for your quick and helpful response. I will certainly look deeper into the recommended source and try the new function in my next study of this kind!

Best regards
Jan Brederecke

Jan Brederecke

Nov 8, 2018, 8:22:02 AM11/8/18
to lavaan
Dear Terrence,

two additional important questions has come to my mind as I was trying to make sense of all my R-outputs:

Is my baseline (configural) model already very poorly fitting because of its high RMSEA and low CFI?
And how come that the more restricted models have a better fit the more restricted they become?

I tried really hard to understand all this but have not come to an acceptable conclusion, maybe you can help me once more!

Thank you so much!
Jan Brederecke

Terrence Jorgensen

Nov 9, 2018, 6:15:02 AM11/9/18
to lavaan
Is my baseline (configural) model already very poorly fitting because of its high RMSEA and low CFI?

Using conventional cutoffs, it appears so.  But those rules of thumb are arbitrary guidelines that don't generalize to all scenarios.  Resampling methods can take into account the properties of your data and model in order to estimate a sampling distribution of fit indices

And how come that the more restricted models have a better fit the more restricted they become?

This can happen whenever the decrement in fit is less than the degrees of freedom.  
Reply all
Reply to author
0 new messages