measurementinvariancecat chisq diff scores

47 views
Skip to first unread message

Jan Brederecke

unread,
Oct 26, 2018, 6:03:09 AM10/26/18
to lavaan
Hello everyone,

I am using semTools' measurementinvariancecat function to look for measurement invariance across groups that filled out a questionnaire.
The only thing that I cannot answer myself concerning this is: Why are my qhisquare-differences not correct?
I understand that there could a) be a mistake in my code or b) something regarding the distribution of chisq when it comes to testing with categorical data. 
It would be great if someone could explain this to me or recommend literature on that topic because my own extensive search has not yet shed light on that for me.

Thank you all for taking the time
Jan
 
PS

Here are my model, syntax and Output




MIModellSelbst <- '

Selbst =~ q4301 + q4302 + q4304u + q4305 + q4307 + q4309u
'
library(semTools)
measurementInvarianceCat(model = MIModellSelbst,
                         data = Subset_SIS,
                         group = "s2")




Scaled Chi Square Difference Test (method = "satorra.2000")

               Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
fit.configural 18         509.03                                  
fit.loadings   23         512.20      3.699       5    0.59353    
fit.thresholds 40         558.36     53.338      17  1.263e-05 ***
fit.means      41         596.05      7.621       1    0.00577 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Fit measures:

               cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
fit.configural      0.899        0.283               NA                 NA
fit.loadings        0.916        0.227            0.018              0.056
fit.thresholds      0.917        0.171            0.001              0.056
fit.means           0.932        0.154            0.015              0.018

Terrence Jorgensen

unread,
Oct 26, 2018, 6:25:00 AM10/26/18
to lavaan
Why are my qhisquare-differences not correct?

They are correct, they are simply not the differences between the naïve test statistics in the column on the left.  A robust chi-sq difference test is not the difference between 2 robustified chi-squared values.  You calculate the naïve chi-squared difference, then robustify that value.


FYI, if this function is too restrictive, there is a more general syntax-writing function you might find useful:

?measEq.syntax


It is already available on CRAN, but you can still install the development version as described at the link above, which fixed some bugs.


Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Jan Brederecke

unread,
Oct 26, 2018, 6:31:04 AM10/26/18
to lavaan

Dear Terrence

thank you for your quick and helpful response. I will certainly look deeper into the recommended source and try the new function in my next study of this kind!

Best regards
Jan Brederecke

Jan Brederecke

unread,
Nov 8, 2018, 8:22:02 AM11/8/18
to lavaan
Dear Terrence,

two additional important questions has come to my mind as I was trying to make sense of all my R-outputs:

Is my baseline (configural) model already very poorly fitting because of its high RMSEA and low CFI?
And how come that the more restricted models have a better fit the more restricted they become?

I tried really hard to understand all this but have not come to an acceptable conclusion, maybe you can help me once more!

Thank you so much!
Jan Brederecke

Terrence Jorgensen

unread,
Nov 9, 2018, 6:15:02 AM11/9/18
to lavaan
Is my baseline (configural) model already very poorly fitting because of its high RMSEA and low CFI?

Using conventional cutoffs, it appears so.  But those rules of thumb are arbitrary guidelines that don't generalize to all scenarios.  Resampling methods can take into account the properties of your data and model in order to estimate a sampling distribution of fit indices


And how come that the more restricted models have a better fit the more restricted they become?

This can happen whenever the decrement in fit is less than the degrees of freedom.  
Reply all
Reply to author
Forward
0 new messages