measurementinvariancecat chisq diff scores

[Jan Brederecke]

Hello everyone,

I am using semTools' measurementinvariancecat function to look for measurement invariance across groups that filled out a questionnaire.

The only thing that I cannot answer myself concerning this is: Why are my qhisquare-differences not correct?

I understand that there could a) be a mistake in my code or b) something regarding the distribution of chisq when it comes to testing with categorical data. 

It would be great if someone could explain this to me or recommend literature on that topic because my own extensive search has not yet shed light on that for me.

Thank you all for taking the time

Jan

 

PS

Here are my model, syntax and Output

MIModellSelbst <- '

Selbst =~ q4301 + q4302 + q4304u + q4305 + q4307 + q4309u
'

library(semTools)
measurementInvarianceCat(model = MIModellSelbst,
                         data = Subset_SIS,
                         group = "s2")

Scaled Chi Square Difference Test (method = "satorra.2000")

               Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
fit.configural 18         509.03                                  
fit.loadings   23         512.20      3.699       5    0.59353    
fit.thresholds 40         558.36     53.338      17  1.263e-05 ***
fit.means      41         596.05      7.621       1    0.00577 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Fit measures:

               cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
fit.configural      0.899        0.283               NA                 NA
fit.loadings        0.916        0.227            0.018              0.056
fit.thresholds      0.917        0.171            0.001              0.056
fit.means           0.932        0.154            0.015              0.018

[Terrence Jorgensen]

Why are my qhisquare-differences not correct?

They are correct, they are simply not the differences between the naïve test statistics in the column on the left.  A robust chi-sq difference test is not the difference between 2 robustified chi-squared values.  You calculate the naïve chi-squared difference, then robustify that value.

https://www.statmodel.com/chidiff.shtml

FYI, if this function is too restrictive, there is a more general syntax-writing function you might find useful:

?measEq.syntax

https://groups.google.com/d/msg/lavaan/oKwP0_6-i1g/i0jGCU-LBwAJ

It is already available on CRAN, but you can still install the development version as described at the link above, which fixed some bugs.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Jan Brederecke]

Dear Terrence

thank you for your quick and helpful response. I will certainly look deeper into the recommended source and try the new function in my next study of this kind!

Best regards

Jan Brederecke

[Jan Brederecke]

Dear Terrence,

two additional important questions has come to my mind as I was trying to make sense of all my R-outputs:

Is my baseline (configural) model already very poorly fitting because of its high RMSEA and low CFI?

And how come that the more restricted models have a better fit the more restricted they become?

I tried really hard to understand all this but have not come to an acceptable conclusion, maybe you can help me once more!

Thank you so much!

Jan Brederecke

[Terrence Jorgensen]

Is my baseline (configural) model already very poorly fitting because of its high RMSEA and low CFI?

Using conventional cutoffs, it appears so.  But those rules of thumb are arbitrary guidelines that don't generalize to all scenarios.  Resampling methods can take into account the properties of your data and model in order to estimate a sampling distribution of fit indices

• bootstrapping: https://doi.org/10.1080/10705511.2015.1118692
  
• permutation: https://www.researchgate.net/project/Permutation-Tests-of-Measurement-Equivalence

And how come that the more restricted models have a better fit the more restricted they become?

This can happen whenever the decrement in fit is less than the degrees of freedom.