Dear community,
I am running a measurement invariance analysis in ordinal data (theta parameterization, identification based on Wu & Estabrook, ordered indicators, scaled and shifted test statistic, robust se and DWLS estimator for all models).
When using semTools::compareFit function the output of the Chi Square Difference test method Satorra 2000 derives to a different chi-square difference than the simple substraction of the standard Chi-Square values (also not the exact difference when using the scaled chi-squared values).
Here my code:
library(semTools)
compareFit.metric.scalar.improv <- compareFit(fit.metric.improv,fit.scalar.improv)
summary(compareFit.metric.scalar.improv)
Results:
Scaled Chi-Squared Difference Test (method = “satorra.2000”)
lavaan->unknown():
lavaan NOTE: The “Chisq” column contains standard test
statistics, not the robust test that should be reported
per model. A robust difference test is a function of two
standard (not robust) statistics.
Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq)
fit.metric.improv 1112 4778.4
fit.scalar.improv 1172 6224.8 996.71 60 < 2.2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
####################### Model Fit Indices ###########################
chisq.scaled df.scaled pvalue.scaled rmsea.scaled cfi.scaled tli.scaled srmr
fit.metric.improv 5059.826† 1112 .000 .042† .941† .937† .063†
fit.scalar.improv 6101.898 1172 .000 .045 .927 .925 .065
################## Differences in Fit Indices #######################
df.scaled rmsea.scaled cfi.scaled tli.scaled srmr
fit.scalar.improv - fit.metric.improv 60 0.004 -0.015 -0.012 0.002
I might misspecify the method of the difference test but could not find the right method bc mostly Satorra, 2000 is recommended. Any idea why the Chi Square Difference is not exactly the difference of the Chi Square values? Is this cause for concern?
I highly appreciate your input.
Best wishes and happy new year!
Claudia