semTools compareFit() error - problems with chi square difference test

115 views
Skip to first unread message

cla...@googlemail.com

unread,
Jan 3, 2025, 7:02:59 AM1/3/25
to lavaan
Dear community,

I am running a measurement invariance analysis in ordinal data (theta parameterization, identification based on Wu & Estabrook, ordered indicators, scaled and shifted test statistic, robust se and DWLS estimator for all models).

When using semTools::compareFit function the output of the Chi Square Difference test method Satorra 2000 derives to a different chi-square difference than the simple substraction of the standard Chi-Square values (also not the exact difference when using the scaled chi-squared values).

Here my code:
library(semTools)
compareFit.metric.scalar.improv <- compareFit(fit.metric.improv,fit.scalar.improv)
summary(compareFit.metric.scalar.improv)

Results:
Scaled Chi-Squared Difference Test (method = “satorra.2000”) lavaan->unknown(): lavaan NOTE: The “Chisq” column contains standard test statistics, not the robust test that should be reported per model. A robust difference test is a function of two standard (not robust) statistics. Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) fit.metric.improv 1112 4778.4 fit.scalar.improv 1172 6224.8 996.71 60 < 2.2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ####################### Model Fit Indices ########################### chisq.scaled df.scaled pvalue.scaled rmsea.scaled cfi.scaled tli.scaled srmr fit.metric.improv 5059.826† 1112 .000 .042† .941† .937† .063† fit.scalar.improv 6101.898 1172 .000 .045 .927 .925 .065 ################## Differences in Fit Indices ####################### df.scaled rmsea.scaled cfi.scaled tli.scaled srmr fit.scalar.improv - fit.metric.improv 60 0.004 -0.015 -0.012 0.002

I might misspecify the method of the difference test but could not find the right method bc mostly Satorra, 2000 is recommended. Any idea why the Chi Square Difference is not exactly the difference of the Chi Square values? Is this cause for concern?

I highly appreciate your input.

Best wishes and happy new year!

Claudia

Terrence Jorgensen

unread,
Jan 23, 2025, 5:03:06 AM1/23/25
to lavaan

Any idea why the Chi Square Difference is not exactly the difference of the Chi Square values?

The output let's you know:
 
lavaan->unknown(): lavaan NOTE: The “Chisq” column contains standard test statistics, not the robust test that should be reported per model. A robust difference test is a function of two standard (not robust) statistics.

Terrence D. Jorgensen    (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen


 
Reply all
Reply to author
Forward
0 new messages