how to interpret output of measurement invariance analysis
173 views
Skip to first unread message
Tina
Nov 12, 2019, 4:33:07 AM11/12/19
to lavaan
I try to test the measure invariance of a scale (with a 1-factor-structure) across two groups ( n= 645; n= 127) and have problems with the application as well as the interpretation. I'm sure they're simple questions and I've already tried to research them, but since I'm doing it for the first time, I'm having trouble transferring the information I find to my case.
I use R (lavaan, semTools). Since the data is ordinal (4-point-scale), I tried to use WLSMV as an estimator first, but got this error message „lavaan WARNING: number of observations (127) too small to compute Gamma in group: 2" because the second group is quite small. Then I did the analysis with MLR and with ML with these results:
ML - Measurement invariance models:
Model 1 : fit.configural
Model 2 : fit.loadings
Model 3 : fit.intercepts
Model 4 : fit.means
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq)
fit.configural 418 26263 26876 1625.9
fit.loadings 439 26302 26818 1706.8 80.911 21 5.684e-09 ***
fit.intercepts 460 26342 26760 1788.9 82.150 21 3.519e-09 ***
fit.means 461 26417 26831 1866.6 77.714 1 < 2.2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Fit measures:
cfi rmsea cfi.delta rmsea.delta
fit.configural 0.868 0.087 NA NA
fit.loadings 0.862 0.086 0.007 0.000
fit.intercepts 0.855 0.087 0.007 0.000
fit.means 0.847 0.089 0.008 0.002
MLR - Measurement invariance models:
Model 1 : fit.configural
Model 2 : fit.loadings
Model 3 : fit.intercepts
Model 4 : fit.means
Scaled Chi-Squared Difference Test (method = “satorra.bentler.2001”)
lavaan NOTE:
The “Chisq” column contains standard test statistics, not the
robust test that should be reported per model. A robust difference
test is a function of two standard (not robust) statistics.
Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq)
fit.configural 418 26263 26876 1625.9
fit.loadings 439 26302 26818 1706.8 80.954 21 5.590e-09 ***
fit.intercepts 460 26342 26760 1788.9 77.652 21 1.985e-08 ***
fit.means 461 26417 26831 1866.6 205.604 1 < 2.2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Fit measures:
cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
fit.configural 0.883 0.069 NA NA
fit.loadings 0.874 0.070 0.008 0.001
fit.intercepts 0.866 0.070 0.008 0.001
fit.means 0.857 0.073 0.009 0.002
Here are the results from the previous CFA:
Estimator ML
Optimization method NLMINB
Number of free parameters 44
Number of observations 772
Model Test User Model:
Standard Robust
Test Statistic 1421.567 894.988
Degrees of freedom 209 209
P-value (Chi-square) 0.000 0.000
Scaling correction factor 1.588
for the Yuan-Bentler correction (Mplus variant)
Model Test Baseline Model:
Test statistic 11305.724 7136.422
Degrees of freedom 231 231
P-value 0.000 0.000
Scaling correction factor 1.584
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.891 0.901
Tucker-Lewis Index (TLI) 0.879 0.890
Robust Comparative Fit Index (CFI) 0.900
Robust Tucker-Lewis Index (TLI) 0.890
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -13845.490 -13845.490
Scaling correction factor 2.020
for the MLR correction
Loglikelihood unrestricted model (H1) NA NA
Scaling correction factor 1.663
for the MLR correction
Akaike (AIC) 27778.981 27778.981
Bayesian (BIC) 27983.536 27983.536
Sample-size adjusted Bayesian (BIC) 27843.815 27843.815
Root Mean Square Error of Approximation:
RMSEA 0.087 0.065
90 Percent confidence interval - lower 0.082 0.062
90 Percent confidence interval - upper 0.091 0.069
P-value RMSEA <= 0.05 0.000 0.000
Robust RMSEA 0.082
90 Percent confidence interval - lower 0.077
90 Percent confidence interval - upper 0.088
Standardized Root Mean Square Residual:
SRMR 0.045 0.045
Parameter Estimates:
Information Observed
Observed information based on Hessian
Standard errors Robust.huber.white
Latent Variables:
Estimate Std.Err z-value P(>|z|)
rage_attacks =~
A301_01neu 1.000
A301_02neu 1.022 0.053 19.202 0.000
A301_04neu 1.584 0.076 20.717 0.000
A301_05neu 1.182 0.052 22.723 0.000
A301_06neu 0.908 0.068 13.403 0.000
A301_07neu 1.349 0.060 22.510 0.000
A301_08neu 1.402 0.077 18.300 0.000
A301_09neu 1.365 0.063 21.669 0.000
A301_10neu 1.061 0.065 16.429 0.000
A301_11neu 1.018 0.065 15.581 0.000
A301_13neu 1.165 0.061 19.213 0.000
A301_14neu 0.806 0.059 13.763 0.000
A301_15neu 1.413 0.076 18.672 0.000
A301_18neu 1.098 0.063 17.374 0.000
A301_19neu 0.985 0.058 17.060 0.000
A301_20neu 1.087 0.063 17.306 0.000
A301_21neu 1.033 0.064 16.015 0.000
A301_22neu 1.170 0.059 19.886 0.000
A301_23neu 0.557 0.061 9.126 0.000
A301_24neu 0.714 0.070 10.208 0.000
A301_25neu 1.154 0.072 15.980 0.000
A301_27neu 0.371 0.059 6.313 0.000
Variances:
Estimate Std.Err z-value P(>|z|)
.A301_01neu 0.186 0.010 19.405 0.000
.A301_02neu 0.309 0.017 18.593 0.000
.A301_04neu 0.358 0.029 12.479 0.000
.A301_05neu 0.323 0.021 15.252 0.000
.A301_06neu 0.208 0.021 10.037 0.000
.A301_07neu 0.282 0.017 16.895 0.000
.A301_08neu 0.283 0.023 12.207 0.000
.A301_09neu 0.322 0.019 16.905 0.000
.A301_10neu 0.466 0.028 16.563 0.000
.A301_11neu 0.193 0.018 10.960 0.000
.A301_13neu 0.232 0.017 13.280 0.000
.A301_14neu 0.148 0.012 12.641 0.000
.A301_15neu 0.396 0.027 14.515 0.000
.A301_18neu 0.413 0.026 16.158 0.000
.A301_19neu 0.203 0.015 13.399 0.000
.A301_20neu 0.244 0.019 12.817 0.000
.A301_21neu 0.291 0.019 15.048 0.000
.A301_22neu 0.278 0.016 17.848 0.000
.A301_23neu 0.109 0.014 7.767 0.000
.A301_24neu 0.297 0.025 11.777 0.000
.A301_25neu 0.216 0.016 13.142 0.000
.A301_27neu 0.227 0.028 8.262 0.000
rage_attacks 0.249 0.024 10.186 0.000
And now my questions:
1) Is it okay to ignore the error message when using WLMSV and use MLR (or ML)?
2) If so, would model 1 already be rejected in because the CFI is lower and the RMSEA higher than in the previous CFA? and if I understood it right, I couldn't interpret anything further anyway, could I?
car...@web.de
Nov 12, 2019, 5:46:38 AM11/12/19
to lav...@googlegroups.com
1) I don't think it's okay: https://psycnet.apa.org/record/2012-18631-001
2) Even if you ignore 1) - which I wouldn't do, your configural model already does not fit well. I would collect more data for group 2 and see what happens.
2) Even if you ignore 1) - which I wouldn't do, your configural model already does not fit well. I would collect more data for group 2 and see what happens.
Am 12.11.19, 10:33 schrieb Tina <martina...@gmail.com>:
--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/33723efe-5fbc-4ca9-9ff2-57613b462806%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages