how to interpret output of measurement invariance analysis

173 views
Skip to first unread message

Tina

unread,
Nov 12, 2019, 4:33:07 AM11/12/19
to lavaan

I try to test the measure invariance of a scale (with a 1-factor-structure) across two groups ( n= 645;  n= 127) and have problems with the application as well as the interpretation. I'm sure they're simple questions and I've already tried to research them, but since I'm doing it for the first time, I'm having trouble transferring the information I find to my case.

I use R (lavaan, semTools). Since the data is ordinal (4-point-scale), I tried to use WLSMV as an estimator first, but got this error message  „lavaan WARNING: number of observations (127) too small to compute Gamma in group: 2" because the second group is quite small. Then I did the analysis with MLR and with ML with these results: 
 
ML - Measurement invariance models:
 
Model 1 : fit.configural
Model 2 : fit.loadings
Model 3 : fit.intercepts
Model 4 : fit.means
 
Chi-Squared Difference Test
 
                Df   AIC   BIC  Chisq Chisq diff Df diff Pr(>Chisq)   
fit.configural 418 26263 26876 1625.9                                 
fit.loadings   439 26302 26818 1706.8     80.911      21  5.684e-09 ***
fit.intercepts 460 26342 26760 1788.9     82.150      21  3.519e-09 ***
fit.means      461 26417 26831 1866.6     77.714       1  < 2.2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
 
 
Fit measures:
 
                 cfi rmsea cfi.delta rmsea.delta
fit.configural 0.868 0.087        NA          NA
fit.loadings   0.862 0.086     0.007       0.000
fit.intercepts 0.855 0.087     0.007       0.000
fit.means      0.847 0.089     0.008       0.002
 
MLR - Measurement invariance models:
 
Model 1 : fit.configural
Model 2 : fit.loadings
Model 3 : fit.intercepts
Model 4 : fit.means
 
Scaled Chi-Squared Difference Test (method = “satorra.bentler.2001”)
 
lavaan NOTE:
    The “Chisq” column contains standard test statistics, not the
    robust test that should be reported per model. A robust difference
    test is a function of two standard (not robust) statistics.
 
                Df   AIC   BIC  Chisq Chisq diff Df diff Pr(>Chisq)   
fit.configural 418 26263 26876 1625.9                                 
fit.loadings   439 26302 26818 1706.8     80.954      21  5.590e-09 ***
fit.intercepts 460 26342 26760 1788.9     77.652      21  1.985e-08 ***
fit.means      461 26417 26831 1866.6    205.604       1  < 2.2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

 

 

Fit measures:

 

               cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta

fit.configural      0.883        0.069               NA                 NA

fit.loadings        0.874        0.070            0.008              0.001

fit.intercepts      0.866        0.070            0.008              0.001

fit.means           0.857        0.073            0.009              0.002

 

Here are the results from the previous CFA: 


Estimator                                         ML
  Optimization method                           NLMINB
  Number of free parameters                         44
                                                     
  Number of observations                           772
                                                     
Model Test User Model:
                                              Standard      Robust
  Test Statistic                              1421.567     894.988
  Degrees of freedom                               209         209
  P-value (Chi-square)                           0.000       0.000
  Scaling correction factor                                  1.588
    for the Yuan-Bentler correction (Mplus variant)
 
Model Test Baseline Model:
 
  Test statistic                             11305.724    7136.422
  Degrees of freedom                               231         231
  P-value                                        0.000       0.000
  Scaling correction factor                                  1.584
 
User Model versus Baseline Model:
 
  Comparative Fit Index (CFI)                    0.891       0.901
  Tucker-Lewis Index (TLI)                       0.879       0.890
                                                                 
  Robust Comparative Fit Index (CFI)                         0.900
  Robust Tucker-Lewis Index (TLI)                            0.890
 
Loglikelihood and Information Criteria:
 
  Loglikelihood user model (H0)             -13845.490  -13845.490
  Scaling correction factor                                  2.020
      for the MLR correction                                     
  Loglikelihood unrestricted model (H1)             NA          NA
  Scaling correction factor                                  1.663
      for the MLR correction                                      
                                                                  
  Akaike (AIC)                               27778.981   27778.981
  Bayesian (BIC)                             27983.536   27983.536
  Sample-size adjusted Bayesian (BIC)        27843.815   27843.815
 
Root Mean Square Error of Approximation:
 
  RMSEA                                          0.087       0.065
  90 Percent confidence interval - lower         0.082       0.062
  90 Percent confidence interval - upper         0.091       0.069
  P-value RMSEA <= 0.05                          0.000       0.000
                                                                 
  Robust RMSEA                                               0.082
  90 Percent confidence interval - lower                     0.077
  90 Percent confidence interval - upper                     0.088
 
Standardized Root Mean Square Residual:
 
  SRMR                                           0.045       0.045
 
Parameter Estimates:
 
  Information                                      Observed
  Observed information based on                     Hessian
  Standard errors                        Robust.huber.white
 
Latent Variables:
                   Estimate  Std.Err  z-value  P(>|z|)
  rage_attacks =~                                     
    A301_01neu        1.000                          
    A301_02neu        1.022    0.053   19.202    0.000
    A301_04neu        1.584    0.076   20.717    0.000
    A301_05neu        1.182    0.052   22.723    0.000
    A301_06neu        0.908    0.068   13.403    0.000
    A301_07neu        1.349    0.060   22.510    0.000
    A301_08neu        1.402    0.077   18.300    0.000
    A301_09neu        1.365    0.063   21.669    0.000
    A301_10neu        1.061    0.065   16.429    0.000
    A301_11neu        1.018    0.065   15.581    0.000
    A301_13neu        1.165    0.061   19.213    0.000
    A301_14neu        0.806    0.059   13.763    0.000
    A301_15neu        1.413    0.076   18.672    0.000
    A301_18neu        1.098    0.063   17.374    0.000
    A301_19neu        0.985    0.058   17.060    0.000
    A301_20neu        1.087    0.063   17.306    0.000
    A301_21neu        1.033    0.064   16.015    0.000
    A301_22neu        1.170    0.059   19.886    0.000
    A301_23neu        0.557    0.061    9.126    0.000
    A301_24neu        0.714    0.070   10.208    0.000
    A301_25neu        1.154    0.072   15.980    0.000
    A301_27neu        0.371    0.059    6.313    0.000
 
Variances:
                   Estimate  Std.Err  z-value  P(>|z|)
   .A301_01neu        0.186    0.010   19.405    0.000
   .A301_02neu        0.309    0.017   18.593    0.000
   .A301_04neu        0.358    0.029   12.479    0.000
   .A301_05neu        0.323    0.021   15.252    0.000
   .A301_06neu        0.208    0.021   10.037    0.000
   .A301_07neu        0.282    0.017   16.895    0.000
   .A301_08neu        0.283    0.023   12.207    0.000
   .A301_09neu        0.322    0.019   16.905    0.000
   .A301_10neu        0.466    0.028   16.563    0.000
   .A301_11neu        0.193    0.018   10.960    0.000
   .A301_13neu        0.232    0.017   13.280    0.000
   .A301_14neu        0.148    0.012   12.641    0.000
   .A301_15neu        0.396    0.027   14.515    0.000
   .A301_18neu        0.413    0.026   16.158    0.000
   .A301_19neu        0.203    0.015   13.399    0.000
   .A301_20neu        0.244    0.019   12.817    0.000
   .A301_21neu        0.291    0.019   15.048    0.000
   .A301_22neu        0.278    0.016   17.848    0.000
   .A301_23neu        0.109    0.014    7.767    0.000
   .A301_24neu        0.297    0.025   11.777    0.000
   .A301_25neu        0.216    0.016   13.142    0.000
   .A301_27neu        0.227    0.028    8.262    0.000
    rage_attacks      0.249    0.024   10.186    0.000

 

And now my questions:

1) Is it okay to ignore the error message when using WLMSV and use MLR (or ML)?

2) If so, would model 1 already be rejected in because the CFI is lower and the RMSEA higher than in the previous CFA? and if I understood it right, I couldn't interpret anything further anyway, could I?

 

 

car...@web.de

unread,
Nov 12, 2019, 5:46:38 AM11/12/19
to lav...@googlegroups.com
1) I don't think it's okay: https://psycnet.apa.org/record/2012-18631-001
2) Even if you ignore 1) - which I wouldn't do, your configural model already does not fit well. I would collect more data for group 2 and see what happens.
Am 12.11.19, 10:33 schrieb Tina <martina...@gmail.com>:
--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/33723efe-5fbc-4ca9-9ff2-57613b462806%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages