MLR estimation - huge difference between P-value and robust P-value

[Shajar]

All my data is continues non-normal  distributed. I use MLR estimation.

I get a huge difference between ML P-value (0.004) and robust P-value (0.599).

Is this result "normal"? i.e. should I trust the robust P-value and accept the model?

Follows is the output:

lavaan 0.6-4 ended normally after 30 iterations

  Optimization method                           NLMINB

  Number of free parameters                         18

                                                  Used       Total

  Number of observations                           376         378

  Estimator                                         ML      Robust

  Model Fit Test Statistic                      51.448      25.531

  Degrees of freedom                                28          28

  P-value (Chi-square)                           0.004       0.599

  Scaling correction factor                                  2.015

    for the Yuan-Bentler correction (Mplus variant)

Model test baseline model:

  Minimum Function Test Statistic             1494.478     819.813

  Degrees of freedom                                42          42

  P-value                                        0.000       0.000

User model versus baseline model:

  Comparative Fit Index (CFI)                    0.984       1.000

  Tucker-Lewis Index (TLI)                       0.976       1.005

  Robust Comparative Fit Index (CFI)                         1.000

  Robust Tucker-Lewis Index (TLI)                            1.005

Loglikelihood and Information Criteria:

  Loglikelihood user model (H0)              -1320.150   -1320.150

  Scaling correction factor                                  8.766

    for the MLR correction

  Loglikelihood unrestricted model (H1)      -1294.426   -1294.426

  Scaling correction factor                                  4.657

    for the MLR correction

  Number of free parameters                         18          18

  Akaike (AIC)                                2676.301    2676.301

  Bayesian (BIC)                              2747.033    2747.033

  Sample-size adjusted Bayesian (BIC)         2689.924    2689.924

Root Mean Square Error of Approximation:

  RMSEA                                          0.047       0.000

  90 Percent Confidence Interval          0.026  0.067       0.000  0.027

  P-value RMSEA <= 0.05                          0.563       1.000

  Robust RMSEA                                               0.000

  90 Percent Confidence Interval                             0.000  0.050

Standardized Root Mean Square Residual:

  SRMR                                           0.017       0.017

[Terrence Jorgensen]

Is this result "normal"?

I don't usually see scaling factors that high, but the further they are from 1, the more different the results will be (i.e., the less you can trust the standard test).  

should I trust the robust P-value and accept the model?

The whole point of a robust test is that it corrects the standard one when its normality assumption is violated. So the Type I error rates are closer to nominal for the robust than the standard test.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Shajar]

Thanks!