Robust/scaled statistics in CFA on ordinal response data

[Keri]

Hello, I am conducting a simple CFA on ordinal response data using lavaan (version 0.6-2) [code below]. There are two issues that are confusing me.

1) when I output my model using summary() [see code below], the summary of the cfa under the "robust" column is reporting the fit statistics that are listed as "scaled" in the output for "fitmeasures". [see attached output]

2) the "robust" fit statistics in the output from fitmeasures() (and in some places in the model output) shows NAs. [see attached output]

My confusion is 1) whether I should be using the scaled or robust statistics -- and 2), why the scaled statistics are reported under the "robust" column in the summary() output? I found the guidance on the lavaan website below regarding an update made to the 0.5-21 lavaan version stating the scaled statistics are incorrect and one should use the robust instead. If this is the case, my robust statistics are all NAs and I'm confused regarding whether this is a bug or I am doing something wrong...? Any thoughts on this would be helpful. Thank you!

From http://lavaan.ugent.be/history/dot5.html: 
"robust RMSEA and CFI values are now computed correctly, following Brosseau-Liard, P. E., Savalei, V., and Li, L. (2012), and Brosseau-Liard, P. E. and Savalei, V. (2014); in the output of fitMeasures(), the 'new' ones are called cfi.robust and rmsea.robust, while the 'old' ones are called cfi.scaled and rmsea.scaled "

R code: 

model1 <- 'ee = ~e1 + e2 + e3 + e4 + e5 + e6 + e7 + e8 + e9

dp = ~ dp1 + dp2 + dp3 + dp4 + dp5

pa = ~p1 + p2 + p3 + p4 + p5 + p6 + p7 + p8'

fit <- cfa(model1, data = mbi, ordered = c("e1", "e2", "e3", "e4", 

       "e5", "e6", "e7", "e8", "e9", "dp1", "dp2", "dp3", "dp4",

       "dp5","p1", "p2", "p3", "p4","p5", "p6", "p7", "p8"),

       std.lv = TRUE, missing = "pairwise") 

summary(fit, fit.measures = TRUE, rsquare = TRUE, standardized = TRUE)

fitmeasures(fit)

[Terrence Jorgensen]

2) the "robust" fit statistics in the output from fitmeasures() (and in some places in the model output) shows NAs. 

The correct calculation of robust fit measures proposed by Brosseau-Liard & Savalei has only been proposed for mean-adjusted chi-squared statistics (estimators MLM, MLR, WLSM, ULSM), not mean- and variance-adjusted (or "scaled & shifted") statistics (estimators MLMV, WLSMV).  WLSMV is the default for ordinal data, so that is why cfi.robust, rmsea.robust, etc., will be NA.  The *.scaled fit measures are simply using the original formulas for non-robust chi-squared, but plugging in the robust chi-squared (which, as the articles explain, will not be consistent estimators of the true population values).

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

[Keri]

Thanks for your response -- given this do you recommend reporting the non-scaled fit statistics/indices over the scaled versions? 

[Terrence Jorgensen]

do you recommend reporting the non-scaled fit statistics/indices over the scaled versions? 

If robust estimation is necessary, then report the scaled statistic and any scaled (or robust, if available) fit indices that are derived from it.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[joh4nd]

@Terrence Jorgensen, do you mind (five years later) expanding a bit on the difference between scaled and robust versions of the fit criteria?

As far as I understand, the criteria labelled "scaled" are old, while the "robust" are new and based in the evidence provided by the two papers by Brosseau-Liard and various colleagues.

Would you agree that the scaled versions are for categorial variables but not necessarily robust estimates, while the robust versions are for robust estimates in general? So that would mean applying any of the below estimators to fit a model would immediately recommend interpreting the robust fitmeasures() described in lavOptions (underlined in the quotation below)?

From lavOptions {lavaan}:

> The estimator to be used. Can be one of the following: "ML" for maximum likelihood, "GLS" for (normal theory) generalized least squares, "WLS" for weighted least squares (sometimes called ADF estimation), "ULS" for unweighted least squares, "DWLS" for diagonally weighted least squares, and "DLS" for distributionally-weighted least squares. These are the main options that affect the estimation. For convenience, the "ML" option can be extended as "MLM", "MLMV", "MLMVS", "MLF", and "MLR". The estimation will still be plain "ML", but now with robust standard errors and a robust (scaled) test statistic. For "MLM", "MLMV", "MLMVS", classic robust standard errors are used (se="robust.sem"); for "MLF", standard errors are based on first-order derivatives (information = "first.order"); for "MLR", ‘Huber-White’ robust standard errors are used (se="robust.huber.white"). In addition, "MLM" will compute a Satorra-Bentler scaled (mean adjusted) test statistic (test="satorra.bentler") , "MLMVS" will compute a mean and variance adjusted test statistic (Satterthwaite style) (test="mean.var.adjusted"), "MLMV" will compute a mean and variance adjusted test statistic (scaled and shifted) (test="scaled.shifted"), and "MLR" will compute a test statistic which is asymptotically equivalent to the Yuan-Bentler T2-star test statistic (test="yuan.bentler.mplus"). Analogously, the estimators "WLSM" and "WLSMV" imply the "DWLS" estimator (not the "WLS" estimator) with robust standard errors and a mean or mean and variance adjusted test statistic. Estimators "ULSM" and "ULSMV" imply the "ULS" estimator with robust standard errors and a mean or mean and variance adjusted test statistic.

Regards,

Johan

[cla...@googlemail.com]

I would also be interested in the question, if reporting scaled fit indices when categorical data is used while robust test statistics are for robust estimates in general?

Looking forward to some insights on that issue!

Regards, 

 Claudia