DWLS v. Robust?

[golo...@gmail.com]

Hi,

In an output from a CFA with categorical variables. 

1. What is the difference between "DWLS" and "Robust"? 
2. The DWLS results are better than Robust results... are DWLS results acceptable for categorical variables?
3. Can I obtain the SRMR value from this output?

Thanks for any guidance,

Alex

[yrosseel]

On 06/29/2013 09:09 PM, golo...@gmail.com wrote:
> Hi,
>
> In an output from a CFA with categorical variables.
>

> 1. What is the difference between "DWLS" and "Robust"?

DWLS is the usual test statistic under diagonally-weighted least
squares. Robust is the 'robust' (or scaled) test statistic. This is the
one you should report.

> 2. The DWLS results are better than Robust results... are DWLS results
> acceptable for categorical variables?

The robust version is more reliable.

> 3. Can I obtain the SRMR value from this output?

The SRMR value is not defined for categorical data.

Yves.

[golo...@gmail.com]

Dear Yves,

Thank you very much!

Cordially,

Alex

[Alan Chan]

Dear Yves,

According to the previous discussion (cfa with categorical variable), may I ask so when I report the 'Robust', what is the exact name of the estimator? Is it WLSMV?

Thank you very much.

Best regards,

Alan

[Yves Rosseel]

On 06/13/2016 11:08 PM, Alan Chan wrote:
> Dear Yves,
>
> According to the previous discussion (cfa with categorical variable),
> may I ask so when I report the 'Robust', what is the exact name of the
> estimator? Is it WLSMV?

That is how it is called by Mplus. But it is a misnomer, for two
reasons. It should be called DWLSSS: we use diagonally weighted least
squares (DWLS) for estimation, and a scaled-shifted test statistic (SS),
which replaced the older mean-variance adjusted (MV) test statistic.

In the literature, the estimator is often referred to as 'three stage
robust diagonally least squares', for example in:

Katsikatsou, M., Moustaki, I., Yang-Wallentin, F., & Jöreskog, K. G.
(2012). Pairwise likelihood estimation for factor analysis models with
ordinal data. Computational Statistics & Data Analysis, 56(12), 4243-4258

The robust standard errors are often called sandwich-type robust
standard errors, and the 'robust' test statistic is the one described in
this document:

http://www.statmodel.com/download/WLSMV_new_chi21.pdf

Yves.

[Alan Chan]

Thank you very much. It is very clear!

[Victor Souza]

I get that the robust test statistic is better, but why did it worsen my CFI and TLI?

Check out this output.

lavaan (0.5-22) converged normally after 180 iterations

  Number of observations                          2280

  Estimator                                       DWLS      Robust

  Minimum Function Test Statistic            28398.270   24288.140

  Degrees of freedom                              6418        6418

  P-value (Chi-square)                           0.000       0.000

  Scaling correction factor                                  1.425

  Shift parameter                                         4362.804

    for simple second-order correction (Mplus variant)

Model test baseline model:

  Minimum Function Test Statistic           1337609.187   78273.008

  Degrees of freedom                              6555        6555

  P-value                                        0.000       0.000

User model versus baseline model:

  Comparative Fit Index (CFI)                    0.983       0.751

  Tucker-Lewis Index (TLI)                       0.983       0.746

  Robust Comparative Fit Index (CFI)                            NA

  Robust Tucker-Lewis Index (TLI)                               NA

Root Mean Square Error of Approximation:

  RMSEA                                          0.039       0.035

  90 Percent Confidence Interval          0.038  0.039       0.034  0.035

  P-value RMSEA <= 0.05                          1.000       1.000

  Robust RMSEA                                                  NA

  90 Percent Confidence Interval                                NA     NA

Standardized Root Mean Square Residual:

  SRMR                                           0.049       0.049

Parameter Estimates:

  Information                                 Expected

  Standard Errors                           Robust.sem

[Terrence Jorgensen]

I get that the robust test statistic is better, but why did it worsen my CFI and TLI?

The formulas for incremental fit indices provide a clue.  Fit fit statistic for your model did get smaller, but not to the same degree that the fit statistic of the baseline model got smaller.

  Estimator                                       DWLS      Robust

  Minimum Function Test Statistic            28398.270   24288.140

Model test baseline model:

  Minimum Function Test Statistic          1337609.187   78273.008

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

 

[Victor Souza]

I did notice that, I suppose my question should be:
Why did the DWLS report better reduction in test statistics than the Robust? If there is a point to making use of a Robust estimator, it is to get a better result than a ML estimator, which the Robust did not. The DWLS did provide a better result, and if I'm well informed is a good estimator for ordinal data.

[Victor Souza]

Also, considering that the actual Robust CFI and Robust TLI return NA, are the Scaled CFI and TLI (those are actually reported) reported in the Robust column good statistics, or are they simply placeholders, since they are the exact same ones as the ML ones? Are they better statistics than the ones in the DWLS column?

[Yves Rosseel]

On 11/09/2016 03:00 PM, Victor Souza wrote:
> I did notice that, I suppose my question should be: Why did the DWLS
> report better reduction in test statistics than the Robust?

As Terry said: the robust X2 went down for the model (which is always
good news), but the robust X2 for the baseline model went down too...

> is a point to making use of a Robust estimator

The point is to get more correct inference. Not to please the user.

Yves.

[Yves Rosseel]

On 11/09/2016 03:15 PM, Victor Souza wrote:
> Also, considering that the actual Robust CFI and Robust TLI return NA

Yes. No literature is available telling me how these values should be
computed.

> are the Scaled CFI and TLI (those are actually reported) reported in the
> Robust column good statistics, or are they simply placeholders

They are what lavaan (and everybody else, including Mplus) has been
reporting until now. I do not know how 'bad/good' they are, in
comparison with the yet-to-come 'new' versions.

Yves.

[Victor Souza]

>As Terry said: the robust X2 went down for the model (which is always 

>good news), but the robust X2 for the baseline model went down too... 

Well, yes. The question is *why* it went down by the same factor if the DWLS estimator did not.  I'm struggling to find this in the PDF you posted above. To put it simply, the way it is now is the same as if I did not use a robust estimator. It's exactly the same as using a maximum likehood estimator.

 

>The point is to get more correct inference. Not to please the user.

Using either statistic won't give me pleasure. What I do need to know is how to justify either position. How can you say it's "more correct" if it's exactly the same inference as the maximum likelihood, whereas vanilla DWLS should be more correct (i.e. yield lower false negatives) according to the statistics literature than it?

>They are what lavaan (and everybody else, including Mplus) has been 

>reporting until now. I do not know how 'bad/good' they are, in 

>comparison with the yet-to-come 'new' versions. 

In hindsight, it was stupid to suggest anyone could compare those statistics to yet-unknown ones. What I did ask as well, though, was why those were better than the DWLS. Especially considering that there isn't even a method for the proper robust CFI and TLI. 

[Yves Rosseel]

On 11/09/2016 06:11 PM, Victor Souza wrote:
>>As Terry said: the robust X2 went down for the model (which is always
>>good news), but the robust X2 for the baseline model went down too...
>
> Well, yes. The question is *why* it went down by the same factor if the
> DWLS estimator did not.

I guess because the baseline model fits really, really bad. The robust
DWLS X2 can better handle misspecified models (well, sort of).

> posted above. To put it simply, the way it is now is the same as if I
> did not use a robust estimator. It's exactly the same as using a maximum
> likehood estimator.

This is where you lost me. Did you use ML with categorical data?

> In hindsight, it was stupid to suggest anyone could compare those

Who you think is stupid?

> statistics to yet-unknown ones. What I did ask as well, though, was why
> those were better than the DWLS. Especially considering that there isn't
> even a method for the proper robust CFI and TLI.

And you lost me again.

As all of this seems unrelated to lavaan, perhaps you should consider
posting to the SEMNET discussion group.

Yves.

[Victor Souza]

Who you think is stupid? 

 

I was referring to my own question. 

As all of this seems unrelated to lavaan, perhaps you should consider 
posting to the SEMNET discussion group. 

 

That makes sense. Initially I was worried it could be related with the package. Thanks for your help.