DWLS v WLSMV totally different results in cfa does DWLS still use polychoric as Yves Rosseel said?

[Leya]

Hi,

I found an earlier question where someone asked if selecting DWLS automatically meant the cfa used the polychoric correlation matrix... Yves Rosseel answered and said it did... I have been doing CFA just using DWLS and was getting excellent model fits and nto havig to make any modifications, I even got a non significant test statistic (unusual with large sample). Anyway I then read I am supposed to declare my variables as ordered when running cfa (my data are ordinal and multivariate non-normal). I did this and ran the cfa and the model/s are nowhere near an acceptable fit and short of allowing all variables to covary which is nonsensical I cannot improve either of my models whih most have strong foundations in EFA based on polychoric correlations and appropriate extraction methods etc.. 

Does anyone have any idea why this second approach produced such bad fit when the DWLS approach had great fit? And 2, is the DWLS estimator still using polychoric correlations (gives me a justification for going back to it!)

Many thanks

[Terrence Jorgensen]

Using DWLS will estimate polychorics among variables declared as ordered.  So your model fit changed because you are technically fitting it to different data (a 5-category indicator is not the same as a continuous indicator that happens to range from 1 to 5).

Depending on how many categories you have, the amount of asymmetry in the distributions, and whether you are more interested in measurement or structural parameters, treating them as continuous using MLR might be a better option (if you have 5+ categories).  Here is an enlightening paper:

http://psych.colorado.edu/~willcutt/pdfs/Rhemtulla_2012.pdf

Terry

[Bruce Weaver]

Yes, that is an enlightening paper.  I have some colleagues who want to carry out two-group CFA, but of course, they have 5-point Likert type items with some missing data.  I was struggling to find a way that could deal simultaneously with the ordinal nature of the data and the missing data.  After reading the Rhemtulla et al article, I decided to try treating the data as ordinal, and ended up using MLR, as it seemed to be the only way to avoid listwise deletion. 

In private correspondence, Mijke Rhemtulla said I would need to use a robust test statistic, and recommended either the mean-adjusted “Satorra-Bentler” or a mean-and-variance-adjusted robust test.  With MLR, one gets "a scaled test statistic that is (asymptotically) equal to the Yuan-Bentler test statistic."  I wonder if Terry or anyone else could comment on the suitability of that robust test (compared to the two Rhemtulla recommended) when treating 5-or-more-point ordinal items as continuous. 

Thanks,
Bruce

[Mark Seeto]

Bruce, I was under the impression that in lavaan, if you have ordinal variables then there's no choice but to use DWLS/WLSMV (not MLR). With DWLS/WLSMV, you can use missing="pairwise".

[Bruce Weaver]

Hi Mark.  I believe that if you have to declare variables as ordered if you want to treat them as ordinal--see http://lavaan.ugent.be/tutorial/cat.html.  Otherwise, they will be treated as continuous. 

Cheers,
Bruce

[Mark Seeto]

Yes, sorry for being unclear. What I meant was that if you declare variables as ordered, then lavaan will use DWLS/WLSMV. So I take it that in your previous post, you meant that you first tried declaring them as ordered, but you ended up not declaring them as ordered and using MLR to avoid listwise deletion. I don't think that's necessary though, because you can declare variables as ordered and use missing="pairwise". I seem to remember that with binary variables, declaring them as ordered gives much better results than using MLR, but for your 5-level ordered variables maybe MLR will be ok.

[yrosseel]

On 10/13/2015 10:21 PM, Bruce Weaver wrote:
> Yes, that is an enlightening paper. I have some colleagues who want to
> carry out two-group CFA, but of course, they have 5-point Likert type
> items with some missing data. I was struggling to find a way that could
> deal simultaneously with the ordinal nature of the data and the missing
> data. After reading the Rhemtulla et al article, I decided to try
> treating the data as ordinal, and ended up using MLR, as it seemed to be
> the only way to avoid listwise deletion.

To be clear: in lavaan (0.5.19), MLR (or ML or MLM) does NOT work for
variables that are declared to be ordered. Attempting to do so, will
result in the error message:

Error in lavaan::lavaan(model = model, data = Data, estimator = "MLR", :
lavaan ERROR: estimator ML for ordered data is not supported yet. Use
WLSMV instead.

However, for 5-point likert type items, it is often not unreasonable to
treat them as if they are continuous, and use MLR in this setting (so
without the ordered= argument).

> In private correspondence, Mijke Rhemtulla said I would need to use a
> robust test statistic, and recommended either the mean-adjusted
> “Satorra-Bentler” or a mean-and-variance-adjusted robust test. With
> MLR, one gets "a scaled test statistic that is (asymptotically) equal to
> the Yuan-Bentler test statistic."

Yes, that is fine. The Yuan-Bentler test statistic is a 'modern' version
of the (older) Satorra-Bentler test statistic, which can handle missing
data. So you can use estimator = "MLR" in combination with missing = "fiml"

Yves.

[yrosseel]

On 10/14/2015 01:43 AM, Mark Seeto wrote:
> Bruce, I was under the impression that in lavaan, if you have ordinal
> variables then there's no choice but to use DWLS/WLSMV (not MLR). With
> DWLS/WLSMV, you can use missing="pairwise".

That is correct (for 0.5-19).

Yves.

[yrosseel]

On 10/14/2015 03:14 AM, Bruce Weaver wrote:
> Hi Mark. I /believe/ that if you have to declare variables as ordered

> if you want to treat them as ordinal--see
> http://lavaan.ugent.be/tutorial/cat.html. Otherwise, they will be
> treated as continuous.

True indeed. But note that if they are declared as 'ordered' in the
data.frame, they will still be treated as ordinal.

Yves.

[Bruce Weaver]

Yves, thanks for clarifying re the Yuan-Bentler test.  Merci!

Also, I now see my wording was a bit careless in the first paragraph below. 

What I meant to say was that I was finding it difficult to treat 5-point Likert items as ordinal and at the same time avoid listwise deletion.  (To be clear, some of those difficulties were arising in other software packages, like Stata 13's 'sem' and 'gsem'.) 

I then discovered the Rhemtulla et al. simulation study, and decided that life would be easier (both for me, and for eventual readers of the work) if I treated the 5-point Likert items as continuous (because methods that avoid listwise deletion seem more available for continuous variables).  But I was still having trouble trying to do that AND get a robust test in Stata 13.  (It's probably possible in Stata 14, which as added the Satora-Bentler test--see links below.)  So at that point, I decided to have a go at trying it with lavaan; and MLR seemed to work quite well.

I apologize for any confusion caused by my less than careful wording in the earlier post. 

Stata 14 Info:

• http://www.stata.com/features/structural-equation-modeling/
• http://www.stata.com/new-in-stata/sem-satorra-bentler/
• http://www.stata.com/features/overview/generalized-sem/
  

Cheers,
Bruce

[yrosseel]

On 10/14/2015 04:43 PM, Bruce Weaver wrote:
> But I was still having trouble trying to do that AND get a
> robust test in Stata 13. (It's probably possible in Stata 14, which as

> added the Satora-Bentler test--see links below.) So at /that point/, I

> decided to have a go at trying it with lavaan; and MLR seemed to work
> quite well.

As a side note, I find it rather interesting to see that Stata's sem
function looks and feels a lot like lavaan. For example, when showing
the parameter estimates, they also (like lavaan) do not show p-values
and z-statistics for (residual) variances...

Yves.