CFA Fit with lavaan: Loading Estimates Do Not Match MPlus

[henn...@umn.edu]

I am performing a congeneric latent Classical Test Theory model fit using the cfa() function in the lavaan package.  The data set concerns a test with 20 items and observations on 500 examinees.  Individual responses are 1/0, with "1" indicating a successful response.

 

I find the factor loading estimates from lavaan to be somewhat different from those obtained from MPlus.  I am hoping you can help me to understand why this is the case.

 

Please find attached a pdf file summarizing input code and parameter estimates and a txt file containing the observations.

 

Thank you for your assistance.

 

[Jeremy Miles]

Was the same estimator used? Do things like the sample size, chi-square and df match?

(If you can send something which allows people to run it easily, that's easier for use to try it, and which contains your complete code so that we can run exactly what you ran, for example we don't know what item.list contains. )

Here's a small example, which gives me the same results from Lavaan and Mplus.

R code:

setwd("c:/temp")

library(lavaan)

set.seed(12345)

df <- as.data.frame(matrix(rbinom(prob=0.4, size=1, 2000), ncol=4))

df.n <- df

names(df.n) <- c("q1", "q2", "q3", "q4")

mod2Fit <- "s =~ q1 + q2 + q3 + q4" 

cong.fit <- cfa(mod2Fit, data=df.n, std.lv=TRUE, 

           mimic="Mplus", parameterization="delta", ordered=names(df.n))

summary(cong.fit)

write.table(df, "test.dat", row.names=FALSE, col.names=FALSE)

Mplus code:

DATA: FILE = c:\temp\test.dat; 

VARIABLE:  NAMES ARE q1-q4; 

CATEGORICAL ARE q1-q4; 

MODEL: 

Skill by q1* q2-q4 ; 

Skill@1;

Hope that helps,

Jeremy

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at http://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

[Lisa Henn]

lavaan reports it uses DWLS.  MPlus reports it uses WLSMV.  As near as I can tell from the documentation, these estimators are the same.

 

Both software packages report having read all 500 records from the data file.

 

MPlus output cautions that the Chi-square value is not to be trusted with the WLSMV estimator.  So I'm loathe to put much weight by a comparison of that particular item.  The degrees of freedom are the same, though.

 

I have also tried different starting conditions and convergence criteria; these seem not to change the estimates any.

 

I originally omitted the code for item.list and itlist because they are generated based on the number of columns in the data set (I am running many other cases), and I thought it would be a distraction.  I have added that code back in to my summary document, though. 

 

Please find attached an updated summary of code and results, lavaan_vs_MPlus_2.pdf.  For convenience, I have re-attached the data set test_data_lavaan_vs_mplus.txt.

 

Thank you again for your assistance.

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/m8qpYSIyqdo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.

[yrosseel]

Which version of Mplus are you using?

Using Mplus 7.11, I get identical results as lavaan (full output in attach)

Chi-Square Test of Model Fit

Value 168.705*
Degrees of Freedom 170
P-Value 0.5137

And for the estimates:


Two-Tailed
Estimate S.E. Est./S.E. P-Value

SKILL BY
ITEM1 0.820 0.050 16.321 0.000
ITEM2 0.832 0.030 28.071 0.000
ITEM3 0.728 0.038 18.966 0.000
ITEM4 0.795 0.030 26.126 0.000
ITEM5 0.774 0.051 15.208 0.000
ITEM6 0.794 0.045 17.530 0.000
ITEM7 0.846 0.029 29.221 0.000
ITEM8 0.816 0.035 23.463 0.000
ITEM9 0.902 0.035 25.524 0.000
ITEM10 0.761 0.037 20.334 0.000
ITEM11 0.850 0.027 31.458 0.000
ITEM12 0.770 0.037 20.698 0.000
ITEM13 0.841 0.028 30.472 0.000
ITEM14 0.809 0.038 21.475 0.000
ITEM15 0.773 0.041 18.647 0.000
ITEM16 0.854 0.039 21.701 0.000
ITEM17 0.942 0.023 40.507 0.000
ITEM18 0.806 0.033 24.302 0.000
ITEM19 0.902 0.026 35.060 0.000
ITEM20 0.861 0.037 23.000 0.000


However, when I use Mplus 4 (again, full output in attach), I get the
same results as you report for Mplus. So, they must have changed
something that is related to the way they deal with empty cells in
bivariate tables. You should ask the Mplus people what they did back
then (and for your version). I know what they do in Mplus 7, because
lavaan is doing the same. And if you get any answers, please let us know.

Yves.

[Jeremy Miles]

That is very weird, because I tried it with Mplus 7.11 and I get the same results as Lisa, and Mplus v4. (I used Yves's syntax and pasted it into Mplus.)

Could it be that Windows and Linux versions of Mplus give different results?

Jeremy

--
You received this message because you are subscribed to the Google Groups "lavaan" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+unsubscribe@googlegroups.com.

[yrosseel]

On 05/14/2014 09:09 PM, Jeremy Miles wrote:
>
> That is very weird, because I tried it with Mplus 7.11 and I get the
> same results as Lisa, and Mplus v4. (I used Yves's syntax and pasted it
> into Mplus.)
>
> Could it be that Windows and Linux versions of Mplus give different results?

That could be, since I used linux for Mplus 7, and windows for Mplus 4
(there is no native linux version of Mplus 4).

I don't have Mplus 7 for windows installed. However, when I used Mplus 6
under windows, I again get the lavaan results... (see attach). Could
this be a 32-bit vs 64-bit issue?

I also tried lavaan on this example on both linux and windows, and the
results are identical. But both machines are 64-bit.

Yves.

[Jeremy Miles]

Ah, I wonder if that's it. I'm using 32 bit. When I tried a smaller sample from the dataset (just the first four variables) I got the same answers from Lavaan and Mplus.

J

Yves.

[Lisa Henn]

Hmmm.  I appear to be running MPlus 7.0 instead of 7.11 (the header of the output simply reads "Mplus VERSION 7" and the Help > About window reads "Mplus Version 7 Base Program and Combination Add-on (64 bit)").  But it's a 64-bit machine, running Windows 7.  (For the sake of completeness, I will note that I am running lavaan 0.5-16 under R 3.0.2 64-bit on the same machine.)

 

I looked through the release notes for Mplus Versions 4 - 7.2 but didn't see anything that was obviously related to this problem. 

 

I too have tried fitting a model with just the first four variables.  The match between lavaan and Mplus is much closer, but not exact.  The loading estimates differ in the third decimal place.  By contrast, the thresholds have always matched to all four digits displayed.  Please see attached the comparison.

 

Did you find your results matched or were very close?

 

It seems like the next step is to contact Mplus support.  I'll hold off for a few hours in case this list has anything to add based on this latest information.

 

Thanks,

Lisa

--

You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/m8qpYSIyqdo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.

[yrosseel]

On 05/15/2014 03:45 PM, Lisa Henn wrote:
> Hmmm. I appear to be running MPlus 7.0

In fact, my original 'linux' version was also version 7 (not 7.11). But
I don't think this makes a difference.

> bit)"). But it's a 64-bit machine, running Windows 7.

So, it appears to be linux vs windows after all.

> It seems like the next step is to contact Mplus support.

Keep us posted.

Yves.

[Lisa Henn]

The Mplus team will investigate why different versions of Mplus produce different answers.  They said they would not be able to do this until after Memorial Day.

 

In the meantime, I hope to understand better how lavaan handles the empty cells.  I see in the cfa() documentation that the lavaan default depends on the mimic setting.  It also appears that the default is to add 0.5 to small tables, but to adjust the values in the other cells to maintain the margins.  Do I understand correctly? 

 

I tried calling cfa() with zero.keep.margins set to FALSE with my test case.  A handful of the parameter estimates are slightly different from before (in the third decimal place), but do not match my Mplus results.

 

Are there other aspects about which I should be aware with regard to how lavaan handles empty cells?

 

Thank you for your continued assistance.

 

Lisa

--

You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/m8qpYSIyqdo/unsubscribe.

To unsubscribe from this group and all its topics, send an email to lavaan+unsubscribe@googlegroups.com.

[yrosseel]

On 05/22/2014 10:11 PM, Lisa Henn wrote:
> In the meantime, I hope to understand better how lavaan handles the
> empty cells. I see in the cfa() documentation that the lavaan default
> depends on the mimic setting. It also appears that the default is to
> add 0.5 to small tables, but to adjust the values in the other cells to
> maintain the margins. Do I understand correctly?

For 2x2 tables (only!), the default is to add 0.5 to the empty cell, and
adjust the other cells, so that the margins remain the same.

For larger tables, the default is to add nothing.

The zero.add= and zero.keep.margins= arguments can be set to change this
behavior.

> I tried calling cfa() with zero.keep.margins set to FALSE with my test
> case. A handful of the parameter estimates are slightly different from
> before (in the third decimal place), but do not match my Mplus results.
> Are there other aspects about which I should be aware with regard to how
> lavaan handles empty cells?

Mplus uses zero.keep.margins=TRUE for 2x2 tables, as far as I can tell.

Yves.

[Lisa Henn]

The Mplus team responded to my inquiry.  They were unable to reproduce the result in which the parameter estimates vary with Mplus version.  This was very surprising and disappointing for me.

 

They mentioned that, by default, Mplus adds 0.5 to empty cells.  They also felt that the WLSMV estimator should not be used in this circumstance in which there are various empty cells, because of the estimator's reliance on tetrachoric correlations.  They thought ML and Bayesian estimation would work better in this case.

 

Is there anything more that I should know about how lavaan handles the empty cells?

 

Thank you for your assistance.

 

Sincerely,

Lisa Henn

Yves.

[yrosseel]

On 06/05/2014 11:18 PM, Lisa Henn wrote:
> The Mplus team responded to my inquiry. They were unable to reproduce
> the result in which the parameter estimates vary with Mplus version.

Do you mean platform? The differences we observed (if I remember
correctly) were over platforms (windows vs linux). I don't think the
version number matters.

> They mentioned that, by default, Mplus adds 0.5 to empty cells.

Only for 2x2 tables.

> also felt that the WLSMV estimator should not be used in this
> circumstance in which there are various empty cells, because of the
> estimator's reliance on tetrachoric correlations. They thought ML and
> Bayesian estimation would work better in this case.

That might be true.

> Is there anything more that I should know about how lavaan handles the
> empty cells?

I think I provided all the information there is to know.

Yves.

[Lisa Henn]

When I reported the issue, I sent them this table, compiled from our correspondence here:

 

Mplus Version    OS              Loading Estimates

4.1              Windows 64 bit  matched my Mplus output

7                Linux 64 bit    matched lavaan

6.1              Windows 64 bit  matched lavaan

7.11             Windows 32 bit  matched my Mplus output

 

I assume that in testing it they tried the combinations that I specified.  But they did not share those details.

 

They did say they would review Mplus output files if anyone from here wanted to send them.  Such persons would need to have a current Mplus license. 

 

Thanks again for your help.

 

Lisa

Yves.