chi square difference test WLSMV

[ptronruin]

Hi all,

Can anyone help me with this? How can I run a chi-square difference test for two nested models fitted with WLSMV from lavaan's fitMeasures() output?

Would it be valid to take this approach: https://stats.idre.ucla.edu/mplus/faq/how-can-i-compute-a-chi-square-test-for-nested-models-with-the-mlr-or-mlm-estimators/

which does the following:

c0 = scaling correction factor for the null model.

c1 = scaling correction factor for the alternative model.

d0 = degrees of freedom for the null model.

d1 = degrees of freedom for the alternative model.

SB0 = Satorra-Bentler scaled chi-square value for the null model.

SB1 = Satorra-Bentler scaled chi-square value for the alternative model.

In order to calculate the test statistic, T, we first need to calculate the value cd:

cd = (d0 * c0 - d1 * c1)/(d0 - d1)

Once we have calculated cd, we can compute:

T = (SB0 * c0 - SB1 * c1)/cd

T is distributed chi-square with degrees of freedom:

df = d0 - d1

From lavaan's fitMeasures output, which one is the Satorra-Bentler chi-square? Is it simply the one that says "chisq.scaled"?

I cannot do this in lavaan directly, because I only have the estimated parameters and the goodness of fit measures.

Any advice would be very highly appreciated!

Thanks!

P

[Mauricio Garnier-Villarreal]

P

If you use the default nested model compariosn function in lavaan lavTestLRT (or the anova() function ), when comparisng models ran with WLS, they will be compared with the Sattorra adjusted method. Lavaan does this for you automatically.

Like in thid example, you an see it says that its doing the Satoora.2000 by default for WLS models, but you can choose other methods

HS.model <- '
    visual  =~ x1 + b1*x2 + x3
    textual =~ x4 + b2*x5 + x6
    speed   =~ x7 + b3*x8 + x9
'
fit1 <- cfa(HS.model, data = HolzingerSwineford1939,
            estimator="wlsmv")
fit0 <- cfa(HS.model, data = HolzingerSwineford1939,
            estimator="wlsmv",orthogonal = TRUE)
lavTestLRT(fit1, fit0)

[Yves Rosseel]

On 9/29/20 12:05 PM, ptronruin wrote:
> Hi all,
>
> Can anyone help me with this? How can I run a chi-square difference test
> for two nested models fitted with WLSMV from lavaan's fitMeasures() output?

For WLSMV, lavaan uses the Satorra.2000 method. Unfortunately, this one
can not be done 'by hand'. It needs access to the Gamma matrix, the
information matrix, etc...

Yves.

[ptronruin]

Thank you Yves!

I have tried running lavTestLRT(), but I get the following error message:

"error in MASS::ginv(Delta1)%*%Delta0: non-conformable arguments"

anova() gives me the same error.

I'm sorry but I cannot post data to reproduce the issue since I'm working in a secure lab.

That's why I thought I could circumvent this by getting the difftest "by hand" using the fitMeasures().

Any idea why I could be getting that error?

Alternatively, would it be too hard to do it by hand?

Thank you!

P

[Yves Rosseel]

On 9/29/20 3:48 PM, ptronruin wrote:
> I have tried running lavTestLRT(), but I get the following error message:
> "error in MASS::ginv(Delta1)%*%Delta0: non-conformable arguments"

> Any idea why I could be getting that error?

Hm. Are the two models nested? Could you try again with A.method = "exact"?

> Alternatively, would it be too hard to do it by hand?

You can see in the source code how it is done. The relevant function is

lavaan:::lav_test_diff_Satorra2000

Yves.

[Mauricio Garnier-Villarreal]

P

Can you share the full code? So we can see the models at least, without seeing the data

[ptronruin]

Thank you Mauricio and Yves!

I think I've just made a rooky mistake!

By following Yves's suggestion of A.method="exact", I could see that the problems was that in the eyes of lavaan my models weren't nested.

I hadn't specified the baseline model with the additional parameters fixed to 0.

I'm leaving this here in case someone else makes the same mistake. So, this is what I had done (borrowing from Mauricio's example):

m0 <- '
    visual  =~ x1 + x2 + x3 +  x4 
    textual =~ x5+ x6 + x7 +  x8
    speed   =~ x8 + x9 + x10 
'

m1 <- '

    visual  =~ x1 + x2 + x3 +  x4 
    textual =~ x5+ x6 + x7 +  x8
    speed   =~ x8 + x9 + x10 + x11
'

lavTestLRT(m0, m1) 

Error!

This is what I should've done:

m0b <- '
    visual  =~ x1 + x2 + x3 +  x4 
    textual =~ x5+ x6 + x7 +  x8
    speed   =~ x8 + x9 + x10 + 0*x11
'

m1b <- '

    visual  =~ x1 + x2 + x3 +  x4 
    textual =~ x5+ x6 + x7 +  x8
    speed   =~ x8 + x9 + x10 + x11
'

lavTestLRT(m0b, m1b) 

m0 is statistically equivalent to m0b, but it needs to be specified in the latter way for difftest to work. I know this is probably obvious to most of you, but it wasn't to me. 

Thanks for your help and apologies for the silly mistake!

P