Comparing non-nested CFA models with estimator WLSMV (DWLS)

[Olivia]

Dear all,

 

I am performing confirmatory factor analyses using the lavaan package in R. I have fitted two models (non-nested) and would like to assess which fits my data best. The models are as follows:

 

Model 1: x1, x2, x3, x4 and x5 are either binary or non-normally distributed categorical variables (specified as ordered in my df).

 

model_1 <- '

latent_var =~ x1 + x2 + x3 + x4 + x5

latent_var~~1*latent_var'

 

fit1 <- cfa(model_1, data=df, std.lv=TRUE)

 

Model 2: x1, x2, x3 are the same as above and x6  is a normally-distributed continuous variable.

 

model_2 <- '

latent_var =~ x1 + x2 + x3 + x6

latent_var~~1*latent_var'

 

fit2 <- cfa(model_2, data=df, std.lv=TRUE)

 

I have used the WLSMV (DWLS) estimator and both models seem to fit on the basis of (X2 significance, TLI, CFI, RMSEA, SRMR and WRMR). I have been looking for advice online about how to compare the fit of these two models, because information criterion indices like the AIC or BIC are unavailable and I understand that using  a chi-square difference test is inappropriate. I have read that the WRMR is a good measure of fit for models using categorical variables, and that the smaller the value the better the fit. But I assume that it is not appropriate to base my comparison of model fit on the WRMR value alone.

 

Does anyone have any advice on how I might compare the fit of these two models? Any suggestions are greatly appreciated.

 

Many thanks in advance,

 

Olivia

[Terrence Jorgensen]

I have fitted two models (non-nested) and would like to assess which fits my data best. 

Not really.  You have models involving entirely different endogenous variables, so there is no comparing them.  You can only compare the fit of different models to the same data.  To do this, you could fit both models using all 6 indicators, but just constrain certain loadings to zero instead of omitting the indicators altogether.  Then you could compare them using CFI or TLI (because the same independence model is nested within each competing model, which are nested within the same saturated model -- that puts the 2 competing models on the same continuum between worst and best fitting models).

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

[Olivia]

Dear Terrence, 

Thank you for your reply and for your suggestion - i'll give that a go.

Many thanks again,

Olivia