CFA results do not fit with EFA results

[Kahloong Chue]

Hi, I have a dataset comprising of 6 dimensions (3 items each). Each item is rated on a scale of 1 to 5. When I run an EFA using the MHRM method, the results show that a 6 factor solution is the best solution (AIC, BIC lowest), with good fit indices and loadings.

> out_efa <- mirt(df_mot, 6, method = "MHRM", calcNull = TRUE)
Stage 3 = 206, LL = -49352.7, AR(0.08) = [0.35], gam = 0.0033, Max-Change = 0.0009

Calculating log-likelihood...
Warning message:
Full table of responses is very sparse. Goodness-of-fit statistics may be very inaccurate 

> M2(out_efa, type = "C2", calcNull = TRUE, QMC = TRUE)
            M2 df          p      RMSEA   RMSEA_5   RMSEA_95
stats 97.23314 60 0.00167579 0.01672663 0.0103047 0.02263885
           SRMSR       TLI      CFI
stats 0.02242275 0.9987046 0.999492

However, when I run the respective CFA, the results show poor fit indices. The loadings are still quite good, but the TLI, CFI and RMSEA are bad.

> model1 <- 'TAP = 1-3
+ TAV = 4-6
+ SAP = 7-9
+ SAV = 10-12
+ OAP = 13-15
+ OAV = 16-18
+ COV = TAP*TAV*SAP*SAV*OAP*OAV'

> out_cfa <- mirt(df_mot, model1, method = "MHRM", calcNull = TRUE)
Stage 3 = 165, LL = -45804.0, AR(0.05) = [0.36], gam = 0.0039, Max-Change = 0.0010

Calculating log-likelihood...
Warning message:
Full table of responses is very sparse. Goodness-of-fit statistics may be very inaccurate 

> M2(out_cfa, type = "C2", calcNull = TRUE, QMC = TRUE)
            M2  df p     RMSEA   RMSEA_5  RMSEA_95     SRMSR
stats 9268.791 120 0 0.1854003 0.1821706 0.1885664 0.3972193
            TLI       CFI
stats 0.8408528 0.8751787

Considering that the full table of responses in both cases are very sparse, is there a reason why there is a large difference?  What can I do to improve the fit indices for the CFA?

Thank you.

[Phil Chalmers]

This can happen because EFA models have numerous loadings on their minor factors, which has the effect of improving the fit statistics due to chance capitalization + overfitting. As these factors all have only three loadings they are generally weakly defined, and it's likely the case there is more complex relationships going on in the response data (cross-loadings, residual covariation, higher-order structures, etc). I don't believe I can offer advice on improving the fit as this requires much theoretical consideration given the test content and causal theory connecting the constructs to the items.

 

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/5522c8c4-75d2-4dba-b9d9-487d0bcc8a29n%40googlegroups.com.

[Kahloong Chue]

Dear Phil, thanks for the response.