Interpretation of mirt result (p-value)/m2 statistics

[Yuanjin Zhou]

Hi everyone,

I am really new to mirt package and I am trying to fit a multidimensional IRT model in my data in an exploratory approach. I have a question about the interpretation of results. 

Here is my code: 

mod7 <- mirt(d2,7,  method = "MHRM",itemtype = '2PL',

             calcNull = TRUE, technical = list(NCYCLES = 5000),verbose = F,QMC=T)

The result: 

Call:

mirt(data = d2, model = 7, itemtype = "2PL", method = "MHRM", 

    calcNull = TRUE, verbose = F, technical = list(NCYCLES = 5000), 

    QMC = T)

Full-information item factor analysis with 7 factor(s).

Converged within 0.001 tolerance after 332 MHRM iterations.

mirt version: 1.32.1 

M-step optimizer: NR1 

Latent density type: Gaussian 

Average MH acceptance ratio(s): 0.4 

Log-likelihood = -6572.182, SE = 0.033

Estimated parameters: 115 

AIC = 13374.36; AICc = 13414.61

BIC = 13910.04; SABIC = 13544.85

G2 (130956) = 3408.32, p = 1

RMSEA = 0, CFI = 0, TLI = -0.017

My question is: what does the p-value hereafter the number for G2?

I have also used the m2 statistic for this model. The result is as follows:

            M2 df         p      RMSEA RMSEA_5   RMSEA_95      SRMSR       TLI       CFI

stats 45.24703 38 0.1951517 0.01565664       0 0.03091931 0.02246948 0.9926648 0.9979505

I am wondering whether m2 would be a useful test for an exploratory model? 

Thank you!

Yuanjin

[Phil Chalmers]

Hi Yuanjin,

The p-value associated with G2 is a goodness-of-fit type detection value based on the complete cell frequencies (very unreliable as full-tables of counts are extremely sparse....hence why this is disabled by default). M2 is likely better since it is doing data collapsing to only the first and second moments to help removing sparceness. And yes, M2 is fine for an exploratory model. 

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/215386c0-27b4-4e3e-8a53-75cc5a74c677%40googlegroups.com.

[Brian Peña Calero]

Hello Phil

I wonder about the difference between TLI and CFI output values in M2 and mirt functions. First, Yuanjin get `CFI = 0, TLI = -0.017` and with `M2` get 0.99 in both. Are they different meanings?

[Phil Chalmers]

In principle these fit statistics have the same meaning, but the former that uses the G2 information is *very* unreliable when the complete tables of counts are sparse (which happens when the test has even a moderate number of items, like 10+ for dichotomous tests of 5+ for polytomous). By default, it makes more sense to trust the results from M2() as these are based on the more dense first and second-order tables of counts. HTH.

Phil

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/11f315af-3960-4011-b493-d7252dbbd64dn%40googlegroups.com.

[Brian Peña Calero]

Now I understand it much better. Thank you very much!