M2 function output differs depending on item type

[Hynek Cígler]

Hello. I realized that the M2 function provides different outputs depending on item type. More specifically, the residual matrices differ.

If I use itemtype = "gpcm" or itemtype = "grm", I got all fit indices including SRMSR. If I use itemtype = "gpcmIRT", SRMSR is not returned.

I tried to find the reason, and I realized that residual matrices differ. With itemtype = "gpcm" and M2(..., residmat = T), M2 function returns squared matrix with residuals in lower triangle. If the same is repeated with itemtype = "gpcmIRT", M2 function provides a list of objects
> ls(fit_config)
[1] "delta"   "e"       "estpars" "p"       "SRMSR"   "Xi2"  
where SRMSR object is NULL.

I wonder if this is an error or if SRMSR cannot be computed with gpcmIRT parameterization (which does not make sense to me). Can I get SRMSR with gpcmIRT parameterization? The reason for that parameterization is invariance testing, where I want to test threshold and scalar invariance levels separately (constraining b parameters while releasing c parameter in the second group).

[Hynek Cígler]

In addition, I realized one more unpleasant problem. However the full-information fit of model built on gpcm and gpcmIRT items are exactly the same (almost; however, the difference is negligible), the restricted information model fits provided by the M2 function differ. Example here:

> mod1 <- mirt(dat[items], 1, itemtype = "gpcm", SE = T)
Iteration: 87, Log-Lik: -6703.188, Max-Change: 0.00010

Calculating information matrix...
> mod2 <- mirt(dat[items], 1, itemtype = "gpcmIRT", SE = T)
Iteration: 67, Log-Lik: -6703.187, Max-Change: 0.00008

Calculating information matrix...
> M2(mod1)
            M2  df           p      RMSEA    RMSEA_5  RMSEA_95      SRMSR       TLI       CFI
stats 332.3749 263 0.002385794 0.02147461 0.01333302 0.0282317 0.03792409 0.9924394 0.9930957
> M2(mod2)
          M2  df           p      RMSEA    RMSEA_5   RMSEA_95       TLI       CFI
stats 343.77 263 0.000591406 0.02317123 0.01563961 0.02968448 0.9918096 0.9925205
> anova(mod1, mod2)
          AIC    SABIC      HQ      BIC    logLik    X2 df   p
mod1 13530.38 13603.31 13635.6 13800.13 -6703.188            
mod2 13530.38 13603.31 13635.6 13800.13 -6703.187 0.001  0 NaN

Dne úterý 13. září 2022 v 9:27:02 UTC+2 uživatel Hynek Cígler napsal:

[Phil Chalmers]

M2's auxiliary information such as SRMSR require certain types of modeling assumptions to work (e.g., ordinal/monotonic response models), which is why it is not always reported. That is odd about the gpcmIRT itemtype though as you're correct it should work fine. Could you provide a reprex to save me some time? Thanks.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/7595b8e7-894c-451e-8594-3924b2f677dfn%40googlegroups.com.

[Phil Chalmers]

This is generally to be expected given that estimation is quite sensitive to parameterization given only a 1 × 10^-4 convergence criteria in the EM. You could lower the TOL argument until they agree better, but there's no guarantee that they will converge to each other exactly without very large sample sizes (this is why parameterization can be tricky, and why the package generally prefers the slope-intercept versions for their improved stability). Here's a script worth experimenting with to see this:

library(mirt)

a <- matrix(rlnorm(20,.2,.3))
diffs <- t(apply(matrix(runif(20*4, .3, 1), 20), 1, cumsum))
diffs <- -(diffs - rowMeans(diffs))
d <- diffs + rnorm(20)

dat <- simdata(a, d, 5000, itemtype = 'graded') # change the sample size/TOL criteria

mod <- mirt(dat, 1, 'gpcm')
mod2 <- mirt(dat, 1, 'gpcmIRT', TOL=1e-6)

coef(mod, IRTpars=TRUE, simplify=TRUE)$items
coef(mod2, simplify=TRUE)$items

M2(mod)
M2(mod2)

--

You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/8867f4ed-d094-4a6e-ae19-28fbe3280026n%40googlegroups.com.

[Phil Chalmers]

Scratch that, I see that the itemtype was not added to an internal check function. It's now been patched on the dev.

Phil