M2 based fit indices in discrete mirt

[Hynek Cígler]

Hello. 

I estimated very basic LCA model (with binary items and two categories) using mdirt function, for example as fit <- mdirt(data, 2), here all data variables were binary. 

Then I estimated model fit using M2 function (M2(fit)), which returned M2 statistics and df, p, and RMSEA with CI. First of all, M2 doesn't estimate null model, and thus doesn't return CFI and TLI. Second, it doesn't return SRMSR; if I asked for residual matrices (M2(fit, residmat = T)), it returns strange list with Xi2, delta, estpars, p, e, and SRMSR (which is NULL) objects. 

1. First of all, why  isn't SRMSR estimated? Its construction should be the same as with traditional IRT (e.g., 2PL) model, or I am wrong?
2. Second, what are these objects which are returned as residual matrices? Why they have 21 rows/columns (6 items per 2 categories, 2 latent classes)?
3. And finally, I don't understand why the null model is not defined. It should be equivalent to the NULL model in 2PL model. I assume I can estimate null model as LCA model with a single group (i.e., fit <- mdirt(data, 1)), get M2 statistic and d.f., and estimate it manually. However, I am afraid this not to be the right approach; otherwise, I suppose it were already implemented. 

Thank you for any advice!

(My data are special-education test with recognition of the first and last phone in a word. Therefore, children either can or cannot discriminate phones, which lead to bimodal distribution of raw scores, perfect 2PL model fit, and alpha > .98 even with five or six items. On the other hand, it makes no sense to estimate 2PL model, if I see this pattern; at least its parameters would be biased. However, the LCA model is still slightly worse compared to 2PL and I need complete fit indices to report in test manual.)

[Phil Chalmers]

Hi Hynek,

On Thu, Oct 19, 2023 at 2:20 PM Hynek Cígler <cigler...@gmail.com> wrote:

Hello. 

I estimated very basic LCA model (with binary items and two categories) using mdirt function, for example as fit <- mdirt(data, 2), here all data variables were binary. 

Then I estimated model fit using M2 function (M2(fit)), which returned M2 statistics and df, p, and RMSEA with CI. First of all, M2 doesn't estimate null model, and thus doesn't return CFI and TLI. Second, it doesn't return SRMSR; if I asked for residual matrices (M2(fit, residmat = T)), it returns strange list with Xi2, delta, estpars, p, e, and SRMSR (which is NULL) objects. 

1. First of all, why  isn't SRMSR estimated? Its construction should be the same as with traditional IRT (e.g., 2PL) model, or I am wrong?

"..., or I am wrong?" I believe that's my line in this case. There's no real reason to omit SRMR in this case, even though covariance statistics themselves are not particularly interpretable (unless the model implies ordering) the ability to reproduce the covariance matrix is still a useful thing for evaluating fit. I have previously limited this to ordinal item types only, but this is unnecessary. I'll send the updated version to the dev shortly. 

 

1. Second, what are these objects which are returned as residual matrices? Why they have 21 rows/columns (6 items per 2 categories, 2 latent classes)?

Not sure what that was, but it shouldn't appear in the output. Now the residual matrix is returned as per usual. 

 

1. And finally, I don't understand why the null model is not defined. It should be equivalent to the NULL model in 2PL model. I assume I can estimate null model as LCA model with a single group (i.e., fit <- mdirt(data, 1)), get M2 statistic and d.f., and estimate it manually. However, I am afraid this not to be the right approach; otherwise, I suppose it were already implemented. 

See point 1., where "am I wrong?" should again fall on yours truly.

-- Yours truly, 

Phil

 

Thank you for any advice!

(My data are special-education test with recognition of the first and last phone in a word. Therefore, children either can or cannot discriminate phones, which lead to bimodal distribution of raw scores, perfect 2PL model fit, and alpha > .98 even with five or six items. On the other hand, it makes no sense to estimate 2PL model, if I see this pattern; at least its parameters would be biased. However, the LCA model is still slightly worse compared to 2PL and I need complete fit indices to report in test manual.)

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/09af4a78-9652-4394-9a42-5a3a3ae038e6n%40googlegroups.com.

[Hynek Cígler]

Dear Phil,

You are amazing! :) You answered me within a day with a complete and helpful response twice in two months. Thank you! 

Anyway, I work with ordered items (and I consider latent classed ordered too), so the comparison of SRMSR between 2PL and LCA model could be useful. 

Dne čtvrtek 19. října 2023 v 21:05:00 UTC+2 uživatel mirt-package napsal: