Assistance with Parameter Calibration and Fit Issues Using the mirt Package

[YogM]

Hello Phil and everyone,

I am new to the mirt package and am currently using it to calibrate a dataset consisting predominantly of dichotomous items. My objective is to extract the a, b, and c parameters along with their standard errors; item characteristic curve values (ICC1, ICC2, ICC3); item information function values (IIF1, IIF2, IIF3) based on theta cut points at 1, 2, and 3; and to obtain associated calibration fit indices, convergence values, and cycle counts.

However, I am encountering challenges with the parameter estimates—specifically, abnormally high values for a, b, and c, as well as their corresponding standard errors. Additionally, the overall model fit indices and convergence diagnostics appear to be suboptimal.

Interestingly, the same dataset produces more stable and interpretable results when analyzed using Parscale. (Furthermore, mirt performs as expected on standard datasets such as LSAT7.) I would greatly appreciate any recommendations or best practices to help align mirt’s output more closely with that of Parscale or to get good results.

Thanks,

YogM

[YogM]

Would someone be able to look into this and advise accordingly?  

[Jay Verkuilen]

Not being able to see the PARSCALE estimates, it's really hard to make useful comparisons. I think you need to post them. Also, put in important things like Ns and such. Just eyeballing it, these look reasonable to somewhat low for a number of items. 

[YogM]

Sharing Parscale vs. mirt results (628 respondents, 20 items, 3pl model, 0.001 TOL, Ncycles=150). Target delta < |0.01|. Please review and advise if any adjustments needed.       

[Phil Chalmers]

I don't have access to parscale, nor have I ever personally used it, but this sentence may provide an indication as to why you're seeing such differences:

"By default, PARSCALE assumes a log-normal prior distribution for a parameters (i.e., slope parameter), a normal prior distribution for b parameters (i.e., threshold parameter), and a β prior distribution for c parameters (i.e., lower asymptote parameter)"

So PARSCALE is using MMAP estimation by default, while mirt is using MMLE. You could get better results if you either remove the priors from PARSCALE or include the same default priors in mirt. Using informative priors by default is not my recommendation, which is why mirt is set up as with MLE criteria first. (see here: https://methods.sagepub.com/ency/edvol/sage-encyclopedia-of-educational-research-measurement-evaluation/chpt/parscale)

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/mirt-package/1b5c8b7f-35df-43c0-9650-ddc6f2ef4553n%40googlegroups.com.