MIRT with small sample size but valid models

[Martin Lacher]

Hi there,

I "boldly" did IRT analysis (mostly unidimensional GRM) on data with a sample size of around N=100. After some model and item wrangling I got valid models (RMSEA around 0.5, CFI and TLI > 0.93, sometimes even 0.97, even Chi2 was ok). Data reliability (empirical) rxx is > 0.7.

All articles I found about minimal sample size for IRT suggest at least N=500. Can my data be valid nontheless or do I have to use other methods?

I wanted to use GRM in the first place to analyse the validity of the item levels, which indeed gave me a lot of information about my code system, but now I'm concerned about the calculated factor scores.

Thanks a lot for your help!

Martin

[Phil Chalmers]

On Fri, Feb 19, 2021 at 4:34 AM Martin Lacher <lacher...@gmail.com> wrote:

Hi there,

I "boldly" did IRT analysis (mostly unidimensional GRM) on data with a sample size of around N=100.

Did you get bold results to, or just italicized? ;-P

 

After some model and item wrangling I got valid models (RMSEA around 0.5, CFI and TLI > 0.93, sometimes even 0.97, even Chi2 was ok).

Data reliability (empirical) rxx is > 0.7.

There's a good amount of sampling variability in each of these estimates, so proceed with caution. It's possible to bootstrap these estimates, though given the sample size that's likely not a great idea, and the n.s. chi2 shouldn't be surprising given the limited information available in the moments (likely underpowered).   

 

All articles I found about minimal sample size for IRT suggest at least N=500. Can my data be valid nontheless or do I have to use other methods?

I wanted to use GRM in the first place to analyse the validity of the item levels, which indeed gave me a lot of information about my code system, but now I'm concerned about the calculated factor scores.

There will no doubt be a good amount of bias and sampling variability given on 100 responses, and unless your test is very long there probably won't be sufficient response data available to compensate either. So it's hard to say if your particular sample is good or not, including the quality of the factor scores. You could of course try a simulation study yourself to see just how variable these parameters tend to be. 

Phil 

 

Thanks a lot for your help!

Martin

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/b4cf704c-45ef-402f-90a7-036d5cd093b5n%40googlegroups.com.