Confusing model fit statistics?

[Emma Cernis]

Hi– 

I’m trying to fit an IRT model to a 5-item single-factor measure and am confused by the model fit statistics. It looks like many of the statistics are ok. There are 5 response options so I am using the graded IRT model. I used other packages as well - the mokken package to evaluate dimensionality using the aisp() and the coefH() and it did not flag up any issues with the items. I also ran the empirical plots and they don’t look anything out of the ordinary. Looking at the ICC curves, does not suggest the need to combine the options. I also remove persons with aberrant responses using personfit(). The table of response frequencies can be found below. I would appreciate some help in understanding why the itemfit() is still flagging up all the items as misfits. 

Thanks in advance!

 

                 CEFSAi CEFSAii CEFSAiii CEFSAiv CEFSAv

       0          592         514         636         730         697

       1          425         447         477         439         459

       2          595         638         503         442         427

       3          177         197         175         177         202

       4            75          68           73            76          79

 

[Emma Cernis]

Sorry! forgot the statistics!

[Phil Chalmers]

Hi Emma,

Your model does indeed look quite good overall in that the first and second moments of the model match the data well (so in that sense, there's little evidence that something is structurally wrong), and visually the two-step item-fit plots seem quite reasonable too (these plots don't reflect the S-X2 statistics btw, though I could probably add support for that in future releases). The significance of the S-X2 statistics likely has to do with the power of these test statistics under large sample sizes, so the small p-values still indicate that each respective item "don't not fit perfectly". But, given the standardized versions of these goodness of fit statistics (RMSEA, SRMSR, CFI, etc) the fit doesn't appear to be systematically too poor, even in the items, so I'd be more inclined to believe the items are being described within reason by the select IRT models. Judging model fit is, unfortunately, a bit of an art, and shouldn't entirely rely on the magnitude of p-values (much like everything else in statistics). HTH.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/6ea6d21c-a273-4631-a82a-2f1483476e39n%40googlegroups.com.

[Emma Cernis]

Thank you so much! That's really helpful.

Is there a type of item fit analysis you would use apart from the S-X2? 

Thanks again -

Emma