goodness of fit X2

[Yutaro Sakamoto]

Dear all

I'm trying to do MIRT analysis;

modelA ; undimensional IRT model

modelB ; confirmatory MIRT model

I estimated item parameter in both model and tested goodness of fit (anova function);

#-----------------------------

Model 1: mirt(data = TIMSS2011jpn_5, model = 1, itemtype = "graded", method = "MHRM")

Model 2: mirt(data = TIMSS2011jpn_5, model = modelB_5, itemtype = "graded", 

    method = "MHRM")

               AIC     AICc         SABIC      BIC       logLik      X2       df   p

modelA 131923.8 132018.7 133319.0 134698.1 -65527.88   NaN  NaN NaN

modelB 131954.4 132050.7 133359.3 134747.9 -65540.20 -24.625   3   1

 

In this, the value of  X2 is negative(-24.625). Could this happen?

Please give me your advice, Thank you.

Yutaro Sakamoto

[Phil Chalmers]

I mention that this can happen in another thread: https://groups.google.com/forum/#!topic/mirt-package/DvAwioqioEA, but this is more extreme than one would usually see, so perhaps a local minimum was hit in one of the models. You could try starting from different starting values (mirt(..., GenRandomPars = TRUE) would generate some automatically) to see if the same result occurs.

However, you really don't need to be using the MHRM anyway here for low dimensional models, so I would recommend switching back to the EM method. The EM generally doesn't have the stochastic approximation issue, so it should be more stable. Cheers.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yutaro Sakamoto]

Dear Phil

Thank you for your response.

I read that article and you mean that if we use MHRM method we should think about information statistics because MHRM method  can give some funny result in likehood ratio testing. 

And I tried  to start from different starting values (mirt(..., GenRandomPars = TRUE). So the result was different but the value of X2 is also negative;

modelA unidimensional IRT

modelB confirmatory MIRT(3factors)

(modelC) bifactor model (3 group factors + 1 general factors)

#-------------------

Model 1: mirt(data = TIMSS2011jpn_5, model = 1, itemtype = "graded", method = "MHRM", 

    GenRandomPars = TRUE)

Model 2: mirt(data = TIMSS2011jpn_5, model = modelB_5, itemtype = "graded", 

    method = "MHRM", GenRandomPars = TRUE)

               AIC     AICc    SABIC      BIC    logLik     X2  df   p

modelA 131929.8 132024.7 133325.1 134704.1 -65530.89    NaN NaN NaN

modelB 131967.0 132063.2 133371.9 134760.5 -65546.48 -31.18   3   1

So, could I use MHRM method in only modelC and use EM in modelA and B?

If I do so, can I use likehood ratio test and information criterion?

(modelA vs modelB etc..)

thanks,

Yutaro

2015年3月4日水曜日 0時55分18秒 UTC+9 Phil Chalmers:

[Phil Chalmers]

Oh I see what you mean now. Yes you should only be looking at information criteria because none of those models are nested within each other, and therefore comparing X2 values for non-nested models is not appropriate or valid. Cheers.

Phil