Convergence problems with three-dimensional 3PL model

[Darrin Grelle]

I am calibrating a large pool of items with substantial data MCAR. I have no trouble estimating a one-factor 2PL or a one-factor 3PL if I set priors for the guessing parameter, but I cannot get the three-factor 3PL to converge after 500 iterations even with priors set. I've tried fixing the guessing parameters to the values that were estimated in the one-factor model, but I don't think I'm doing it correctly. The model still isn't converging and when I look at the estimates, the guessing parameters don't match the values I fixed them to.

Could you please take a look at my syntax and let me know if I've made an error somewhere? Thanks!

cmodel <- mirt.model('
                     F1 = 1-197
                     F2 = 198-383
                     F3 = 384-554
                     COV = F1*F2*F3')

guessfix = c(0.22, ..[554 values]..., 0.13)

cmod3F3PLfix <- mirt(mirtData, cmodel, method = 'QMCEM', SE = TRUE,
                  guess = guessfix)

[Phil Chalmers]

On Mon, Mar 23, 2015 at 2:48 PM, Darrin Grelle <darrin...@gmail.com> wrote:

I am calibrating a large pool of items with substantial data MCAR. I have no trouble estimating a one-factor 2PL or a one-factor 3PL if I set priors for the guessing parameter, but I cannot get the three-factor 3PL to converge after 500 iterations even with priors set. I've tried fixing the guessing parameters to the values that were estimated in the one-factor model, but I don't think I'm doing it correctly. The model still isn't converging and when I look at the estimates, the guessing parameters don't match the values I fixed them to.

The fixed values are changing? That sounds peculiar. Could you provide a reproducible example of this (email me the data directly if it is sensitive). 

 

Could you please take a look at my syntax and let me know if I've made an error somewhere? Thanks!

cmodel <- mirt.model('
                     F1 = 1-197
                     F2 = 198-383
                     F3 = 384-554
                     COV = F1*F2*F3')

guessfix = c(0.22, ..[554 values]..., 0.13)

cmod3F3PLfix <- mirt(mirtData, cmodel, method = 'QMCEM', SE = TRUE,
                  guess = guessfix)

The syntax looks fine to me, so I'm not sure what the issue is.

Phil

 

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Darrin Grelle]

Hello again,

I got the syntax to work, but my solution still won't converge. I have no trouble estimating a three-factor 2PL, so I'm not sure why a model with fixed guessing parameters wouldn't converge. Is there a way to input starting values to aid in estimation? Do you have any other suggestions?

Thanks!
Darrin

[Phil Chalmers]

How are the values themselves looking in your model? Perhaps the inter-factor covariance terms are very high, causing multicollinearity, or some slopes have become too large? In any case, you can increase the number of iterations by passing technical = list(NCYCLES = #), or continue at the previous model by passing pars = mod2values(oldmodel). 

Phil

--

[Darrin Grelle]

Thank you for the advice, Phil. Even fixing the guessing parameters and passing the 2PL estimates as starting values, I still can't get the 3PL to converge in 500 iterations. Just watching the max change values, it doesn't look like increasing the number of iterations would help all that much. The solution appears to be getting stuck in a local maximum.

Using the 2PL, my fit statistics suggest that the 3-factor model fits better than the 1-factor model, but the factors are highly correlated. I assume this is why my 3PL won't converge. Do you have any guidance on factor correlations when choosing between MIRT and unidimensional IRT? It seems that if the the factors are too highly correlated that the complexity of MIRT estimation might not be worth the extra effort.

Thanks!

[Phil Chalmers]

On Thu, Mar 26, 2015 at 2:10 PM, Darrin Grelle <darrin...@gmail.com> wrote:

Thank you for the advice, Phil. Even fixing the guessing parameters and passing the 2PL estimates as starting values, I still can't get the 3PL to converge in 500 iterations. Just watching the max change values, it doesn't look like increasing the number of iterations would help all that much. The solution appears to be getting stuck in a local maximum.

Using the 2PL, my fit statistics suggest that the 3-factor model fits better than the 1-factor model, but the factors are highly correlated. I assume this is why my 3PL won't converge. Do you have any guidance on factor correlations when choosing between MIRT and unidimensional IRT? It seems that if the the factors are too highly correlated that the complexity of MIRT estimation might not be worth the extra effort.

If the factors are highly correlated (like 0.85+) then you really don't gain much by including them in the model, and it really just builds additional complexity. I would check out how the itemfit()/residuals() statistics look for the unidimensional model to see if the items are behaving well, and just keep with the simpler model. Otherwise, you could try your luck at a bifactor model if there really are packets of inter-correlating items, but you may reach the same stability issues. Cheers.

Phil 

[Darrin Grelle]

Hello again, Phil

I really appreciate all the help that you've provided so far. I have two more questions and then hopefully I won't need to bug you anymore. I have asked two of my psychometrician colleagues about this first issue and we are all stumped:

I ran my model of 545 items and 3 factors (each item loading on one factor and correlated factors) using both 2PL and 3PL. As I've mentioned, the 2PL converged easily, but my 3PL can only get down to a max change of about .00064. The factor correlations in the 2PL were .75, .78, and .85. The factor correlations in the 3PL were .95, .98, and .999. I cannot think of a reason why the estimation of the guessing parameter would affect the factor correlations so much, but I thought you might have some idea.

Second, do you happen to know if R-MIRT max-change is the same as the convergence criterion used in BILOG? I would assume it is, but wanted to check.

Thanks again!
Darrin

[Phil Chalmers]

On Tue, Mar 31, 2015 at 12:36 PM, Darrin Grelle <darrin...@gmail.com> wrote:

Hello again, Phil

I really appreciate all the help that you've provided so far. I have two more questions and then hopefully I won't need to bug you anymore. I have asked two of my psychometrician colleagues about this first issue and we are all stumped:

I ran my model of 545 items and 3 factors (each item loading on one factor and correlated factors) using both 2PL and 3PL. As I've mentioned, the 2PL converged easily, but my 3PL can only get down to a max change of about .00064. The factor correlations in the 2PL were .75, .78, and .85. The factor correlations in the 3PL were .95, .98, and .999. I cannot think of a reason why the estimation of the guessing parameter would affect the factor correlations so much, but I thought you might have some idea.

3PL models cause instability everywhere, i.e., can cause item slopes to increase/decrease dramatically, as well as shift the intercept parameters at the same time (think of an item with a 10% correct endorsement with a guessing parameter that you set manually to 0.2; what would the intercept be then, and how would the slope look? It seems that would be extremely unstable, and probably not what you want). Unless you have extremely large sample sizes (like 2000-5000+) it's pretty hard to get decent estimates with the 3PL. 

 

Second, do you happen to know if R-MIRT max-change is the same as the convergence criterion used in BILOG? I would assume it is, but wanted to check.

Yes.

Phil

[Nicolas Neef]

Hi Phil,

I ran into the same problem. I fixed my guessing parameter for a 3PL model and added contrains.

model_5 <- 'F = 1-30,
            CONSTRAIN = (1-30, a1)'

guessfix <- c(0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.33,0.33,0.5,0.33,
          0.5,0.25,0.5,0.5,0.33,0.5,0.33,0.33)

tPL_5 <- mirt(ds[,c(11:13,18:23,25,27:30,32,35:36,38,40:42,44:50,52,53)], model = model_5, itemtype = '3PL', method = "EM",
              guess = guessfix)

What I now learned. If I change the itemtype to "2PL" it works fine! I hope this might help others running into a similar problem.

Cheers,

Nicolas