2PL versus 3PL

[Juliane Venturelli]

 Hi Phill,

 I appreciate your time to help us all.

 I am trying to decide if I fit a 2PL or a 3PL model. The test is a two-choice item. I have used a prior to the g parameter

as I was having convergence problems.

model <- 'F = 1-40 

               PRIOR = (1-40, g, norm, 0, 3)'

Does the prior seem ok?

My main question, how could I compare the 2PL with the 3PL model? 

I apologize if you have answered to this question before. I have spent a while looking for the answer, but I haven't found it.

Thank you again.

Juliane 

[Phil Chalmers]

Hi Juliane,

On Thu, Jan 7, 2016 at 1:50 PM, Juliane Venturelli <juliane.v...@gmail.com> wrote:

 Hi Phill,

 I appreciate your time to help us all.

 I am trying to decide if I fit a 2PL or a 3PL model. The test is a two-choice item. I have used a prior to the g parameter

as I was having convergence problems.

model <- 'F = 1-40 

               PRIOR = (1-40, g, norm, 0, 3)'

Does the prior seem ok?

It depends on how strong you need/want it to be and where you want it to be centred, but currently this looks off and very weak and probably won't help much with non-convergence. The current value centres around g = .5 (found via qnorm(0)), which is pretty high for a guessing parameter. You may wan to try something centred around .2, which can be obtained with qnorm(.2). 

If you're more familiar with the beta prior distribution from other software it's easy to compare their effects side by side with the following:

#############

library(ggplot2)

pts <- seq(.01, .5, length.out=1000) #between g of 0 and .5

normd <- dnorm(qnorm(pts), qnorm(.2) , .22, log = TRUE)

#plot(pts, normd)

betaden <- function(pts, size, mean){

    stopifnot(mean > 0 | mean < 1)

    compliment <- 1 - mean

    dbeta(pts, size * mean, size * compliment, log = TRUE)

}

betad <- betaden(pts, 35, .2)

#plot(pts, betad)

df <- data.frame(g=pts, type=factor(rep(c('norm', 'beta'), each = 1000)),

                 den=c(normd, betad))

ggplot(df, aes(g, den, colour = type)) + geom_line() + geom_vline(xintercept = .2, col = 'red')

#############

This shows a particular weight of 35 for the beta distribution (where its mean = alpha/ (alpha + beta))  corresponds pretty closely to the shape of the normal prior when the mean is qnorm(.2), sd = .22, though the normal is still a bit weaker. By comparison, with an sd=3 the prior weight would be extremely weak, so the model would converge much closer to the ML estimates and therefore have very little influence during estimation. Adjust the script accordingly to suit your needs.

 

My main question, how could I compare the 2PL with the 3PL model? 

This is harder to answer. Though the models are technically nested, likelihood ratio/Wald tests are very difficult when freeing the g parameters and only behave well in very large samples (where convergence is less of an issue anyway). You may want to just compare the models via model fit criteria with the M2() function, which will work regardless of the priors included. Cheers.

Phil

 

I apologize if you have answered to this question before. I have spent a while looking for the answer, but I haven't found it.

Thank you again.

Juliane 

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Juliane Venturelli]

Hi Phil,

Thank you for your quick answer (as always).

My choice to center the prior around 0.5 is because my items are only two-choices. Do you still think it is 

not a good idea? Also, is there a very practical IRT book you recommend? Quite often I get lost with the

practical aspect of all this. 

I appreciate all your work.

Juliane 

[Phil Chalmers]

On Thu, Jan 7, 2016 at 3:42 PM, Juliane Venturelli <juliane.v...@gmail.com> wrote:

Hi Phil,

Thank you for your quick answer (as always).

My choice to center the prior around 0.5 is because my items are only two-choices. Do you still think it is 

not a good idea?

I would say no, but that's kind of the problem with two choice response data. How much information can you really get from items when people are so likely to guess for a better grade? This is mainly the reason for multiple choice data and especially for fill-in-the-blank style responses. For other kinds of response data (yes/no, true/false) it's not likely there is 'guessing' so a 2PL is fine. 

 

Also, is there a very practical IRT book you recommend? Quite often I get lost with the

practical aspect of all this. 

Don't know of any practical books yet, but some are in development. Maybe I'll put one together some day when I'm finally finished grad school. Cheers.

Phil

[alexand...@gmail.com]

Hi everyone!

I would like to understand more about the g prior distribution. If I use PRIOR = (1-40, g, beta, 7, 28) I will center the distribution around .2. Is it possible to have this same result using PRIOR = (1-40, g, expbeta, xxx, xxxx) or PRIOR = (1-40, g, norm, xxx, xxx)?

In the BILOG-MG manual, they talk about the weight of n observations. They say

alpha = np + 1

beta = n(1-p) + 1

where p = 1/(number of choices)

By default, they use n = 20. For a five-choice item (p=0.2), alpha=5 and beta=17. But in this case, the center of the distribution is 0.23 rather than 0.20, which is strange for me. What do you think?

If I wanted to replicate BILOG-MG results in mirt, would I have to use PRIOR = (1-40, g, beta, 5, 17)?

What is the default in mirt, in respect to g prior distribution?

About the d prior. I am more familiar with the b parameter. So, if I want a normal prior distribution for b, let's say mean=0 and sd=1, do I simply use this information for d prior ( PRIOR = (1-40, d, norm, 0, 1) )?

Thanks!

Best regards,

Alexandre Jaloto

[Phil Chalmers]

On Sun, Nov 15, 2020 at 1:34 PM alexand...@gmail.com <alexand...@gmail.com> wrote:

Hi everyone!

I would like to understand more about the g prior distribution. If I use PRIOR = (1-40, g, beta, 7, 28) I will center the distribution around .2. Is it possible to have this same result using PRIOR = (1-40, g, expbeta, xxx, xxxx) or PRIOR = (1-40, g, norm, xxx, xxx)?

The former will set up the value at .2 in the logit of the g parameter, which is automatically transformed under-the-hood by the package, and likely isn't what you want. The latter approach with expbeta likely gives what you want since it undoes the logit transformation before applying this prior. That's also why the norm prior is useful here, as logits are unbounded and so a normal prior is reasonable for constraining either positive or negative logits.

 

In the BILOG-MG manual, they talk about the weight of n observations. They say

alpha = np + 1

beta = n(1-p) + 1

where p = 1/(number of choices)

By default, they use n = 20. For a five-choice item (p=0.2), alpha=5 and beta=17. But in this case, the center of the distribution is 0.23 rather than 0.20, which is strange for me. What do you think?

I think it's just a prior, and may be based on their personal beliefs/past observations. It's doubtful that guessing parameters would be exactly equal to 1/#categories in any given application, so setting priors that depend on the subject matter and test analyst's expertise seems like a more rational course of action than setting a formulaic prior structure. 

 

If I wanted to replicate BILOG-MG results in mirt, would I have to use PRIOR = (1-40, g, beta, 5, 17)?

PRIOR = (1-40, g, expbeta, 5, 17) 

  

What is the default in mirt, in respect to g prior distribution?

None. mirt uses marginal maximum likelihood (MML) by default, so no priors are utilized unless explicitly declared. Including priors will result in marginal maximum a posteriori estimates. 

 

About the d prior. I am more familiar with the b parameter. So, if I want a normal prior distribution for b, let's say mean=0 and sd=1, do I simply use this information for d prior ( PRIOR = (1-40, d, norm, 0, 1) )?

Unfortuantely, priors are not invariant to transformation, so the d = -b*a form would depend on your believe of the slope/discrimination parameters too. Though if you believe that all slope parameters are actually equal to 1 then yes, your prior would be sufficient. HTH.

Phil

 

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/bc23326e-51ee-402f-95ef-4473dbe0d079n%40googlegroups.com.