Can 3pl Item Response Theory model be estimated in Stan?

[Yong Luo]

Hi there,

I am having troubling getting Stan to estimate the 3pl item response theory model using the similar syntax as in 1pl and 2pl model since inv_logit is not vectorized.

Is there a way to estimate the 3pl model in Stan?

Thanks!

[Bob Carpenter]

It should be possible to estimate 3PL in Stan. At least
as possible in any other Bayesian estimation.

What kinds of problems are you having? If you send us the model,
we might be able to help.

The 3PL problem does introduce another degree of freedom at
the item level and it should correlate negatively with
item difficulty in the posterior, because there are now two
ways to account for a lot of students getting it right, either
an easy problem or an easy problem to guess. So you're almost
certainly going to need reasonable priors if you don't have
a lot of data, especially if you have problems that almost
every student got right (if they all got them wrong, it's not
such an issue, because the guessing rate is bounded below at zero).

So I'd suggest looking at a hierarchical model with covariance among
the three item-level parameters, guessing rate, difficulty, and discrimination.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Yong Luo]

Really appreciate your response, Bob.

I attached two versions of the Rstan codes I used with data included. Would it be possible for you to take a look?

A million thanks!

Yong

[Bob Carpenter]

The problem with your coding of the model is that you need to use
constraints to guarantee your parameters are within bounds. You have:

parameters {
...
vector[n_item] guessing; // guessing for item
...
model {
...
y[k] ~ bernoulli(guessing[k]*ones+(1-guessing[k])*p[k]);
...
}


You need to declare

vector<lower=0,upper=1>[n_item] guessing;

because guessing[i] has to be in the interval [0,1]. In Stan, if
a variable is constrained in the log probability function (say by being
drawn from a beta or being part of a bernoulli parameter)n, then you need
to declare it with those constraints.

And rather than taking log_alpha, you can just declare

vector<lower=0>[n_item] alpha; // discrimination for item

and then give it a lognormal distribution if you want. Or you can
do it the way you did it.

You probably don't need to constrain both beta and theta to be normal(0,1)
to deal with the additive and multiplicative invariances. Just taking one
to be normal(0,1) solves additive invariance by setting the location of that
parameter to be centered on 0 and setting its scale to be 1. The other one
could be normal(0,5) or normal(0,10).

Unlike some of our other distributions, there's really no benefit
speed-wise to vectorizing the bernoulli because there aren't any computations
that can be shared across the vectorized instances. So it'd probably be
clearer to just write it as your version two (and I'd suggest always using the
same variables to loop over the same indices):

for (i in 1:n_item)
for (j in 1:n_student)
y[i,j] ~ bernoulli(guessing[i] + (1 - guessing[i]) * inv_logit(alpha[i] * (theta[j] - beta[i])));

Because of the alpha[i] * beta[i] term, it can be more well behaved to reparameterize
to

inv_logit(gamma[i] * theta[i] - xi[i])

and then recover the alpha[i] and beta[i] with some algebra from the gamma[i] and xi[i].

- Bob

> stan-users+...@googlegroups.com <javascript:>.
> > For more options, visit https://groups.google.com/groups/opt_out <https://groups.google.com/groups/opt_out>.

[Yong Luo]

Thank you, Bob! This is really helpful advice. Now the code version 2 is running!

To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/GMgofkUqvr4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Yong Luo]

Bob,

One more question about your advice on reparameterizing alpha[i] * beta[i].

Should I do it using the following:

transformed parameters {

vector[n_item] xi;

   for (i in 1:n_item)

                        xi[i]<-alpha[i]*beta[i];

}

Thanks again!

On Thu, Feb 13, 2014 at 8:51 AM, Bob Carpenter <ca...@alias-i.com> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

You received this message because you are subscribed to a topic in the Google Groups "stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/GMgofkUqvr4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Bob Carpenter]

You'd add a parameter for xi, remove the parameter beta, and
define a generated quantities block:

generated quantities {
vector[n_item] beta;

for (i in 1:n_item)

beta[i] <- xi[i] / gamma[i];
}

Or you could replace the loop with elementwise operations,

beta <- xi ./ gamma;

- Bob

On 2/13/14, 4:13 PM, Yong Luo wrote:
> Bob,
> One more question about your advice on reparameterizing alpha[i] * beta[i].
> Should I do it using the following:
> transformed parameters {
> vector[n_item] xi;
> for (i in 1:n_item)
> xi[i]<-alpha[i]*beta[i];
> }
>
> Thanks again!
>
>
>
>
> On Thu, Feb 13, 2014 at 8:51 AM, Bob Carpenter <ca...@alias-i.com <mailto:ca...@alias-i.com>> wrote:
>
> The problem with your coding of the model is that you need to use
> constraints to guarantee your parameters are within bounds. You have:
>
> parameters {
> ...
> vector[n_item] guessing; // guessing for item
> ...
> model {
> ...

> y[k] ~ bernoulli(guessing[k]*ones+(1-__guessing[k])*p[k]);

> ...
> }
>
>
> You need to declare
>

> vector<lower=0,upper=1>[n___item] guessing;

> stan-users+...@googlegroups.__com <mailto:stan-users%2B...@googlegroups.com> <javascript:>.
> > For more options, visit https://groups.google.com/__groups/opt_out
> <https://groups.google.com/groups/opt_out> <https://groups.google.com/__groups/opt_out

> <https://groups.google.com/groups/opt_out>>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-users+unsubscribe@__googlegroups.com <mailto:stan-users%2Bunsu...@googlegroups.com>.
>
> For more options, visit https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>.
>
>
> --

> You received this message because you are subscribed to a topic in the Google Groups "stan users mailing list" group.

> To unsubscribe from this topic, visit https://groups.google.com/d/__topic/stan-users/GMgofkUqvr4/__unsubscribe
> <https://groups.google.com/d/topic/stan-users/GMgofkUqvr4/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@__googlegroups.com
> <mailto:stan-users%2Bunsu...@googlegroups.com>.
> For more options, visit https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>.

>
>
> --
> You received this message because you are subscribed to the Google Groups "stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.