Help with Graded Response Model

[Christopher Fisher]

Hi-

I am trying to set up a basic graded response model closely following  the IRT examples on Github. Unfortunately, convergence is very poor and there seems to be a lot of autocorrelation. I attached the Stan model code, the output, and simulated data, which has 20 items and 100 persons. 

I tried tweaking the priors to smaller and larger ranges and adding a delta (mean person ability) as done in simpler examples on Github, but to no avail. So I'm not sure if its an issue with the priors, identifiability or mispecification.  Any advice would be greatly appreciated. 

~Chris 

[Dr. Hans Hansen]

Hi Chris,

there are some issues in your code:

1. you seem to have items and personids mixed up in the data (100 items & 20 persons).

2. i think that beta should be the threshold variable in the ordinal_logistic sampling statement.

3. minor: alpha should be constrained to be positive.

i have a GRM, which i am not entirely sure whether its correct, since i use stan just a bit so far :P, priors are not fully specified and it takes the data in the wide format rather than in long format. however, it samples 2000 iterations in 4 chains in 30seconds.

hope this helps, felix

    data{

      int P;                        //nr items

      int N;                       //nr persons

      int K[P];                   //nr categories per item -1

      int Y[N, P];                  // data

    }

    

    parameters {

      ordered[max(K)] kappa[P];              // intercepts

      real theta[N];                                    //person parameter

      real<lower=0> alpha[P];                  // slope

    }

    

    model {

      theta ~ normal(0,1);

      alpha ~ normal(2,1);

      for (p in 1:P){

        kappa[p][1:K[p]] ~ normal(0,5);

    

        for (n in 1:N){

          Y[n, p] ~ ordered_logistic(theta[n] * alpha[p], kappa[p][1:K[p]]);

[Christopher Fisher]

Thanks for the advice. I'll see what I can do!

[Bob Carpenter]

We'll be rolling out a GRM case study by Daniel Furr at Berkeley very
very soon (it's in press, so to speak):

It'll go here when it shows up and I'll also make an announcement
overall:

http://mc-stan.org/documentation/case-studies.html

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Christopher Fisher]

Thanks Bob. I'm still experiencing difficulty so a case study should be helpful. 

[Daniel Furr]

Hello. The case study I have in the works is actually for the partial credit model, not the graded response model.

The Stan model that Hans Hansen suggested above looks about right to me, except that there may be problems if the items have differing numbers of response categories. If there are differing numbers of response categories, the items with fewer response categories will have extra kappa parameters that are unused. I have found that having unused paramters lead to convergence problems. I think there is a solution to this problem, but programming it would take some time and creativity.

On the other hand, if the items have the same number of response categories, Hans' model should work.

[Dr. Hans Hansen]

Hi Daniel,

is it something reasonable to do to set unused kappas for nonexistent categories to an arbitrary high value by using a sharp prior, e.g. normal(30, .0001)? I don't use them in ordered_logistic() then.

best, felix

[Bob Carpenter]

No. You're much better off removing the unused parameters
if you can. If you can't do that, choose something easy
to sample from, like normal(0, 1).

- Bob

> On Mar 13, 2016, at 7:43 AM, Dr. Hans Hansen <drhans...@gmail.com> wrote:
>
> Hi Daniel,
>
> is it something reasonable to do to set unused kappas for nonexistent categories to an arbitrary high value by using a sharp prior, e.g. normal(30, .0001)?
>

> best, felix

[Daniel Furr]

I'm attaching the Stan program I came up with. I figured I'd code the GRM sooner or later, so I might as well do it now given that you've asked about it. I haven't vetted it thoroughly, so please check that it gives reasonable results before relying on it. It's coded for long form data, and the response y should have a lowest score of zero. Items do not have to have the same number of response categories.

I follow Bob's advice on the matter of unused parameters, which is reflected in the Stan program.

[Bob Carpenter]

Thanks! May I suggest replacing this:

for(k in 1:m[i]) {
if(k == 1) {
kappa[pos_kappa[i]] <- mu[i];
}
if(k > 1) {
kappa[pos_kappa[i]+k-1] <- kappa[pos_kappa[i]+k-2] +
delta[pos_delta[i]+k-2];
}
}

with

if (m[i] > 0)
kapa[pos_kappa[i]] <- mu[i];
for (k in 2:m[i])
kappa[pos_kappa[i]+k-1]
<- kappa[pos_kappa[i]+k-2] + delta[pos_delta[i]+k-2];

It's almost always a good idea to reduce nesting (from 2-deep to
1-deep here) when possible. And if you know that m[i] > 0, then
you can drop the conditional (there isn't a constraint to that effect
in the transformed data variable declaration, but there should probably
at least be a lower=0)

I thought these

segment(kappa, pos_kappa[ii[n]], m[ii[n]]))

would get phased out in favor of the R/MATLAB style, but this
is really unreadable:

kappa[pos_kappa[ii[n]]:(pos_kappa[ii[n]] + m[ii[n]])]

so would require a different computation for m as end-position
rather than size.

- Bob

> <graded_response.stan>

[Daniel Furr]

Thanks, Bob, I'll make use of your suggestion when I write up a case study for this. Also, an earlier version of the model I wrote used the R/MATLAB style of subsetting kappa, but I found it too confusing to read that way while troubleshooting.

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/viQXrMU7vU0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.