Manual Example Multilevel 2PL IRT Model

[Rick Farouni]

Hi,


In the example in section 9.2 of the manual (found here: https://github.com/stan-dev/stan/blob/develop/src/models/misc/irt/irt2_multilevel.stan), how is the multiplicative aliasing problem solved by the following model specification?

model {
  alpha ~ normal(0,sigma_alpha); 
  beta ~ normal(0,sigma_beta);   
  log_gamma ~ normal(0,sigma_gamma);
  delta ~ cauchy(0,5);
  sigma_alpha ~ cauchy(0,5);
  sigma_beta ~ cauchy(0,5);
  sigma_gamma ~ cauchy(0,5);
  for (n in 1:N)
    y[n] ~ bernoulli_logit( exp(log_gamma[kk[n]])
                            * (alpha[jj[n]] - beta[kk[n]] + delta) );
}

According to Bafumi, Joseph, et al. (2005), http://www.stat.columbia.edu/~gelman/research/published/171.pdf, the authors recommend that the alpha be assigned a fixed population distribution (e.g, N(0,1)) or that the normalization and scaling method be used. I see that the additional parameter "delta" serves as the mean of the alphas, but what about scaling?

Thanks

[Andrew Gelman]

Hi, I’d recommend writing the parameter as gamma and _not_ constraining it to be positive, possibly giving it a normal(1,sigma_gamma) distribution.  Writing it as log_gamma seems wrong to me because, in practice, it is possible to have items with negative discrimination.

Regarding identifiability of scaling, the proper priors on alpha, beta, gamma assure (Bayesian) identifiability.  The aliasing problem I think is resolved because the mean of the gammas is not zero.  But I agree that it probably will be better to give the alphas a fixed variance of 1.

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rick Farouni]

Hi,

Does giving a gamma a normal(1,sigma_gamma) distribution resolve reflective indeterminacy? If so what about the more general case of rotation indeterminacy in a two -dimensional model with  gamma1 and gamma2, for example. In the case of independent alphas, what do you  think would be the best way to  deal with the resulting  8  component  posterior mixture density? Would a gamma~MVN( [1,1], I) do the trick?

[Bob Carpenter]

It's more efficient to write

gamma ~ normal(1,1);

and rely on the vectorization. Using multi_normal is relatively
inefficient.

And we really need to rewrite some of those models. Using exp(log_gamma)
is a bad idea. If you're going to make something positive, it's
cleaner to just declare the variable as:

vector<lower=0>[K] gamma;

Rather than having a log_gamma without a constraint and then exponentiating.
Then the lower-bound constraint takes care of the exponentiation in
the background for you.

Identifiability in theory is not the same as in practice. In practice,
it may take a lot of samples to read convergence if a parameter is
not well identified in the posterior. So cranking down one of the
parameters to normal(0,1) will identify the scale.

- Bob

[sp_r...@yahoo.it]

Sorry, but I'd say that the model (2.3.0 Manual, p. 34) should be

> model {

>    sigma_alpha ~ cauchy(0,5);
>    sigma_beta ~ cauchy(0,5);
>    sigma_gamma ~ cauchy(0,5);

>    alpha ~ normal(0,sigma_alpha);
>    beta ~ normal(0,sigma_beta);
>    log_gamma ~ normal(0,sigma_gamma);
>    delta ~ cauchy(0,5);

>    for (n in 1:N)
>      y[n] ~ bernoulli_logit( exp(log_gamma[kk[n]])
>                              * (alpha[jj[n]] - beta[kk[n]] + delta) );
> }

shouldn't it?
Shouldn't the prior for a parameter always appear before sampling statements that use that parameter?

Thanks

Sergio

[Marcus Brubaker]

The ordering of sampling statements doesn't matter.  Assignment statements need to come before other statements which use them, but otherwise the ordering of sampling statements doesn't matter.

[Bob Carpenter]

The key points are:

* every variable needs to be defined before if is used
* data, transformed data, parameters, and transformed parameters
variables are automatically defined before the model block
is executed

We need to rename "sampling statement" to something more like
"distribution log probability increment statement" to make it
a bit clearer that it doesn't literally do sampling of the left-hand
side variable from the right-hand side distribution.

Having said that, I prefer the style where the priors come before
the likelihood if your model's programmed in terms of priors and
likelihood.

- Bob