Problems with a Dynamic IRT Bayesian Model

79 views
Skip to first unread message

Guilherme Jardim Duarte

unread,
Jul 7, 2016, 2:26:05 PM7/7/16
to Stan users mailing list
Hi everyone,

I am trying to run a Dynamic IRT Bayesian Model with lots of parameters (around 40 thousand) and data points (1 Million) on a Mac OSx Computer with 12GB Ram.

I know this is a sort of "heavy" model. It turns out that sampling starts but it can't even run 1 iteration.

I also tried to run a model with only three terms in order to test if there's a problem with my code, but, even so, I couldn't get any success.

Did someone manage to run heavier models on RStan? Is there a problem with my code?

My code is here and my data is on dropbox: https://www.dropbox.com/s/ddapyibfz3f3d43/stan_data.RData?dl=0

stan_code <- '
/**
*  Dynamic Ideal Point Estimation via
*  Hamiltonian Monte Carlo for the Brazilian Chamber of the Deputies, 1991--2015.
*
*/
data {
int<lower=0> N;                 // number of votes
int<lower=0> P;                 // number of different parties
int<lower=0> J;                 // number of different legislators
int<lower=0> R;                 // number of different rollcalls
int<lower=0> T;                 // number of different terms

int<lower=0, upper=1> y[N]; // vote (yea = 1, nay = 0)
int<lower=0> p[N]; // parties
int<lower=0> j[N]; // legislators
int<lower=0> r[N]; // rollcals
int<lower=0> t[N]; // terms

vector[P] theta0;               // prior mean (M&Q: mu0)
vector<lower=0>[P] delta0;      // initial t sd (M&Q: sqrt of C[0,])

vector[2] b0;                   // prior mean for pos and discrim
cov_matrix[2] B0;               // prior covar for pos and discrim
}
parameters {
vector[2] alpha_beta[R];        // position and discrim of Rollcall r
vector[J] rho[T];            // position in term t of legislator j
vector[P] theta[T];            // position in term t of party p
}
model {
for (n in 1:N) {
  y[n] ~ bernoulli(Phi_approx(alpha_beta[r[n], 1] + alpha_beta[r[n], 2] * rho[t[n], j[n]]));
  rho[t[n]] ~ normal(theta[t[n], p[n]], .3);
}
alpha_beta ~ multi_normal(b0, B0);

theta[1] ~ normal(theta0, delta0);
for (i in 2:T)
  theta[i] ~ normal(theta[i - 1], .2);
}
'

My sessionInfo():

> sessionInfo()
R version 3.3.0 (2016-05-03)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.4 (El Capitan)

locale:
[1] pt_BR.UTF-8/pt_BR.UTF-8/pt_BR.UTF-8/C/pt_BR.UTF-8/pt_BR.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base    

Bob Carpenter

unread,
Jul 7, 2016, 2:49:20 PM7/7/16
to stan-...@googlegroups.com
We ran a one-million data point hierarchical IRT model with
around 40K (or maybe slightly more) parameters. It took about four
hours to fit. But that was a standard IRT model where we'd
identified what was correct. Ideal-point modeling is more
problematic because you don't know which way the issues go,
so the sign of rho can change. Andrew and Jennifer discuss some
tricks for identification in their regression book.

Some comments on the code:

> for (n in 1:N) {
> y[n] ~ bernoulli(Phi_approx(alpha_beta[r[n], 1] + alpha_beta[r[n], 2] * rho[t[n], j[n]]));

Logistic regression is a lot faster. So I'd suggest

y[n] ~ bernoulli_logit(alpha_beta[r[n], 1] + alpha_beta[r[n], 2] * rho[t[n], j[n]])

It's too bad we don't have more general lambda abstraction to vectorize the
algebra on the inside.

That puts the priors on a slightly different scale

> rho[t[n]] ~ normal(theta[t[n], p[n]], .3);

You do realize our normal priors use scale, not precision parameters
(i.e., like R, unlike BUGS/JAGS)?

Also, this doesn't make sense, as it's going to include priors
for rho[t] multiple times. And they somehow depend on n. I don't
know what you're trying to do, but I don't think this can be right.

Finally, you can vectorize this:

> for (i in 2:T)
> theta[i] ~ normal(theta[i - 1], .2);
> }

but you'll have to tear apart the terms to build up
the y and the mu terms. It'll be much more efficient
that way, but uglier to code.

> alpha_beta ~ multi_normal(b0, B0);

Multi-normal is relatively slow. It's a little more efficient with
multi_normal_cholesky, but still you'll find it much more
efficient with independent priors.

- Bob
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Guilherme Jardim Duarte

unread,
Jul 7, 2016, 3:41:18 PM7/7/16
to Stan users mailing list
Bob,

you're a legend.

The problem was with this line:

>   rho[t[n]] ~ normal(theta[t[n], p[n]], .3);

Now I changed it to:
>  rho[t[n], j[n]] ~ normal(theta[t[n], p[n]], .3);


It's working fast.

Thank you so much,

G.
Reply all
Reply to author
Forward
0 new messages