Indicies: Aggregating lower level predictors, for group-level models

[Brendan]

Dear List,

Firstly, I must make the usual apologies & disclaimers that I'm new to Stan, MCMC, and posting on mailing lists, and appreciate your time for reading this far!

There's a long and a short way to ask my question. Hopefully the short version suffices!

Short version
In the model below, I've tried to aggregate a longer data vector (wlp, which goes from i...n), to be used in a group level (j...J) regression. The outcome of interest is at the group level only. However, the approach I used in WinBUGS (using from and to index variables, of length J, giving the start and end positions of the subsets of wlp which correspond to each group), hasn't worked in Stan.

model {
    for(j in 1:J)
        index <- from[j]:to[j];
        wlp <- mean(i_lp[index]);
        y[j]  ~ normal(a + b * wlp, sigma);
}

#### EXPECTATION FAILURE LOCATION: file=input; line=21, column=25
####
####         index <- from[j]:to[j];
####                         ^-- here

Putting aside why on earth I'd want to for a second, is there a way to subset vectors mid-loop like this?

[Bob Carpenter]

On 6/26/13 11:02 AM, Brendan wrote:

> ...

> The outcome of interest is at the group level only. However, the approach I used in WinBUGS
> (using from and to index variables, of length J, giving the start and end positions of the subsets of wlp which
> correspond to each group), hasn't worked in Stan.

...

> index <- from[j]:to[j];
> wlp <- mean(i_lp[index]);

...

> Putting aside why on earth I'd want to for a second, is there a way to subset vectors mid-loop like this?

No, there's not. It's another thing on our to-do list, but
even if we build it, it won't be the fastest way to do this in Stan.

In Stan 2.0, we've introduced a way to do it as follows:

wlp <- mean(sub_vector(i_lp,from[j],to[j]));

For now, the only way to do it is with a temporary
variable into which the sub-vector is written:

{
vector[to[j] - from[j]] temp;
for (i in from[j]:to[j])
temp[i - to[j] + 1] <- i_lp[i];
wlp <- mean(temp);
}

You need the brackets to make "temp" a local variable.

- Bob

[Brendan]

Cheers Bob! I'll try and get this going.

Just out of interest, do you guys have any rough ideas about when 2.0 might be ready?

On Wednesday, 26 June 2013 16:02:12 UTC+1, Brendan wrote:

Dear List,

Firstly, I must make the usual apologies & disclaimers that I'm new to Stan, MCMC, and posting on mailing lists, and appreciate your time for reading this far!

There's a long and a short way to ask my question. Hopefully the short version suffices!

Short version
In the model below, I've tried to aggregate a longer data vector (wlp, which goes from i...n), to be used in a group level (j...J) regression. The outcome of interest is at the group level only. However, the approach I used in WinBUGS (using from and to index variables, of length J, giving the start and end positions of the subsets of wlp which correspond to each group), hasn't worked in Stan.

model {
    for(j in 1:J)

        index <- from[j]:to[j];
        wlp <- mean(i_lp[index]);

        y[j]  ~ normal(a + b * wlp, sigma);
}

#### EXPECTATION FAILURE LOCATION: file=input; line=21, column=25
####

####         index <- from[j]:to[j];
####                         ^-- here

[Bob Carpenter]

On 6/26/13 2:01 PM, Brendan wrote:
...

> Just out of interest, do you guys have any rough ideas about when 2.0 might be ready?

Maybe in a couple of weeks, but no promises.

We're making a lot of changes, so we want to make sure
everything works smoothly so we don't have to make even more
changes going forward.

The development branch on GitHub is stable under our new
development process, but I'll warn you that the command structure
will change. After that change, RStan will follow, and we'll
be pretty much ready to release. Much of the doc is going to
need to change, too.

Overall, the interface through RStan shouldn't be that
different. It'll mainly be an under-the-hood change with
more options in terms of samplers, adaptation, and optimization.

We might roll out a beta first to work out the kinks with
a wider pool of users.

- Bob