samples of derived parameters allocated but not stored

[Ben Goodrich]

Hi,

If you run Wawro's model with something like

goodrich@CYBERPOWERPC:/opt/stan$ demo/wawro --data_file=/opt/wawro/
stan/wawro.dump

and read samples.csv into R (code below), the first 18 columns
(corresponding to parameters) have a lot of unique values and the
remaining 50 columns (corresponding to derived parameters) have one
value, zero. I assume it's a bug to not store the derived parameters
but otherwise it's a feature request to store them (or maybe have the
option).

Ben

P.S. It would be nice if samples.csv had headers rather than having to
infer them from the order of the parameters.

P.P.S. R code

> samples <- read.csv("samples.csv", header = FALSE)
> as.matrix(apply(samples, 2, FUN = function(x) length(unique(x))))
[,1]
V1 784
V2 785
V3 785
V4 786
V5 785
V6 786
V7 786
V8 786
V9 783
V10 786
V11 785
V12 786
V13 786
V14 786
V15 786
V16 785
V17 786
V18 784
V19 1
V20 1
V21 1
V22 1
V23 1
V24 1
V25 1
V26 1
V27 1
V28 1
V29 1
V30 1
V31 1
V32 1
V33 1
V34 1
V35 1
V36 1
V37 1
V38 1
V39 1
V40 1
V41 1
V42 1
V43 1
V44 1
V45 1
V46 1
V47 1
V48 1
V49 1
V50 1
V51 1
V52 1
V53 1
V54 1
V55 1
V56 1
V57 1
V58 1
V59 1
V60 1
V61 1
V62 1
V63 1
V64 1
V65 1
V66 1
V67 1
V68 1

[Bob Carpenter]

Now that I added local variables, the idea
is to print all the derived parameters as well
as the parameters. It was working, but my latest
patch must've broken it. We need better tests
around the behavior of the models, not just their
compilability.

Re P.S. I plan to generate code that'll let
you read the whole thing into R as a list.
How do I specify the header? If I have a
simple scalar x and a 2-D array y, I'd like to write
a header like:

# x, y[,1], y[,2]

but I can't figure out how to get R to accept
anything like this or manipulate it once it's
written. So I was thinking I'd just do something
roughly as follows (only getting all the R right):

read_file <- function(path) {
sample_matrix <-
as.array(scan(path)),
dims=c(num_samples,num_params),
byrow=T);

y[,1] <- sample_matrix[,1];
y[,2] <- sample_matrix[,2];

params <- list(y=y,...);

return params;
}

The right thing to do would be to generate C++ code
to write directly into R. But I probably want the
file-based thing, too.

- Bob

[Ben Goodrich]

On Jan 5, 11:44 am, Bob Carpenter <c...@alias-i.com> wrote:
> Now that I added local variables, the idea
> is to print all the derived parameters as well
> as the parameters.  It was working, but my latest
> patch must've broken it.  We need better tests
> around the behavior of the models, not just their
> compilability.

OK.

> Re P.S.  I plan to generate code that'll let
> you read the whole thing into R as a list.

I would say that an R function should return an object of class "mcmc"
from the coda library, rather than a list. If you don't want to depend
on the coda library (not really a big deal IMO), then I would say a
data.frame is preferable to a plain list.

> How do I specify the header?  If I have a
> simple scalar x and a 2-D array y, I'd like to write
> a header like:
>
> # x, y[,1], y[,2]

I was thinking that the parser would make the write_csv function
engrave the parameter names into the first line of the .csv file
automagically with something like this near the top

if(writer__.lines() == 0) {
// have the parser construct a vector of names called names
for(unsigned int k_0 = 0; k_0 < dim_names; ++k_0) {
writer__.write(names[k_0]);
}
writer__.newline();
}

Then, inside R you can just call

samples <- read.csv(path, header = TRUE)

at which point samples is a data.frame with column names that could be
converted to something else fairly easily with as.*() functions, e.g.

as.matrix(samples)
as.mcmc(as.matrix(samples))
as.list(samples)

Ben