rstan request

[Andrew Gelman]

Hi, this has come up before, I think… when running Stan from R, I’d like it, by default, to pull in the data from objects of the same name in the R workspace. That way I don’t have to include data(y=y,X=X,N=N) etc in my stan call. Or maybe this feature already exists and I don’t realize it?
Thanks!
A

[Ben Goodrich]

stan(..., data = .GlobalEnv) should already work if y, X, N etc. are all in the global environment. The problem is if you have additional objects in the global environment that stan() can't process. Then I think it throws an error.

Ben

[Bob Carpenter]

I'm not keen on what Andrew wants, which I believe is something
that has no "data = ..." argument and just looks in the global
environment by default. Globals are error prone due to the possibility
of accidentally overwriting them (e.g., by downloading some other
package).

But if Ben and Jiqiang think it's OK to do, we could potentially
wait until the model is compiled, then read the variables off
that, then use them as the equivalent of

data = c("y", "X", "N", ...)

to create the var_context used to read in data.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Avraham Adler]

What exactly is the issue? If it is one on convenience, I tend to agree with Bob in that it's safer not to play with globals. Have you considered created a named list (DATA = c(...)) and calling stan(..., data = DATA)? This way, if anything changes, you just change the elements of DATA but your stan call remains the same. Or is there something else?

Avi

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

[Richard McElreath]

I share Bob's hesitation about working with globals. But it's worth noting that R is meant to be interactive, with the intent that users work with globals. So Andrew's request finds comfort from the design doc for the R Environment.

It has taken me years to get over my resistance to using global variables in R. One way to think about R is that everything about it that is annoying to "real" programmers makes it more useful for the majority of users.

[Andrew Gelman]

Avi:

Yes, I’ve done that thing with the named list but I find it a bit of a pain in the ass to have code that says y=y,X=X,N=N, etc.  It just looks ugly to me, also it’s awkward that when I change the model I have to do 2 different things:

1.  Change the Stan program

2.  Change the data input to the stan call in R.

To me, #2 is redundant, and I find it to be one more piece of bookkeeping that I have to keep track of in these models.

A

[Tamas Papp]

I agree --- this is one of those DWIM features that look useful, but can
be a potential source of bugs in more complicated code, especially if
code and variables are reused.

If this is added, please make this optional (eg have a data = TRUE
argument that would make Stan pull in variables from the globals, etc,
if this is feasible).

Best,

Tamas

[Krzysztof Sakrejda]

How about a function with a short-and-sweet name that you could throw around your data expression, se something like data=f(a,b,z,q), that would pick put the correct things from a given environment. 

It could be implemented as:

 f <- function(..., e=.GlobalEnv) { x <- all.names(substitute(list(...))); x <- x[2:length(x)]; o<-lapply(x, get); names(o) <- x; return(o) }
a <- 5
b <- 3
f(a, b, letters)

you should be able to do: stan(..., data=f(a, b))

That way the Stan call would name the required object without having to say from where (global by default) and it would avoid having to name the same thing twice (  list(a=a, b=b)   ).  All we need is a nice short memorable name.  It's even useful if you want to use another environment, but I usually just do:

with(data, stan(.....)) which makes it clear and short.

Krzysztof

On Tuesday, December 9, 2014 10:43:46 AM UTC-5, Bob Carpenter wrote:

[Avraham Adler]

Understood, but it isn't exclusive to Stan. I use nloptr all the time, and am used to passing in the data values used in the function being minimized in the call to nloptr (or optim) itself(x=x, y=y, etc.).

Avi

[Bob Carpenter]

I think it's at least three different things, because if
you change the name of the data variables in the Stan program,
then you are also going to have to

3. (Re)assign the variables in the global environment.

So if you decide in the Stan program to rename a data variable sigma_mu
instead of just sigma, then you'd have to do the following in the global
environment:

> sigma_mu <- sigma;

Punctuation is of course optional

- Bob

>> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

>> > For more options, visit https://groups.google.com/d/optout.
>>
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Bob Carpenter]

We should do more of the with() construction --- I really
like it because of the explicit scope.

- Bob

[Andrew Gelman]

Just by analogy, if I want to add two numbers in R, I can do the following:

c <- a + b

I don’t have to do this:
c <- plus(data=c(“a”,”b”))
or
c <- plus(data=list(a=a,b=b))

Similarly, in Stan I just want to fit the damn model. If my Stan program used data x,y,z,n, I’d like R do to the following, for each variable:
- If I directly supply it in the Stan call, use it.
- If I don’t supply it, pull it from my R session.
- If I don’t supply it in my R function, return an error.
If you think it would be a good idea, the R call could print a statement letting the user know which variables were being pulled in from the R session, so that it wouldn’t happen to users without them realizing it.

A

[Ben Goodrich]

On Wednesday, December 10, 2014 4:28:13 PM UTC-5, Andrew Gelman wrote:

Just by analogy, if I want to add two numbers in R, I can do the following:

c <- a + b

I don’t have to do this:
c <- plus(data=c(“a”,”b”))
or
c <- plus(data=list(a=a,b=b))

By your analogy, you would also want

x <- rnorm(100)
mean()

to work because the first argument to the mean function is x and an object called x exists in the global environment. Your preferred behavior relies on the fact that the declared names of the variables in the data block of the Stan program are the same as their symbol names in R and that you are calling stan() from the global environment instead of calling stan() from another function. Subtle errors and undefined behavior can crop up otherwise.

R only behaves the way you want when you utilize an environment, as in

lm(y ~ x1 + x2, data = dataset)
with(dataset, lm(y ~ x1 + x2))
do.call(lm, args = list(formula = y ~ x1 + x2), envir = .GlobalEnv)

Similarly, Stan would behave the way you want it to if you just called

stan(file, data = .GlobalEnv)
stan(file, data = dataset)

except that you if the data argument includes any objects that Stan cannot handle, then an error is thrown currently. We could possibly not throw that error if the object does not correspond to any variable declared in the data block of a Stan program.

Ben
 

>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

[Andrew Gelman]

Doing stan(file, data = .GlobalEnv) or stan(file, data = dataset), that would work.

A

[Ben Goodrich]

It always has worked (unless you have an object lurking in .GlobalEnv or dataset that stan() can't process, in which case it does not work).

[Bob Carpenter]

If we do this (which I'll go on record as objecting to because of
its lack of transparency and its dependence on globals), then we
shouldn't leave it the way it is now where it will pull everything
out of the blog environment into a var_context --- it's going to
make all the behavior depend not only on the relevant global vars,
but also irrelevant ones.

Let's try to at least remain as modular as possible.

So what I'd propose (if we do this) is to add a new code-generated
method to the generated model class

std::vector<std::string> data_variable_names();

that returns a vector of data variable names. This should only take
me a day or so, including testing.

Then, we would create a var_context from the model and an environment
(defaulting to the global environment) that would pull out all the
variables named in the model from the global environment to create
a var_context.

My main objection to all of this is that it's not clear from the call
stan(...) which dependencies there are on the global environment. You
need to look inside the file/string defining the Stan model for that.

I can also anticipate Andrew's next request, which will be to pull out of the
list just the variables given by the list, and pull the rest out of the
global environment, but maybe we should punt on that (though the
stacking/chainging/whatevering var_context should allow it pretty easily).

- Bob

>> >> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

>> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

>> > For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Jiqiang Guo]

As Ben pointed out, this is some data preprocessing in stan().  So even you can pass .GlobalEnv, it might not be a good idea to do it.  In particular, it needs to check if doubles can be converted to integers without losing precision.

Jiqiang 

[Jiqiang Guo]

On Wed, Dec 10, 2014 at 5:20 PM, Bob Carpenter <ca...@alias-i.com> wrote:

If we do this (which I'll go on record as objecting to because of
its lack of transparency and its dependence on globals), then we
shouldn't leave it the way it is now where it will pull everything
out of the blog environment into a var_context --- it's going to
make all the behavior depend not only on the relevant global vars,
but also irrelevant ones.

I agree with Bob.  I don't like the idea of looking for objects in an environment without being told. 

Jiqiang