Stan v2.3.0 tagged: CmdStan, RStan, PyStan -- once we're all ready, we'll announce.

[Daniel Lee]

v2.3.0 (16 June 2014)
======================================================================
Bug Fixes
------------
* fixed poisson_rng, neg_binomial_rng
* integer overflow now throws errors
* allow < and > within range constraints
* support MS Visual Studio 2008
* fixed bug in lkj


Features
------------
* functions added to Stan language!
* turned off eigen asserts
* CmdStan is now in its own repository
* added quad_form_sym, multi_gp_log, multi_gp,
  multi_normal_cholesky_rng, to_matrix, to_vector, to_array, flatten,
  qr_Q, qr_R
* added JSON parser to Stan
* a^b syntax for pow(a,b)

[Daniel Lee]

I've gotten CmdStan up. We can announce all at once.

[Marco Inacio]

It's actually to_matrix, to_vector, to_row_vector, to_array_1d, to_array_2d instead of to_matrix, to_vector, to_array, flatten

Also added neg_binomial_2

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Gelman]

Please, let’s not release a new Rstan until we have the new functions that we’ve discussed, so we can just run stan() and stan_optimize() the way we want.  Thanks.

A

[Bob Carpenter]

We have a number of critical bugs that the pending release
fixes. These include errors in correlation matrix Jacobian,
buggy Cholesky factor parser, several RNG bugs, and memory
leaks in the categorical distributions and softmax functions.
We also have a gazillion more features in this release.

Is there a design everyone agrees on somewhere for which new functions
you want? Is it just changes to existing functions (for what I've been
calling "make-like" and you've been calling "Sebastian's function(s?)").

And is there someone in line to code, test, and document it? I
don't understand enough R to get anywhere near this. I'm butting
out of the design and doc, as well.

If so, what's the time estimate on getting it implemented (not hours
it'll take, but time from now in calendar days before it'll be done)?

If the answer to that last question isn't something like a week
or less, then I think we should wait until after 2.3 in order to
get the bug fixes out.

I hope to then get L-BFGS done soon and put out another release
soon after 2.3. (Speaking of which, is the finite diffs Hessian
ready to go with that?)

Then after that, I have the ODE solver to do. But that probably won't
be ready for at least five or six weeks.

This is just my opinion, of course --- what do others think?
I know Marcus wants to get L-BFGS in ASAP and I think Andrew wants
it, even if he doesn't know it.

- Bob

[Ben Goodrich]

On Monday, June 16, 2014 5:44:37 PM UTC-4, Bob Carpenter wrote:

Is there a design everyone agrees on somewhere for which new functions
you want?  Is it just changes to existing functions (for what I've been
calling "make-like" and you've been calling "Sebastian's function(s?)").

No. But more than that, not everyone thinks it is a good idea.

I think I'm outvoted, but I will just say it one more time: If stan() were to automatically save the stanfit object to foo.RData, then it would be in violation of CRAN's policy against malicious behavior

http://cran.r-project.org/web/packages/policies.html

which states "The code and examples provided in a package should never do anything which might be regarded as malicious or anti-social. The following are illustrative examples from past experience ... Packages should not write in the users' home filespace, nor anywhere else on the file system apart from the R session's temporary directory". If packages were to do that, it would be easy for users to unwittingly overwrite results that they explicitly saved to the disk.

So, if we do this, rstan will never be able to go onto CRAN. I think the lack of binary installation via the usual mechanism from CRAN is a problem that is two or three orders of magnitude worse than the problem that people who do not know enough R to save() and load() their own stuff have to wait an extra minute while it compiles in the relatively rare case that they want to run the same model in different R sessions.

We could implement the autoloading behavior without the autosaving behavior, although I don't know if that would justify delaying the release. Whatever we decide to do, we should do it now because we have the Meetup downtown next Tuesday with about 25 people signed up to learn how to install various flavors of Stan.

Ben

[Bob Carpenter]

I don't think anyone's provided any feedback other than Andrew
and Ben --- so as far as that goes, I'd say the "vote" is a draw.
And I'm not sure Andrew understood the repercussions w.r.t. CRAN,
but hopefully does now.

Would you (Ben) be OK with it saving to the temp directory so
that it wouldn't work cross session? I also think cross sessions
will be problematic because of potentially changing versions of
Stan or R.

Personally, I'd like to obstain from voting. Now that I know
the primitive functions are in, my own preferred way of working
will be to use those directly. And nothing's to prevent a user
from saving something to their own home directory using the
primitive functions.

- Bob

[Jiqiang Guo]

Thanks for Ben's explanation about CRAN's policies.  I did not even

know about that.  Good to know that.  

Jiqiang 

[Ben Goodrich]

On Monday, June 16, 2014 6:44:36 PM UTC-4, Bob Carpenter wrote:

Would you (Ben) be OK with it saving to the temp directory so
that it wouldn't work cross session?  I also think cross sessions
will be problematic because of potentially changing versions of
Stan or R.

What would be the purpose of autosaving to the temp directory? The stanfit object is already in RAM so the user can just pass that to the fit argument of stan(). If we assume the users don't know that they can pass that to the fit argument of stan(), then I guess doing it for them is slightly helpful. It does take a minute to save a large object with compression though.

Ben

[Michael Betancourt]

I think what’s important is that this issue is nowhere near settled enough

to warrant a delay in 2.3.

[Andrew Gelman]

Hi all.  If Bob and I have a chance to meet on Thurs we can discuss.  There’s no question in my mind that we want a version of these functions that does not require recompilation each time.  If what we need to do is too complicated to do right away, then at the very least we can add some stuff to the rstan documentation to explain to people how to run things.

But the current status quo, in which I need to use Sebastian’s code and give my functions new names (and with lack of parallelism of stan and stan_optimize) is no good.  I do want to do some basic fixes on this right away.

A

[Bob Carpenter]

I don't know much about R, so I doubt I'm going to be any help
on Thursday.

My only input is that I don't want to delay 2.3 until we come
up with a design and then a coder, tester, and documenter.

First task is the design. Which basic fixes do you want, Andrew?
If it writes into the temp directory and doesn't compile across
sessions, is that OK? If you have to explicitly provide a path with
a default of the temp directory, would that be OK? Or would you
rather just give up on CRAN? I don't really care one way or another,
but it'd probably help everyone to lay out:

* functional requirements

* technical specs on how to achieve the above

- Bob

[Andrew Gelman]

P.S.  One intermediate step we could take, while still arguing about the rest, is to standardize default names.

For example, suppose we run the following function call:

object <- stan(foo.stan,…)

or

object <- stan_optimize(foo.stan,…)

The function can handle this as follows by first looking for a file “foo.stan” in the working directory and an object “foo.stan” in the workspace,  Then:

1.  If the object foo.stan in the working directory exists and has been updated more recently than the foo.stan file (or if there is no foo.stan file), then Stan will just use the foo.stan object as the compiled model and run it.

2.  If the object foo.stan in the working directory exists but the file foo.stan has been updated more recently, then Stan will recompile the model from the file and overwrite it as foo.stan in the working directory, then run it.

3.  If there is no object foo.stan in the working directory but the file foo.stan exists, then Stan will compile the model from the file, save the compiled model as foo.stan, and run it.

4.  If there is no object foo.stan in the working directory and no file foo.stan, then Stan will spit out an error.

The above can be default behavior.  We can have an option that saves into the directory also.

But it’s important for me to have some of this functionality.  As it is, my code is a mess.  Running things with 0 chains, etc., it’s horrible.  When people use Bugs/Jags, they have to do hack after hack.  I don’t want Stan users to be hacking in this way.

Also, we need some verbose options.

A

[Ben Goodrich]

On Tuesday, June 17, 2014 10:28:50 AM UTC-4, Andrew Gelman wrote:

But the current status quo, in which I need to use Sebastian’s code and give my functions new names (and with lack of parallelism of stan and stan_optimize) is no good.  I do want to do some basic fixes on this right away.

Attached is a function that does all that and would work on everything but Windows.

Ben