Stan 1.2.0 & RStan 1.2.0 released

[Bob Carpenter]

Stan 1.2.0 and RStan 1.2.0 are now available for download.
See:

http://mc-stan.org/

Here are the highlights.


1. Full Mass Matrix Estimation during Warmup

Yuanjun Gao, a first-year grad student here at Columbia (!),
built a regularized mass-matrix estimator. This helps
for posteriors with high correlation among parameters
and varying scales. We're still testing this ourselves, so
the estimation procedure may change in the future.

It's not the default option. The major reason is the matrix
operations required are expensive, raising the algorithm
cost to O(K * M * N^2 + log(M) N^3), where K is the average
number of leapfrog steps, M is the number of iterations, and
N is the number of parameters.

Yuanjun did a great job with the Cholesky factorizations
and implemented this about as efficiently as is possible.
(His homework for Andrew's class was also the inspiration
for the Gaussian process models in the manual.)

It's integrated with NUTS.


2. Cumulative Distribution Functions

The practical upshot is that Stan supports more truncated
distributions, and hence more truncated and censored data models.

Michael Betancourt did the heavy lifting here, which involved a crazy
amount of "special function" derivative calculations and implementations.
Everyone knows that the derivative of a distribution function with respect
to the variate is the density. But what about the partials with respect to the other
parameters? We'll be documenting all of the functions and derivatives
in the manual.

Daniel Lee generalized the entire density and distribution function
testing framework to generate code for tests. We're doing much more
extensive tests of the vectorizations and derivatives. Also, Daniel
implemented efficient vectorized derivatives for many more of the density
functions.


3. Model Log Probability and Derivatives in R

Jiqiang Guo, who's at the helm of RStan, wrote code to allow users
to access the log probability function in a Stan model and its gradients
directly. The functions are parameterized with the unconstrained
parameterization of a Stan model with support on all of R^N. He
also exposed the model functions to convert back and forth between
the constrained and unconstrained parameterizations for initialization
and interpretation of the samples.

David Blei suggested that if we added this feature, people
could do interesting things in R with it, such as optimization.
Let us know if you find it helpful.


4. Print Posterior Summary Statistics from Command Line

Daniel Lee wrote a program to print a summary of one or more chains
from the command line, mirroring the print() command of RStan.


5. Bug Fixes

We also fixed a bad memory leak in multivariate operations that
was introduced in the last release when we optimized the matrix
operations for derivative calculations. We also fixed the Windows
issue with conservative matrix resizing which caused multivariate
models to crash under Windows at optimization levels above 0.


There's also been a lot going on in various branches that haven't
been merged into the trunk yet, so stay tuned.

- Stan Development Team


P.S. Here are the release notes.


v1.2.0 (6 March 2012)
======================================================================

Enhancements
----------------------------------
* full mass matrix estimation during warmup
* expose model log_prob and gradient functions in RStan for use
in other packages (such as optimizers)
* command-line program to display output from multiple chains
with parameter-by-parameter mean, se, sd, quantiles, and R-hat
* probability function speed improvements with vectorization
* created Stan contributed repositories for user-contributed
and experimental features (first entry is an emacs mode)
* modified makefiles so targets are the same under Windows,
Linux, and Mac

New Functions
----------------------------------
* most of the cumulative distribution functions (see the documentation
index for the full list of supported functions)
* added monitor() function in RStan

Bug Fixes
----------------------------------
* disabled Boost asserts in parser to quiet R's warnings
* enabled prints in generated quantities block
* various documentation patches
* fixed memory leak in matrix operations leading to leaks in
multivariate probability function use
* wrapped call to gradient log prob to catch unexpected exceptions
* fixed matrix resize issue on Windows that caused models to fail
at optimization levels above 0
* fixed bug in print preventing hyphens or grave accents from
priting
* fixed issue preventing matrix rows from being assigned on the
left side of an assignment statement
* clearer error messages on matrix and other function arguments

[akhil]

Hi Stan devs, 

Where can I find more detail about the mass matrix estimation routine?  Also, I was wondering if the Riemannian approach in future releases will allow users to define their own metric or if the default will be Fisher-Rao?

Thanks,

akhil

[Bob Carpenter]

The mass matrix estimation routine is Yuanjun's.

It regularizes toward the diagonal matrix of variances and
toward the identity matrix. Like the diagonal estimation that's the default,
it works during warmup by (roughly) using the last half of the
warmup to estimate (it actually does this as it goes along, so
the mass matrix gets updated during warmup). Yuanjun's writing
up a paper that'll have all the details.

We'll start with Michael Betancourt's SoftAbs metric and maybe
the diagonal approximate SoftAbs, too:

http://arxiv.org/pdf/1212.4693.pdf

We're working on refactoring the HMC code to make it all
much more configurable at the C++ level.

The metric will definitely be plug-and-play at the C++ level.
Any suggestions on how to make it user configurable through
the Stan command and RStan? My inclination would be to name a class
that is used in a template somewhere and instantiated in the code.

We also need to work on user-defined functions of other kinds, so
maybe we can do both of these at the same time.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Andrew Gelman]

Incidentally, this is another reason for Yuanjun to write up his paper.  Once it's written up, others can read it and maybe come up with improvements!

[Richard Alexander]

Hi,

I'd appreciate if you could provide a bit more information related to this new function in version 1.2 and it's execution from R.  I've not been able to find it in the manual.

 

   * command-line program to display output from multiple chains
      with parameter-by-parameter mean, se, sd, quantiles, and R-hat

 

In addition, related to processing the results from multiple chains, are the functions (scripts) available for computing Rhat and the effective sample size.  I'd like to check these against my code.  Because of RAM limitations on my Windows PC (8GB), I've been experimenting with executing separate chains in R and storing fewer results from the stan object (parameter values and only the mean and SS of the predictions, excluding the burnin samples).  The predictions from the generated quantities block are excessively large in this particular case.  My interest in looking at the functions is that I've encountered some larger than expected discrepancies between my calculated Rhats and Effs, based on comparisons with the standard multiple-chain results.

 

Many thanks for your help.

Richard

[Bob Carpenter]

On 3/15/13 4:53 PM, Richard Alexander wrote:
> Hi,
> I'd appreciate if you could provide a bit more information related to this new function in version 1.2 and it's
> execution from R. I've not been able to find it in the manual.

http://stan.googlecode.com/files/stan-reference-1.2.0.pdf

See Chapter 5, "Print Command for Output Analysis", p. 28

> * command-line program to display output from multiple chains
> with parameter-by-parameter mean, se, sd, quantiles, and R-hat

It's a command-line program, so it's not directly accessible from
R.

RStan has the same functionality, though, so you can run
chains in parallel. Run help("rstan") in R and look through
the index.

> In addition, related to processing the results from multiple chains, are the functions (scripts) available for computing
> Rhat and the effective sample size.

If you can get your data into the format that Stan outputs,
then you can run the above analysis from the command line.
There are also ways to do this in R.

And other packages, like Coda, that compute R-hat and
effective sample size. They all do this differently,
as it requires an estimate. Stan splits the chains in
half for R-hat and uses an R-hat-like variance estimator
for effective sample size, which is more conservative
for poorly mixed chains than others.

> I'd like to check these against my code. Because of RAM limitations on my Windows
> PC (8GB), I've been experimenting with executing separate chains in R and storing fewer results from the stan object
> (parameter values and only the mean and SS of the predictions, excluding the burnin samples). The predictions from the
> generated quantities block are excessively large in this particular case. My interest in looking at the functions is
> that I've encountered some larger than expected discrepancies between my calculated Rhats and Effs, based on comparisons
> with the standard multiple-chain results.
> Many thanks for your help.

Let us know what you find.

- Bob

[Richard Alexander]

Many thanks Bob. I decided to use your suggestion of using CODA routines rather than trying to reinvent the wheel with my own functions. I'm applying the CODA routines to half the chain samples as done in Stan. So far, it's working fine but I'll see how the results for Neff and Rhat compare with those from Stan as I look at different datasets and models.
Richard.

[Andrew Gelman]

I don't think you should be using coda routines for monitoring convergence. Our R.hat and n_eff are much better.
A

On Mar 17, 2013, at 10:02 PM, Richard Alexander wrote:

> Many thanks Bob. I decided to use your suggestion of using CODA routines rather than trying to reinvent the wheel with my own functions. I'm applying the CODA routines to half the chain samples as done in Stan. So far, it's working fine but I'll see how the results for Neff and Rhat compare with those from Stan as I look at different datasets and models.
> Richard.
>

[Richard Alexander]

Thanks Andrew,

Per your advice, I've been able to correct my prior calculations of Neff and Rhat (found the coding error), based on the recommended Stan functions.  From some test runs with fake data, I can now see that they are typically more conservative measures compared to those computed according to Coda. 

When you say that these are "better" measures, is that because they are conservative (and thus likely to lead to more cautious model estimation with additional simulations) and/or is this based on the outcomes of systematic evaluations of test datasets (with known parameters and error) or experiences with field datasets?  Are there references for such evaluations?

Thank you.

Richard

[Andrew Gelman]

Hi, Richard.  See attached excerpt from BDA3.  The short story is that the new measures account for between and within chain behavior, whereas the old measures ignored the time-series information within each chain.  We've also done some work finding the new measure doing better in some examples.  I'm cc-ing my collaborator on this work, Kenny Shirley, as we are in the middle of trying to write this all up.

Andrew