extracting Rhat, n_eff and total sample size in Rstan

[Tamas Papp]

Hi,

I find it useful to plot a histogram of Rhat, and n_eff / total sample
in rstan, and then eyeball it for outliers (for long vectors, the plot
is not always informative since it only shows the first few values).

I found that I could extract Rhat and n_eff using

summary(fit)$summary[,"Rhat"]
summary(fit)$summary[,"n_eff"]

and the total sample size using

length(extract(fit, pars = "lp__")[[1]])

from a stanfit object `fit`, so I can do what I need, but I am wondering

1. if there is a method in the API, and if yes, what it is (the above
seems so ad hoc that I am sure I am missing something)

2. if not, are there plans to include a few convenience functions?

Best,

Tamas

[Bob Carpenter]

What would you like to see in the way of utility functions?

We've been rethinking the entire interface for RStan, so now's
a good time to bring up suggestions.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Andrew Gelman]

Hi, I’m ccing Michael A. and Jonah G. on this as they are preparing ShinyStan.
A

[Jonah Gabry]

Histograms of Rhat, and n_eff / total sample is definitely something we can include in SHINYstan.  

In terms of convenience functions, I was wondering if maybe I should export more of the behind-the-scenes functions that I've written for SHINYstan (e.g., functions for extracting various bits of information from the stanfit object) so that loading the SHINYstan package gives access to a library of convenience functions (in addition to the shiny app). 

Jonah

[Allen Riddell]

Hi Bob, Tamas,

I agree that adding an rhat and an ess method to fit objects would be desirable.
Perhaps we could add such things to the proposed Stan 3 spec?
(https://github.com/stan-dev/rstan/wiki/Interfaces-3.0-Spec)

Thanks,

Allen

[Tamas Papp]

Hi Bob,

Currently just what I had in the e-mail.

I opened an issue: https://github.com/stan-dev/rstan/issues/116

which has simple implementations and use case examples.

Best,

Tamas

[Jonah]

In the SHINYstan R package I'm working on (which contains an app for interactively exploring Stan models), I use a function I wrote called 

stan_get which takes arguments stanfit and what(as well as optional arguments like pars for restricting the parameters used)  and is used like this

rhats <- stan_get(stanfit, what = "rhat") # returns rhats 

q_95 <- stan_get(stanfit, what = "quantile", probs = c(0.025, 0.975)) # returns 2.5% and 97.5% quantiles

etc. 

This way you can use one function and just change the what argument instead of having 10 or 20 functions. Right now it can return Rhats, effective samples sizes, total sample size, means, SDs and any combination of quantiles. 

I've also written the function so that the what argument allows the same thing to be specified in many different ways, since different people use different conventions. So, for example, all of the following will return the effective sample sizes:

stan_get(stanfit, what = "ess")

stan_get(stanfit, what = "ESS")

stan_get(stanfit, what = "n_eff")

stan_get(stanfit, what = "N_eff")

stan_get(stanfit, what = "neff")

stan_get(stanfit, what = "Neff")

[Jonah]

Forgot to say that the histograms of Rhat and n_eff/total sample size will be included in SHINYstan within the next few days. You'll also be able to access the stan_get function. 

On Friday, November 21, 2014 3:55:39 AM UTC-5, Tamas Papp wrote: