As far as we can tell, Stan 2.09 is the latest version of Stan
with a properly functioning sampler.
Versions from 2.10 on are producing biased samples
that slightly underestimate posterior variance. Thanks to
Matthew R. Becker for filing the issue:
https://github.com/stan-dev/stan/issues/2178
Stan 2.10 changed the NUTS algorithm from using slice sampling
along a Hamiltonian trajectory to a new algorithm that uses
multinomial sampling:
https://arxiv.org/abs/1601.00225
We are mortified that after all of our nagging to get people
to use samplers that worked and weren't biased, we released
a biased sampler. The 2.10 version had a major bug which was
easy to see and fix, but that apparently didn't solve the
bigger problem.
Michael and I are poring over the proofs and the code, but
it's unfortunate timing with the holidays here as everyone's
traveling. We'll announce a fix and make a new release as soon
as we can. Let's just say this is our only priority at the moment.
Until then, the only thing I can recommend is using straight
up static HMC (which is not broken in the Stan releases)
or using something other than Stan or rolling back to Stan 2.09.
I'm not even sure how to do the latter for versions other than CmdStan,
which is just a source download and doesn't require any
installation.
If all else fails, we'll roll back the sampler to the 2.09 version
in a couple days.
- Bob