Nans in R-hat for parameters that don't move

[Andrew Gelman]

> On Apr 22, 2015, at 2:42 AM, Bob Carpenter <ca...@alias-i.com> wrote:
>
> If it's a correlation matrix, that's by design. The 1 values
> on the diagonals don't move, so you get a 0/0 in the formula.
> It's been confusing users, though --- what should we do here?
> The problem is that we don't know if it's because a parameter is
> stuck because it's not sampling or stuck by design, so I'd be
> reluctant to just put it to the max value by default.

Aahhh, interesting. I think if none of the chains move and they’re all equal to the same value, we should set n_eff to #samples and R_hat to 1, by definition. If none of the chains move but they are not all equal, we should set n_eff to 0 and R_hat to infinity. Does this make sense?

[Ben Goodrich]

On Wednesday, April 22, 2015 at 4:25:00 PM UTC-4, Andrew Gelman wrote:

Aahhh, interesting.  I think if none of the chains move and they’re all equal to the same value, we should set n_eff to #samples and R_hat to 1, by definition.  If none of the chains move but they are not all equal, we should set n_eff to 0 and R_hat to infinity.  Does this make sense?

I think RStan at least does all of those already, except for returning NaN for the R_hat when there is no within-chain or between-chain variation, which I like because it makes it easier to mentally filter out the constants. But I think it would be better to have the summary not report on constants in the first place.

Ben

[Bob Carpenter]

So not print the diagonals of correlation matrices? That'd require
knowing the constraints on all the variables as they're printed. That's
something we're hoping to expose in the refactoring.

- Bob

[Andrew Gelman]

I don’t think this is a good solution because we could also have generated quantities that happen to have zero variance. That’s why I suggested those definitional settings for n_eff and R_hat.
A

[Ben Goodrich]

I guess that could happen for generated quantities that are discrete. But then I am skeptical of reporting the minimal value of R_hat and the maximal value of n_eff. It depends on whether the continuous things that the generated quantity is a function of are mixing or not.

For parameters, transformed parameters and continuous generated quantities, the constant margins are just a waste of space in the summary.

Ben

[Andrew Gelman]

No, even for continuous generated quantities, for example we set z <- max(0,theta) and theta happens to be all-positive.

[Ben Goodrich]

Okay, piecewise functions too, but a perfect R_hat / n_eff for z is not plausible if theta is not mixing.

[Allen B. Riddell]

Another case where this arises is when you pin a parameter to 0 for
identifiability (e.g., multinomial logit). I agree that 1 seems a little
weird to me. 'nan' isn't much better though. Python currently reports
'nan' if var_within is near zero.

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.