diagnostic design

[Ilya Goldin]

I've been toying around with some design ideas for the diagnostic
plot. Here's a version that incorporates an Rhat display; it
essentially combines information from the left and right sides of the
R2WinBUGS figure that Andrew pointed out. Any comments?

This is a diagnostic plot, not a comprehensive display. For a vector
parameter like theta (say, theta[1]..theta[100]), we may not be able
to show estimates for each cell in the array. (Is vector the right
word?) My intuition is to always show caterpillars / Rhats for all
cells (up to the maximum that fits on the screen) that have Rhat
problems. If there's room, we would also randomly choose additional
cells.

Because it's an on-screen plot (you wouldn't include it in an academic
paper), I think use of color is fine.

One design choice here is that Rhats for all parameters would not be
grouped together in this kind of display (theta gets its own group of
Rhats, lambda underneath would get its own group, etc.)

Also, do we need to display the multivariate PSRF or no? Is that on
the same scale as univariate Rhat?

Best,
Ilya

-- Ilya Goldin <www.pitt.edu/~goldin>

[Bob Carpenter]

On 7/9/12 9:52 PM, Ilya Goldin wrote:
> I've been toying around with some design ideas for the diagnostic
> plot. Here's a version that incorporates an Rhat display; it
> essentially combines information from the left and right sides of the
> R2WinBUGS figure that Andrew pointed out. Any comments?
>
> This is a diagnostic plot,

The attachment may have been scrubbed. I didn't get anything.
Some comments below on what doesn't need graphics.

> not a comprehensive display. For a vector
> parameter like theta (say, theta[1]..theta[100]), we may not be able
> to show estimates for each cell in the array.

Right.

> (Is vector the right
> word?)

In Stan, there are vectors, row vectors, matrices, reals, ints,
and arrays of arbitrary dimensionality of any of these.

> My intuition is to always show caterpillars / Rhats for all
> cells (up to the maximum that fits on the screen) that have Rhat
> problems. If there's room, we would also randomly choose additional
> cells.

I don't have a preference here. Andrew will have an opinion.
R2WinBUGS just did the first 40 or so. The problem I see is in
displaying a variety of entries in a 10 x 10 x 10 matrix.

> Because it's an on-screen plot (you wouldn't include it in an academic
> paper), I think use of color is fine.

As long as there's no red/green and enough contrast to
see distinctions, I think it'll be OK.

> One design choice here is that Rhats for all parameters would not be
> grouped together in this kind of display (theta gets its own group of
> Rhats, lambda underneath would get its own group, etc.)
>
> Also, do we need to display the multivariate PSRF or no? Is that on
> the same scale as univariate Rhat?

I don't thin we need the multivariate version.

One issue is that it's problematic with large numbers of variables.
We're imagining regularly fitting models with hundreds or thousands
of parameters.

- Bob