balance between tight priors and posteriors - what is unbiased

[Linas Mockus]

Hi,

Can somebody explain the balance between tight priors and unbiased posteriors? From Gelman Bois paper on PK analysis I inferred that if posterior is within 3 sd of prior mean then estimate should be unbiased. What about if 95% CI of posterior is within 95% CI of prior? Attached are some posterior/prior density plots. Which estimates could be considered unbiased (if any). Posterior is solid while prior is dashed.
 

[Bob Carpenter]

Biased relative to what? If you have the right model,
Bayesian estimation using posterior means is unbiased
by definition. I should've added that note to the post about
calibration and overfitting:

http://andrewgelman.com/2017/04/12/bayesian-posteriors-calibrated/

- Bob

> On May 2, 2017, at 6:36 PM, Linas Mockus <linasm...@gmail.com> wrote:
>
> Hi,
>
> Can somebody explain the balance between tight priors and unbiased posteriors? From Gelman Bois paper on PK analysis I inferred that if posterior is within 3 sd of prior mean then estimate should be unbiased. What about if 95% CI of posterior is within 95% CI of prior? Attached are some posterior/prior density plots. Which estimates could be considered unbiased (if any). Posterior is solid while prior is dashed.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <density.pdf>

[Andrew Gelman]

Bayes posterior mean is _calibrated_: If we label thata_hat = E(theta|y), then E(theta | theta_hat) = theta_hat, where the expectation is taken over the joint distribution of y and theta.

But Bayes posterior mean is not _unbiased_. Unbiased is E(theta_hat | theta) = theta. Bayes posterior mean can never be unbiased under this definition.

Unbiasedness is an important principle in _measurement_, not so much in _estimation_.

[lmockus]

Thanks a lot for the explanation. I am still slightly confused. If we
have tight priors then it is difficult for the sampler to explore space
not defined by the prior. It means that posterior is sort of "biased" by
the prior. Perhaps this is an argument for an uninformative prior.

Thank you,
Linas

>--
>You received this message because you are subscribed to a topic in the

>Google Groups "Stan users mailing list" group.

>To unsubscribe from this topic, visit
>https://groups.google.com/d/topic/stan-users/sTGBHiggiBc/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to

[Bob Carpenter]

> On May 2, 2017, at 7:37 PM, Andrew Gelman <Gel...@stat.columbia.edu> wrote:
>
> Bayes posterior mean is _calibrated_: If we label thata_hat = E(theta|y), then E(theta | theta_hat) = theta_hat, where the expectation is taken over the joint distribution of y and theta.

You mean this:

E[theta | theta-hat] = INTEGRATE theta * p(y, theta | theta-hat) d.y d.theta

= theta-hat

I'm still trying to understand how to think about p(y, theta | theta-hat).

> But Bayes posterior mean is not _unbiased_. Unbiased is E(theta_hat | theta) = theta. Bayes posterior mean can never be unbiased under this definition.

I'm pretty confident this is right

E[theta-hat | theta] = INT theta-hat(y) p(y | theta) d.y

= INT (INT theta' p(theta' | y) d.theta') p(y | theta) d.y

!= theta

but then my calc isn't really up to going further. If everything distributed nicely
it'd be

INT theta' p(theta' | y) p(y | theta) d.theta' d.y

But then I'm stuck in understanding why this can't be theta.

Sort of the flip side of posterior predictive---it's the posterior estimative
or something like that.

- Bob

[Bob Carpenter]

But that biases in a different way. There's a vast literature on
this, some of the more relevant of which Andrew wrote and we reference
in the manual. He also wrote about it in Bayesian Data Analysis.
It's why we don't recommend improper or gamma(0.001, 0.001) priors.

- Bob

> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Linas Mockus]

Thanks, Bob. Somebody is telling that my priors are too tight and I have to rerun my 5 day PK estimation again. Their argument is that tails are not captured properly. Frankly I got those priors by running PK estimation with loose priors and then set up thight prior = posterior of previous estimation. Obviously posterior of previous estimation had to be approximated with a standard distribution but it seems lognormal worked well (at least visually) for PK parameters. Not a perfect way but... I think my friend is right that I may have cut tails but is there any way to estimate the error introduced by this shortcut?

Thank you,

Linas

[Bob Carpenter]

I don't know how to do that (or really anything!) other
than by using simulation.

- Bob