get variational fullrank cov

[marmaduke woodman]

hi

How would I obtain the estimated full rank covariance from the
variational algorithm?

As I understand it, the first 'sample' is the mean field, and the rest
are samples from the variational distribution.

Should I just try to estimate a covariance from the samples from the
variational distribution? In which case I should need to adjust the
number of samples according to number of parameters.. This seems a
little roundabout, unless I've severely misunderstood something.

Thanks,
Marmaduke

[Alp Kucukelbir]

hi marmaduke,

thanks for your message. i think your understanding is spot on.

there is unfortunately no direct way of accessing Sigma = L^T L at the moment. the reason for this is that the normal approximation is in the unconstrained (transformed) space, and not in the original latent variable space. so Sigma is not of interest to the typical ADVI user.

you are correct: the first line are the mean estimates of the parameters. these are in the original latent variable space.

what we've done in the past is to compute empirical covariances based on samples. that's worked OK for us, but let me know if you require something more.

cheers

alp

[Bob Carpenter]

Although you can't access Sigma directly, you can
do inference wit the covariance you care about
directly using the draws from the posterior the same
way you would for any Monte Carlo algorithm. The
draws build in the covariance and transform back to
the constrained space (at least I'm pretty sure that's
what's going on --- haven't tried it myself yet).

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[marmaduke woodman]

On 11/03/2015 06:26 PM, Alp Kucukelbir wrote:
> there is unfortunately no direct way of accessing Sigma = L^T L at the
> moment. the reason for this is that the normal approximation is in the
> unconstrained (transformed) space, and not in the original latent variable
> space. so Sigma is not of interest to the typical ADVI user.

ah indeed I forgot this part, perfectly reasonable

a propos, I have seen that this transform can result in bimodal
posteriors in the constrained space, which seems possible if the
variational density in the unconstrained space places significant
density outside the near-linear part of the transform function (not sure
how to phrase this mathematically, density gets squeeze out instead of
inward). Is this correct? Is this a sign that the constraints are
perhaps too tight?


thanks for all the great answers,
marmaduke

[Alp Kucukelbir]

hi marmaduke,

bob is spot on.

re your question: could you elaborate a bit more? which transformation
are you speaking of exactly?

cheers
alp

> --
> You received this message because you are subscribed to a topic in the

> Google Groups "Stan users mailing list" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/stan-users/VP0W4twLKRA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Bob Carpenter]

There's a chapter near the end of the manual that describes
al the transforms. It's inv_logit() for upper/lower bounds
and exp() for one-sided constraints (plus translation and sign).
I don't think either of those can transform unimodal to bimodal.
I'm sure there's a simple convexity argument, but I'd need time
and some paper to work it out. I don't know offhand about the
other transforms.

- Bob

> --

> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.