advi multiple chains

[Andrew Gelman]

Hi, does ADVI use multiple chains? It should. This will help in assessing convergence.

[Dustin Tran]

Not yet but I agree. It should and will be necessary as we develop an R-hat for variational inference later in the future. Added as a Github issue to stan-dev/stan: https://github.com/stan-dev/stan/issues/1552

Dustin

> On Jul 17, 2015, at 10:05 PM, Andrew Gelman <gel...@stat.columbia.edu> wrote:
>
> Hi, does ADVI use multiple chains? It should. This will help in assessing convergence.
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Michael Betancourt]

Just to be the pedant — R-hat is a diagnostic of distributional behavior,
whereas the variational optimization of ADVI is a point behavior. An
R-hat-like diagnostic can diagnose the lack of convergence to a unique
optimum, but there is no theory arguing that there _should_ be a unique
optimum. We’re flying on heuristics here!

[Andrew Gelman]

Bob:

No, ADVI converges to a distribution, it’s just an approximate distribution that’s part of a parametric family. If different chains converge to different distributions, that doesn’t mean there’s anything wrong with the algorithm—but the user should still know it.

A

[Bob Carpenter]

That was Michael, not me, by the way --- though I do
realize most of the messages starting with "No, ..." are
aimed at me :-)

- Bob

[Andrew Gelman]

Ahhh, excellent. It’s so much more satisfying to disagree with Michael!

[Michael Betancourt]

There’s a big difference between converging “in distribution”
and converging “to a distribution” and my point stands. My
feeling is that most ADVI problems will be multimodal so we
need to figure out the difference between good multi modality
and bad multi modality before we assume that the users can.

[Andrew Gelman]

Yes, and I think R-hat can be a good tool in helping us figure out the difference between good multi modality
and bad multi modality .

[Sebastian Weber]

How come you think that ADVI problems are multi-modal? Also: Why not simply run an anova on the results from on the unconstrained parameter space; should this be sufficient. I don't see the point for running Rhat stuff which is geared towards chains, but ADVI is a parametric approximation, so what would an anova miss?

Sebastian

[Dustin Tran]

Problems in ADVI on the optimization level share all problems with nonconvex optimization, and a big one is the multi-modality of most interesting models, where one would like to use variational inference in the first place.

Another is that even many convex optimization problems are hard, and while the “chains” may be converging to the same solution eventually, we’d like to assess the quality of its convergence at finite runtime. This latter note is more analogous to the setting of R-hat for MCMC, where the chains are somewhere inbetween the starting distribution and the target distribution. If I recall correctly, the original paper also references that the problem applies more generally to “iterative maximization” and not just “iterative simulation”.

Dustin

[Sebastian Weber]

Ah, I see. And yes, now that you say it: I work with non-linear models and most of my time I spent with figuring out appropriate parametrizations coupled with clever priors to rule out the multi-modality. Does this approach make sense or should I live with the possibility that depending on my starting point I end up in a different local extremum? I never trust these multi-modal solutions or should I? To  my understanding algorithms are not built to deal with this which is why I prefer to rule them out.

Sebastian