ADVI

[Janne Sinkkonen]

I guess some are willing to try the ADVI algorithm, http://arxiv.org/pdf/1506.03431.pdf, currently available in Github in some developer branches for the cmdStan interface. Below is what I did to get it apparently working:

1. git clone https://github.com/stan-dev/cmdstan.git -b feature/advi-command

2. In the cmdstan directory, "make stan-revert". (With "make stan-update/develop" I ended up having no math library.)

3. "make manual", then read the short chapter on variational estimation.

4. "make examples/bernoulli/bernoulli"

5. Within the bernoulli directory, first " ./bernoulli sample data file=bernoulli.data.R" to make sure Stan is working in general, then "./bernoulli variational data file=bernoulli.data.R"

I understand ADVI is experimental and any kind of support is limited, etc. I'm just eager to try it with some models that are slow with NUTS.

Thanks for making it available.

[Andrew Gelman]

Janne:

I think ADVI will be part of Stan 2.7 which is coming out very soon!

A

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Maxwell Joseph]

Just wanted to say thanks to Dustin and Stan team for working on this.

[Dustin Tran]

Thanks! You should thank Alp Kucukelbir though. I was only around for the last few steps, while he took the mammoth task of launching it all and making it happen(!).

Dustin

[Maxwell Joseph]

Woops - big thanks to Alp as well!

[Janne Sinkkonen]

Intialization with the current ADVI (develop branch) seems to be difficult at times. I have a model with a Cholesky-factor correlation matrix, and while NUTS sometimes complains at the beginning, it samples eventually fine. The LBFGS optimiser similarly reports a few non-finite gradient at first, then starts to iterate.

With this model, the mean field ADVI seems to evaluate ELBO to nan quite often, then eventually crashes with 

"Exception: stan::variational::normal_meanfield::set_mu: Input vector[1] is nan, but must not be nan!"

This is with init=0, which works better than random initialisation. Playing with ADVI parameters (number of gradient samples, eta) does not help much. More diagonal LKJ prior maybe helps a bit. 

Gradient seems to be fine when I look at it with "diagnose init=0". At least there are no infinities or eye-popping differences between the gradient and the finite approximation. Some gradient components are zero, for missing value "parameters" of the model. (They are not zero with init=1.)

With init=2, ADVI constantly fails without reporting any iteration steps (even when eval_elbo=1). 

I don't know what happens at the initialisation phase in general, but ADVI code does not seem to have any retries or checks for failed gradients? (I only took a quick look at the advi.hpp file, so could be wrong.)

(This is not critical for me in any way, just wanted to share the observation in case it helps with development.)

[Bob Carpenter]

I hope our warning that

*** variational inference is experimental ***

was prominent enough!

That's because we haven't run it on very many models yet.
This would be a great project if someone wants to help out.
It should lead to at least some kind of paper and hopefully
more robust algorithms for Stan's variational inference.

Until then, we really appreciate feedback on what works and what
doesn't. So thanks very much. And thanks for running with diagnose,
HMC and L-BFGS, because that would've been my first comment!

- Bob

[Dustin Tran]

Yup, this is especially great info to know! There’s been recent developments on more robust algorithms for VI, e.g., Lucas Theis and Matt Hoffman’s paper on a trust region method. But it’s still quite in the early stages (and the current algorithm in Stan as well).

On calculating the gradient, we rely on the internals of stan::model::gradient(), which I’m not sure if it does the checks inside.

Dustin

[Bob Carpenter]

> On Jul 13, 2015, at 10:25 PM, Dustin Tran <dustinv...@gmail.com> wrote:
>
> Yup, this is especially great info to know! There’s been recent developments on more robust algorithms for VI, e.g., Lucas Theis and Matt Hoffman’s paper on a trust region method. But it’s still quite in the early stages (and the current algorithm in Stan as well).

At least that's our Matt Hoffman and not one of the
other ones:

http://arxiv.org/pdf/1505.07649

Poor Matt's having to use a middle initial now!

It also looks like Matt's been working on astrophysics (or
at least astronomy):

http://arxiv.org/pdf/1506.01351.pdf

> On calculating the gradient, we rely on the internals of stan::model::gradient(), which I’m not sure if it does the checks inside.

Calculating densities and/or gradients can throw exceptions.
What do you do with those exceptions?

In initialization for MCMC, we retry up to 100 random inits.
Absolutely not necessary if the user model is specified properly
for MCMC, but ADVI mileage may vary.

- Bob

[Dustin Tran]

> Calculating densities and/or gradients can throw exceptions.
> What do you do with those exceptions?

Nothing so far. The checks/error messages written in the ADVI code have only been on dimension-matching.

> In initialization for MCMC, we retry up to 100 random inits.

> Absolutely not necessary if the user model is specified properly
> for MCMC, but ADVI mileage may vary.

We can certainly do this too; the analog of this for variational inference is to redo the optimization up to 100 times if none of the previous initializations converged. How do you guys determine when to restart the MCMC again?

[Janne Sinkkonen]

To me, with the experience of a single model :), this seems similar kind of initialisation issue that sometimes happens with optimisation and NUTS, and is cured by restarts. It has two parts. 

One is failed elbo evaluation: it reports a NaN, maybe because one or more elbo samples return nan. The iteration still continues. Would it make sense to reject the nan samples, and maybe continue sampling until the configured number of (elbo) samples succeeds? This would introduce a bias, but that's ok if the problem disappears when the iteration continues. 

The other issue that crashes the iteration with the set_mu exception looks like a gradient failure. That would be a place for a restart.

Ordinary non-convergence is usually a reason to tune eta or other parameters. 

[Ben Goodrich]

On Wednesday, July 15, 2015 at 5:08:38 AM UTC-4, Dustin Tran wrote:

> Calculating densities and/or gradients can throw exceptions.
> What do you do with those exceptions?

Nothing so far. The checks/error messages written in the ADVI code have only been on dimension-matching.

I tried this with a student's project and we were running into exceptions where it looks like the gradient had NaNs or Infinities. Then we tried increasing grad_samples to something greater than 1 in hopes that would overcome the problem but it seems to just average the gradient samples, in which case if there is a NaN in any of them then it will throw. Is it possible to make a quick patch somewhere to exclude non-finite gradients from the average?

Ben

[Dustin Tran]

I submitted a pull request, which adds the checks for NaN/infinity immediately after calls to the model’s log probability or gradient. 

On throwing NaN/infinity-evaluated samples: This is certainly doable, although I’m not sure if we should be doing this. In principle, this shouldn’t happen for any samples; and if one occurred, then there’s something wrong with the underlying parameter space we’re searching over. (Or perhaps it’s a numerical issue?) We can throw a warning perhaps. Not sure what the convention is for the rest of Stan?

Dustin

[Ben Goodrich]

On Wednesday, July 15, 2015 at 4:19:22 PM UTC-4, Dustin Tran wrote:

On throwing NaN/infinity-evaluated samples: This is certainly doable, although I’m not sure if we should be doing this. In principle, this shouldn’t happen for any samples; and if one occurred, then there’s something wrong with the underlying parameter space we’re searching over. (Or perhaps it’s a numerical issue?) We can throw a warning perhaps. Not sure what the convention is for the rest of Stan?

Stan's MCMC stuff presumes that it is a numerical issue, assigns zero probability to those parameters, and continues on. In this case, I'm pretty sure it is an numerical issue with the gradient (albeit perhaps an intractable one) because it always occurred between 100 and 200 optimization steps and none of the unconstrained parameters were outside (-3,3) when it stopped.

Ben

[Dustin Tran]

Cool, I found the relevant code on the MCMC source. I'll add that to the pull request soon.

Dustin