VB issues

[Ben Goodrich]

I'm just about to flip the switch on VB in rstan, and I have some questions.

First, even for a simple model like

parameters {

  real theta;
}
model {
  theta ~ normal(0,1);
}

it hits its maximum number of iterations with the default arguments

  9900         0.0             4.825            2.701   MAY BE DIVERGING... INSPECT ELBO

  10000       -0.1             4.864            2.701   MAY BE DIVERGING... INSPECT ELBO
MAX ITERATIONS REACHED

or even with many more iterations. Is there some guidance as to what "INSPECT ELBO" entails? The resulting samples are decent:

Inference for Stan model: normal_model

1 chains: each with iter=(1001); warmup=(0); thin=(0); 1001 iterations saved.
 
Warmup took (0.00) seconds, 0.00 seconds total
Sampling took (0.00) seconds, 0.00 seconds total
 
            Mean   MCSE  StdDev    5%    50%    95%  N_Eff  N_Eff/s  R_hat
lp__    -1.3e+00  0.018    0.56  -2.4   -1.1  -0.92    999      inf   1.00
theta   -9.8e-02  0.029    0.87  -1.5  -0.12    1.4    886      inf    1.0
 
Samples were drawn using meanfield with .
For each parameter, N_Eff is a crude measure of effective sample size,
and R_hat is the potential scale reduction factor on split chains (at  
convergence, R_hat=1).

I realize bin/print does not customize by algorithm, but I'm trying to figure out the best way to show the output in rstan. In general, rstan is treating it like it would for HMC with 1 chain. But in my opinion MCSE, N_Eff, N_Eff/s, and R_hat should not appear in the VB output because the samples are iid.

Thoughts?

Ben

P.S. The "Samples were drawn using meanfield with ." thing is messed up in CmdStan.

[Andrew Gelman]

It would be good for vb to have some adaptation, so that, instead of recommending that people try different tuning parameters, it could auto-tune.  I think this could be hugely important in the success of this algorithm, just as it was important in the success of Nuts.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dustin Tran]

> Even for a simple model it hits its maximum number of iterations with the default arguments

That’s… rather unfortunate. I think for such simple models, this can easily be solved by doing the adaptation of the learning rate hyperparameter as Andrew mentioned. Alp and I are working on that now. The general idea is to write this as part of a new “warm up” stage for ADVI. Not sure when we’ll get it done.

>  Is there some guidance as to what "INSPECT ELBO" entails? 

This warning occurs whenever the change in the ELBO is too large—greater than a constant threshold. It might signify that the gradients are blowing up and thus the optimization routine is not converging. This almost always occurs at the beginning of ADVI, which is not surprising because changes in the ELBO are expected to be drastic in the beginning. Unfortunately, we haven’t found a good heuristic to make that threshold be large in the beginning so that such warnings aren’t outputted, and then have it decrease over time; the problem is equivalent to trying to predict the progress of the optimization within some “window". For now we leave the threshold as a constant.

> But in my opinion MCSE, N_Eff, N_Eff/s, and R_hat should not appear in the VB output because the samples are iid. 

Yeah, I don’t think these make much sense in the context of approximate inference yet. The only reason they exist at the moment is because they’re required for the output functions.

* MCSE is a bit pointless for variational inference because we may as well write down the analytic variance coming from the variational distribution instead.

* Getting estimates for N_Eff and N_eff/s for a posterior approximation sounds really really hard. It’s also not well-posed what this means in the context of analytic approximating distributions (VB) rather than empirical distributions (MCMC). I think this needs to be studied more closely first.

* Same as above for R_hat, which Andrew, Alp and I would like to work on soon.

> The "Samples were drawn using meanfield with ." thing is messed up in CmdStan.

Thanks for the heads-up. I’ll submit an issue for this in CmdStan.

Dustin

[Bob Carpenter]

So do you think we should not put it in RStan until there's
auto tuning?

- Bob

> On Aug 15, 2015, at 7:39 PM, Andrew Gelman <gel...@stat.columbia.edu> wrote:
>

[Dustin Tran]

I think we should be fine trying to put it in RStan now. The user interface will remain the same as the autotuning happens internally. The only difference at the interface level will be printed info about the warm-up stage.

Dustin

[Andrew Gelman]

Yes, my point was that some sort of auto-tuning would be a huge help.

[Ben Goodrich]

On Saturday, August 15, 2015 at 8:08:28 PM UTC-4, Dustin Tran wrote:

> Even for a simple model it hits its maximum number of iterations with the default arguments

That’s… rather unfortunate. I think for such simple models, this can easily be solved by doing the adaptation of the learning rate hyperparameter as Andrew mentioned. Alp and I are working on that now. The general idea is to write this as part of a new “warm up” stage for ADVI. Not sure when we’ll get it done.

In the case where it hits the maximum number of iterations, does it draw from a multivariate normal based on the state of the variational approximation at that time or from the state when the ELBO was at its highest?

Ben 

[Dustin Tran]

The state of the variational approximation at that time.

Dustin

[Ben Goodrich]

On Sunday, August 16, 2015 at 10:06:28 PM UTC-4, Dustin Tran wrote:

The state of the variational approximation at that time.

That seems suboptimal. On one hand, I could see how it might be safer to not even draw from the multivariate normal if it hits the iteration limit. On the other hand, we could cache the best ELBO evaluation. But either seems better than taking a chance that the last state is randomly awful and then letting the user make inferences from the result.

Also, even though exceptions are caught now, by default grad_samples is still 1 so a single exception causes it to halt. I think we should either make grad_samples default to 2 or to do one gradient evaluation more than whatever grad_samples is if there are exceptions.

Ben

[Dustin Tran]

> That seems suboptimal. On one hand, I could see how it might be safer to not even draw from the multivariate normal if it hits the iteration limit. On the other hand, we could cache the best ELBO evaluation. But either seems better than taking a chance that the last state is randomly awful and then letting the user make inferences from the result.

Generally I would agree although we have to be a bit careful about this. This is because that answer naturally extends to what we would do in the case when it does converge. And that latter question is foundational (and unaddressed) in optimization: should we output the last set of parameter estimates or the best set of parameter estimates according to the objective function?

In monotonic sort of optimization routines the two are equivalent. In stuff likes stochastic approximations, however, there are good reasons for either. For instance, in non convex optimization SGD might so happen as to land at a high point of one hill but later jump away and end up converging at the top of a lower hill. Do we output the parameters leading to the higher point—which is likely not even a local optima—or do we output the local optima that we converged too?

For now I think the best solution is to output a warning if (1) the maximum number of iterations is reached; and/or (2) the ELBO is higher at a set of parameters which isn’t the last set.

Thanks for the heads-up. Throwing this on the laundry list of things to do for ADVI.

> Also, even though exceptions are caught now, by default grad_samples is still 1 so a single exception causes it to halt. I think we should either make grad_samples default to 2 or to do one gradient evaluation more than whatever grad_samples is if there are exceptions.

Yeah I noticed that too. I didn’t want to throw down a constant threshold for the max number of dropped gradients at the time. But yeah this needs to be addressed. Thinking a bit more about it now, I’ll just submit a pull request which sets the max number to be 5*grad_samples—the intuition being that we allow 5 dropped gradients per grad_sample, and so that it still scales with the size of the mini batch.

Dustin

[Ben Goodrich]

On Monday, August 17, 2015 at 12:51:07 AM UTC-4, Dustin Tran wrote:

> That seems suboptimal. On one hand, I could see how it might be safer to not even draw from the multivariate normal if it hits the iteration limit. On the other hand, we could cache the best ELBO evaluation. But either seems better than taking a chance that the last state is randomly awful and then letting the user make inferences from the result.

Generally I would agree although we have to be a bit careful about this. This is because that answer naturally extends to what we would do in the case when it does converge. And that latter question is foundational (and unaddressed) in optimization: should we output the last set of parameter estimates or the best set of parameter estimates according to the objective function?

I think a lot of people would be perplexed if vb returned a point that was known to be worse than some other point with respect to the ELBO, although the best known point may itself be dubious if there is no mode there. That said, when its stopping point coincides with the smallest recorded ELBO, it seems to be really useful.

Ben

[Ben Goodrich]

meant largest recorded ELBO

 

[Alp Kucukelbir]

jumping in a bit late here.

i didn't quite understand the motivation behind ben's model.

we're drawing from the prior and the "likelihood" doesn't exist (it's a deterministic identity function). in addition, there are no observed variables (data).

as for the warning "INSPECT ELBO", that's a total heuristic. that being said, for this "model", the warning is warranted since the ELBO is not even defined (there is no evidence, i.e. p(data), to lower-bound, no?). i think i'm a bit confused.

i agree with the rest of the discussion -- dustin seems to have taken action on the important points.

cheers
alp

[Ben Goodrich]

On Monday, August 17, 2015 at 7:50:40 AM UTC-4, Alp Kucukelbir wrote:

i didn't quite understand the motivation behind ben's model.

It was an unmotivated model for the reasons you mentioned. I was just trying to get the R interface merged and was wondering what should be output, warnings to throw, etc. But I separately used it on a multivariate normal over 270 latent variables mapped onto binary / ordinal test questions with a lot of missingness and it converged fast.

Ben

[Alp Kucukelbir]

ok, got it. i just wanted to make sure i wasn't missing something here.

that's great. dustin and i are implementing the autotuning for eta_adagrad right now, so that should make advi even more robust. *crossing fingers*

--
You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/IB9BIGS1F_8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Bob Carpenter]

I agree it'd be good to output a warning if the final result isn't
the maximum value hit during optimization.

I think the whole notion of what to do with multi-modal posteriors
like LDA for either optimization or sampling is a wide-open problem.
It's all heuristic driven, and it's not even clear what the success
criteria should be. Given that you can't do full Bayes or anything
even remotely like it on these models means everything's going to
be a hack (aka "heuristic" if you want to be polite).

SGD's not even guaranteed to converge to a local optimum without
a proper Robbins-Monro schedule of learning rates.

- Bob

[Alp Kucukelbir]

with https://github.com/stan-dev/stan/pull/1590 ADVI should converge to a local optimum

You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/IB9BIGS1F_8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Andrew Gelman]

Let me emphasize that, in my experience, multimodality typically isn’t the problem, but slow convergence is. So I think we have to be careful not to consider multimodality as the paradigm for slow convergence.