Using init values with ADVI (vb) or BFGS (optimizing) to compute a regularization path

[Dean Eckles]

Hi Stan experts,

I have a model that includes what might be normally described as a random effects component. That is, there is a batch of coefficients that are modeled (in this case, with t-distribution), which induces some shrinkage or regularization of those coefficients.

I'm interested in using the exact same model, but where I just set the scale parameters for that distribution, rather than learning them from the data. And, in fact, I want to fit variations on the model for a whole set of possible values of those scale parameters. You can think of this as wanting to fit an entire regularization path where we are treating those scale parameters as tuning parameters. These would then be tuned using a held out sample / cross-validation as part of a larger procedure. 

Is this something others have used Stan for? It seems like I could use the init parameter to vb/optimizing to use the results from some prior point in the path to initialize the next run.

However, looking at how the ELBO convergence diagnostic works, I wonder if this will end up resulting in much longer runs than needed in order for convergence to be detected, since tol_rel_obj is defined with respect to the change in ELBO from the start, correct? Maybe BFGS would work better.

Thanks for any pointers on this topic. I know what I'm doing isn't very Bayesian, but seemed like a nice way to do regularization with an arbitrary model.

Best,

Dean

Dean Eckles

Assistant Professor

MIT Sloan School of Management

[Daniel Lee]

On Fri, Sep 23, 2016 at 12:53 AM, Dean Eckles <eck...@mit.edu> wrote:

Is this something others have used Stan for? It seems like I could use the init parameter to vb/optimizing to use the results from some prior point in the path to initialize the next run.

Yes. For this type of model, I'd highly suggest thinking about the model and building it up very carefully. After that, run NUTS. I would not trust the vb/optimizing estimates because that's not what you're looking for.

 

However, looking at how the ELBO convergence diagnostic works, I wonder if this will end up resulting in much longer runs than needed in order for convergence to be detected, since tol_rel_obj is defined with respect to the change in ELBO from the start, correct? Maybe BFGS would work better.

Eh. The problem you described should be fit with full Bayes. If you specify it correctly, the full Bayes model in Stan will do a good enough job and quickly enough.

 

Thanks for any pointers on this topic. I know what I'm doing isn't very Bayesian, but seemed like a nice way to do regularization with an arbitrary model.

That's fine. If that's what you want to do, the language is flexible enough to specify that particular model. If you understand the properties of the model implemented and the inference algorithms, we encourage it.

Daniel

 

Best,

Dean

Dean Eckles

Assistant Professor

MIT Sloan School of Management

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Gelman]

Hi, Dean.  Believe it or not, your idea is very close to something I've been interested in for awhile, using Stan to fit models with preset values of key hyperparameters, then doing some sort of approximate integration and model averaging.   Most of the others in the Stan team hate this idea but I think it has a lot of promise.  So maybe we can talk about this one together and either prove 'em all wrong or, more likely, go down in flames!

Andrew

P.S.  There's also been some talk about instituting a conjugate gradient optimizer in Stan.

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Dustin Tran]

BFGS does sound like what you’re looking for at the moment. This would also be most closely related to how others calculate parameter estimates along a regularization path.

The convergence diagnostics for MCMC and VI will be more conservative than necessary for this type of calculation, so it would be relatively slow (if that’s a concern on your model+dataset). There’s also a practical issue, which is that with the current ADVI interface, you can only initialize the mean parameters of the normal approximation. The standard deviation parameters will always be learnt from scratch.

Dustin

[Dean Eckles]

Thanks for these replies, Daniel and Andrew. I appreciate the guidance about worrying about whether to trust the variational results. (Maybe I should see how well they agree with NUTS in my cases. I did that, but pretty informally.)

Two pieces of additional background that might clarify my choice not to use NUTS:

1. While fitting a single model doesn't take terribly long, I also want to use this as part of a simulation study, so I intend on running this (that is, the whole regularization path) many thousands of times. And NUTS is pretty slow by comparison (hey, I know it is actually fast by other comparisons!). And I really only need the point estimate right now.

2. The likelihood in this model doesn't correspond to my loss function. I'm doing causal inference and have another cross-validation procedure I'm using to try to make some bias–variance tradeoffs with respect to the relevant loss function.

Andrew, glad to hear you're interested. Using a Bayesian framework like Stan can make is easy to implement lots of arbitrary models, some parts of which I might want to pick via some non-Bayesian data-driven method. ML folks are always noting the nice Bayesian interpretations of their regularization paths, maybe we can note the nice regularization paths generated by tuning a hyperparameter in our otherwise fully Bayesian models. Happy to chat more when I'm next at Columbia if not before then.

Best,

Dean

[Dean Eckles]

Thanks, Dustin. I hadn't focused on the BFGS method as much as the ADVI one because the R wrappers seems to have lots of warnings. But maybe those warnings are for other use cases :)

Best,

Dean

[andre.p...@googlemail.com]

Hi Dean,

that reminds me of some two-stage modelling of diesel engines optimization in Matlab. 

I fancy in STAN I would not go the cross-validation route, but build a 

unified model with global hyper-priors and included local models referring to them.

 

Andre

[Avraham Adler]

On Thursday, September 22, 2016 at 11:51:13 PM UTC-4, Dean Eckles wrote:

Thanks, Dustin. I hadn't focused on the BFGS method as much as the ADVI one because the R wrappers seems to have lots of warnings. But maybe those warnings are for other use cases :)

Best,

Dean

Hello, Dean.

I'm curious, what do you mean by not using BFGS "because the R wrappers seems to have lots of warnings"? Stan itself is throwing a slew of errors? R is misbehaving?

Thanks,

Avi

[Bob Carpenter]

The procedure Dean is describing is standard operating procedure in machine
learning. For example, it's what you see in the Hastie,
Tibshirani and Friedman book and their papers on elastic net. It
avoids the "problem" of putting priors on parameters. I find the parameter
plots vs. regularization strength they produce very enlightening. It's usually
done with straight-up penalized maximum likelihood, which you'd get
with L-BFGS in Stan. With optimization, there's probably an advantage to starting
each optimization at the previous optimum for speed---at least
that's what Hastie et al. report. I'd always suspected that a lot of the motivation
for doing things this way was that the software in play wouldn't generalize
to hierarchical priors (no optimum, which is why you need max marginal
likelihood, aka "empirical Bayes"). For example, if you want to throw Vowpal
Wabbit at a ton of data or many of the logistic regression optimization packages
out there.

Like Avi, I'm not sure what Dean meant about warnings---the R wrappers
for L-BFGS have worked well for me when I've tried them.

I'll add a warning that this makes sense only when the coefficients for
which you are generating priors are on the same scale. If you have effects
on the order of 1e6 and others on the order of 1e-6, you can't motivate
a one-size-fits-all prior or regularization. So it might help to at least
standardize the predictors.

- Bob

> On Sep 22, 2016, at 9:16 PM, Andrew Gelman <gel...@stat.columbia.edu> wrote:
>

[Dean Eckles]

Bob, yes, that's exactly what I have in mind — as in penalized regression. And usually we do it from most penalized to least (i.e. relaxing the penalty).

But I wonder whether even if you could use a hierarchical prior, would it really be better? Maybe if the model is correctly specified and your "real" loss function is the likelihood. Maybe not otherwise.

On the warnings, I meant (what I remembered at least) as a warning in the documentation that this might not be fully baked. But now I'm not seeing that, so maybe it disappeared as Stan and rstan have been updated. However, I would say I haven't figured out how to get vb to be quiet ('refresh' isn't an argument to vb). I'm running 40 processes of this in parallel and it prints out a lot of junk.

Yes, I'm standardizing the priors according to some side information.

Best,

Dean

>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

>> To post to this group, send email to stan-...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/R0CC91MElFY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Dean Eckles]

A brief follow-up on this topic.

I now recall why I was using VB rather than BFGS: BFGS seemed to be quite variable from run-to-run in its results, and often inconsistent with the MCMC and VB results. So that's why I was using VB even when seemingly just using BFGS would make sense. My guess is that the penalized likelihood is just really flat and perhaps not convex.

One concrete remaining issue is how tolerance is specified for VB:

However, looking at how the ELBO convergence diagnostic works, I wonder if this will end up resulting in much longer runs than needed in order for convergence to be detected, since tol_rel_obj is defined with respect to the change in ELBO from the start, correct?

That is, it would be nice to be be able to specify other ways to detect convergence in VB. At least if I'm understanding this correctly that if I provide very helpful initialization, VB may run for much longer than really needed. I don't see any open issues on Github about this yet — maybe that is worth opening?

Best,

Dean

[Alp Kucukelbir]

hi dean,

if i recall correctly i wrote up a little heuristic procedure that determines an appropriate "running window" for tracking convergence. `tol_rel_obj` is then defined with respect to averaged ELBO values within that window.

my experience has been, at least for the default values of `tol_rel_obj`, that ADVI stops prematurely rather than running longer than necessary. but that might just be due to the types of models i've played around with.

all this said: there's definitely room for improvement in assessing convergence for VI; how to best do this is arguably an open research area.

cheers

alp

>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

>> To post to this group, send email to stan-...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/R0CC91MElFY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

[Bob Carpenter]

Was it BFGS or L-BFGS?

Was the problem the lack of modes in hierarchical models?

Also, they fit different things---optimization is finding
the mode of the penalized likelihood whereas ADVI is finding
the posterior mean of the Bayesian model. They only agree
when the posterior is symmetric.

- Bob

> >> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> >> To post to this group, send email to stan-...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/R0CC91MElFY/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Dean Eckles]

Bob, it was L-BFGS. Though I experimented with larger values of history_size with little effect. But perhaps I should try BFGS just to be sure.

This was a hierarchical model to start with (for fully Bayesian inference), but then I am fixing the scale at a series of values. That's how I am creating the regularization path.

But this is also a measurement error model. The model has the form:

Y ~ normal(alpha + X_true * beta, sigma);
for (i in 1:N) { 

    X_true[i] ~ student_t(nu_x, mu_x, sigma_x);
  X[i] ~ multi_normal(X_true[i], tau);
}
beta ~ normal(0, sigma_beta);

With nu_x, sigma_x, tau, and sigma_beta all fixed as "data" (in addition to Y and X). tau is really known and then some subset of the others can be used to form a regularization path / grid.

So maybe this ends up being mode-less some how, but that wasn't an obvious case of the hierarchical model mode-lessness to me.

Best,

Dean

> >> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> >> To post to this group, send email to stan-...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/R0CC91MElFY/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/R0CC91MElFY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Dean Eckles]

Thanks for this context, Alp. I tried poking around github to see how this works, but didn't quite manage to find where that might be implemented.

[I guess my concern was that if I provide quite good initialization (because it is from a prior run with a barely different penalty), my interpretation of this convergence detection was that it would still want a large relative improvement from that starting point, which might take a long time to find. But it sounds like that is a misunderstanding of how the convergence detection works.]

Best,

Dean

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Dustin Tran]

Source for the convergence diagnostic can be found here:

https://github.com/stan-dev/stan/blob/develop/src/stan/variational/advi.hpp#L401

> [I guess my concern was that if I provide quite good initialization (because it is from a prior run with a barely different penalty), my interpretation of this convergence detection was that it would still want a large relative improvement from that starting point, which might take a long time to find. But it sounds like that is a misunderstanding of how the convergence detection works.]

Your intuition is right. Even if you initialize at the global optima, ADVI will continue to wander around as the convergence diagnostic is not tripped yet. Moreover, while it’s wandering, ADVI can depart from the global optima due to the variance of its stochastic gradients. (This is also a problem that appears in a benchmark test that Bob mentioned here (https://groups.google.com/forum/#!category-topic/stan-users/general/ZNBMJgNAWmo).)

Dustin

[Andrew Gelman]

Hi, I think the key to the convergence diagnostic will be using multiple chains.

A

[Bob Carpenter]

There are multiple kinds of convergence here. The previous discussion
was about whether the optimizer is considered to have converged. Then
Andrew raised the question of whether when you run this all multiple times,
each run produces the same answer, which helps diagnose whether you've converged
to the wrong answer if the runs don't agree. I don't think there are any
Markov chains involved here. We can just inspect the posterior normal approximations
in multiple runs---we don't have to draw a sample from each normal approximation
and use R-hat.

- Bob

>>> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.