advi dev thread

[Alp Kucukelbir]

hi folks,

i had a fantastic meeting with daniel today. here's a summary. this is mainly for the both of us, to keep track on short-term development goals. but it might also be useful for everyone else.

+ create new branch to develop stochastic advi (with data subsampling).
+ hopefully merge feature/bbvb into develop in near future.

+ move entropy calculation into vb_params classes (meanfield and fullrank)
+ template calc_ELBO functions accordingly

+ comment all writes to streams and declare in words where i want things to go. for now just use std::cout and files as necessary

+ get rid of base_vb.hpp

+ implement daniel's trick for calculating the ELBO with <double>log_prob calls.

+ bring the tests up to date.
+ write new tests.

+ figure out samples -> transformations -> quantiles reporting business

cheers
alp

[Bob Carpenter]

> On Mar 17, 2015, at 4:09 AM, Alp Kucukelbir <akucu...@gmail.com> wrote:
>
> hi folks,
>
> i had a fantastic meeting with daniel today. here's a summary. this is mainly for the both of us, to keep track on short-term development goals. but it might also be useful for everyone else.

Please keep this kind of thing in the issue tracker, not in
e-mail. Then you can have checkboxes to tick off when it's
done and it can be edited rather than just appended to.

Thanks.

- Bob

[Bob Carpenter]

> On Mar 17, 2015, at 4:09 AM, Alp Kucukelbir <akucu...@gmail.com> wrote:
>

> ...

> + figure out samples -> transformations -> quantiles reporting business

This is going to be more general than advi (is the "ad" for autodiff?), because
we'll also need it for (p)MLE, MAP (if Andrew gets to say "sampling through adaptive
neighborhoods", I get to say "MAP", and MML, and any other TLA beginning with "max"
we come up with :-)

So I think it'd be best if it went in its own branch. Efficient software
dvelopment and maintenance is largely about good modularity and separation
of concerns. This feature can be developed and tested independently of
variational inference and thus should be. Design top down, but develop bottom
up, building up from things that work.

Lecture mode off.

- Bob

[Alp Kucukelbir]

thanks bob.

luckily, i dealt with most of the items on the list already.

a tracker would be useful for the test functions i need
to write. i'll figure out how to do that on github soon.

re: samples -> trans -> quantiles. this is not a priority
for me. just reporting the mean is fine for now. advi output
looks like the output from "optimize", though it means
something quite different.

re: advi. yup! bbvb/bbvi is starting to mean something rather
specific (or more general, depending on your stance) than our
algorithm.

cheers
alp

[Bob Carpenter]

> On Mar 17, 2015, at 12:28 PM, Alp Kucukelbir <akucu...@gmail.com> wrote:
>
> thanks bob.
>
> luckily, i dealt with most of the items on the list already.

That's the best kind of list!

> a tracker would be useful for the test functions i need
> to write. i'll figure out how to do that on github soon.

Just click on "Issues" on the right nav bar, then on
the big green button that says "New Issue". Then give
it an informative name and write whatever you want.

> re: samples -> trans -> quantiles. this is not a priority
> for me. just reporting the mean is fine for now.

Rob's going to need that function for MML, so you can probably
just wait until he does it.

> advi output
> looks like the output from "optimize", though it means
> something quite different.

That's good --- it's best not to have too many formats for
dumping out parameter values.

>
> re: advi. yup! bbvb/bbvi is starting to mean something rather
> specific (or more general, depending on your stance) than our
> algorithm.
>
> cheers
> alp
>

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>