pystan ADVI

[Sachinthaka Abeywardana]

Hi all,

I am hoping to implement a HMM model for which Variational inference would be useful considering the label switching problem. Just wondering if ADVI is already implemented in pystan and if so what would the call look like? (Apologies if it is mentioned in documentation but I couldn't find it).

Would the call look something like:

pystan.fit(model_code = "model.stan", method="ADVI")

Thanks in advance

[Dustin Tran]

Hi Sachinthaka, If I recall there was recent renewed interest in opening up ADVI to the PyStan interface. The major code has already been developed to do so and was waiting on a few other things at the time. I think it’s reasonable for ADVI to be available on PyStan as it’s already on RStan and CmdStan.

For now, I think you’ll have to resort to one of these other interfaces.

Dustin

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sachinthaka Abeywardana]

Just so that we are on the same page, you are referring to the vb(m) call in R right? or is there a seperate function for ADVI?

Cheers,

Sachin

[Dustin Tran]

Yup.

[Bob Carpenter]

How do you think variational inference will help with label
switching? If you have multiple modes and run it multiple times,
you'll wind up in multiple modes and with label switching.

But it doesn't look like we're going to get ADVI on PyStan any
time soon because there's nobody with the combined Python/C++ expertise
and time to do it. If somebody wants to volunteer, we're always
happy to accomodate new developers.

- Bob

[Sachinthaka Abeywardana]

Oh, that's a shame. But I understand. As for why it would, variational Bayes is mode seeking. It probably would end up in a different mode when I run it again but the permuted parameters might be similar.

The other thing is with VB I just have to look at the last converged value of a parameter, since that would describe the posterior whereas I would need to look at the entire chain in normal mcmc which would be plagued with multiple modes.

Thanks,
Sachin

You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/a7fQHHDV0VA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

[Bob Carpenter]

That doesn't solve your problem, it just masks it!

If you really have multiple modes, as in mixture models like K-means,
LDA, or unsupervised HMMs, nothing is going to solve your problem.
The technology doesn't exist to do full Bayesian inference in these
settings and the approximations (like taking a variational approximation
of a single mode) are a long way from the actual model people write
down in their papers.

The command-line version works and should be easy to call
from Python. The output is CSV so it should be easy to read
back in. The only hassle is the input data format, which is
an easy one to generate programatically. And given that you don't
want any of the posterior analysis tools, I'd recommend just going
the command-line route.

- Bob