AVDI - different estimates compared to "classical sampling"

[Stefano]

Hello,

just a curiosity. With the same model I get this estimates with NUTS (which are correct) taking 45 minutes

and this with ADVI (which are orders of magnitude incorrect) taking 2 minutes

Any way I can let the "magic" happen? (model attached)

[Andrew Gelman]

Hi, in our experience ADVI has worked best when parameters are on unit scale (so, estimates not too far from 0 with uncertainties not too far from 1).  And it should be checked using multiple chains and fake data simulation.

Andrew

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<ARMET_s_fixed_sd.stan>

[Bob Carpenter]

Not sure what magic you're expecting. Andrew and Alp and others are
working on figuring out why ADVI doesn't work, but I don't think they
have any concrete suggestions so far other than to keep things on a unit
scale and the more data you have so that the more multivariate normal
the unconstrained posterior is, the better.

The big problem is that we don't know how to diagnose ADVI failures
when it thinks it's converged without running MCMC, which has better
failure diagnostics.

We're always open to suggestions on how to make ADVI (or any other pieces
of Stan) better.

- Bob

> On Mar 16, 2017, at 9:21 PM, Stefano <mangiol...@gmail.com> wrote:
>

> Hello,
>
> just a curiosity. With the same model I get this estimates with NUTS (which are correct) taking 45 minutes
>
>
>
>
>

> and this with ADVI (which are orders of magnitude incorrect) taking 2 minutes
>
>
>
>
>

> Any way I can let the "magic" happen? (model attached)
>
>
>

[stefano]

Thanks,

I was just wondering if I missed something obvious. From online reading missed that some adjustment on the value ranges were needed.

I will try to test this aspect, and give feedback if helpful.

Bw.

2017-03-22 6:42 GMT+11:00 Bob Carpenter <ca...@alias-i.com>:

Not sure what magic you're expecting.  Andrew and Alp and others are
working on figuring out why ADVI doesn't work, but I don't think they
have any concrete suggestions so far other than to keep things on a unit
scale and the more data you have so that the more multivariate normal
the unconstrained posterior is, the better.

The big problem is that we don't know how to diagnose ADVI failures
when it thinks it's converged without running MCMC, which has better
failure diagnostics.

We're always open to suggestions on how to make ADVI (or any other pieces
of Stan) better.

- Bob

> On Mar 16, 2017, at 9:21 PM, Stefano <mangiol...@gmail.com> wrote:
>
> Hello,
>
> just a curiosity. With the same model I get this estimates with NUTS (which are correct) taking 45 minutes
>
>
>
>
>
> and this with ADVI (which are orders of magnitude incorrect) taking 2 minutes
>
>
>
>
>
> Any way I can let the "magic" happen? (model attached)
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <ARMET_s_fixed_sd.stan>

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/z3NjVUVaYGs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Bob Carpenter]

I don't think there's any advice in any doc anywhere. It's
still early days of the exploration of what's going on with
the algorithm.

- Bob

> On Mar 21, 2017, at 7:52 PM, stefano <mangiol...@gmail.com> wrote:
>
> Thanks,
>
> I was just wondering if I missed something obvious. From online reading missed that some adjustment on the value ranges were needed.
>
> I will try to test this aspect, and give feedback if helpful.
>
> Bw.
>
>
> 2017-03-22 6:42 GMT+11:00 Bob Carpenter <ca...@alias-i.com>:
> Not sure what magic you're expecting. Andrew and Alp and others are
> working on figuring out why ADVI doesn't work, but I don't think they
> have any concrete suggestions so far other than to keep things on a unit
> scale and the more data you have so that the more multivariate normal
> the unconstrained posterior is, the better.
>
> The big problem is that we don't know how to diagnose ADVI failures
> when it thinks it's converged without running MCMC, which has better
> failure diagnostics.
>
> We're always open to suggestions on how to make ADVI (or any other pieces
> of Stan) better.
>
> - Bob
>
> > On Mar 16, 2017, at 9:21 PM, Stefano <mangiol...@gmail.com> wrote:
> >
> > Hello,
> >
> > just a curiosity. With the same model I get this estimates with NUTS (which are correct) taking 45 minutes
> >
> >
> >
> >
> >
> > and this with ADVI (which are orders of magnitude incorrect) taking 2 minutes
> >
> >
> >
> >
> >
> > Any way I can let the "magic" happen? (model attached)
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> > <ARMET_s_fixed_sd.stan>
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/z3NjVUVaYGs/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.