Problems with tiny stepsizes and (lower precision) Differential Equation models?

[David Manheim]

Hi all,

I'm running a differential equation model, using the bdf solver, and want to set my precision fairly low (1.0e-4, instead of the default 1.0e-6). When I do so, STAN spends a very long time with tiny stepsizes (>1.0e-16) during warmup. This seems to be exploring above the precision of the diff_eq solver, unhelpfully...

1) Is that a possible problem, or am I misunderstanding something?

2) Is there a way around this, other than many fewer warmup steps?

Thanks,

David Manheim

[Bob Carpenter]

I take it you mean Stan's step sizes are small in the Hamiltonian
simulation, not that the step sizes in the BDF integrator are small (both
are doing stepwise solutions of a system of diff eqs).

Yes, it's a possible problem if you initialize somewhere that's
poorly conditioned. Tighter priors and more reasonable inits
will help. One thing to watch out for is interval-constrained
parameters. They initialize in the middle of the range, but people
like to do uniform(0, 1e4) for parameters and don't realize they'll
get 5e3 as the center of initialization.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Michael Betancourt]

You’re decreasing the precision to really low levels (the default
in the original implementation was 1e-8) which causes really
inaccurate solves that immediately obstruct the validity of the
Metropolis correction. In other words, the transitions are
corrupted by noise and you end up getting stuck in the tails
instead of moving towards the typical set. The adaptation
tries to compensate the only way it knows how, by decreasing
the step size, but because the inaccuracies are not due to
HMC integrator error this doesn’t do anything and you get stuck
in a loop where the step size goes to really small values.

Lesson: nothing good happens when you try to make things
faster by sacrificing accuracy. Unless you have extreme
simplifications (i.e. everything is Gaussian), high-dimensional
problems are too fragile and the only hope you have is to do
things correctly.

If that means that your model runs too slowly then you have
to limit the scope of your model to something that will run in
the required time.

[David Manheim]

Hi Michael, Bob,

Thanks for the excellent feedback. Based on this feedback, I changed 2 things to test if it works better;

  1) I increased precision of bdf_integrate, but allowed it to do fewer iterations (1,000 iterations, both precision are 1e-7.)

  2) I decreased the adapt_window (the slow adaptation interval), and left the initial and final adaptation intervals higher.

The result is that it allows the model to finish the 200 warmup iterations about a hundred times faster than the original (hours instead of days). The values it arrives look good, and the confidence intervals are (comparatively) small.

The next question I have is how to check if this is converging well, especially if there is anything you'd suggest in particular given my approach;

The Rhat values are all in (0.97,1.02), it's not exceeding the max treedepth anywhere, and the pairs plot looks great. I'm running multiple chains - I ran 2 in the run on my laptop for this data, and I ran 8 on the server that's ~halfway done.

What else should I be trying or checking? (Or is this a horrible approach, and why?)

Thanks so much,

David

[Bob Carpenter]

> On Sep 27, 2016, at 12:35 PM, David Manheim <davidm...@gmail.com> wrote:

...

> The Rhat values are all in (0.97,1.02), it's not exceeding the max treedepth anywhere, and the pairs plot looks great. I'm running multiple chains - I ran 2 in the run on my laptop for this data, and I ran 8 on the server that's ~halfway done.


> What else should I be trying or checking? (Or is this a horrible approach, and why?)

That's all we provide out of the box for convergence monitoring
other than making sure n_eff is not super low compared to the number
of iterations and that there aren't divergent transitions.

- Bob