how use integrate_ode_cvode in Cmdstan-develop

[Adam Darwich]

I am just starting out with Stan and am hoping to use the stiff ODE solver that I read had been implemented in Cmdstan-develop:
http://andrewgelman.com/2015/12/16/working-stiff/

I devised a simulated toy problem that works ok with integrate_ode in the Cmdstan-develop version but can however not get it to work with integrate_ode_cvode.
When running 'make model.exe' in cmd window the header file seems to compile okay, however when compiling the .exe file it throws several errors related to not finding the cvode functions (?),
such as: "undefined reference to: 'N_VMake_Serial', 'CVodeCreate', 'CVodeSetE...', 'CVodeInit...', CVodeSetU..." etc.
I think I have all necessary dependencies downloaded at: ...\cmdstan-develop\cmdstan-develop\stan\lib\stan_math\lib\cvode_2.8.2\

Am calling the function using: integrate_ode_cvode(PKPDmodel, y0, t0, ts, theta, x_r, x_i, rel_tol, abs_tol, max_num_steps);

Would be thankful for help on this, it is probably a trivial problem but am struggling to figure it out.

I've attached a printout of the error I'm getting in the cmd window, .stan and data-file.

[idontgetoutmuch]

If I have time I will try to reproduce your error but in the meantime this does not look stiff to me but maybe setting y2 = 0 causes it to be stiff? Did you try it with different initial conditions (non-stiff) with the existing solver? Ah I see you did. Perhaps you could post the code you used for this?

[Adam Darwich]

Thank for your reply. I'm planning on analysing a stiff problem but have started out simple to see if I could get cvode to work, I've attached the stan file using the integrate_ode that does compile and run in cmdstan-develop.

[Adam Darwich]

Ok, so I found the following post by Michael Betancourt giving a couple of good examples to try out with cvode:
https://groups.google.com/forum/#!topic/stan-dev/HPkzeTFKjBg

I tried these examples just replacing "integrate_ode(...)" with "integrate_ode_cvode(..., ,1e-10, 1e-10, 1e6)", again it ran fine with the default solver but gave the same errors as before with cvode.
Thinking that I must have missed out on something during the setup. Do I need to setup the cvode library in any particular way in order to use it or can it be used as is on the develop branch of github?

[idontgetoutmuch]

I didn't even get as far as you. I tried Michael Betancourt's example but

make examples/sir/sir
./examples/sir/sir sample data file=examples/sir/sir.data.R

gave lots of errors like

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

validate transformed params: y[k0__][k1__] is -1.20367e-09, but must be greater than or equal to 0

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Rejecting initial value:

  Error evaluating the log probability at the initial value.

then a message

Gradient evaluation took 0.002367 seconds

1000 transitions using 10 leapfrog steps per transition would take 23.67 seconds.

Adjust your expectations accordingly!

then more messages like

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Exception thrown at line 68: integrate_ode: parameter vector[1] is inf, but must be finite!

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

After 5 minutes I killed the process. I am guessing it was going to take something like 1000 x 25 seconds ~ 7 hours to finish but maybe I misunderstood what the messages were telling me.

I don't think this problem is intrinsically difficult. I just coded up a Sequential Monte Carlo implementation for an even simpler SIR model and got a reasonable result quite quickly. I could try your model using SMC but maybe we should wait until someone more knowledgeable can comment on what we are both seeing. In the meantime I will try `integrate_ode_cvode` and report back.

I am using `27e078f` on the `develop branch`. Let me know if you need more details.

[idontgetoutmuch]

lol - with `integrate_ode_cvode` I get

Gradient evaluation took 0.030436 seconds

1000 transitions using 10 leapfrog steps per transition would take 304.36 seconds.

Adjust your expectations accordingly!

I also get messages such as

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Exception thrown at line 67: CVode failed with error flag -1

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

On the other hand I am seeing

Iteration:  500 / 2000 [ 25%]  (Warmup)

as I write this response so maybe I will get an answer soon.

BTW I don't think you have to set up the CVODE library in any particular way. It is bundled in with the cmdstan repo so I presume there are no issues with someone using a different version to the one that the stan devs use.

[Adam Darwich]

Good to see that the CVODE example runs, I have tried downloading stan-dev from both GitHub and via the website and still get the same error message of undefined reference to: 'N_VMake_Serial', 'CVodeCreate', 'CVodeSetE...', 'CVodeInit...', 'CVodeSetU...' etc. Am clueless of what the problem can be.

[Michael Betancourt]

1. Go to a new working directory.
2. Checkout a fresh copy of cmdstan from github,

git clone g...@github.com:stan-dev/cmdstan.git

3. Update submodules

git submodule update —init —recursive

4. Make and run

make pkpdmodel
./pkpdmodel sample algorithm=hmc stepsize=0.001 data file=pkpdmodel.data.R init=0.25

If the compilation in (4) fails then there’s something weird going on with your compiler.
In that case report back with you compiler settings.

You’re missing positivity constraints on your parameters. If I update your model to read

parameters {
real<lower=0> theta[3];
real<lower=0> sigma;
}

then it runs in two minutes on my laptop.

On Apr 5, 2016, at 1:14 PM, Adam Darwich <adam.s....@gmail.com> wrote:

> Good to see that the CVODE example runs, I have tried downloading stan-dev from both GitHub and via the website and still get the same error message of undefined reference to: 'N_VMake_Serial', 'CVodeCreate', 'CVodeSetE...', 'CVodeInit...', 'CVodeSetU...' etc. Am clueless of what the problem can be.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Adam Darwich]

Michael, unfortunately I got the same error messages at (4) "make...". So then most likely a compiler issue?
Not quite sure, is that the g++ or stanc settings you are referring to? I've attached print out from g++ and stanc. Let me know if you meant something else.
Thanks.

[Daniel Lee]

Adam, 

You're on Windows. Hopefully your compiler is the RTools provided one. That's the only one we've tested. 

For (4), add .exe at the end. Something like:

    make foo.exe

I'd also suggest skipping the step of building bin/stanc.exe

If the differences in calls to make executables across platforms isn't in the CmdStan manual, please let us know. It should just be mentioned at the front once and not called out everywhere in the manual. 

Daniel

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

<Print_GppInfo.txt>

<Print_StancInfo.txt>

[Adam Darwich]

Daniel, yes am using Gcc, provided in RTools (GppInfo.txt file in previous post).
I can build ("make model.exe") and run the model with integrate_ode function without issues it is just when I switch to integrate_ode_cvode that the compilation issues occur.
Thank you.
Adam

[Daniel Lee]

The text files attached don't show the problem. Can you double-check that you attached the correct file?

[Adam Darwich]

Ok, I've attached the error messages from compiling the model file, is that what you need?
Thanks.

[Daniel Lee]

Yes, that helps, but what did you do before that step? Did you start with a fresh clone of CmdStan?

On Apr 5, 2016, at 11:46 AM, Adam Darwich <adam.s....@gmail.com> wrote:

Ok, I've attached the error messages from compiling the model file, is that what you need?
Thanks.

--

You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

<Print_ErrorMessages.txt>

[Adam Darwich]

Yes, following Michael's advice:

1. Go to a new working directory.

2. Checkout a fresh copy of cmdstan from github:

[Daniel Lee]

But that's not exactly what you would have done, right? This should fail since you're on Windows:

  make pkpdmodel

So... what did you do? We need to dig into the details since it's a matter of config. What version of RTools are you running?

Daniel

[Daniel Lee]

I'll try this out on a Windows machine this afternoon. It looks like it's not linking the library file, so either the build is broken in CmdStan or you're not up to date. I'll get back to you after verifying that it's ok on develop. 

Daniel

[Daniel Lee]

I've definitely reproduced it... it's a Windows only problem -- I'll report back with what the issue is.

Daniel

[Daniel Lee]

Somehow the current development version isn't working. Adam, can you go and check out the git commit hash right after the pull request was merged? (I don't know it off hand, but you should be able to spot it in the logs.)

I'll get it fixed on the develop branch, but will need a little more time to figure out what make is doing. 

Daniel

[idontgetoutmuch]

I have this working too. The accuracies of the results are

impressive. Good job Stan devs! I have some questions.

1. I generated some data with known parameters. What integrator is being

used and what error control? I'm assuming RK4 but neither was I able

to see anything in the documentation and nor did searching the repo

https://github.com/stan-dev/stan for RK4 yield any results. I tried

another solver (RKf45) in a stand-alone program but got sufficiently

different results to make me feel I didn't understand enough of what

Stan was doing under the covers. If I get time I will try a pure RK4

but I feel I'm shooting in the dark a bit until I know what Stan

*actually* uses.

2. It's imperative to use

./examples/pkpdmodel/pkpdmodel sample algorithm=hmc stepsize=0.001 data file=examples/pkpdmodel/pkpdmodel.data.R init=0.25

If you don't specify the stepsize and the init then the running the model produces

Iteration:    1 / 2000 [  0%]  (Warmup)

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Exception thrown at line 50: integrate_ode_cvode: parameter vector[2] is inf, but must be finite!

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,


but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

and nothing further (in a reasonable time).

I assume that the stepsize is specified as 0.001 as the parameters

being estimated are of that order? I was unable to find any clues as

to why init needs set to 0.25 in this case though.

3. The estimation took

Warmup took (469) seconds, 7.8 minutes total
Sampling took (717) seconds, 12 minutes total

Is this to be expected? It seems quite quite big to me given the state

update (solving the ODE) should really be quite cheap.

Thanks, Dominic.

[Bob Carpenter]

We're about to roll out CVODES in Stan 2.10, which should be
both more robust and faster. See below for some details.

> On Apr 9, 2016, at 12:59 PM, idontgetoutmuch <idontge...@gmail.com> wrote:
>
> I have this working too. The accuracy of the results are

> impressive. Good job Stan devs! I have some questions.
>
> 1. I generated some data with known parameters. What integrator is being
> used and what error control?

Adams Moulton. See

https://computation.llnl.gov/casc/sundials/description/description.html#descr_cvode

The original solver was an RK45 version from Boost.

> I'm assuming RK4 but neither was I able
> to see anything in the documentation and nor did searching the repo
> https://github.com/stan-dev/stan for RK4 yield any results. I tried
> another solver (RKf45) in a stand-alone program but got sufficiently
> different results to make me feel I didn't understand enough of what
> Stan was doing under the covers. If I get time I will try a pure RK4
> but I feel I'm shooting in the dark a bit until I know what Stan
> *actually* uses.
>
> 2. It's imperative to use
>
> ```
> ./examples/pkpdmodel/pkpdmodel sample algorithm=hmc stepsize=0.001 data file=examples/pkpdmodel/pkpdmodel.data.R init=0.25
> ```
>
> If you don't specify the stepsize and the init then the running the model produces
>
> ```
> Iteration: 1 / 2000 [ 0%] (Warmup)
>
> Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
> Exception thrown at line 50: integrate_ode_cvode: parameter vector[2] is inf, but must be finite!
> If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
>
> but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
> ```
>
> and nothing further (in a reasonable time).
>
> I assume that the stepsize is specified as 0.001 as the parameters
> being estimated are of that order? I was unable to find any clues as
> to why init needs set to 0.25 in this case though.

It takes steps in stepsize * gradient, and that needs to be small enough
to follow curvature in the posterior.

init=0.25 means that parameters are randomly initialized in (-0.25, 0.25)
on the unconstrained scale. All parameters are transformed to being
unconstrained (details in the manual).

>
> 3. The estimation took
>
> ```
> Warmup took (469) seconds, 7.8 minutes total
> Sampling took (717) seconds, 12 minutes total
> ```
>
> Is this to be expected? It seems quite quite big to me given the state
> update (solving the ODE) should really be quite cheap.

CVODES should be faster. These runs can be slow because either the
solver hits a stiff region it can't deal with or because the posterior
parameter density is complicated, and we need small step sizes in the HMC
algorithm. The algorithms and adaptations are also described in the manual.

- Bob

[Adam Darwich]

Daniel, thank you for looking into this.
Looking at Git shell, it is ge1fb49d. Is that what you need?

[Daniel Lee]

Hi Adam,

Sorry for the delay. That git hash didn't pull up anything, but I did track it down. It's a simple fix, but make is tough to manage cross-platform.

Anyway, I have an open issue now: https://github.com/stan-dev/cmdstan/issues/371

It should be fixed later today. (hopefully)

Daniel

On Mon, Apr 11, 2016 at 9:33 AM, Adam Darwich <adam.s....@gmail.com> wrote:

Daniel, thank you for looking into this.
Looking at Git shell, it is ge1fb49d. Is that what you need?

--

[Adam Darwich]

Hi Daniel,

I saw that the issue has been updated to closed on GitHub so re-downloaded cmdstan-dev.
The cvode model file now compiles and runs, so can confirm that it is now working.

Many thanks to you all for all the help.

Adam