Using Rcpp function or C++ libraries in Nimble model

[Colton Padilla]

Hi everyone,

I am currently running a simple ODE as part of model by wrapping a deSolve function into a nimbleRcall(). However, even with a very simple version of the ODE and very few observations being used the model is taking an exorbitant amount of time to run (110 seconds for 100 iterations). The project I am currently working on is invested in using nimble and having an ODE as part of this model we are working on. However, the amount of time that a simple version of this model is taking is very daunting. We (my PI) has experience using Rcpp functions and C++ libraries and we are wondering if there is a way to leverage these within a nimble model. For example, could we use a function from the C++ "boost" library within the nimbleModel to lessen the overhead time from moving back and forth between C++ and R? 

Thanks for any info on this.

Colton

[Perry de Valpine]

Hi Colton,

For quick answers: 

1. Look at nimbleExternalCall. It is somewhat rudimentary but is designed to let you call C/C++ code directly (from compiled nimble code), without going through the R interpreter. But to use Rcpp, you may have to code the C++ and create the #includes etc yourself and rely less on Rcpp's slickness (e.g. sourceCpp) directly from R. You need to have access to both the .o and .h files.

2. I am not a deSolve expert but I think it wraps layers of R processing around a call to a C++ internal that does the actual integration. It may be the case that all the time in the R interpreter is what is so costly. If nimbleExternalCall is too clunky, you could do the following: Use Rcpp completely independently of nimble, so that you have an R function (created by Rcpp) that calls directly to compiled code to compute what you want. Then call that function via a nimbleRcall. The overhead for the nimbleRcall should be much less (but not nothing) if you're not spending time executing R other than calling back into compiled code.

I'm happy to help look at details or sketch ideas if you start exploring one of these.

HTH

Perry

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/e57aa846-5bb4-4e43-8d98-ae87ea382d5an%40googlegroups.com.

[Brian Hatfield]

Hi Colton, Perry,

A little late to the party, but I'm trying to do a similar thing (using an Rcpp fn in nimble). One note that might be helpful:  if you make a package out of your rcpp function this will generate the .o file for you (you will have to make the header file, but that is straight forward).  It might seem daunting to create a package from an rcpp file (or files), but it is actually pretty simple thanks to some rcpp functionality; there are built in functions to create the package file structure, and to build the package once the .cpp/.h files are added to the source folder.  A couple of links with info on building packages:

https://stackoverflow.com/questions/50354302/rcpp-sourcecpp-undefined-symbol

https://www.lancaster.ac.uk/stor-i-student-sites/ziyang-yang/2021/03/13/3-steps-to-build-own-r-package-rcpp/

[Perry de Valpine]

Thank you! This is very helpful to know.

To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/f8a3a502-f0f7-4700-b773-127b4cd9c45bn%40googlegroups.com.

[Colton Padilla]

Hey everyone,

I am sending this back up for discussion since I have ran into some issues. Perry and Brian, thank you for the prompt responses last year, I got caught up with some things and just forgot to respond. I was able to get some ODEs to run using Rcpp and the nimbleRcall function relatively easily. I am now using a much more complex ODE and am having trouble with the Rcpp function crashing during the MCMC process and stopping the MCMC from running. It will make it through some repetitions and move the progress bar but then it will stop somewhere in there. I am guessing this has to do with some errant values being thrown within the MCMC causing the ODE to move to 0 or even into the negatives for some classes. The issues is, it is not crashing the MCMC process. It does not error and stop the MCMC despite adding in stop messages within my Rcpp function, so I am unsure if my assessment of the issue is correct or not. I am trying to move the function to full C++, but I am very limited in my knowledge of C++ so it is a struggle. I found a tutorial (https://github.com/oliviergimenez/ODEnimble/tree/master) and attempted to use a similar pipeline, but when I try to compile the externalCall my computer tells me it can't find the header files for Rcpp. Has anyone else tried to get something similar to work? And if not, is there any way to get the Rcpp function to error and stop the MCMC fully rather than just leaving it stuck? 

[Colton Padilla]

Hi everyone, 

I was able to sort this out and it had to do with some errant values within my rates. Essentially at times the MCMC threw a value for the rate that caused the derivative of the ODE to be nonfinite and would then crash the ODE. This would then cause the function to error within the MCMC and never return the value. I am going to post a separate thread to see what may be happening with the Rcalls and why it appears the function is running multiple times during the MCMC.