Nimble post-processing capabilities

[Dave Ramsey]

Hi,   I have recently discovered Nimble and have been playing with it over the last week and thought I would share my experiences so far.

I installed Nimble version 0.4, using RStudio with R ver 3.2.0 x64 on a WIN 10 machine with 64gb of ram.  The package installed out of the box and I ran the examples with no issues.    I am a long-time BUGS/JAGS user so thought I would throw one of my recent projects at it.  Basically it consists of two (interacting) state space models.   Anyone who has worked with SSMs in BUGS will probably know that mixing is often slow and these models can take a  _long_ time to converge.   JAGS was struggling with this model so I didn't hold out much hope for nimble

Well, my nimble model appeared to work pretty well.  It ran as least as fast, if not faster than JAGS.  I did not have to do any fiddling with initial values, like you often have to do with JAGS to get the thing to run.  The most pleasing part was the model seemed to be converging quicker than in JAGS (less reliance on conjugate gibbs sampling?).   So I am pretty happy with my nimble experience so far.   I think there is heaps of upside for a platform like this which allows so much customisation of the sampling process. The fact I could apply block updates to certain nodes in my model so easily was fantastic.  Also the ability to calculate log posterior densities for particular nodes (calcNodes,getLogProbs) opens up a whole new world for customising BUGS type models and using them in a whole bunch of different ways.  In fact I reckon Nimble could be the future of Bayesian modelling and the logical successor to WinBUGS/JAGS.   So congrats to the development team!

  

Obviously, nimble is very much a work in progress but a couple of things that I would like to see in future versions would be:

- At the moment it seems you can only return your posterior samples in one big matrix.  This is fine for small models with few parameters, but when you have a more complex model with lots of multivariate nodes its a real pain.    It would be good  if nimble could return posterior samples as a list similar to functionality found in coda with one element for each monitored variable.

- The ability to run multiple chains in one call of the model  (i.e.    Model$run(n.iter= 1000, n.chains=3).  Maybe allow thinning at this point too rather than have to specify this in configureMCMC

- The ability to update the posteriors and discard the samples accumulated up to that point. I realise you can use Model$run(n.iter, reset =F) but this retains all the samples.   When you have a big model requiring a large burn-in, this can be problematic.

- A progress bar (like JAGS).

- Post processing functionality like in coda or tighter integration with coda

 

- I guess the library of samplers  will be added to over time.   It is great to see nimble has a basic particle filter and hope development of that (e.g. SMC, PMC) continues.

There were a few other minor issues.   Sometimes the build and/or compilation process would fall over crashing my session but overall things were fairly stable.

keep up the good work!

regards

Dave

[Perry de Valpine]

Hi Dave,

Many thanks for sharing your experience trying NIMBLE and all of these ideas.  These make a lot of sense.

As you can imagine we have a pretty big development agenda!  I’ll email you separately to see if you’d be game to try implementing some of these ideas :).  We could get to them, but we’d love to help others get familiar with the code base and able to contribute, and some of your ideas could be good entry points for that. 

A few comments below:  

On Sep 29, 2015, at 10:50 PM, Dave Ramsey <rams...@gmail.com> wrote:

Hi,   I have recently discovered Nimble and have been playing with it over the last week and thought I would share my experiences so far.

I installed Nimble version 0.4, using RStudio with R ver 3.2.0 x64 on a WIN 10 machine with 64gb of ram.  The package installed out of the box and I ran the examples with no issues.    I am a long-time BUGS/JAGS user so thought I would throw one of my recent projects at it.  Basically it consists of two (interacting) state space models.   Anyone who has worked with SSMs in BUGS will probably know that mixing is often slow and these models can take a  _long_ time to converge.   JAGS was struggling with this model so I didn't hold out much hope for nimble

Well, my nimble model appeared to work pretty well.  It ran as least as fast, if not faster than JAGS.  I did not have to do any fiddling with initial values, like you often have to do with JAGS to get the thing to run.  The most pleasing part was the model seemed to be converging quicker than in JAGS (less reliance on conjugate gibbs sampling?).   So I am pretty happy with my nimble experience so far.   I think there is heaps of upside for a platform like this which allows so much customisation of the sampling process. The fact I could apply block updates to certain nodes in my model so easily was fantastic.  Also the ability to calculate log posterior densities for particular nodes (calcNodes,getLogProbs) opens up a whole new world for customising BUGS type models and using them in a whole bunch of different ways.  In fact I reckon Nimble could be the future of Bayesian modelling and the logical successor to WinBUGS/JAGS.   So congrats to the development team!

  

Obviously, nimble is very much a work in progress but a couple of things that I would like to see in future versions would be:

- At the moment it seems you can only return your posterior samples in one big matrix.  This is fine for small models with few parameters, but when you have a more complex model with lots of multivariate nodes its a real pain.    It would be good  if nimble could return posterior samples as a list similar to functionality found in coda with one element for each monitored variable.

This makes sense.  Anyone interested in writing it?  I think if you look at the source code for the function as.matrix.CmodelValues in the package source (nimble/packages/nimble/R) it would be relatively easy to go from there and construct the list format you are suggesting.

- The ability to run multiple chains in one call of the model  (i.e.    Model$run(n.iter= 1000, n.chains=3).  Maybe allow thinning at this point too rather than have to specify this in configureMCMC

- The ability to update the posteriors and discard the samples accumulated up to that point. I realise you can use Model$run(n.iter, reset =F) but this retains all the samples.   When you have a big model requiring a large burn-in, this can be problematic.

Point of clarification: Is the situation that you want to run some samples, extract them, then run more samples, extract those, and then combine them in R?  Or is the main point that you’d like to clear the samples but skip the burnin on the second time?  

- A progress bar (like JAGS).

This would be relatively easy to do for the MCMC run part, less easy for the building and compilation parts.  Would anyone be interested in extending our MCMC nimbleFunction to be able to display a progress bar?

- Post processing functionality like in coda or tighter integration with coda

Sounds great, but do you have a more specific suggestion other than creating a coda mcmc object from a sample?

 

- I guess the library of samplers  will be added to over time.   It is great to see nimble has a basic particle filter and hope development of that (e.g. SMC, PMC) continues.

Yes we have more sequential Monte Carlo (particle filters) in development to be released soon.  If anyone is interested in contributing to this please let us know.  Also we hope that users will implement their own samplers and let us know.  We’ve added an adaptive random walk on a log scale (“RW_log”), which can really help for standard deviation parameters that can get stuck near zero and also an elliptical slice sampler, but we’d love to see more.

There were a few other minor issues.   Sometimes the build and/or compilation process would fall over crashing my session but overall things were fairly stable.

If you have a reproducible crash situation please let us know!

keep up the good work!

Thanks so much.  We’d be really keen to hear other user experiences and suggestions.

Perry

regards

Dave

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To post to this group, send email to nimble...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/8210b3a7-a69d-4d17-9bff-6698910313df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Ramsey]

Hi Perry,  

Point of clarification: Is the situation that you want to run some samples, extract them, then run more samples, extract those, and then combine them in R?  Or is the main point that you’d like to clear the samples but skip the burnin on the second time?  

To clarify, I guess I was asking if it could be made possible to clear the burnin samples when you run the model a second time 

Sounds great, but do you have a more specific suggestion other than creating a coda mcmc object from a sample?

This request was a bit vague but yes, I guess I was asking if it was possible to return an mcmc.list object compatible with coda.

If you have a reproducible crash situation please let us know!

I think my issues have mainly been with nimble not playing nicely with Rstudio (global environment issue notwithstanding).  I will to investigate this a bit more and will let you know,

cheers

Dave