Help profiling MCMC code

[Cole Monnahan]

Hi developers,

I need some help with my MCMC work in ADMB. I've got the no-U-turn sampler working in ADMB on this fork: 

https://github.com/colemonnahan/admb

I'm still smoothing out some small things but it is very close. However, It doesn't seem to be working that much better (ESS/time) than the current RWM option (default). It mixes really nicely, but is very slow. I did a couple speed tests today and found a big problem. If I run 10,000 iterations for a simple assessment model it takes:

RWM algorithm (-mcmc 10000)

1 min 21 sec

hybrid algorithm (-mcmc 10000 -hybrid -hynstep 1 -hyeps .001)

2 min 46 sec 

Static HMC method (-mcmc 10000 -hmc -hynstep 1 -hyeps .001)

5 min 34 sec for 

NUTS (-mcmc 10000 -nuts -max_treedepth 0 -hyeps .001)

22 min 21 sec for NUTS

The first 3 algorithms should all evaluate the user function 10000 times (ignoring optimization which is negligible here) to get the NLL and gradients. For technical reasons, NUTS has to calculate it twice for each iteration. So we'd expect NUTS to be about twice as slow, but it is like 10 times slower!

Bottom line, there must be some code inefficiency going on since the overhead for the algorithms should be negligible (e.g., writing output files or testing a U-turn which requires dot products of parameter vectors). For a stock synthesis model the user function should dominate run time.

Could someone help me profile this code to find what is going on? Or have a look through the core files (NUTS and helper functions particularly the build_tree recursive function) and see if I'm doing anything really stupid? I'm a beginner C++ programmer so rookie mistakes likely abound.

Thanks,

Cole