How does HMC work with dconstraint?

[dirkdouw...@gmail.com]

Hello,

I saw in the release list that dconstraint now works with HMC. I'm just wondering how this works in practice when the sampler steps outside the constrained region. I saw in the documentation that it says 

 "NaN values may be encountered in the course of the leapfrog procedure. In particular, when the stepsize (epsilon) is too large, the leapfrog procedure can step too far and arrive at an invalid region of parameter space, thus generating a NaN value in the likelihood evaluation or in the gradient calculation. These situation are handled by the sampler by rejecting the NaN value, and reducing the stepsize."   

Is this like MH, where the current sampled values are set to the previous ones and then the MCMC moves to the next iteration? Or does it keep going and try to find a valid parameter?

Thanks!

[Daniel Turek]

Dirk, this is a good question about how HMC handles stepping out-of-bounds in the parameter space.  No, it's not exactly like the MH accept/reject step, where if a MH sampler proposed value goes "out of bounds," then the proposal is automatically rejected, and the MH sampler proceeds to the next iteration (where it tries a different proposal value).

Rather, the HMC sampler creates trajectories (sequences of locations) through the (possibly multi-dimensional) parameter space.  These trajectories are generated by generating a path approximately following the gradient of the log-likelihood surface, which is done using finite-sized steps (repeatedly calculating the gradient at each location, then taking a small "step" in the direction of the gradient).  Therefore, a (self-tuning) parameter of the HMC sampler is the "stepsize" which is used when generating these trajectories.  In the absence of any out-of-bounds steps, at the conclusion of generating a trajectory (a sequence locations in the parameter space), the HMC sampler selects one of the locations making up the trajectory as the "next sample" - the "next sample" is not necessarily the final location at the terminal end of the trajectory, but rather a random point selected from all locations making up the trajectory, which is selected in a random manner so as to maintain the forward/backward reversible nature of the HMC sampling process, ensuring a reversible Markov chain (as is necessary for valid MCMC sampling).

However, sometimes if the "stepsize" is too large, these finite-sized steps used to generate the trajectory might go out-of-bounds, reaching an invalid location in parameter space.  In that event (in contrast to how the MH sampler operates), the HMC ceases extending the trajectory; some recent part of the trajectory (which itself might be in valid parameter space) may be immediately discarded, as is necessary to ensure reversibility; the "next sample" is selected (still in a random manner) from the remaining part of the trajectory.  And finally, at the next adaptation step (which occurs periodically during the warm-up phase of the HMC sampler), the HMC's internal "stepsize" parameter may be adjusted to be smaller, to help avoid future steps going out-of-bounds.

I hope I've explained this clearly, and it helps answer your question.  You can view the source code the nimble's "NUTS" HMC sampler here, although looking at the source code may or may not be elucidating.

Cheers,

Daniel

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/nimble-users/f0c1b92a-4e6a-4fb0-95a5-65001a51e39cn%40googlegroups.com.

[dirkdouw...@gmail.com]

Thanks, Daniel! I think I understand it.

My basic understanding is that a NUTs iteration is made up of a bunch of doublings. So you are saying that if it steps out of bounds, it discards the last doubling?