Issues Resetting HMC Chains in NIMBLE Across Multiple Datasets

[Aaron Kauffeldt]

Hi everyone,

I'm currently working on a NIMBLE project where I need to run HMC for multiple datasets using the same model structure but maintaining control over chains, iterations, and resets.

To do this, I compile two instances of the same model (identical code, constants, and initial data) and then update them in a loop with new datasets from a list. Here's the key part of my code:

Rmodel1 <- nimbleModel(code = pumpCode, name = "model1", constants = pumpConsts,
                       data = pumpData, buildDerivs = TRUE, calculate = FALSE)
conf1 <- configureHMC(Rmodel1, monitors = c("beta1Clim", "beta1Hab", "beta1Weather", "beta1yearWeather", "beta1Sampling", "beta2Sampling", "y_pred", "y_sim"))
mcmc1 <- buildMCMC(conf1)
Cmodel1 <- compileNimble(Rmodel1)
Cmcmc1 <- compileNimble(mcmc1, project = Rmodel1)

Rmodel2 <- nimbleModel(code = pumpCode, name = "model2", constants = pumpConsts,
                       data = pumpData, buildDerivs = TRUE, calculate = FALSE)
conf2 <- configureHMC(Rmodel2, monitors = c("beta1Clim", "beta1Hab", "beta1Weather", "beta1yearWeather", "beta1Sampling" ,"beta2Sampling", "y_pred", "y_sim"))
mcmc2 <- buildMCMC(conf2)
Cmodel2 <- compileNimble(Rmodel2)
Cmcmc2 <- compileNimble(mcmc2, project = Rmodel2)

for (i in 1:10) {
  rawdata <- data_list[[i]]
  pumpData <- list("y_sampling" = as.numeric(rawdata$pres_abs_data))

  Cmodel1$resetData()
  Cmodel1$setData(pumpData)
  Cmodel1$setInits(inits(pumpConsts))
  Cmcmc1$run(nburnin = 500, niter = 500, reset = TRUE)
  Cmcmc1$run(niter = 100, thin = 4, reset = FALSE, resetMV = FALSE)

  Cmodel2$resetData()
  Cmodel2$setData(pumpData)
  Cmodel2$setInits(inits(pumpConsts))
  Cmcmc2$run(nburnin = 500, niter = 500, reset = TRUE)
  Cmcmc2$run(niter = 100, thin = 4, reset = FALSE, resetMV = FALSE)
}

The issue is: the first iteration works fine and samples correctly. But from the second iteration onward, the sampling appears to only return the initial values repeatedly. It runs very quickly and doesn’t seem to be drawing proper samples anymore.

When using runMCMC with multiple chains on a single model, I don’t face this problem—but I need more control than that function offers.

Has anyone else run into similar problems when trying to reset and rerun an HMC-compiled model with new data? Is there something else I need to do to fully reinitialize the model or sampler state between runs?

Any help or insights would be greatly appreciated!

[Perry de Valpine]

Hi Aaron,

Thanks for posting a question. This looks like it might be the same issue discussed on this thread recently. On that thread Daniel posted code for installing from a branch of nimble on GitHub that should fix the problem. Could you try installing from that branch and letting us know if it works? Thanks.

Perry

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/nimble-users/f7352939-04a7-4559-9f9d-b2414f153a1cn%40googlegroups.com.

[Aaron Kauffeldt]

Hi Perry,

Yes, it turned out to be the same issue. I installed the version from the GitHub branch, and that resolved the problem—everything is now running without any issues. Thanks for the quick help!

Aaron