How to save posterior iterations portion by portion?

78 views
Skip to first unread message

Fengqing Chao

unread,
Nov 22, 2016, 10:12:27 AM11/22/16
to nimble-users
Hi all,
May I know if there's an equivalent function in nimble to save the chain of iteration portion and keep on producing more iterations? Like the jags function "update()"? In jags, I have something like below:
## the jags model first
mod <- jags(data, parameters.to.save, model.file, n.chains, n.iter)
model.update <- mod
save(model.update)
  
  #--- update MCMC ----------
    for (i in 2:N.STEPS) {      
      
      model.update <- update(model.update,
                             parameters.to.save, n.iter)
      save(model.update)      
    }#end of i loop (steps)

The code above updates MCMC and save some iterations at multiple times.

Thanks.
Fengqing

Daniel Turek

unread,
Nov 22, 2016, 10:44:48 AM11/22/16
to Fengqing Chao, nimble-users
Yes, assuming you have a compiled MCMC algorithm called "Cmcmc", you can continue running it to produce more iterations using the argument "reset = FALSE".  For example:

> Cmcmc$run(10000)   ## run for initial 10,000 iterations
> samples <- as.matrix(Cmcmc$mvSamples)   ## extract matrix of first 10,000 samples
> 
> Cmcmc$run(50000, reset=FALSE)      ## continue running the MCMC for an additional 50,000 iterations
> samples60 <- as.matrix(Cmcmc$mvSamples)   ## extract matrix of all 60,000 samples
> 

Hope that helps!




--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users+unsubscribe@googlegroups.com.
To post to this group, send email to nimble...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/6039ef23-89af-4e1d-9602-edff07e2a725%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Message has been deleted
Message has been deleted

Fengqing Chao

unread,
Nov 23, 2016, 1:47:37 AM11/23/16
to nimble-users, pinwhee...@gmail.com
Hi Daniel,
Thanks for your help!

It seems to me that the Cmcmc will become bigger and bigger when I continue to run more iterations. Sometimes I need a very long burnin for my model to get converge. The machine will run out of space/memory to continue the run since Cmcmc will become too big. Is there a way to get by this?


I need at lease 1 million iterations to get convergent, with about 5k parameters. I'm not sure when my model will converge. So I need to let the mcmc run for 2 or 3 millions and monitor the iterations from time to time. I'm wondering if nimble can discard the past iterations (to free up the space) after I save each portion of iterations and keep on running the mcmc (like what jags did as shown in the code in my 1st post)?


Thanks!

Fengqing

To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.

Daniel Turek

unread,
Nov 23, 2016, 9:42:53 AM11/23/16
to Fengqing Chao, nimble-users
Fengquing, there's no immediate muilt-in mechanism to do what you're asking: to stop recording samples, then start recording samples at a later point.   The best built-in approach would be to use the option for thinning the samples, which would only record every 100th or 1000th sample (the frequency is up to you).  To specify the MCMC to only record every 1000th sample (of all the parameters), you would set the thinning interval using the MCMC configuration object as:

> conf$setThin(1000)

Then proceed to build the MCMC algorithm from the conf object, compile it, and run it:

> Rmcmc <- buildMCMC(conf)
> Cmcmc <- compileNimble(Rmcmc, ...)
> Cmcmc$run(...)

There are other options for what you're trying to do, which would require some custom NIMBLE code, but could certainly be done.  I'll followup with you about other possibilities, too.

Keep in touch,
Daniel


To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users+unsubscribe@googlegroups.com.

To post to this group, send email to nimble...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages