WAIC vs Rhat

[Sankar Ananthanarayanan]

Good day! I hope this finds you all well.

Apologies for a perhaps rather basic question - I'm just getting started in nimble. I'm doing a mark-recapture study of geckos with monthly capture data and SVL (snout-vent length) at first capture. I want to see if SVL with random effects has an effect on monthly (and yearly) survival probability.

> MCMCsummary(mcmc.phisvlrep, round = 2) mean sd 2.5% 50% 97.5% Rhat n.eff beta[1] -0.48 1.58 -3.59 -0.55 2.58 1.09 1146 beta[2] 0.66 0.88 0.03 0.12 3.10 2.28 767 p 0.23 0.04 0.17 0.22 0.31 3.46 1128 sdeps 4.77 3.79 0.17 5.03 9.92 7.00 47

Here's the output for the model, but the Rhat is extremely high, which I believe indicates problems with convergence (I get extremely different results when I set different seeds). While this would lead me to mistrust this model, when I run the WAIC = True code, here is the output.

> WAIC2 model WAIC 1 (phisvlre,p) 492.6639 2 (phi,pe) 498.2916 3 (phi,p) 505.8322 4 (phisvl,p) 543.7731

The Phi ~SVL + random effects model performs the best here. Why would that be? And what am I doing wrong?

Many many thanks for the help! My code is attached for scrutiny.

Best,
Sankar

[Dr. Heather Gaya]

Hi Sankar,

I don't know if I fully understand what you're asking, but I'll do my best. WAIC can be calculated (and produce a numeric answer) at any point during the model run. This means you can get a WAIC table and model rankings even if the model is not yet converged. However, the results are not valid until all the candidate models have converged. 

So, my take on this is that you need to run all your models longer until you see a suitable Rhat for all variables and also visually check the chains for decent convergence. Once all models have converged, you can trust their WAIC scores. Note that for 'sdeps' you have an effective sample size of 47. This suggests you'll need to run this model for much, much longer, likely 2 -3 X the number of iterations you have right now. 

As a side note, you don't have to first get the model to converge and then separately run WAIC as a secondary run. I would do this at the same time to save computation time. 

As a second side note, have you looked at the NimbleEcology package? You may find that the mixing is better and the convergence is faster using the CJS functions. 

-Heather 

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/nimble-users/5cf82699-4dc9-40a9-aa2e-63f2f25c9cd4n%40googlegroups.com.

[Bassim Rabea]

Dear member

I am looking forward to finding the geco.csv file related to this message, as I would like to see how the Excel sheet was constructed. I am currently working on a similar project involving a different species, so if anyone could kindly share this file, I would greatly appreciate it.

Kind Regards,

Basem Rabia, MSc
Ecologist, Nature Conservation Sector
Former Graduate Assistant, Biology Dept., Missouri State University
📞 +966573561836 | +20 1271135630
✉️ bassi...@yahoo.com
🔗 linkedin.com/in/BasemRabia

---

From: nimble...@googlegroups.com <nimble...@googlegroups.com> on behalf of Sankar Ananthanarayanan <a.san...@gmail.com>
Sent: Thursday, October 9, 2025 9:49 AM
To: nimble-users <nimble...@googlegroups.com>
Subject: WAIC vs Rhat

 

--

[Sankar Ananthanarayanan]

Thanks for your reply Heather - this was very helpful!

I tried increasing my iterations 5 fold and burn in and it greatly improved the Rhat and n.eff values but the trace plot does not show any convincing signs of convergence.

I'll take a look at nimbleEcology as well. Thank you for the suggestion! Unfortunately, there are not as many resources for how to use it (as compared to nimble).

Sankar

[Sankar Ananthanarayanan]

Hi Bassim,

I'm not able to share all my data at this stage but I can give an example with some dummy data of what my csv looks like. hope this helps!

Sankar

[Simon Bonner]

Hi Sankar,

If your sample data is representative, then it looks like you have high survival but low capture probability. If this is the case, then the model including the random effects may be close to unidentifiable.  It will be very difficult to estimate individual differences in the survival probability if the capture probability is small. In the output you showed, the mean of sdeps is 4.77 and the sd is 3.79. These are close to the theoretical values for U(0,10) (mean of .5 and sd of 2.89. This suggests that the MCMC sampler is essentially drawing from the prior and that there is little information about random effects in your data. This can lead to the type of behaviour you are seeing. Perhaps you can provide some summary statistics on your data. E.g. How many individuals? How often were the individuals captured? What is the average time between first and last capture?

One thing I did notice with your model is that the definition of z[i,first[i]] is not necessary: z[i,first[i]]=1 by definition because the individual must have been alive when it was first captured. You can include this as data. It shouldn't affect the performance of the model, but it makes things a little simpler.

I hope that helps.

Cheers,

Simon