Issues with GOF testing for an MSOM implemented with occuMulti

[johann...@mac.com]

Hi everyone!

For my MSc research, I am using two multi-species occupancy models (summer + winter) implemented with the package unmarked in R (using the approach from Rota et al. 2016) to look at habitat associations for wolves and deer, but am running into a problem with my goodness-of-fit results. 

I'm using parboot() with 1000 simulations to assess goodness-of-fit for my global models, using three statistics (SSE, Pearson chi-square, and Freeman–Tukey) using code from Andrade-Ponce et al. 2022. I am doing so to ensure there are no serious issues with the models and to calculate ĉ (overdispersion parameter). My understanding is that for these models, a 1 < ĉ < 3 is common and indicates moderate overdispersion which I planned to address by using QAICc for model selection and inflating my SEs for my top model predictions by √ĉ. 

The issue is with the winter global model (attached the plot below; Figure 1d, 1e, and 1f). The result for the simulated chi-sq (Fig. 1e) is being skewed by a few extreme values making the mean an enormous value and giving a ĉ value (overdispersion parameter) a value of 0 which I know is incorrect (indicating no overdispersion). My understanding is that this could happen because the Pearson statistic divides by the expected value, (observed - expected)^2 / expected, so cells with very small fitted expected values can dominate the statistic. I suspect the issue is the following: our detections are much more rare in winter compared to summer (fewer detections of our species) so when a detection occurs (observed value = 1) in a cell where the fitted expected value is very small, the chi-squared statistic is dividing by nearly 0 making the resulting value huge. I tested how the distribution looks if I drop the top 2.5-5% of simulated values and the distribution reflects closer to what I'd expect (attached plots below). I also tested how the calculation of the p-value and c-hat changes if these values are dropped and it is closer to what I'd expect as well (c-hat = 1.16 when top 5% of values are dropped). However, it doesn't seem defensible to actually drop these values to calculate the overdispersion parameter, but otherwise the chi-squared result is not informative. 

For my summer models (plots a, b, and c), I calculated ĉ from the chi-sq statistic, but given the result for my winter model, I’m considering using the SSE-based ĉ of approximately 1.11 for winter QAICc and SE adjustment, while reporting all three GOF results transparently. Does this seem appropriate? 

I would be grateful for any advice or thoughts. 

Thanks,

Johanna