AIC comparison before penalized likelihood, or vice versa?

[Amelia]

Hi there,

I have a bit of a conceptual question. I'm looking at occupancy of three species from camera trap data. I've just used a (a) null model, (b) model with 'distance to grazing land' as a covariate on the first-order parameter, with second-order set to 1, and third-order to 0; (c) model with 'distance to grazing land' as the covariate on the first- and second-order parameters, and third-order set to 0.

All models give the warning of high SE values. I have 20 sites across 90 days (with occasion length = 1), and two of the species are seen across all sites. As per the instructions in the occuMulti vignette, I can use penalized likelihood to address this.

I just don't know what the appropriate process is. Should I:

* Fit the models, compare AIC values and choose model with lowest AIC, and then fit this model with penalized likelihood

* Fit the models, also fit them with penalized likelihood, compare AIC values of both penalized and non-penalized models, and choose the lowest AIC model (even if this is a model with high SE values?)

* Only run AIC comparison on the penalized models, and choose the lowest AIC model.

Any help is much appreciated!

Kind regards,

Amelia

[Ken Kellner]

Hi Amelia,

You shouldn't use AIC in combination with penalized likelihood. One approach is to fit several penalties for each model and then rank the resulting models based on the cross validation metric. I've attached a brief example of this.

However, in your specific case, I would strongly advise you to not use the multispecies occupancy model. Penalized likelihood may "fix" the estimates, but I think the issues with your models come from two more fundamental limitations. First your sample size is unfortunately too small. In most cases you need 100 or more sites to have adequate power to detect any effects with this model, it's very data hungry. Second and more importantly, if 2 of 3 species are always detected, the model isn't going to be able to accurately estimate any interactions regardless of your sample size. For any given pair of species, at least one of the species will be at every site. The model needs information about where both species occur, where neither species occurs, and where each species occurs on its own to be able to estimate anything, and that's not possible in this case.

Ken

On Sun, Jun 15, 2025 at 11:18:00PM -0700, Amelia wrote:
> Hi there,
>
> I have a bit of a conceptual question. I'm looking at occupancy of three
> species from camera trap data. I've just used a (a) null model, (b) model
> with 'distance to grazing land' as a covariate on the first-order
> parameter, with second-order set to 1, and third-order to 0; (c) model with
> 'distance to grazing land' as the covariate on the first- and second-order
> parameters, and third-order set to 0.
>
> All models give the warning of high SE values. I have 20 sites across 90
> days (with occasion length = 1), and two of the species are seen across all
> sites. As per the instructions in the occuMulti

> <https://cran.r-project.org/web/packages/unmarked/vignettes/occuMulti.html#penalized-likelihood> vignette,

> I can use penalized likelihood to address this.
>
> I just don't know what the appropriate process is. Should I:
> * Fit the models, compare AIC values and choose model with lowest AIC, and
> then fit this model with penalized likelihood
> * Fit the models, also fit them with penalized likelihood, compare AIC
> values of both penalized and non-penalized models, and choose the lowest
> AIC model (even if this is a model with high SE values?)
> * Only run AIC comparison on the penalized models, and choose the lowest
> AIC model.
>
> Any help is much appreciated!
> Kind regards,
> Amelia
>

> --
> *** Three hierarchical modeling email lists ***
> (1) unmarked (this list): for questions specific to the R package unmarked
> (2) SCR: for design and Bayesian or non-bayesian analysis of spatial capture-recapture
> (3) HMecology: for everything else, especially material covered in the books by Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2016, 2021) and Schaub & Kéry (2022)
> ---
> You received this message because you are subscribed to the Google Groups "unmarked" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/unmarked/898bc7f0-4918-4832-bcb2-268ec053fa8bn%40googlegroups.com.

[Amelia J]

I really appreciate your response Ken, it’s very helpful! 

Does this mean that single- or multi-occupancy modelling is not particularly appropriate for a species detected at all sites (regardless of how many times per site)? I had high SE values for the single-species models too and have been trying to decipher the cause (hence trying the multispecies in case I was missing an important interaction effect).

Thank you again, these forums are really helpful for beginner modellers like myself :)

To view this discussion visit https://groups.google.com/d/msgid/unmarked/aFAVnasNTKtuD3Bt%40BADGER.

[Ken Kellner]

Hi Amelia,

In general, that's correct. If you detected a species at every site, you already know occupancy probability is 1, and it's not possible to identify covariate effects on occupancy. So occupancy modeling (either single or multispecies) would not give you any useful information in that case. If you already know the species occurs at every site, then it would make more sense to look other metrics such as abundance or how activity varies over time, if you have the required data.

The model will often still "run" but you will get huge estimates and SEs, as you saw.

Ken

> To view this discussion visit https://groups.google.com/d/msgid/unmarked/CAM8njjmHhQ%2Be3-kBg-C8irh-MDS11%3DYyLZv%2BnbRKshT8MOJxrA%40mail.gmail.com.

[Amelia J]

Great, thanks so much :)

To view this discussion visit https://groups.google.com/d/msgid/unmarked/aFAdYBHqJ5hhi32t%40BADGER.