Different AIC for null model between PRESENCE and unmarked

54 views
Skip to first unread message

Goul 56

unread,
Jun 1, 2022, 11:31:13 AM6/1/22
to unmarked
Hello, 

I need some help for analyses that i compared between PRESENCE and unmarked. 
I create sames models on PRESENCE and unmarked, i got sames AIC for all model except for the null model. The difference is quite important. Does anyone have any idea where the difference might come from? If you need an overview of a part of my script, my data or anythhing else, tell me. 


Capture.PNG


                                                      nPars      AIC        delta      AICwt    cumltvWt
psi(.)p(vent + Temp + Temp²)     5         376.14    0.00       0.6370     0.64
psi(.)p(vent)                                    3        377.82    1.69       0.2742     0.91
psi(.)p(Temp)                                 3        382.37     6.23       0.0283     0.94
psi(.)p(soleil)                                  3        382.49     6.35       0.0267     0.97
psi(.)p(nuage)                                 3        382.51    6.37       0.0264     0.99
psi(.)p(.)                                           2       385.03     8.90       0.0075     1.0

Thanks for your help
Goulvenn.

Ken Kellner

unread,
Jun 1, 2022, 11:50:40 AM6/1/22
to unmarked
Can you show the coefficient estimates for the null model from unmarked and from PRESENCE?

Do you have missing values in any of your covariates? If so unmarked might be dropping sites where detection covariates in a given model are missing, meaning the datasets for the null model and the rest of the model would not be the same. I'm not sure if/how PRESENCE handles that.

Ken

Goul 56

unread,
Jun 1, 2022, 12:16:26 PM6/1/22
to unmarked
I have exactly the same number of values for all covariates. 

Of course,
For unmarked : 

Call:
occu(formula = ~1 ~ 1, data = umf)

Occupancy:
 Estimate  SE   z P(>|z|)
     13.5 NaN NaN     NaN

Detection:
 Estimate    SE     z  P(>|z|)
    -1.06 0.125 -8.46 2.61e-17

AIC: 385.0337



For PRESENCE :
Untransformed Estimates of coefficients for covariates (Beta's)
======================================================================
                                          estimate    std.error
A1   psi.a1                           :   1.499966    0.552676
B1   P[83].b1                         :  -0.798877    0.166805

============================================================

   Individual Site estimates of <psi>
                Site               estimate  Std.err   95% conf. interval
psi             1 site1           :  0.8176   0.0824     0.6027 -  0.9298

============================================================

   Individual Site estimates of <P[1]>
                Site               estimate  Std.err   95% conf. interval
P[1]            1 site1           :  0.3103   0.0357     0.2449 -  0.3842
P[2]            1 site1           :  0.3103   0.0357     0.2449 -  0.3842
P[3]            1 site1           :  0.3103   0.0357     0.2449 -  0.3842
.............................


Thanks for your help,
Goulvenn

Richard Chandler

unread,
Jun 1, 2022, 12:32:52 PM6/1/22
to unmarked
The likelihood might have a ridge or be multimodal, and one of the two software programs might be getting stuck on a local solution. I would try several sets of starting values for both unmarked and PRESENCE. What happens if you do this:

occu(formula = ~1 ~ 1, data = umf, starts=c(1.5, -0.8))
 

--
Richard Chandler
Associate Professor
Wildlife Ecology and Conservation
Warnell School of Forestry and Natural Resources
University of Georgia
From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Goul 56 <goulv...@gmail.com>
Sent: Wednesday, June 1, 2022 12:16 PM
To: unmarked <unma...@googlegroups.com>
Subject: [unmarked] Re: Different AIC for null model between PRESENCE and unmarked
 
[EXTERNAL SENDER - PROCEED CAUTIOUSLY]

--
*** Three hierarchical modeling email lists ***
(1) unmarked (this list): for questions specific to the R package unmarked
(2) SCR: for design and Bayesian or non-bayesian analysis of spatial capture-recapture
(3) HMecology: for everything else, especially material covered in the books by Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2016, 2021) and Schaub & Kéry (2022)
---
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/f1bd1697-90a2-4a78-9241-07f5dca9b0f9n%40googlegroups.com.

Richard Chandler

unread,
Jun 1, 2022, 12:36:10 PM6/1/22
to unmarked
You could also use: engine='TMB', which uses exact gradients (rather than finite difference approximations) to find the MLEs. TMB should be the most stable option. Thanks to Ken for adding this new functionality!


From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Richard Chandler <rcha...@warnell.uga.edu>
Sent: Wednesday, June 1, 2022 12:32 PM
To: unmarked <unma...@googlegroups.com>
Subject: Re: [unmarked] Re: Different AIC for null model between PRESENCE and unmarked
 

Goul 56

unread,
Jun 1, 2022, 3:52:53 PM6/1/22
to unmarked
Hey, 

Thank you for your response. 

I tried the both solution. The first (start = c((1.5, -0.8)) give the same coefficient and AIC than PRESENCE but the second (engine = 'TMB) give different coefficient and AIC like at the beginning.

Goulvenn

Richard Chandler

unread,
Jun 2, 2022, 8:09:32 AM6/2/22
to unmarked
You should use the fit with the highest likelihood, which is going to be the one you got in PRESENCE and the one in unmarked when you used starting values. We should improve the default starting values in unmarked. They are usually just set to 0, which isn't the best option. I'll open an issue on github.


--
Richard Chandler
Associate Professor
Wildlife Ecology and Conservation
Warnell School of Forestry and Natural Resources
University of Georgia
Sent: Wednesday, June 1, 2022 3:52 PM

To: unmarked <unma...@googlegroups.com>
Subject: Re: [unmarked] Re: Different AIC for null model between PRESENCE and unmarked

Goul 56

unread,
Jun 2, 2022, 8:59:46 AM6/2/22
to unmarked
Okay, thank a lot for your help !!!

aw...@scenichudson.org

unread,
Jun 2, 2022, 9:34:16 AM6/2/22
to unmarked
Interesting and intimidating thread for those of us whose knowledge of MLE algorithms and the behind the scenes working of these programs isn't as strong.

If someone weren't comparing unmarked and PRESENCE, but just running models in unmarked, they would have missed this issue, right?  I feel like I would have just gone with the unmarked results using starting values of 0, assuming decent goodness of fit.  Until default start values are addressed, is there a best practice for unmarked to test for this?  E.g., should one always rerun models with informed starting values after running models with starting values of 0?  Do we know how common this issue is?

Richard Chandler

unread,
Jun 2, 2022, 9:51:32 AM6/2/22
to unmarked
Starting values are unlikely to be an issue in most cases, but it's always a good idea to try a few sets of starting values anytime you use maximum likelihood, regardless of the software program being used.

Sensitivity to starting values is most likely to occur when the model has lots of parameters, or when the data suggest that occupancy or detection probability is close to 0 or 1. These boundary cases aren't defined on the logit scale. You will usually notice a problem when this occurs. For example, in Goulvenn's case, the estimate of occupancy was essentially 1, and the SE was NaN. That should prompt you to try other starting values.


--
Richard Chandler
Associate Professor
Wildlife Ecology and Conservation
Warnell School of Forestry and Natural Resources
University of Georgia
From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of aw...@scenichudson.org <aw...@scenichudson.org>
Sent: Thursday, June 2, 2022 9:34 AM

Alex Wolf

unread,
Jun 2, 2022, 9:56:19 AM6/2/22
to unma...@googlegroups.com
Very helpful and practical advice - thank you, Dr. Chandler!

You received this message because you are subscribed to a topic in the Google Groups "unmarked" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unmarked/xqNhgSo1S_w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to unmarked+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/BL3PR02MB811530FFF2EA795B9646043DFBDE9%40BL3PR02MB8115.namprd02.prod.outlook.com.

Eduardo Silva

unread,
Jun 2, 2022, 11:05:16 AM6/2/22
to unma...@googlegroups.com
Hi all,

The same problem reported for occupancy models in this thread is also happening in N-mixture models. Although estimates for a given model are similar between these packages, AIC values are very different. We have seen the same problem with different datasets. Anyone experiencing the same issues?

Saludos!

Eduardo A. Silva-Rodriguez, Med.Vet., PhD
Facultad de Ciencias Forestales y Recursos Naturales
Universidad Austral de Chile


To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/CAJBv2bpne86yR8XGjOU2pZ8-XyogyAdup%3D5yPWdnGgxxy8a5nA%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages