A few incorrect results using the defaults for occu

138 views
Skip to first unread message

Jim Baldwin, USDA Forest Service

unread,
Aug 18, 2014, 4:39:38 PM8/18/14
to unma...@googlegroups.com
I have found a few instances where using the defaults for occu (and no covariates) where there is convergence and no indication there is a problem but the results for the maximum likelihood estimates for psi and p (probability of presence and detection probability, respectively) are not close enough to the true maximum likelihood estimates.  I can get occu to obtain close enough estimates by either making the relative tolerance parameter (reltol) very small or putting in starting values other than the defaults (which appear to me to be zero).

For the example attached I have 30 sites with 3 visits each (no missing values).  occu with the defaults gives estimates of psi-hat = 0.9996363 and p-hat 0.3668276.  By changing reltol = 1E-15 I get psi-hat = 0.8660276 and p-hat 0.4233891.  By added in "starts=c(0.1,0.1)" I get psi-hat = 0.8660271 and p-hat 0.4233893.  (The latter two sets are essentially the correct values.) 

My question is what in the output should I have noticed that would have told me I had the wrong result?  I've looked at "str" of the output object but I don't what to look at to know I should have changed the defaults.

I appreciate any guidance.

Thanks,

Jim




occu eample.r

Jeffrey Royle

unread,
Aug 18, 2014, 4:50:50 PM8/18/14
to unma...@googlegroups.com
hi Jim,
 This is an interesting as it seems to be an outright failure of optim() for a very simple data set.  Worse, the convergence code (if you look at the summary(output) ) says "0" which means convergence was achieved so maybe it looks good at first glance.
 The only thing I see that is suspect is the NaN for the standard errors resulting from the boundary estimate which might prompt one to dig a little deeper.

 I'm not sure what the best approach would be to dealing with this. Maybe we need some specific warning in the summary output but its not clear to me that catching NaN standard errors would solve the problem. I'm curious what other thoughts are on this and whether other people have observed these kinds of failures.

regards
andy



--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Richard Chandler

unread,
Aug 18, 2014, 9:34:48 PM8/18/14
to Unmarked package
Hi Jim,

I have noticed this problem too, and it seems as though the "Nelder-Mead" option can be more reliable when you are working with small datasets:

> d4 <- occu(~1 ~1,umf,method="Nelder-Mead",se=FALSE)
> plogis(coef(d4))
 psi(Int)    p(Int) 
0.8661981 0.4233863 

--
Richard Chandler
Assistant Professor
Warnell School of Forestry and Natural Resources
University of Georgia

Jim Baldwin

unread,
Aug 18, 2014, 11:12:55 PM8/18/14
to unma...@googlegroups.com
Thank you!  That works so much better.  That takes care of almost all of the differences from the exact values that I noticed and those differences that remain are much, much smaller.

I'm assuming that if I use both methods (or more than two methods) and choose the estimates from the method with the largest likelihood, then those estimates must be closer to the true maximum likelihood estimate.  Might that be more successful than trying different starting values?

Jim




Jim

Jim Baldwin
Station Statistician
Pacific Southwest Research Station
USDA Forest Service
 

Richard Chandler

unread,
Aug 19, 2014, 2:27:39 PM8/19/14
to unma...@googlegroups.com, jbal...@fs.fed.us
Hi Jim,

What you suggest makes sense to me. And in general, I think it is always good to double check things when working with small datasets. The recent paper by Gurutzeta et al. (2014) discusses some of these issues:


Richard
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--
Richard Chandler
Assistant Professor
Warnell School of Forestry and Natural Resources
University of Georgia

--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Kery Marc

unread,
Aug 19, 2014, 3:02:51 PM8/19/14
to unma...@googlegroups.com, jbal...@fs.fed.us
Guillera-Arroita et al. :)



From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Richard Chandler [richard....@gmail.com]
Sent: 19 August 2014 20:27
To: unma...@googlegroups.com
Cc: jbal...@fs.fed.us
Subject: Re: [unmarked] A few incorrect results using the defaults for occu

To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.

Richard Chandler

unread,
Aug 19, 2014, 3:05:37 PM8/19/14
to Unmarked package
Doh! Thanks, Marc. I started to write Gurutzeta's paper... 

Jim Baldwin

unread,
Aug 19, 2014, 5:20:49 PM8/19/14
to unma...@googlegroups.com
Thanks Richard and Marc for the link to that paper.  Gurutzeta is presenting that very paper at the Graybill meetings in Ft. Collins which I'll be at and I assume several folks in the unmarked group will also be there.

Jim

Brent P

unread,
Feb 26, 2016, 7:20:03 PM2/26/16
to unmarked
Any suggestion if the listed corrective methods do not fix the issue? I have a similarly sized dataset and am having issues with one species, despite that species being detected at 40% of the sites...only trying to fit the null model.

Thanks for thoughts and help,
Brent

Brent P

unread,
Feb 26, 2016, 10:14:50 PM2/26/16
to unmarked
All,

The issue appears to be resolved...seems like a matter of low detections across all sites.

Thanks for your time,
Brent

Reply all
Reply to author
Forward
0 new messages