Duration of search effort as observation covariate in pcount

447 views
Skip to first unread message

Shane Siers

unread,
Nov 8, 2013, 6:45:30 PM11/8/13
to unma...@googlegroups.com
Hi, all.  I'm looking for guidance on how to account for variation in search effort (duration of search) among sampling occasions in a single-season (closed) pcount survey.

On each search occasion, an observer searches for snakes for a variable amount of time (dictated by logistical constraints, including weather, NOT by capture rates), with most non-urban habitat searches ( forest & savanna, 15 sites) being 100 to 180 minutes long and most urban searches (3 sites) being 45 to 120 minutes in duration.

What is the proper method for accounting for this variation in observation duration?  Some candidates:


minutes  <-pcount(~min ~site, data=umf)           #Duration as obsCov

intercept<-pcount(~min+h3 ~site, data=umf)        #Different intercepts by habitat

interact <-pcount(~min*h3 ~site, data=umf)        #Diff. slope/int by habitat

offset   <-pcount(~1+offset(min) ~site, data=umf) #Duration as offset

 

                       nPars     AIC   delta    AICwt cumltvWt

interact (min*habitat)    24 3831.55    0.00  1.0e+00     1.00

intercept(min+habitat)    22 3854.97   23.42  8.2e-06     1.00

minutes  (min)            20 3857.27   25.72  2.6e-06     1.00

offset   (offset(min))    19 4885.42 1053.87 1.4e-229     1.00

 

Allowing a different slope and intercept for each habitat type “fits” better, by far…but is this appropriate?  Does this allow the 1180 observations to explain an inordinate amount of the variation in encounter rates as opposed to abundance covariates at just 18 sites, or are observation and abundance so decoupled that this should not matter?


I’ve seen discussion in previous unmarked posts of including an "offset" term in the GLM formula as a constraint on the relationship (more time = higher counts, e.g. “exposure time”, more area = higher counts), but I’ll confess to not fully understanding how this works, including whether it should be offset(min) or offset(log(min)).  Further, including duration as an offset is nowhere in the ballpark with respect to AIC values.  Are models with an offset and without comparable in an AIC framework?


Thanks in advance.



--
Shane Siers


 

Kery Marc

unread,
Nov 8, 2013, 10:08:52 PM11/8/13
to unma...@googlegroups.com
Hi Shane,

re. your questions in this mail, let me make a stab at a couple of answers. Most important perhaps, there is usually not a single proper way of doing things in modeling and modeling has elements of both a science and of an art (to paraphrase McCullagh & Nelder). So, in modeling as in real life, one should try not to get lost among a multitude of options.

In your case, all four candidates make sense to me (at least "looking from afar"), with the exception of the model with the offset: the offset argument should normally contain the natural log of the variable with which you want to "standardise" your counts (converting the counts to a "density" with respect to one unit of the offset). Then, if you let AIC play the role of a referee, the verdict is clearly in favour of one model. Now, if this model passes a goodness of fit test (e.g., using the parametric bootstrap) and yields estimates that make sense from what you know about the system, then I would go with this model. Except you or a knowledgable colleague comes up with a model that makes more sense based on a priori thinking about the biological system and the way in which your data were collected.

Kind regards  --  Marc



--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Shane Siers

unread,
Nov 10, 2013, 1:09:08 PM11/10/13
to unma...@googlegroups.com
Thanks for your thoughtful response, Marc, and for steering me clear of my misunderstanding of the offset.

Best,
Shane
--
Shane Siers

Please watch my underwater videos at www.macronesia.net
 

Dan Linden

unread,
Nov 10, 2013, 2:17:23 PM11/10/13
to unma...@googlegroups.com
The offset function in R simply sets a model coefficient to 1, preventing the model from spending a parameter to estimate a beta coefficient for the given offset covariate.  Like Marc said, typically an offset is used to account for differences in sampling area or time to convert a count into a density or a rate, this while still making use of the Poisson distribution for the observed counts.  Since a Poisson regression models the linear predictor of the expected count on the log scale (i.e., the log link), the offset also needs to be on the log scale: 

log(E[y/offset]) = lambda
log(E[y]) = log(offset) + lambda

So in your situation, by putting an offset in the observation model of pcount you are stating that capture probability scales linearly (on the logit scale) at a rate of 1.0 with the time spent searching.  Seems like a pretty restrictive assumption.  But otherwise yes, an offset term should be comparable in a model selection framework.

Shane Siers

unread,
Nov 10, 2013, 2:21:42 PM11/10/13
to unma...@googlegroups.com
Thanks, Dan.


--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Giancarlo Sadoti

unread,
Nov 10, 2013, 11:18:41 PM11/10/13
to unma...@googlegroups.com
I agree with Marc and Dan; an offset appears most appropriate here. In addition, I think it should be emphasized that an offset converts your response to individuals per unit time (or space if you were to use search area), so it is not appropriate to compare (e.g. via AIC) an offset model to a model without an offset.

Giancarlo

Benedikt

unread,
Nov 11, 2013, 2:15:35 AM11/11/13
to unma...@googlegroups.com
I disagree with Giancarlo. You can use AIC to compare models with and without offset. You can compare models with different structure (that is what AIC is for). However, you have to make sure that you always compare the same data set. For example, there can be problems if you have missing covariates. For example, assume you have 50 sites and no missing values for covariate A. You only have values for covariate B for 40 sites. Then you should not use AIC to compare models occu(~1 ~A) and occu(~1 ~B).

Best, Benedikt

Kery Marc

unread,
Nov 11, 2013, 9:13:31 AM11/11/13
to unma...@googlegroups.com
Dear Beni,

like you, I would have thought that you can compare GLMs that have an offset with GLMs that have not. As you say, one cannot use AIC to compare model sets of which some have, say, a log-transformed response and others have the response on the original scale. But in the GLM with an offset, the response remains the same: it's the counts, it's NOT the density. Only you add in one predictor variable with its coefficient fixed at 1. This then results in an equivalent model as if you were modeling density (but with Poisson errors), though you are not actually treating density as a response.

I have a couple more things to say perhaps about the offset issue. I think you should not put the offset on the detection model, but on the abundance model (if at all). I think a little more.

Regards  --  Marc



From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Benedikt [benz.s...@gmx.ch]
Sent: 11 November 2013 08:15
To: unma...@googlegroups.com
Subject: Re: [unmarked] Re: Duration of search effort as observation covariate in pcount

--

Kery Marc

unread,
Nov 11, 2013, 11:58:44 AM11/11/13
to unma...@googlegroups.com
Hi all,

I would *not* use an offset for the detection part of the model. An offset is used if the expected count has a known relationship with one of the covariates. This would for instance be the case if Shane's sites differ in their area (I forgot whether they do, but this is important). Then, the expected abundance N(i) at a site i would be A(i) * lambda(i), you would specify log(A(i)) as an offset in your Poisson GLM or, equivalently, in the abundance part of your mixture model. lambda(i) would then be interpreted as the density per unit of A.

However, the searching time spent does not seem to have such a relationship to the expected observed count. That is, it is not reasonable to assume that 10 mins search time will produce twice as many snakes as 5 mins. Rather, I would estimate the relationship between the search time and the numbers of snakes caught and this is what you do if you treat "minutes" as a simple covariate (not as an offset). And this is the model that AIC says is the best.

Alternatively, perhaps your "sites" are not well-defined (say, they aren't all 1ha) and search time is a proxy for the area searched. Then, you could use search time in the model for abundance. The expected abundance would then scale to search time because a longer search would mean a larger area searched, but probably not linearly. One way how search time (or similar, route length) could be incorporated in a hierarchical model as a proxy for the area searched is sketched out in the paper by Royle and Dorazio in JABES (2006). They describe a three-level HM for M, N and the observed counts. M(i) is the population size at a site that is incompletely covered by sampling, so that the "exposed population size" is N(i), with N(i) <= M(i). You can treat N(i) ~ Binomial(M(i), theta), with theta being a function of route length. This model can be collapsed to a two-level model that can be fitted in unmarked by using 1/route length or 1/sampling effort as a covariate on abundance. See the attachment for a description by written by Andy Royle and Richard Chandler in their webinar on HMs.

So if people have searched increasingly larger areas at what is considered a "site" in Shane's study, then perhaps the reciprocal of search time could be incorporated as a covariate in the model for abundance.

Kind regards  --  Marc





From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Benedikt [benz.s...@gmx.ch]
Sent: 11 November 2013 08:15:
To: unma...@googlegroups.com
Subject: Re: [unmarked] Re: Duration of search effort as observation covariate in pcount

--
3-level HM.pdf

Murray Efford

unread,
Nov 11, 2013, 2:27:31 PM11/11/13
to unma...@googlegroups.com
I do not disagree with Marc's specific advice, but we argued earlier this year that it *is* appropriate to include an offset-like adjustment in detection models (Efford, Borchers & Mowat 2013 MEE 4:629; see 'General comments' in Discussion), in preference to a covariate. The appropriate adjustment is 'offset-like' because it is linear on a cumulative hazard scale, not the arbitrary link scale (one reason a simple covariate is *less* appropriate). This is a powerful and general tool, but as far as I know there is no implementation in non-spatial software. The user must take responsibility for assuming a constant detection rate per unit time is a reasonable model, but the cumulative hazard scale takes care of Marc's worry that 10 min != 2 x 5 min.
Murray


From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Kery Marc [marc...@vogelwarte.ch]
Sent: Tuesday, 12 November 2013 5:58 a.m.
To: unma...@googlegroups.com
Subject: RE: [unmarked] Re: Duration of search effort as observation covariate in pcount

Shane Siers

unread,
Nov 11, 2013, 8:22:27 PM11/11/13
to unma...@googlegroups.com
Thanks, Marc, Dan, Giancarlo, Benedikt and Murray, for weighing in.

As it stands now, I'm provisionally satisfied with using duration (minutes) as an observation covariate with habitat interaction terms as indicated by AIC model selection.

Marc, our scenario is indeed "sites are not well-defined ... and search time is a proxy for the area searched".  Brown Treesnakes are of such high density and low detectability that we do not approach any sort of "census" of a given site on a given night, that is, our counts of new animals per hour do not drop as the search progresses through the night.  From that standpoint, I *would* expect counts to be linearly related to time spent searching, i.e. at a capture rate of one snake per hour, I would expect to get one snake in one hour and two snakes in two hours.  

Your suggestion of a three-level HM appears appropriate, but I'm not sure that our preliminary assessment of relative abundance warrants the time investment in the more complex "not in the toolbox" model at this point, given the acceptability of the proposed models (pending an acceptable GOF).  I like the idea of collapsing this to a two-level model to be fit in unmarked, but the "exposure level" (1/sampling duration) varies by observation and not by site (siteCovs only accepts one row per site) so I don't see how I can include it in the abundance model (am I wrong there?).

Shane


Giancarlo Sadoti

unread,
Nov 12, 2013, 5:50:09 AM11/12/13
to unma...@googlegroups.com
FWIW, I misspoke earlier when I cautioned about comparing AIC in offset vs. non-offset models of this fllavor. I've never had occasion to use a (well-formulated) offset as intended, so it has always appeared (in looking at the AIC or LL) that the offset was changing my response to a different scale.  G

Kery Marc

unread,
Nov 12, 2013, 6:37:02 PM11/12/13
to unma...@googlegroups.com
Dear Murray,

thanks for that suggestion. I haven't yet had the time to read your paper, but please let me ask you a question. I just talked with somebody who was using an N-mixture model for estimation of abundance based on data that ha
d observation occasions that differed in length (e.g., between 1 and 10 min, from what I understood). What would you suggest is the best way to "standardise" detection probability for this unequal effort ?

Thanks for comments, kind regards  ---  Marc



Murray Efford

unread,
Nov 13, 2013, 6:10:40 AM11/13/13
to unma...@googlegroups.com
Hi Marc

The idea is to think in terms of cumulative hazard of detection (lambda) rather than p (where lambda = -log(1-p) and p = 1 - exp(-lambda)). Other things being equal (!), lambda increases linearly with duration (effort). For any observed count there is a known duration ts; we parameterize in terms of the hazard lambda0 for a reference duration t0, so the detection probability for duration ts is 1 - exp(-ts/t0 * lambda0). This fits into the likelihood for a Poisson N-mixture model with R replicate populations sampled T times as follows:

neglogLik <- function (beta, n, ts, t0 = 1, maxN = 100) {
    ## beta is parameter vector (lambda0, N-bar) on log scale
    ## both n and ts should be matrices with dim (R,T)
    lambda0 <- exp(beta[1]);  N <- exp(beta[2])
    pt <- 1 - exp(-ts/t0 * lambda0)
    tmp <- sapply(0:maxN, dbinom, x = n, prob = pt)
    dim(tmp) <- c(nrow(n), ncol(n), maxN+1)
    tmp <- apply(tmp, c(1,3), prod)
    tmp <- sweep(tmp, MAR = 2, FUN ='*', STATS = dpois(0:maxN, N))
    -sum(log(apply(tmp,1,sum)))
}

I don't know if this can be done in unmarked.
Murray


Sent: Wednesday, 13 November 2013 12:37 p.m.

Richard Chandler

unread,
Nov 13, 2013, 9:46:08 AM11/13/13
to Unmarked package
Thanks for the code Murray. I put this on the to-do list: https://github.com/rbchan/unmarked/issues/78. 
Richard Chandler
Assistant Professor
Warnell School of Forestry and Natural Resources
University of Georgia

Kery Marc

unread,
Nov 15, 2013, 7:54:03 AM11/15/13
to unma...@googlegroups.com
Dear all,

since Murray's more mechanistic model for detection as a function of time duration cannot currently be fitted in unmarked, I was wondering how to best approximate this as a linear function of time or of a function of time (e.g., the log). The attached code does this. I think I would simply try a linear, quadratic and possibly cubic polynomial of time or possibly log(time).

Kind regards  --  Marc


P <- function(ts, t0, lambda0){ 1 - exp(-ts/t0 * lambda0)}
ts <- 1:100
t0 <- 1
lambda0 <- 0.01


for(lambda0 in c(0.001, 0.01, 0.1, 1, 2, 4, 6)){
par(mfrow = c(1,2))
plot(ts, P(ts, t0, lambda0), xlab = "Duration", ylab = "Detection probability",
   main = paste("lambda0:", lambda0), ylim = c(0,1), type = "l")
plot(log(ts), P(ts, t0, lambda0), xlab = "log(Duration)", ylab = "Detection probability",
   main = paste("lambda0:", lambda0), ylim = c(0,1), type = "l")
browser()
}



From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Richard Chandler [rcha...@warnell.uga.edu]
Sent: 13 November 2013 15:46
To: Unmarked package

Dan Linden

unread,
Nov 15, 2013, 10:29:03 AM11/15/13
to unma...@googlegroups.com
Similar to Marc, I had plotted some figures to see how the cumulative hazard function for survey length compared to some logit-scale models for detection.  The code below shows how an offset for log(time) (beta coefficient = 1) and an estimated coefficient for log(time) (here I plotted beta = 2) compares to the cumulative hazard.  The plots are across different values of detection probability for the minimum survey length (p0).  Obviously the benefit of Murray's approach is that you do not have to estimate a separate parameter for the slope of the relationship, similar to an offset.

# mininum survey
t0<-1 
# survey lengths
ts<-t0:10

for(p0 in round(10^(seq(-1,-.1,by=.1)),2)){ 
  lambda0 <- -1*log(1-p0)
  # detection over survey length
  pt <- 1-(exp(-ts/t0*lambda0))
  plot(ts,pt,type="l",xlim=c(0,10),ylim=c(0,1),main=paste("p0 =", p0),
     ylab="Detection probability",xlab="Survey length",las=1)
  lines(ts,plogis(qlogis(p0) + 1.0*log(ts)),col="blue")
  lines(ts,plogis(qlogis(p0) + 2.0*log(ts)),col="red")
  legend(6,.3,c("Hazard","1*log(time) [offset]","2*log(time)"),
         col=c("black","blue","red"),lty=1)
  browser()
}

Marc, I poached your use of the browser function.  Never seen that before, it's handy!
hazard.png
Reply all
Reply to author
Forward
0 new messages