AICc and effective sample size

[TDS]

Hello everyone,

I carried out avian point counts at 48 points, on 8 transects (6
points per transect, each 200 m apart). I am currently running
occupancy models in unmarked, and due to my small sample size, I
believe I should be using AICc rather than AIC. However, as mentioned
in a previous thread, it is not clear what the effective sample size
is for these models. I can try to create my own script to use AICc,
but am not sure what the effective sample size should be in this
situation, 48 or 8. Do you have any recommendations on what the
effective sample size should be, and how I could efficiently use AICc
instead of AIC?

Thank you,
Taza

[Richard Chandler]

Hi Taza,

I don't think anyone has worked on this problem for these classes of
models. I would personally just stick with AIC, but if you want to
explore alternatives, perhaps you or someone else could conduct a
simulation study to determine which information criterion picks the
"true" model the most frequently.

Richard

[Chris Nagy]

Hi Taza

Maybe I am misunderstanding but...Are you trying to infer to the transect (ie your 8 transects are your rows/sites and your 6 points on each are the columns/visits) or are you inferring about the 48 points (ie you went to each point multiple times and "transect" might be a covariate for geographic location/larger scale "site"/etc)?  Depending on which you are doing you would have, in my opinion, 8 (in the former case) or 48 (in the latter)...although in the latter situation your sites along a single transect may not be closed or independent depending on how close they are.

I use the number of sites - although obviously there is discussion over this - since modeling site occupancy is what I'm usually really interested in, not super-precise estimates of detection or something like that.  Using # of visits instead of # of individual sites seems wrong to me.  In my mind a similar situation is using total # of animal locations instead of # of individual animals in a telemetry study as your sample size if you are trying to infer to the animal population in general.  If you have 3 animals but you locate them 5000 times, your sample size of this population is still 3.  Similarly, I would think for drawing conclusions about patterns of site occupancy, the number of sites you have - and their occupancy states - stays the same no matter how many times you survey them.  Survey more times is certainly better in terms of your knowledge about each site, but your sample size stays the same.

chris

--
"I have always suspected that if our economic and political problems are ever really solved, life will become simpler instead of more complex, and that the sort of pleasure one gets from finding the first primrose will loom larger than the sort of pleasure one gets from eating an ice to the tune of a Wurlitzer." - George Orwell

[Richard Chandler]

In case someone wants to investigate the properties of these ICs, here is some code to get you started. You could run 1000 simulations for various values of nSites and nOccs and see which IC does best. In this example they all pick the correct model, and the rank order is the same too.



library(unmarked)

nSites <- 30
nOcc <- 3

set.seed(7)
x1good <- rnorm(nSites)
x2junk <- rnorm(nOcc)
x3good <- matrix(rnorm(nSites*nOcc), nSites, nOcc)
x4junk <- matrix(rnorm(nSites*nOcc), nSites, nOcc)

phi <- plogis(-1 + 1*x1good)  # Truth
p <- plogis(1 + -1*x3good)    # Truth
Z <- rbinom(nSites, 1, phi)
y <- matrix(NA, nSites, nOcc)
for(i in 1:nSites) {
    y[i,] <- rbinom(nOcc, Z[i], p[i,])
    }

umf <- unmarkedFrameOccu(y=y,
    siteCovs=data.frame(x1good=x1good, x2junk=x2junk),
    obsCovs=list(x3good=x3good, x4junk=x4junk))

fms <- list()
fms$Null <- occu(~1 ~1, umf)
fms$x1. <- occu(~1 ~x1good, umf)
fms$.x3 <- occu(~x3good ~1, umf)
fms$x1.x3 <- occu(~x3good ~x1good, umf)
fms$x1x2.x3x4 <- occu(~x3good + x4junk ~ x1good + x2junk, umf)

fitlist <- fitList(fits=fms)
modSel(fitlist)

# n = number of sites
AICc1 <- function(fit) {
    K <- length(coef(fit))
    y <- getY(fit@data)
    n <- nrow(y)      # Assumes no missing data
    AIC <- fit@AIC
    AICc <- AIC + 2*K*(K+1) / (n-K-1)
    return(AICc)
    }
# n = total number of observations   
AICc2 <- function(fit) {
    K <- length(coef(fit))
    y <- getY(fit@data)
    n <- length(y)    # Assumes no missing data
    AIC <- fit@AIC
    n <- nSites*nOcc
    AICc <- AIC + 2*K*(K+1) / (n-K-1)
    return(AICc)
    }


(AICrank <- sort(sapply(fms, function(x) x@AIC)))
(AICc1rank <- sort(sapply(fms, AICc1)))
(AICc2rank <- sort(sapply(fms, AICc2)))

[TDS]

Thanks again for your replies to my previous email! I did want to use
n=48, treating each of the 48 points as independent.

However, now I am concerned that the 48 points may not actually be
independent -- there might be some spatial correlation between the
points that are close together on a transect. (Eight transects each
have 6 points, 200m apart.) The habitat is very patchy, and therefore
I don't think all the points on one transect would be more closely
related then all the points on another transect. (Therefore, it
wouldn't make sense to just include the transect name as a categorical
covariate.) However, it is possible that two points 200m apart might
be spatially correlated. Is there any way to check for this, and if I
find that the points are spatially correlated, is there any way to
account for this in the model.

Thank you for your help.

Cheers,
Taza



On Oct 29, 10:36 am, Richard Chandler <richard.chandl...@gmail.com>
wrote:

> > On Tue, Oct 26, 2010 at 3:17 PM, TDS <tazascham...@gmail.com> wrote:
>
> >> Hello everyone,
>
> >> I carried out avian point counts at 48 points, on 8 transects (6
> >> points per transect, each 200 m apart). I am currently running
> >> occupancy models in unmarked, and due to my small sample size, I
> >> believe I should be using AICc rather than AIC. However, as mentioned
> >> in a previous thread, it is not clear what the effective sample size
> >> is for these models. I can try to create my own script to use AICc,
> >> but am not sure what the effective sample size should be in this
> >> situation, 48 or 8. Do you have any recommendations on what the
> >> effective sample size should be, and how I could efficiently use AICc
> >> instead of AIC?
>
> >> Thank you,
> >> Taza
>
> > --
> > "I have always suspected that if our economic and political problems are
> > ever really solved, life will become simpler instead of more complex, and
> > that the sort of pleasure one gets from finding the first primrose will loom
> > larger than the sort of pleasure one gets from eating an ice to the tune of

> > a Wurlitzer." - George Orwell- Hide quoted text -
>
> - Show quoted text -

[Richard Chandler]

Hi Taza,

Search for the word "spatial" in the Google Group and you will find several discussions of this. Currently, there is no way to model spatial dependence in unmarked. It's on the to-do list, if that's any consolation. Chapter 9 of Royle and Dorazio (Academic Press 2008) suggest some ways of doing this using WinBUGS.

Richard

[Taza Schaming]

Thanks! I had searched for "spatial", but was hoping a way to model
spatial dependence in unmarked had recently been developed.

Thanks again,
Taza

--
Taza Schaming
Cornell University
Department of Natural Resources
Fernow Hall
Ithaca, NY 14853
Phone: 508-277-5084
Cornell Laboratory of Ornithology

[Giancarlo Sadoti]

I'm a little late to the party, but I'd check out Jim Hines' paper
"Tigers on trails" in the July 2010 Ecol Apps (if you haven't already;
pardon me if anyone has addressed this in another topic).

Jim talked about this study at an occupancy workshop last year (and is
implemented in PRESENCE), but this paper finally (mostly) cleared it
up for me.

As I understand it, this spatial dependence is that between
observations (within sites) in a space-for-time point-on-transect
sampling approach. For example, the detection of a tiger on one point
means you're likely to detect it on the next given the space use of a
tiger (on trails). It's not the spatial dependence people worry about
between sites in most analyses (which Richard mentions is tricky to
tackle just yet in these models; e.g., what is a residual when we're
modeling a latent state?).

I know there's lots on Richard's to-do list for unmarked's
functionality, but Jim's spatial dependence model might be a valuable
addition to the occu framework (if enough people think so, I
suppose!). A multi-season (colext) extension (if it even possible)
would be even better! ;)

And thank you again Richard for all you do to tweak and upgrade
unmarked!

Giancarlo

On Dec 3, 6:45 am, Taza Schaming <td...@cornell.edu> wrote:
> Thanks! I had searched for "spatial", but was hoping a way to model
> spatial dependence in unmarked had recently been developed.
>
> Thanks again,
> Taza
>
> On Fri, Dec 3, 2010 at 8:21 AM, Richard Chandler
>
>
>
> <richard.chandl...@gmail.com> wrote:
> > Hi Taza,
>
> > Search for the word "spatial" in the Google Group and you will find several
> > discussions of this. Currently, there is no way to model spatial dependence
> > in unmarked. It's on the to-do list, if that's any consolation. Chapter 9 of
> > Royle and Dorazio (Academic Press 2008) suggest some ways of doing this
> > using WinBUGS.
>
> > Richard
>

[Richard Chandler]

Spatial models like the Hines et al (2010) one are on the to-do list, but first we're probably going to tackle "open population" models for various types of count data. Hopefully Ian will be able help again at some point. He deserves most of the credit for unmarked, and I know he has experience with spatial models. Any chance Ian?

Richard

[Ian Fiske]

I do agree that spatial models would be a valuable addition to unmarked.  That said, it's a pretty big project and I won't have time for this until I finish my dissertation, which is still my current side project.  Sorry folks, sounds fun though!

Ian