model selection and large SEs on p

[James Saracco]

Dear list members,

I am analyzing data from a point-count based monitoring program for which=
distance estimation is part of the field protocol and for which distance=
-detection modeling is to be used to 'correct' count oberservations for i=
mperfect detection. I am dealing with a bunch of species (61 with > 75 de=
tections, but with about 1/4 of them with > 1000 detections) and potentia=
l models (12 for each species with different combinations of key and adju=
stment functions and covariates). Small sample sizes in some cases (belo=
w 150 or so, suggest that more complicated covariate models; e.g., models=
with observer effects, where there might be 20 or more observers) may no=
t be appropriate. Nevertheless, given the number of specie to be conside=
red, as an initial step, I just ran all species-model combinations with a=
for loop in R using the Distance package. If I look at just the lowest A=
IC models, I get strongest support in most cases for the more complicated=
covariate models, yet standard errors for one or more scale covaraiates =
in these models can sometimes be wonky (ie very large), suggesting overfi=
tting. And the reported standard errors for 'average p' can sometimes be =
> > 1, which I don't understand. The detection histograms, and predicted p'=
s look reasonable though (although there is often evidence of fewer than =
expected observations at nearest distances). My initial solution was to e=
liminate models with CVs on the mean detection probability > 100% from co=
nsideration. But I don't have any real theoretical or empirical grounds f=
or doing so. Part of the problem is that I can't figure out how the stand=
ard errors of the average detection probability estimates (presumably cal=
culated using eq. 6.50 in the Buckland et al 2004 book) are calculated an=
d how they can be > 1.=20

So, my questions are 1) how can the average detection probabilities be > =
1?, and 2) how might one proceed in a model-selection framework when prec=
ision seems to be low, but for which AIC suggests you have a good model (=
with maybe one or 2 parameters not estimated well, but the rest of which =
look good).

Any insights or suggestions would be appreciated. Thanks!

--
distance-sampling list home page:
http://www.jiscmail.ac.uk/lists/distance-sampling.html
To unsubscribe, go to the home page and click "Join or leave the list",
or email jisc...@jiscmail.ac.uk with the following message body:
leave distance-sampling
Before posting to the list, read the list guidelines at
http://www.jiscmail.ac.uk/files/distance-sampling/
distance-sampling.welcome
Any questions, email the list owner at
distance-sam...@jiscmail.ac.uk
--

[Eric Rexstad]

James:

I suspect you have fitted a number of models that have not converged=20
(resulting in the 'wonky' standard errors you describe). I have the=20
suspicion that this automated approach "all species all keys all=20
adjustments all covariates" is too blunt an instrument for the analyses=20
you wish to perform.

As a first step, check to see whether your models have indeed=20
converged. You may need to increase the value of the "debug.level"=20
argument to see more detail about the optimisation process for your model=
s.

On 24/10/2013 19:46, James Saracco wrote:
> > Dear list members,
> >

> > I am analyzing data from a point-count based monitoring program for whi=
ch distance estimation is part of the field protocol and for which distan=
ce-detection modeling is to be used to 'correct' count oberservations for=
imperfect detection. I am dealing with a bunch of species (61 with > 75 =
detections, but with about 1/4 of them with > 1000 detections) and potent=
ial models (12 for each species with different combinations of key and ad=
justment functions and covariates). Small sample sizes in some cases (be=
low 150 or so, suggest that more complicated covariate models; e.g., mode=
ls with observer effects, where there might be 20 or more observers) may =
not be appropriate. Nevertheless, given the number of specie to be consi=
dered, as an initial step, I just ran all species-model combinations with=
a for loop in R using the Distance package. If I look at just the lowest=
AIC models, I get strongest support in most cases for the more complicat=
ed covariate models, yet standard errors for one or more scale covaraiate=
s in these models can sometimes be wonky (ie very large), suggesting over=
fitting. And the reported standard errors for 'average p' can sometimes b=
e > 1, which I don't understand. The detection histograms, and predicted =
p's look reasonable though (although there is often evidence of fewer tha=
n expected observations at nearest distances). My initial solution was to=
eliminate models with CVs on the mean detection probability > 100% from =
consideration. But I don't have any real theoretical or empirical grounds=
for doing so. Part of the problem is that I can't figure out how the sta=
ndard errors of the average detection probability estimates (presumably c=
alculated using eq. 6.50 in the Buckland et al 2004 book) are calculated =
and how they can be > 1.

> >
> > So, my questions are 1) how can the average detection probabilities be =

> > 1?, and 2) how might one proceed in a model-selection framework when pr=
ecision seems to be low, but for which AIC suggests you have a good model=
(with maybe one or 2 parameters not estimated well, but the rest of whic=
h look good).

> >
> > Any insights or suggestions would be appreciated. Thanks!
> >
> > --
> > distance-sampling list home page:
> > http://www.jiscmail.ac.uk/lists/distance-sampling.html

> > To unsubscribe, go to the home page and click "Join or leave the list",=

> > or email jisc...@jiscmail.ac.uk with the following message body:
> > leave distance-sampling
> > Before posting to the list, read the list guidelines at
> > http://www.jiscmail.ac.uk/files/distance-sampling/
> > distance-sampling.welcome
> > Any questions, email the list owner at
> > distance-sam...@jiscmail.ac.uk
> > --
> >
> >
> >

--=20
Eric Rexstad
Research Unit for Wildlife Population Assessment
Centre for Research into Ecological and Environmental Modelling
University of St. Andrews
St. Andrews Scotland KY16 9LZ
+44 (0)1334 461833
The University of St Andrews is a charity registered in Scotland : No SC0=
13532