Power Analysis sample size

140 views
Skip to first unread message

mitchfirman

unread,
Mar 19, 2020, 1:52:11 PM3/19/20
to distance-sampling

Hi All,

 

I have some combined survey data (photographic and distance sampling) that I want to run a power analysis on, to examine what change in abundance we can detect given the current abundance and precision.  I’m using a single abundance estimate to run the power analysis, assuming repeated surveys will have the same level of precision. The R function I’d like to use requires a sample size (function tTestLnormAltPower from the EnvStats package).  

 

Can I use the df computed from Satterthwaite’s (1946) method, for the combined survey data as an approximation of the sample size?  Or can someone recommend an alternative approach for approximating sample size or an alternative type of power analysis that doesn’t rely on sample size?

 

Thanks,

Mitch

Eric Rexstad

unread,
Mar 20, 2020, 4:56:02 AM3/20/20
to mitchfirman, distance-sampling
Hello again Mitch

Chapter 5 of the Advanced Distance Sampling text (Buckland et al. 2004) is entitled "Temporal inferences from distance sampling surveys".  That chapter discusses means of assessing power to detect change from a series of surveys.  The computations described therein are not dependent upon sample size determination, but rather upon precision in the resulting density or abundance estimates.

There is companion R code used to create some of the figures in the chapter to be found here at Len Thomas' web site:

Eric Rexstad
Senior Research Fellow, CREEM, Univ. of St Andrews, Charity SC013532
--
You received this message because you are subscribed to the Google Groups "distance-sampling" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.

mitchfirman

unread,
Apr 14, 2020, 2:07:49 PM4/14/20
to distance-sampling

Hi Eric,

 

I had a look at Chapter 5 but, to be honest, it was generally above my comprehension level.  Thanks for Len’s R code in regards to Fig 5.6, that helps me sort things out.

 

I’m also interested in a comparison between abundance values from two different years’ data.  I followed the methods in section 3.6.5 from Introduction to Distance Sampling (Buckland et al. 2011).  Specifically, I followed the formula for a shared source of variance (equations 3.108 to 3.111).  I noted that this slightly increased the precision to detect differences when compared to the method that assumes complete independence between the two estimates (eqns 3.99 to 3.101).  Intuitively this did not make sense since I was used to variance calculations, that included common sources of variance, to slightly increase the CV when adding separate abundance estimates.  Conceptually, it makes sense that a common source of variance would increase the precision for determining the difference between two estimates but it would be nice to have some sort of affirmation that I haven’t messed up the calculations.

 

I would also assume that the calculation of degrees of freedom would also benefit by taking the same common sources of variance into account – is this what “Inference can now proceed as before, either with additional applications of Satterthwhaite’s approximation…” is referring to? (page 86 first paragraph)     

 

Thanks,

Mitch



On Friday, 20 March 2020 01:56:02 UTC-7, Eric Rexstad wrote:
Hello again Mitch

Chapter 5 of the Advanced Distance Sampling text (Buckland et al. 2004) is entitled "Temporal inferences from distance sampling surveys".  That chapter discusses means of assessing power to detect change from a series of surveys.  The computations described therein are not dependent upon sample size determination, but rather upon precision in the resulting density or abundance estimates.

There is companion R code used to create some of the figures in the chapter to be found here at Len Thomas' web site:


Eric Rexstad
Senior Research Fellow, CREEM, Univ. of St Andrews, Charity SC013532
On Mar 19 2020, at 5:52 pm, mitchfirman <mitchel...@gmail.com> wrote:
Hi All,
 

I have some combined survey data (photographic and distance sampling) that I want to run a power analysis on, to examine what change in abundance we can detect given the current abundance and precision.  I’m using a single abundance estimate to run the power analysis, assuming repeated surveys will have the same level of precision. The R function I’d like to use requires a sample size (function tTestLnormAltPower from the EnvStats package).  

 

Can I use the df computed from Satterthwaite’s (1946) method, for the combined survey data as an approximation of the sample size?  Or can someone recommend an alternative approach for approximating sample size or an alternative type of power analysis that doesn’t rely on sample size?

 

Thanks,

Mitch

--
You received this message because you are subscribed to the Google Groups "distance-sampling" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distance...@googlegroups.com.

Eric Rexstad

unread,
Apr 15, 2020, 12:41:29 PM4/15/20
to mitchfirman, distance-sampling
Mitch

You are right to scratch your head when performing the calculations in Section 3.6.5 of Buckland et al. (2001).  Because with a pooled detection function there are parameters of only a single detection function to estimate rather than two sets of parameters.  That can result in some reduction in uncertainty, but in most line transect survey situations that reduction is modest, because the greatest proportion of uncertainty in density estimates derive from variability in encounter rates between transects.

Use of the Sattherthwaite correction makes a difference in the calculations when there are a small number of transects.  Hopefully, if you are in the design stage, you can design to have a sufficient number of replicate transects.

Eric Rexstad
Senior Research Fellow, CREEM, Univ. of St Andrews, Charity SC013532
The Observatory, Buchanan Gardens, St Andrews KY16 9LZ
To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.

mitchfirman

unread,
Apr 21, 2020, 8:45:56 PM4/21/20
to distance-sampling

Thanks Eric, I think I’ve sorted the calculations out but I’m hoping you can help resolve my confusion regarding interpretation of the results:

 

When using the equations in the Introduction to Distance Sampling Section 3.6.5 for estimating a change in density for independent estimates (Buckland et al. 2001), the difference between estimates becomes significant when the second abundance estimate drops 14% compared to the first (when both estimates have a CV of %5 and df=30)...

 

I’m also interested in power analysis for the same research and compared these results to Fig 5.6a from Advanced Distance Sampling (Buckland et al. 2004). When the R code is adjusted to include an annual rate of change of up to  50%, it is estimated that between 4 and 5 years of data at a CV of 5% is required to detect a log-linear population trend of 0.5 per year (power = 0.8, alpha = 0.05).  I understand that the comparison of two values (mentioned above) and a power analysis are asking two different questions but it seems that the results are somewhat dissimilar.  I know that the power analysis includes an uncertainty in regards to the ability to measure the actual abundance, but the need for 4 or 5 years of data to detect an annual population change rate of 0.5 seems overly conservative.  Can someone please explain this discrepancy or point me to the error in my ways?

 

Thanks,

Mitch  



On Wednesday, 15 April 2020 09:41:29 UTC-7, Eric Rexstad wrote:
Mitch

You are right to scratch your head when performing the calculations in Section 3.6.5 of Buckland et al. (2001).  Because with a pooled detection function there are parameters of only a single detection function to estimate rather than two sets of parameters.  That can result in some reduction in uncertainty, but in most line transect survey situations that reduction is modest, because the greatest proportion of uncertainty in density estimates derive from variability in encounter rates between transects.

Use of the Sattherthwaite correction makes a difference in the calculations when there are a small number of transects.  Hopefully, if you are in the design stage, you can design to have a sufficient number of replicate transects.

Eric Rexstad
Senior Research Fellow, CREEM, Univ. of St Andrews, Charity SC013532

Mark Wilson

unread,
Apr 22, 2020, 4:19:06 AM4/22/20
to mitchfirman, distance-sampling
Hi Mitch,

I'm maybe (very plausibly!) misunderstanding something about the analyses you're describing, but the numbers you quote for the Power Analysis don't seem right. A CV of 5% for a population estimate sounds tight. A decline in abundance of 50% p.a. is catastrophic. Starting from a relative abundance index of 1, 5 years of decline of this scale takes you to a year 5 index of 0.03. The notion that you could estimate population size to within 5% but not have the power to detect population decline until you were at 3% of your initial level seems wrong. I don't think actual abundances matter very much (the effect of actual abundance on your power to detect declines will be through encounter rates, which will be a product of detectability and abundance. But this should, I think, be wrapped up in your CV of 5%.

Apologies if I've misunderstood anything here. But it seems to me like the power analysis is off-kilter here. Eric or others on the list may have more erudite views.

Best wishes,

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distance-sampling/e9c25547-ca9b-4682-9b63-fb08f3d5be49%40googlegroups.com.


--
Dr Mark Wilson
Senior Research Ecologist
BTO Scotland

Unit 15
Beta Centre
Innovation Park
University of Stirling
FK9 4NF

Tel: 01786 458024

Registered Charity No SC039193 (Scotland) and No 216652 (England & Wales)
Company Limited by Guarantee No 357284 (England & Wales)

Stephen Buckland

unread,
Apr 22, 2020, 4:42:59 AM4/22/20
to Mark Wilson, mitchfirman, distance-sampling

Yes, I agree with Mark’s comment.  From fig 5.6a of the 2004 book, it looks like, with a cv of 5%, a lambda value of about 0.87 requires 5 years data to give power of 80% with alpha of 5%.

 

Steve Buckland

mitchfirman

unread,
Apr 24, 2020, 11:27:33 AM4/24/20
to distance-sampling
Hi Everyone,

Apologies, I should have mentioned that my actual CV is closer to 20% but I'm using 5% as an example since it more readily illustrates my confusion regrading Figure 5.6a.  When I alter the figure's R code to extend lambda to 0.1, it tells me that 4 years of data at a 5%CV can only detect a decline if lambda is at 0.25? (extended figure attached).  Even a 20% CV requiring 5 years of data to detect a lambda of less than 0.4 seems off?  I appreciated the comments.

Thanks,
Mitch

Fig5-6a.PNG

Stephen Buckland

unread,
Apr 24, 2020, 12:05:20 PM4/24/20
to mitchfirman, distance-sampling

Mitch, bear in mind that this is based on a t-test with just 3df for 5 years of data, and 2 df for 4 years of data.  So you are testing whether a slope differs significantly from zero on the basis of just 4 or 5 points, and for this analysis you estimate the cv from a very small sample size.  If you assumed the cv were known, or used the annual estimates of cv in your analysis, you would get very different answers.  The power calculation just estimates the cv from the departures of the estimates from a fitted line.  This is fine for a reasonably long time series of estimates, but probably not a good strategy for very short sequences.

 

Steve

 

From: distance...@googlegroups.com [mailto:distance...@googlegroups.com] On Behalf Of mitchfirman
Sent: 24 April 2020 16:28
To: distance-sampling <distance...@googlegroups.com>
Subject: Re: [distance-sampling] Power Analysis sample size

 

Hi Everyone,

To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distance-sampling/9cd5da7a-1bda-4b7d-b70e-2be819557cd2%40googlegroups.com.

mitchfirman

unread,
Apr 27, 2020, 4:04:41 PM4/27/20
to distance-sampling
Thanks Steve, that makes sense.  As you have probably guessed, I have to assess the power of a short time series to detect a decline.  The CV of the current (or future) abundance estimate is not likely to vary greatly as is appropriate for attempting this power analysis. You mention that an analysis performed using a known CV would have different results...  could you point me in the right direction as to what the appropriate method might be?

Thanks,
Mitch   

Stephen Buckland

unread,
Apr 28, 2020, 5:12:08 AM4/28/20
to mitchfirman, distance-sampling

The 2001 distance sampling book gives an example of a pairwise comparison.  This approach could be extended to more than 2 estimates.  E.g. if you have 3 estimates E1, E2 and E3, you could take -1.E1 + 0.E2 + 1.E3, find its variance, and hence test whether the value differs significantly from zero.  To test for curvature in the trend, you could take -1.E1 + 2.E2 -1.E3, again finds its variance, then test whether it differs significantly from zero.

 

There are no doubt other solutions!

To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distance-sampling/bc10da2c-0ebb-4f8d-9039-54138abbb2b5%40googlegroups.com.

mitchfirman

unread,
Apr 28, 2020, 6:23:09 PM4/28/20
to distance-sampling
Thanks Steve, this actually makes sense (for a change).

Based on a known CV, I could predict an effect size that could be detected and spread the effect size over multiple years as an annual rate, achieving the known effect size on the last year when compared to the first year.

But I'd also like to get an idea of the power to detect this decline (i.e. include the uncertainty regarding the measurement of the actual abundance). Can I do this as well?

Thanks,
Mitch     

Stephen Buckland

unread,
Apr 29, 2020, 4:32:10 AM4/29/20
to mitchfirman, distance-sampling

I tend to prefer to work with estimated confidence interval length rather than power, but yes, you can take a given effect size, and then estimate the probability that you would reject H0 of no effect.  If you know the precision of the test statistic (e.g. one of those I suggested), then you can do the power calculation.

To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distance-sampling/cf890312-6acb-4f9b-88c1-430658875fa5%40googlegroups.com.

mitchfirman

unread,
May 3, 2020, 12:53:11 PM5/3/20
to distance-sampling

Thanks Steve,

 

If I’m understanding this correctly, I would complete many iterations of the comparison of the two abundances… using a R function like ‘nlnorm’ to simulate values of the two estimates, with D1 based on the field data estimate and with D2 being a reduction of D1 based on the effect size being tested (sd and df for both D1 and D2 based on the field data estimate). I would then look at the percentage of the comparisons that rejected the H0 to determine the power of the test in relation to the effect size and alpha value that was used.

 

I hope I’m correct as this seems to make sense.

 

Thanks again,

Mitch       

Stephen Buckland

unread,
May 3, 2020, 1:31:24 PM5/3/20
to mitchfirman, distance-sampling

You could take the expected value of your test stat corresponding to a given effect size, then find the prob of rejecting H0, as you have the variance of the test stat.  But your way should also work fine.

To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distance-sampling/f904411c-d36b-476c-861b-4f3bd508a816%40googlegroups.com.

mitchfirman

unread,
May 5, 2020, 10:27:51 PM5/5/20
to distance-sampling
Thanks Steve, I also tried your method and thankfully the results matched mine.

Mitch
Reply all
Reply to author
Forward
0 new messages