Statistical significance of decreasing occupancy in colext?

[Matthew Hyde]

Hi all,

I'm running a dynamic occupancy model with three years of data following disturbance with colext function. As we expected, we see large extinction rates and decreasing occupancy after the first year. Occupancy is clearly decreasing , but with overlapping confidence intervals, i.e. psi for year is 0.734 (SE: 0.104, CIs: 0.531-0.937) and psi for year 3 is 0.459 (SE: 0.130, CIs: 0.205-0.713). However, with the sizeable overlap of the 95% CIs, I was wondering what statistical method would be most useful to determine whether the difference is statistically significant.

We have acceptable c-hat values and M-B GOF values for the model. 

I appreciate any help or advice.

Best,

Matt

[Marc Kery]

Dear Matt

I could see three approaches to compute the SE and CIs for the difference between psi1 and psi3:

1. The delta rule
2. The bootstrap.
3. Go Bayesian and add one line in your code which computes that difference, and then you can enjoy the power of posterior inference.

Staying in the frequentist part of the world, and with unmarked, I'd suggest #2. You can run a non-parametric bootstrap as follows. You should probably treat as resampling unit all the data from a site. Then, you repeat the following a large number of times (e.g., 1000 or 10,000; how much depends on the available number of sites in your data set):

1. Draw from your data set with n sites a sample of the same size of n sites, with replacement. Thus, the resampled data set will contain some sites multiple times and others not at all.
2. Fit your model to the resampled data set, compute psi1 and psi3 and take the difference, and save that.

And that's already it ! The SD of these bootstrapped differences is your bootstrapped SE of the difference between psi1 and psi3, and the 2.5% and 97.5% percentiles represent a 95% CI for the difference in psi between year 1 and year 3. If 0 is outside of it, it's your lucky day.

The bootstrap (nonparametric as described, or also the parametric variant) is a tremendously powerful technique for variance estimation, especially for derived quantities. Plus, it's often ridiculously easy to compute. It belongs into the tool-box of any ambitious quantitative ecologist.

Best regards  --- Marc

---

From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Matthew Hyde <mhy...@gmail.com>
Sent: Saturday, September 23, 2023 00:04
To: unmarked <unma...@googlegroups.com>
Subject: [unmarked] Statistical significance of decreasing occupancy in colext?

 

--
*** Three hierarchical modeling email lists ***
(1) unmarked (this list): for questions specific to the R package unmarked
(2) SCR: for design and Bayesian or non-bayesian analysis of spatial capture-recapture
(3) HMecology: for everything else, especially material covered in the books by Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2016, 2021) and Schaub & Kéry (2022)
---
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/920e63e2-7e2c-4d35-a148-d05ef6f01993n%40googlegroups.com.

[Matthew Hyde]

Marc,

Thanks so much for your quick response. I'll give #2 a try and I'll let you know if I have any difficulty with this approach. I appreciate your quick and helpful feedback.

Best,

Matt

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/ZR0P278MB0869E15B516FB53DAF1912EEEBFFA%40ZR0P278MB0869.CHEP278.PROD.OUTLOOK.COM.

[Marc Kery]

Excellent. BTW, how may sites do you have ?

---

From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Matthew Hyde <mhy...@gmail.com>

Sent: Saturday, September 23, 2023 04:00
To: unma...@googlegroups.com <unma...@googlegroups.com>
Subject: Re: [unmarked] Statistical significance of decreasing occupancy in colext?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/CAEdMJ0_1pLk%2BaHHJJOB8rJQy0KpBrDRuRxoB5BrnSC%3DVnCJ21A%40mail.gmail.com.

[Matthew Hyde]

We have 44 sites over 3 years, so not a ton of data. We're running dyn occ for 8 species to see how a fire between years 1 and 2 affected occupancy in years 2 and 3.

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/ZR0P278MB08695F09815EDE325D75F7F9EBFEA%40ZR0P278MB0869.CHEP278.PROD.OUTLOOK.COM.

[Marc Kery]

OK, that's not a ton indeed. I am not exactly sure about what the rules are for choosing the number of bootstrap replicates in relation to the size of a data set, but perhaps I'd run it for 1000 and 10,000 reps and compare.

One thing: if you really want to show/test for an effect of fire on all species, then a multispecies approach might be valuable, where you combine the analysis of all species, and treat their params (on the logit scale) as random effects drawn from a normal distribution. These models allow you to make inferences about the community (or "the average species" or "all species") as well as about the individual species. The community-level fire effect may be "sharper" than the fire effects estimated for each individual species. Drawback is that you have to go Bayes, typically JAGS or NIMBLE.

We show such "dynamic community models", DCMs, in chapter 5 of the AHM2 book, and you can download all the code from the website (unless you want to buy the book of course 😉..)

Marc

---

From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Matthew Hyde <mhy...@gmail.com>

Sent: Saturday, September 23, 2023 10:58

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/CAEdMJ09Va%3Dzvs5g%3DJ7jcWc4spaT9Hxk8i-zi23nOrnD7-a8qhQ%40mail.gmail.com.

[Matthew Hyde]

Thank you Marc! I'll take a look at chapter 5, I brought the book back from thr workshop at Patuxent and it's proved quite helpful. 

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/ZR0P278MB08695AA3B957BCCA7F272719EBFEA%40ZR0P278MB0869.CHEP278.PROD.OUTLOOK.COM.

[Matthew Hyde]

Hi Marc, I was just wondering what the one line of code would be if we were to go bayesian to compute the differences between CIs. You mentioned this as option 3 and I was just thinking that might be helpful for future projects.

Best,

Matt

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/ZR0P278MB0869E15B516FB53DAF1912EEEBFFA%40ZR0P278MB0869.CHEP278.PROD.OUTLOOK.COM.

--

Matthew Hyde

Cell: (+1) 224-545-9634

Correo: Mhy...@gmail.com

[Marc Kery]

Dear Matt,

on re-reading part of my response I see it may be confusing. I did not mean to say that you can add one line in YOUR existing code and then that would give you a Bayesian solution. In 3, I meant to say that you first have to implement the model in some Bayesian software such as JAGS, NIMBLE, Stan, and THEN it's just one additional line.

best regards  – Marc

---

From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Matthew Hyde <mhy...@gmail.com>

Sent: Friday, February 9, 2024 14:24
To: unma...@googlegroups.com <unma...@googlegroups.com>
Subject: Re: [unmarked] Statistical significance of decreasing occupancy in colext?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/CAEdMJ09KBAs%3D%2Bxyamx_RryZPSGvEARoct0_H3MnD2Lr-%3Dy%2BEaw%40mail.gmail.com.

[Matthew Hyde]

Hi Marc,

Thanks for the clarification. That's how I interpreted it too, I know we would have to convert the whole analysis to Bayesian, not just adding one line. I was just thinking that it might be good for a future analysis where we would do a bayesian analysis in jags instead of using unmarked.

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/ZR0P278MB0869CDAF007BEC5A4B5AA6D9EB4B2%40ZR0P278MB0869.CHEP278.PROD.OUTLOOK.COM.

[Marc Kery]

Dear Matt,

once you have a model in JAGS or NIMBLE, to get a 95% CRI for the difference between the psi's in years 3 and 1, you simply add that one line:  diff <- psi3 - psi1

Then, you can do the old frequentist's trick and check whether 0 is inside or outside of the 95% CRI for diff to say whether psi1 and psi3 are "significantly" different.

Best regards  -- Marc

---

From: unma...@googlegroups.com <unma...@googlegroups.com> on behalf of Matthew Hyde <mhy...@gmail.com>

Sent: Friday, February 9, 2024 14:31

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/CAEdMJ09divoeDeYt1y-XgokaQw1BAwGAhUeMdnvU%2BcrgGEnMzw%40mail.gmail.com.

[Matthew Hyde]

Thanks Marc! That's way easier than I expected.

You received this message because you are subscribed to a topic in the Google Groups "unmarked" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unmarked/918UEKmIBvk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to unmarked+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/ZR0P278MB086919D97A120DDAFCCE0BD7EB4B2%40ZR0P278MB0869.CHEP278.PROD.OUTLOOK.COM.