Evaluating data deficiency for individual species in an MSOM

[Mary Clapp]

Hi all,

My question is more about model interpretation than it is about model building, and has to do with interpreting species-level parameter estimates whose [say, 95%] BCI overlap 0 in an MSOM. In some cases, this might indicate a "true" lack of effect, whereas in others, it may simply indicate a paucity of data. I'm aware that in MSOMs with a hyperprior on community membership, parameter estimates for species without much data will be drawn toward the community mean (zero, if the variable is standardized) in the absence of a very strong directional signal. So I am curious about how one might approach parsing of species whose parameters of interest overlap 0 between "no-effect" and "data deficient" species. I suppose the question would also apply to many single-species models, where the aim of the study is to identify species whose occupancy is/is not affected by some variable (management action, perturbation, habitat feature, etc).

But let's say for convenience that the submodel for psi in a (static, stacked by year) MSOM is as follows:

    for (i in 1:nspec) {
    w[i] ~ dbern(omega) # metacommunity membership hyperprior
      for (j in 1:nsite) {
        for (t in 1:nyear) {

          logit(psi[j,t,i]) <- b0[i] + b1[i]*Variable1[j,t]  + gamma[t]

          mu.psi[j,t,i] <- w[i] * psi[j,t,i]
          z[j,t,i] ~ dbern(mu.psi[j,t,i])
            

Some species will truly have no relationship to Variable1, whereas for others, we may simply not have enough data to say one way or another. 

To parse the difference, I've considered using some arbitrary threshold on n.eff (number of effective samples) or standard error of the posterior estimate of bBACI, or number of detections in the raw data, as I've seen in elsewhere in the literature. Perhaps it is an intrinsically arbitrary pursuit, but I am curious how others well-versed in these models would approach the question and potentially in a less subjective/more repeatable way. 

Any thoughts, suggestions, or useful resources are welcome (and apologies in advance if I have missed some seminal contribution any of you have already made to this topic!). 

Thanks, and thanks to everyone who contributes to this valuable resource.

Best,

Mary

[Matthijs Hollanders]

Hey Mary,

If the BCI is large there is a lot of uncertainty about that parameter, which may be driven by data deficiency. If the BCI overlaps 0, there is no evidence of effect. But irrespective of whether it's data deficiency or truly no effect, the uncertainty in your parameter estimate reveals that, given your model conditioned on your observed data, you did not observe strong evidence for an effect. If the BCI is squarely centered on 0 but spreads wide, the model is saying there's non-negligible possibility of strong positive or negative effects. But at the end of the day, it's the uncertainty in the parameter that will determine your ability to make any inference on that parameter.

I think ultimately your n.eff or MCSE suggestions also boil down to assessing the uncertainty in the parameter.

Does this help?

Matt

--
*** Three hierarchical modeling email lists ***
(1) unmarked: for questions specific to the R package unmarked
(2) SCR: for design and Bayesian or non-bayesian analysis of spatial capture-recapture
(3) HMecology (this list): for everything else, especially material covered in the books by Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2016, 2021) and Schaub & Kéry (2022)
---
You received this message because you are subscribed to the Google Groups "hmecology: Hierarchical Modeling in Ecology" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hmecology+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hmecology/9852e09e-95c0-4e06-a28e-9db5638b1e15n%40googlegroups.com.

[Liam Berigan]

Hi Mary,

To elaborate on Matt’s point, this could also be quantified with a Region of Practical Equivalence (ROPE). See the blog post linked below: the idea is that you pick an interval that you believe is essentially equivalent to zero effect (e.g., -0.1 to 0.1), and evaluate the probability that your parameter of interest falls within that interval. This allows for distinguishing between parameters with 95% CIs that overlap zero due to poor sample size, and those that have a “true” value of zero.

Best,

Liam Berigan

To view this discussion visit https://groups.google.com/d/msgid/hmecology/CAKMRE2tQ92igF4UVYrzmbOtGpJnS4_zTq%3DDr9-tzHFn5pp%3Dyaw%40mail.gmail.com.