Deriving Goodness of Fit and c-hat from OccuMS

[Benjamin Becker]

Dear List,

Is there any guidance for calculating c-hat and goodness of fit from an occuMS() model? I see that parboot() does not support the condbinom parameterization and when I run the multinom parameterization I get: 

> parboot(mod04b_stacked_fit)
Error in sample.int(length(x), size, replace, prob) :
  NA in probability vector

The model object works fine with predict(), SSE(), plot(), and gives reasonable errors, plots, and parameter results, but not ranef().

> ranef(mod04b_stacked_fit)
Mean Mode      2.5% 97.5%
[1,] NA   integer,0 NA   NA  
[2,] NA   integer,0 NA   NA  

[3,] NA   integer,0 NA   NA

Thanks for any guidance.

-Ben

[Ken Kellner]

Hi Ben,

parboot and ranef work for me with the example code in the occuMS help file. Do you have missing values in your y-matrix and/or covariates? I suspect these may not be being handled properly.

Ken

On Thu, Jul 21, 2022 at 05:07:43PM -0700, Benjamin Becker wrote:
> Dear List,
>

> Is there any guidance for calculating *c-hat *and *goodness of fit *from an
> *occuMS()* model? I see that parboot() does not support the condbinom

> parameterization and when I run the multinom parameterization I get:
>
> > parboot(mod04b_stacked_fit)
> Error in sample.int(length(x), size, replace, prob) :
> NA in probability vector
>
> The model object works fine with predict(), SSE(), plot(), and gives
> reasonable errors, plots, and parameter results, but not ranef().
>
> > ranef(mod04b_stacked_fit)
> Mean Mode 2.5% 97.5%
> [1,] NA integer,0 NA NA
> [2,] NA integer,0 NA NA
> [3,] NA integer,0 NA NA
>
> Thanks for any guidance.
>
> -Ben
>
>

> --
> *** Three hierarchical modeling email lists ***
> (1) unmarked (this list): for questions specific to the R package unmarked
> (2) SCR: for design and Bayesian or non-bayesian analysis of spatial capture-recapture
> (3) HMecology: for everything else, especially material covered in the books by Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2016, 2021) and Schaub & Kéry (2022)
> ---
> You received this message because you are subscribed to the Google Groups "unmarked" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/50e90e60-8809-43a1-831c-34f66a3435c6n%40googlegroups.com.

[Benjamin Becker]

Thanks so much Ken - You are correct that we have lots of NAs in both the  y-matrix and covariates since we have different numbers of visits for each site and year.  Collapsing/pooling surveys to biweekly or monthly sample periods doesn't help since not all sites are visited each year, and some sites only visited a few times early in the year to establish occupancy, but not revisited later in season if no occupancy.  (problem of highly unbalanced sampling design).

We currently have a survey level covariate for pre-nestling or post-nestling period based on date (0/1).Would it be acceptable to add a third factor level of "not visited" (so (0/1/2) which would remove all NAs from the covariates (but we would still would have NAs in the y's)?

Thanks,

-Ben

You received this message because you are subscribed to a topic in the Google Groups "unmarked" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unmarked/G8kWsLPNe6A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to unmarked+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/YtnvmtIYSQu/hu0x%40COYOTE.

[Ken Kellner]

Hi Ben,

unmarked should handle the NAs itself. I was able to find the problem with ranef but I can't replicate your issue with parboot. Can you email me your model object (mod04b) as an .Rdata or .Rds file off-list (to con...@kenkellner.com) so I can take a closer look at it?

Ken

On Mon, Jul 25, 2022 at 07:57:30PM -0700, Benjamin Becker wrote:
> Thanks so much Ken - You are correct that we have lots of NAs in both the
> y-matrix and covariates since we have different numbers of visits for each
> site and year. Collapsing/pooling surveys to biweekly or monthly sample
> periods doesn't help since not all sites are visited each year, and some
> sites only visited a few times early in the year to establish
> occupancy, but not revisited later in season if no occupancy. (problem of
> highly unbalanced sampling design).
>

> We currently have a survey level covariate for *pre-nestling* or
> *post-nestling* period based on date (0/1).Would it be acceptable to add

> To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/CADm2a3pA8Tf0apfL6-9rEJEJgoiKJqKt7kODpzxchhiBQdkWWg%40mail.gmail.com.

[Ken Kellner]

I think I figured it out. Can you try the attached patch to see if it helps?

library(unmarked)
source("occuMS_patch.R")

#...your code

> To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/Yt/dK1W0sfCCYQfs%40COYOTE.

[Benjamin Becker]

Ken, 

This patch worked for both ranef() and parboot().

Thanks so much !

-Ben

To view this discussion on the web visit https://groups.google.com/d/msgid/unmarked/Yt/lCtWBsSAbuTiI%40COYOTE.