Goodness of fit test for multi-session model

[Charlotte Patterson]

Hi all,

I am wanting to calculate goodness of fit values (using secr.test) for five multi-session models. However, I am struggling with the 'secr manual help page' documentation as it relates to translating the example code to a multi-session model.

From the example, I need to first define the statfn:

bothfn <- function(object) {

  CH <- object$capthist

  f1 <- sum(apply(abs(CH) > 0, 1, sum) == 1) / nrow(CH)

  devdf <- deviance(object) / df.residual(object)

  c(f1 = f1, devdf = devdf)

}

test <- secr.test(my_model, nsim = 19, statfn = bothfn, fit = TRUE, seed = NULL, ncores = 1, tracelevel = 1)

How would I modify this for a multi-session model?

Thank you,

Charlotte 

[Murray Efford]

It works fine with fit = TRUE and the default statistic deviance(fit) / df.residual(fit), and with fit = FALSE and the default f1 statistic (proportion of detected animals with only one detection, computed separately for each session).

The default f1 statistic for multi-session data is computed session by session. You can roll the session-specific values into a single number and compute alongside the deviance statistic like this:

bothfn <- function(object) {
    CH <- object$capthist

    onecapt <- function(ch) sum(apply(abs(ch) > 0, 1, sum) == 1)
    f1 <- sum(sapply(CH, onecapt)) / sum(sapply(CH, nrow))

    devdf <- deviance(object) / df.residual(object)
    c(f1 = f1, devdf = devdf)
}

test <- secr.test (ovenbird.model.1, nsim = 3, statfn = bothfn, fit = TRUE)
test

Murray

[Charlotte Patterson]

Hi Murray,

Thanks for your reply. I've tried running the code you've suggested below, however I'm still having an issue that might be related to my data, but I can't understand the error I'm getting despite some extensive googling.

Let me know if you need more information.

Thanks,

Charlotte

---

From: secr...@googlegroups.com <secr...@googlegroups.com> on behalf of Murray Efford <murray...@gmail.com>
Sent: Monday, 28 September 2020 11:37 AM
To: secr <secr...@googlegroups.com>
Subject: Re: Goodness of fit test for multi-session model

 

--
You received this message because you are subscribed to the Google Groups "secr" group.
To unsubscribe from this group and stop receiving emails from it, send an email to secrgroup+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/secrgroup/96307673-6e23-4e79-8e9a-f8f799efb33en%40googlegroups.com.

[Murray Efford]

This appears to be an unrelated problem, presumably in the simulation step. That should show up if you run simulate(fit). I can't investigate right now. There's really no need to be using bothfn rather than default, but should be OK.

Murray