Log-score calculation for occupancy model

72 views
Skip to first unread message

Mike Allen

unread,
Dec 22, 2020, 1:25:57 PM12/22/20
to hmecology: Hierarchical Modeling in Ecology

Hello everyone,
I'm trying to calculate the "log scoring rule" from the Stevens & Conway (2019) paper* using their code and simulated occupancy data (325 sites, 4 years, 6 visits per site per year). I split the data into 4 parts (260 sites) for training and 1 part (65 sites) for testing. The idea is to ultimately do this five times for a 5-fold cross validation version of this metric. The code runs fine if I comment out the log-scoring calculation part. But when it is included I get this error:
"Error in checkForRemoteErrors(val) :
  3 nodes produced errors; first error: RUNTIME ERROR:
Failed check for discrete-valued parameters in function :"


The code to simulate the data in R and run the model in JAGS are below (only takes a minute or two to run).  Can anyone help me figure out what is causing this error? 

Thanks very much,

Mike

*Stevens, B. S., & Conway, C. J. (2019). Predicting species distributions: unifying model selection and scale optimization for multi‐scale occupancy models. Ecosphere, 10(5), e02748.

#####################################
###### SIMULATE 4 YEARS OF 6-VISIT DETECTION / NON-DETECTION AT 325 SITES
###### WITH ONE COVARIATE FOR PSI AND ONE FOR P
#####################################

# (data simulation code adapted from Kery & Royle AHM1)

# Choose sample sizes and prepare obs. data array y
n.sites <- 325                      # Number of sites
J <-  6                        # Number of presence/absence measurements per site per year
y1 <-
  matrix(NA, nrow = n.sites, ncol = J) # to contain the obs. data, year 1
y2 <-
  matrix(NA, nrow = n.sites, ncol = J) # to contain the obs. data, year 2
y3 <-
  matrix(NA, nrow = n.sites, ncol = J) # to contain the obs. data, year 3
y4 <-
  matrix(NA, nrow = n.sites, ncol = J) # to contain the obs. data, year 4

# Create a covariate called vegHt
set.seed(110)
vegHt <- runif(n.sites,-1, 1)

# Choose parameter values for occupancy model and compute occupancy
beta0 <- -2                    # Logit-scale intercept
beta1 <- 2                    # Logit-scale slope for vegHt
psi <- plogis(beta0 + beta1 * vegHt) # Occupancy probability
# plot(vegHt, psi, ylim = c(0,1), lwd = 3) # Plot psi relationship

# Now visit each site and observe presence/absence perfectly
set.seed(120)
z1 <- rbinom(n.sites, 1, psi)        # True presence/absence year 1
set.seed(130)
z2 <- rbinom(n.sites, 1, psi)        # True presence/absence year 2
set.seed(140)
z3 <- rbinom(n.sites, 1, psi)        # True presence/absence year 3
set.seed(150)
z4 <- rbinom(n.sites, 1, psi)        # True presence/absence year 4

# Create a covariate called wind
set.seed(160)
wind1 <- array(runif(n.sites * J,-1, 1), dim = c(n.sites, J))
set.seed(170)
wind2 <- array(runif(n.sites * J,-1, 1), dim = c(n.sites, J))
set.seed(180)
wind3 <- array(runif(n.sites * J,-1, 1), dim = c(n.sites, J))
set.seed(190)
wind4 <- array(runif(n.sites * J,-1, 1), dim = c(n.sites, J))

# Choose parameter values for measurement error model and compute detectability
alpha0 <- -1                        # Logit-scale intercept
alpha1 <- -3                        # Logit-scale slope for wind
p1 <- plogis(alpha0 + alpha1 * wind1) # Detection probability
p2 <- plogis(alpha0 + alpha1 * wind2) # Detection probability
p3 <- plogis(alpha0 + alpha1 * wind3) # Detection probability
p4 <- plogis(alpha0 + alpha1 * wind4) # Detection probability

#plot(p1 ~ wind1, ylim = c(0,1))     # Look at relationship

# Take J = 6 presence/absence measurements at each site in each year
set.seed(200)
for (j in 1:J) {
  y1[, j] <- rbinom(n.sites, z1, p1[, j])
}

set.seed(210)
for (j in 1:J) {
  y2[, j] <- rbinom(n.sites, z2, p2[, j])
}

set.seed(220)
for (j in 1:J) {
  y3[, j] <- rbinom(n.sites, z3, p3[, j])
}

set.seed(230)
for (j in 1:J) {
  y4[, j] <- rbinom(n.sites, z4, p4[, j])
}

#####################################
###### FORMAT DATA FOR THE STEVENS & CONWAY, ECOSPHERE (2019) MODEL
###### RUN THE MODEL IN JAGS
#####################################

T = 4 # number of years sampled
n = c(n.sites, n.sites, n.sites, n.sites)
y = cbind(c(t(y1)), c(t(y2)), c(t(y3)), c(t(y4)))
sites.list = list()
for (s in 1:n.sites) {
  sites.list[[s]] = rep(s, J)
}
sites <- matrix(rep(do.call("c", sites.list), T), ncol = T)
# the site-level covariate
vegHt <- vegHt
# obs-level covariate
wind = cbind(c(t(wind1)), c(t(wind2)), c(t(wind3)), c(t(wind4)))

# Now separating out into "training" and "testing" data set
frac <-
  4 / 5 # fraction to use for training data set (must split data set evenly)
n.sites.train <- n.sites * frac
n.sites.test <- n.sites * (1 - frac)
T <- T
T2 <- T
n.train <- n * frac
n.test <- n * (1 - frac)
y.train <- y[1:(nrow(y) * frac), ]
y.test <- y[((nrow(y) * frac) + 1):nrow(y), ]
sites.train <- sites[1:(nrow(sites) * frac), ]
sites.test <- sites[((nrow(sites) * frac) + 1):nrow(sites), ]
vegHt.train <- vegHt[1:(length(vegHt) * frac)]
vegHt.test <- vegHt[((length(vegHt) * frac) + 1):length(vegHt)]
wind.train <- wind[1:(nrow(y) * frac), ]
wind.test <- wind[((nrow(y) * frac) + 1):nrow(y), ]
total.visits <- matrix(rep(J, n.sites * frac * T), ncol = T)

jags_data = list(
  n.sites = n.sites.train,
  T = T,
  n = n.train,
  y = y.train,
  sites = sites.train,
  vegHt = vegHt.train,
  wind = wind.train,
  T2 = T,
  sites2 = n.test,
  total.visits = total.visits,
  vegHt2 = vegHt.test,
  wind2 = wind.test,
  y2 = y.test
)

# save the model file
sink("SC_mod.txt")

cat(
  "
  model {

    ####################
    #Priors
    ####################

    #detection parameters
    alpha[1] ~ dt(0, 1/6.25, 1)    #cauchy(0, 2.5) priors
    alpha[2] ~ dt(0, 1/6.25, 1)

    #occupancy covariates
    beta[1] ~ dt(0, 1/6.25, 1)
    beta[2] ~ dt(0, 1/6.25, 1)

    ####################
    #Likelihood
    ####################

    #likelihood for latent states

    for(s in 1:n.sites){                        #n.sites = no. of field sampling locations
      for(t in 1:T){                        #T = no. of years sampled
        z[s,t] ~ dbern(psi[s,t])            #z[s,t] = latent occupancy state at site s in year t

        #site-specific occupancy probability
        l.psi[s,t] <- beta[1] + beta[2]*vegHt[s]
        psi[s,t] <- exp(l.psi[s,t])/(1+exp(l.psi[s,t]))
      }
    }

    #likelihood for observations in vertical format (not all sites sampled every year: i = ith observation in year t)

    for(t in 1:T){
      for(i in 1:n[t]){                    #n = vector of number of sites sampled by year

        #sampling-event specific detection probability
        l.p[i,t] <- alpha[1] + alpha[2]*wind[i,t]
        p[i,t] <- exp(l.p[i,t])/(1+exp(l.p[i,t]))
        eff.p[i,t] <- z[sites[i,t],t]*p[i,t]        #sites = matrix of site numbers (s indices from above) for observation i in year t
        y[i,t] ~ dbern(eff.p[i,t])
      }
    }

    #Calculating integrated likelihood and log-posterior predictive scores for testing data set

     for(t in 1:T2){                            # T = no. of years sampled in testing data set
         for(s in 1:sites2[t]){                    # sites2 = vector of no. of sites in testing data by year
                 l.psi2[s,t] <- beta[1] + beta[2]*vegHt2[s]
                 psi2[s,t] <- exp(l.psi2[s,t])/(1+exp(l.psi2[s,t]))
             for(v in 1:total.visits[s,t]){

                 #total.visits = matrix with total number of visits for site s in year t in testing data
                 #calculate p for each visit as a function of covariate values and regression coefficients

                 l.pnew[s,v,t] <- alpha[1] + alpha[2]*wind2[s,v,t]
                 p.new[s,v,t] <- exp(l.pnew[s,v,t])/(1+exp(l.pnew[s,v,t]))
                 prob1[s,v,t] <- pow(p.new[s,v,t], y2[s,v,t])*pow((1-p.new[s,v,t]),(1-y2[s,v,t]))
                 prob2[s,v,t] <- 1-p.new[s,v,t]
             }

             #see Appendix S1 for integrated likelihood and log-score formulae
             term1[s,t] <- indicator[s,t]*psi2[s,t]*prod(prob1[s,1:total.visits[s,t],t])
             term2[s,t] <- (1-indicator[s,t])*((1-psi2[s,t]) + psi2[s,t]*prod(prob2[s,1:total.visits[s,t],t]))
             y.prob[s,t] <- term1[s,t] + term2[s,t]
             lprob.y[s,t] <- log(y.prob[s,t])
         }
         score.t[t] <- sum(lprob.y[1:sites2[t],t])        #see S1.2
     }
     l.score <- -2*sum(score.t)

  }
"
)

sink()

# Inits
z.wide <- as.matrix(aggregate(y, by = list(sites[, 1]), max)[2:5])
z.wide.train <- z.wide[1:(nrow(z.wide) * frac), ]
inits <- function() {
  list(
    z = z.wide.train,
    alpha = c(-1,-3) + rnorm(2),
    beta = c(-2, 2) + rnorm(2)
  )
}

# Parameters monitored
params <- c("alpha", "beta", "l.score")

# MCMC settings
ni <- 20000
nt <- 10
nb <- 10000
nc <- 3

# Call JAGS from R and summarize posteriors
SCtest <-
  jagsUI::jags(
    jags_data,
    inits = inits,
    params,
    "SC_mod.txt",
    n.chains = nc,
    n.thin = nt,
    n.iter = ni,
    n.burnin = nb,
    parallel = TRUE
  )


kkellner

unread,
Dec 22, 2020, 1:49:31 PM12/22/20
to hmecology: Hierarchical Modeling in Ecology
Because of the way the supplied data 'sites2' is being calculated, JAGS is interpreting it as a float instead of an integer (though it looks like an integer in R), and thus not allowing it to be used as an index. You can fix this by wrapping sites2 in as.integer() when creating the input data:

jags_data = list(
  n.sites = n.sites.train,
  T = T,
  n = n.train,
  y = y.train,
  sites = sites.train,
  vegHt = vegHt.train,
  wind = wind.train,
  T2 = T,
  sites2 = as.integer(n.test), #change here
  total.visits = total.visits,
  vegHt2 = vegHt.test,
  wind2 = wind.test,
  y2 = y.test
)

With this fix your model got past this particular error but ran into another because data 'indicator' is missing.

Ken

Mike Allen

unread,
Dec 22, 2020, 4:15:49 PM12/22/20
to kkellner, hmecology: Hierarchical Modeling in Ecology
Thanks! I fixed the integer problem as suggested and added the missing 'indicator' data (updated code pasted below).
Now it is hung up on this line:" prob1[s,v,t] <- pow(p.new[s,v,t], y2[s,v,t])*pow((1-p.new[s,v,t]),(1-y2[s,v,t]))" with the error:
"Error in checkForRemoteErrors(val) :
  3 nodes produced errors; first error: RUNTIME ERROR:
Compilation error on line 58.
Dimension mismatch taking subset of y2
"

The matrix y2 in jags_data looks to be the correct dimensions to me (390 x 4). Any further help in debugging this example script would be much appreciated!

Mike
n.sites.train <- as.integer(as.character(n.sites * frac))
n.sites.test <- as.integer(as.character(n.sites * (1 - frac)))

T <- T
T2 <- T
n.train <- as.integer(as.character(n * frac))
n.test <- as.integer(as.character(n * (1 - frac)))

y.train <- y[1:(nrow(y) * frac), ]
y.test <- y[((nrow(y) * frac) + 1):nrow(y), ]
sites.train <- sites[1:(nrow(sites) * frac), ]
sites.test <- sites[((nrow(sites) * frac) + 1):nrow(sites), ]
vegHt.train <- vegHt[1:(length(vegHt) * frac)]
vegHt.test <- vegHt[((length(vegHt) * frac) + 1):length(vegHt)]
wind.train <- wind[1:(nrow(y) * frac), ]
wind.test <- wind[((nrow(y) * frac) + 1):nrow(y), ]
total.visits <- matrix(rep(J, n.sites.test*T), ncol = T)

z.wide <- as.matrix(aggregate(y, by = list(sites[, 1]), max)[2:5])
z.wide.train <- z.wide[1:(nrow(z.wide) * frac), ]
z.wide.test <- z.wide[((nrow(z.wide) * frac) + 1):nrow(z.wide),]


jags_data = list(
  n.sites = n.sites.train,
  T = T,
  n = n.train,
  y = y.train,
  sites = sites.train,
  vegHt = vegHt.train,
  wind = wind.train,
  T2 = T,
  sites2 = as.integer(n.test),
  total.visits = total.visits,
  vegHt2 = vegHt.test,
  wind2 = wind.test,
  y2 = y.test,
  indicator = z.wide.test
inits <- function() {
  list(
    z = z.wide.train,
    alpha = c(-1,-3) + rnorm(2),
    beta = c(-2, 2) + rnorm(2)
  )
}

# Parameters monitored
params <- c("alpha", "beta", "l.score")

# MCMC settings
ni <- 20000; nt <- 10; nb <- 10000; nc <- 3

# Call JAGS from R and summarize posteriors
SCtest <-
  jagsUI::jags(
    jags_data,
    inits = inits,
    params,
    "SC_mod.txt",
    n.chains = nc,
    n.thin = nt,
    n.iter = ni,
    n.burnin = nb,
    parallel = TRUE
  )
--
*** Three hierarchical modeling email lists ***
(1) unmarked: for questions specific to the R package unmarked
(2) SCR: for design and Bayesian or non-bayesian analysis of spatial capture-recapture
(3) HMecology (this list): for everything else, especially material covered in the books by Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2016, 2020)
---
You received this message because you are subscribed to a topic in the Google Groups "hmecology: Hierarchical Modeling in Ecology" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hmecology/myskwk3AV2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hmecology+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hmecology/4bc12b26-842d-497d-8de2-30605d7745bbn%40googlegroups.com.

kkellner

unread,
Dec 23, 2020, 12:28:54 PM12/23/20
to hmecology: Hierarchical Modeling in Ecology
The line of code with the error is expecting y2 to have 3 dimensions (y2[s,v,t]) but as you said it only has 2 in your data. Based on a quick look at the code it seems like your input data is organized in a long/vertical form [site * occasion, year] but the "calculating integrated likelihood" section of the JAGS code is expecting things organized as an array [site, occasion, year]. I'm not sure which way of organizing is the way you want it but I'd go back and check through each line of the JAGS model and make sure the dimensions are what you expect them to be.

Ken

Reply all
Reply to author
Forward
0 new messages