Issues with right-censored data

[Alexander Jack]

Hi all, 

I am having issues with right-censored data. I have followed the section on how to format censored data from the nimble user guide. I have also provided valid initial values for my censored data so I'm unsure why my mcmc sampler is giving me errors for every single censored data point. I sadly can't share data, but I will share code below.

Thanks in advance for your help!

library(nimble)
library(coda)
nimbCode <- nimbleCode({
shape  ~ dunif(0, 10)
for(i in 1:ninds) {
lambda[i] <- exp(beta0 + beta2*x[i])
censored[i] ~ dinterval(t[i], c[i])
t[i] ~ dweib(shape, lambda[i])
}
beta0 ~ dunif(-50, 50)
beta1 ~ dnorm(0, 1)
beta2 ~ dnorm(0, 1)
})
# Build NIMBLE model
nimbMod <- nimbleModel(
nimbCode,
const = list(ninds = nrow(dat),
 c = dat$c),
data = list(t = dat$daysMonitored,
x = dat$treatment,
censored = dat$censored),
inits = list(shape = 0.5,
 beta0 = 5,
 beta1 = 0,
 beta2 = 0,
 t = dat$daysMonitoredInit)) # initialize censored data with valid values

comp.mod <- compileNimble(nimbMod)
conf.mod <- configureMCMC(comp.mod, monitors = c('beta0', 'beta2', 'shape'),
enableWAIC = TRUE)
mod.mcmc <- buildMCMC(conf.mod)
c.mod <- compileNimble(mod.mcmc)
mcmc.out <- runMCMC(c.mod,
niter=10000,
nburnin=1000,
nchains=4,
thin = 1,
WAIC=TRUE)

post.samples <- mcmc.list(sapply(mcmc.out$samples,
 as.mcmc,simplify=FALSE))
sum <- summary(post.samples)
plot(post.samples, trace=TRUE, density=FALSE)
gelman.plot(post.samples)
autocorr.plot(post.samples)

## posterior summary
summary(post.samples)
## model assessment using WAIC value
mcmc.out$WAIC

# simulate new datasets
deps <- comp.mod$getDependencies(c("beta0", "beta1", "beta2", "shape"),
 downstream = TRUE, self = FALSE)
comp.mod$simulate(deps, includeData = TRUE)

[Chris Paciorek]

hi Alexander,

We're going to need more information here.

- please say specifically what you mean by "giving me errors"

- before you start the MCMC, please check the values of `censored` and `t` in the model are what you expect, in particular that for `t` you have a mix of initial values and actual data values

- before the MCMC, check `nimbMod$isData('t')` and check that is the expected mix of TRUEs and FALSEs.

- before the MCMC, run `comp.mod$calculate('t')` and `comp.mod$calculate('censored')` to check you get numerical values and not NA or NaN. If not, you haven't initialized something, which is not always a problem, but often can be.

If a bit of discussion of the above doesn't reveal the problem, then given you can't share the data, can you give us a reproducible example with simulated data?

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/nimble-users/1192a94f-2855-4b85-ac8e-125fb50af8adn%40googlegroups.com.

[Alexander Jack]

Hi Chris,

Yes sorry that was vague. 

For example, every node where I have asked nimble to impute censored data gives me an error such as 

warning: logProb of data node censored[2117]: logProb is -Inf.

in addition to errors like this:

MCMC sampling encountered a log probability density value of infinity. Results of sampling may not be valid.

For nodes where I am not asking NIMBLE to impute data it works fine.

I also checked my inits and they gave me the all clear

Note] All model variables are initialized.

When I ask the model to calculate my response (nimbMod$calculate('t') ) I get back a normal log probability ( -8518494 )

But when I ask it to calculate my column corresponding to whether or not the data are censored (a binary column) nimbMod$calculate('censored') I get back NA  so I now am wondering if this binary column might be my issue, but I'm unsure why that would be as it's just a vector of 0's and 1's. 

[Chris Paciorek]

If it's the logProb of `censored` that is giving invalid values, it must be that either `t` or `c` in the model are causing problems. 

1. Please check the current value of `t` in the model. Perhaps `dat$daysMonitored` and `dat$daysMonitoredInit` are somehow inconsistent/invalid.

2. Since `c` is set via `constants`, there is no 'c' variable in the model. But do check again that `dat$c` is what you expect.

You can probably focus on item 2117 as an example: `comp.mod$t[2117]`,  `comp.mod$censored[2117]`, `dat$c[2117]`

To view this discussion visit https://groups.google.com/d/msgid/nimble-users/526f8869-43ae-4cdf-8bc3-843a74d23522n%40googlegroups.com.