Hello, I am trying to run a negative binomial regression and succeed until I try to implement truncation of the criterion variable.
I started with the following simple negative binomial regression:
data {
int<lower=1> N; // number of data points
int<lower=0> y[N]; // observations
int<lower=1> K; // number parameters
matrix[N,K] x; // individual predictors
}
parameters {
vector[K] beta;
real<lower=0.01, upper=50> nb_beta; // beta (inv. scale) parameter of NB
}
model {
for (k in 1:K){
beta[k] ~ normal(0,1);}
nb_beta ~ cauchy(0,1);
for (n in 1:N) {
y[n] ~ neg_binomial(exp(x[n] * beta)*nb_beta,nb_beta);
}
}
This model works fine and converges in less than 10 seconds and within 250 samples for a data set with N = 1000 and K=15 predictors.
Next I tried to adapt the basic model for a truncated outcome variable, where all y >= 3, by changing only the sampling statement from
y[n] ~ neg_binomial(exp(x[n] * beta)*nb_beta,nb_beta);
to
y[n] ~ neg_binomial(exp(x[n] * beta)*nb_beta,nb_beta) T[2,];
However, this led to a few of the following messages:
Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
stan::prob::neg_binomial_log(N4stan5agrad3varE): Shape parameter is inf:0, but must be finite!
If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
And stan does not complete 10% of the samples within 20+ minutes.
When I initialize the sampling with regression weights obtained from the full data set (i.e. outcomes y >= 0), I can fit the model with truncation.
My questions:
- Is my general approach to modeling truncated outcome variable sound?
- Is it to be expected that truncation makes sampling so much harder, or could I improve my model to achieve convergence also without initialization?
Thanks in advance!
Guido
Thanks for taking time to look at this Bob,
(and apologies for my longish reply)
The derivatives might indeed be the problem. When I run a model without truncation the Gradient evaluation takes 0.06 seconds, and when I run a model with truncation the time taken jumps to 0.45 seconds. These numbers are for a hierarchical regression model with N = 33 000, three groups, and K = 17 predictors.
(with cmdstan--compiled with intel MKL, linux redhat, 2.6 GHz intel processors, 8 or 16GB RAM)
Given the large N and mostly only weakly correlated predictors, the model should not generally be hard to estimate, I assume.
The hard constraint on nb_beta (the beta parameter for the negative binomial) is likely not the problem, because the posterior mean is around 0.9.
I have following ideas to speed up the code, maybe you have an opinion about which is most likely to improve speed (the full model is below)?
(a)
Use more informative priors (i.e. normal(0,1) for betas and chauchy(0,1) for variances.
(I know that nb_beta and all all betas are below 1.) I hope this can address one factor Marc mentioned in his discussion of what influences speed, namely how fast the sampler towards the region of the parameter space with high posteriors.
(b)
Speed up calculations by vectorizing (1) (the second factor Marc mentioned).
In particular, I think of splitting the input by group, i.e. I could submit all relevant variables split by group
Ngrp1, Ngrp2, Ngrp3, ygrp1, ygrp2, ygrp3, xgrp1, xgrp2, xgrp,
which would allow to calculate the means as e.g.
mugrp1 <- beta[1] * xgrp1[1]
instead of looping through all N and do mu[n] <- beta[g[n]] * x[n]
(c)
Speed up by vectorizing (2):
Inspired by Marco (Inacio’s) description of a zero-altered NB regression I thought of doing something along these line:
First increment_log_probs gobally as if there were no truncation:
y ~ neg_binomial(alpha, beta)
Then correct for truncation by incrementing again
increment_log_prob(sum(neg_binomial_cdf_log(SC-1, alpha,beta) * is_equalabove_SC))
increment_log_prob(sum(neg_binomial_ccdf_log(SC, alpha,beta) * is_below_SC))
whereby SC is the truncation point and is_equalabove_SC is a vector with zeros for criterion values below the cutoff SC and ones otherwise, so that the sum is calculated only over the relevant rows (the line below is then for below cutoff values).
Somehow I fear that this is a naïve approach, but I think it should be possible to calculate the cdf and ccdf in vectorized form. But this requires that alpha and beta in neg_binomial_ccdf_log(SC, alpha,beta) can be vectors. I didn't try it, but given that the stan reference describes the function as "neg_binomial_cdf(ints n, reals alpha, reals beta)" I hope that the use of plural means that alpha and beta can be vectors.
Lastly, I found that the stan dev branch of includes functions for cdfs and ccdfs for neg_binomilal_2. I have now simply build stan 2.5 after replacing the original neg_binomial_2.hpp with the dev-version. Is this a safe thing to do?
What do you, or anybody else following this discussion, think?
Cheers – Guido
* without going into to many details: In each groups participants over and at/above the truncation/cutoff point are either over or under represented. Think of running a binomial regression to predict number of symptoms in a group that consists to 80% of cases that have more symptoms than the 90th percentile of the population, and 20% of controls that have fewer symptoms.
#################### Model ########################
data {
int<lower=1> N; // number of data points
int<lower=0> y[N]; // observations
int M; // number of groups
int SC; // cut off value for cases (all cases >= cutoff)
int g[N]; // group for individual
int<lower=1> K; // number predictors (including intercept)
int<lower=1> L; // num group predictors
matrix[N,K] x; // individual predictors
matrix[M,L] u; // group predictors
}
parameters {
matrix[K,M] z;
cholesky_factor_corr[K] L_Omega;
vector<lower=0>[K] tau; // prior scale
matrix[L,K] gamma; // group coeffs
real<lower=0.01> nb_beta; // beta (inv scale) parameter of negative binomial
}
transformed parameters {
matrix[M,K] beta;
beta <- u * gamma + (diag_pre_multiply(tau,L_Omega) * z)';
}
model {
vector[N] mu; // expected number of symptoms
tau ~ cauchy(0,1); // orig scale: 2.5
to_vector(z) ~ normal(0,1);
L_Omega ~ lkj_corr_cholesky(2);
to_vector(gamma) ~ normal(0,1); // orig scale: 5
nb_beta ~ cauchy(0,1);
for (n in 1:N) {
mu[n] <- exp(dot_product(x[n],beta[g[n]]));
if (y[n] < SC)
y[n] ~ neg_binomial(mu[n]*nb_beta,nb_beta) T[,SC];
else
y[n] ~ neg_binomial(mu[n]*nb_beta,nb_beta) T[SC-1,];
}
}
thanks Bob,
in had updated my response to reflect what to do when parameters vary by individual. I see you replied to my original responses, but not my updated response (I'm can update response directly on the google groups web page, these updates are apparently not distributed via email).
I'll try out if the approach I describe there can be implemented/improves speed and let you know.
guido
--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/QYBfP4n3N44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.