Hi,
I have a problem where I want to sample from a distribution that is loosely speaking a "two-sided Gaussian" -- it is composed of the left half of a gaussian of width sigma2 and the right half of a gaussian of width sigma 1. The right half is to be scaled so that its peak is the same as that of the left. So the distribution looks like a Gaussian that is asymmetric about its peak.
I think a way to do this is to use a mixture of two truncated Gaussians with mixture fractions equal to sigma1/(sigma1+sigma2) and sigma2/(sigma1+sigma2)
The distribution for the left side is y ~ normal(mu,sigma1) T[,mu], so that it has support only on the left side of the peak
The distribution for the right side is y ~ normal(mu,sigma2) T[mu,], so that it has support only on the right side of the peak
I implemented the mixture by drawing a uniform random variable called mixingParam; if it's less than sigma1/(sigma1+sigma2), I draw from the first distribution and otherwise I draw from the second.
The complication is that y and mu and sigma are really vectors.
After reading the sections in the Stan manual about truncated distributions and mixtures, I implemented this in my Stan model.
The PyStan code is included below: I've included the entire Stan model, but the relevant part is in the parameters and model blocks.
The model compiles, but it doesn't sample, returning the error below.
Any help to correctly implement this sampling would be appreciated!
Cheers,
Joe Catanzarite
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-28-9777225edb81> in <module>()
25 nSamplesPerChain = int(round(nTotalPerChain/2))
26 nChains = 5
---> 27 fit = sm.sampling(data=data, iter=nTotalPerChain, chains=nChains, n_jobs=-1)
28 #fit = sm.sampling(data=data, iter=2000, chains=5, init=init, n_jobs=-1)
29 #fit = pystan.stan(model_code=stan_model_occ, data=data, iter=100, chains=2, n_jobs=-1, verbose=True);
/Users/jcatanza/anaconda/lib/python2.7/site-packages/pystan/model.pyc in sampling(self, data, pars, chains, iter, warmup, thin, seed, init, sample_file, diagnostic_file, verbose, algorithm, control, n_jobs, **kwargs)
719 call_sampler_args = izip(itertools.repeat(data), args_list, itertools.repeat(pars))
720 call_sampler_star = self.module._call_sampler_star
--> 721 ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
722 samples = [smpl for _, smpl in ret_and_samples]
723
/Users/jcatanza/anaconda/lib/python2.7/site-packages/pystan/model.pyc in _map_parallel(function, args, n_jobs)
79 n_jobs = None
80 pool = multiprocessing.Pool(processes=n_jobs)
---> 81 map_result = pool.map(function, args)
82 pool.close()
83 pool.join()
/Users/jcatanza/anaconda/lib/python2.7/multiprocessing/pool.pyc in map(self, func, iterable, chunksize)
249 '''
250 assert self._state == RUN
--> 251 return self.map_async(func, iterable, chunksize).get()
252
253 def imap(self, func, iterable, chunksize=1):
/Users/jcatanza/anaconda/lib/python2.7/multiprocessing/pool.pyc in get(self, timeout)
565 return self._value
566 else:
--> 567 raise self._value
568
569 def _set(self, i, obj):
RuntimeError: Rejecting initial value:
Log probability evaluates to log(0), i.e. negative infinity.
Stan can't start sampling from this initial value.
Rejecting initial value:
==========================================
from pystan import StanModel
stan_model_occ = """
functions {
real[,] population_model_insolation(int nxGrid, int nRadiusGrid,
real lnf0, real alpha, real beta, real[] x_grid, real[] planet_radius_grid,
real[] planet_radius_rng, real[] x_rng) {
/* Population inference with an independent power law model
Using modified code above that computes
completeness in the parameter space of [ insolation , planet radius ],
A double power law model for the population.
:param nxGrid: size of insolation grid
:param nRadiusGrid: size of radius grid
:param lnf0: normalization,
:param beta: exponent of insolation power law,
:param alpha: exponent of radius power law
:param x_grid: //1D grid
:param planet_radius_grid: //1D grid
:param x_rng: // lower and upper boundary values of insolation grid
:param radius_rng: // lower and upper boundary values of radius grid
*/
// Declare internal variables
real powerLawNumberDensity[nRadiusGrid,nxGrid];
// Loop over phase space bins, compute number density in each bin,
// from the power law using the input lnf0, alpha and beta parameters
for (j in 1:nxGrid)
for (i in 1:nRadiusGrid) {
powerLawNumberDensity[i,j] <- ( exp( lnf0 ) * ( alpha+1 ) * ( beta+1 ) * ( planet_radius_grid[i]^alpha ) * ( x_grid[j]^beta ) ) / ( ( planet_radius_rng[2]^( alpha+1 ) - ( planet_radius_rng[1]^( alpha+1 ) ) ) * ( ( x_rng[2]^( beta+1 ) ) - ( x_rng[1]^( beta+1 ) ) ) );
}
return powerLawNumberDensity;
}
real lnlike(real alpha, real beta, real lnf0, int nKois, real[] planet_radius_rng,
real[] x_rng, real[,] vol, real[] x_grid, real[] planet_radius_grid,
real[] koi_x, real[] koi_rps_true, int nRadiusGrid, int nxGrid, real e, real[,] new_completeness ) {
/* log likelihood function
:param alpha:
:param beta:
:param lnf0:
:param nKois
:param planet_radius_rng
:param x_rng
:param vol:
:param x_grid:
:param planet_radius_grid:
:param koi_x:
:param koi_rps_true:
:param nRadiusGrid
:param nxGrid
:param e
:param new_completeness
*/
// Declare internal variables
real pop[nRadiusGrid,nxGrid];
real pop1[nRadiusGrid,nxGrid];
real pop2[nRadiusGrid,nxGrid];
real populationDensity[nRadiusGrid-1,nxGrid-1];
real sumExpectedCounts;
real logPop[nKois];
real lnLikelihood;
real sumLogPop;
real lnLikelihood0;
real popModelInsol0[nRadiusGrid,nxGrid];
real popModelInsol[nKois,nKois];
// Compute the population model for a grid over [ insolation, radius ] parameter space
popModelInsol0 <- population_model_insolation(nxGrid, nRadiusGrid, lnf0, alpha, beta, x_grid, planet_radius_grid, planet_radius_rng, x_rng);
// Compute the expected density of detections per bin in the [ insolation, radius ] parameter space,
// based on the population model and the completeness
for (i in 1:nRadiusGrid)
for (j in 1: nxGrid) {
//sum_completeness_0[i,j] <- new_completeness[j,i];
pop[i,j] <- popModelInsol0[i,j] * new_completeness[j,i];
}
// Initialize
sumExpectedCounts <- 0.0;
sumLogPop <-0.0;
for (i in 1:nRadiusGrid-1)
for (j in 1: nxGrid-1) {
// Compute the number density of detections on a grid of bin centers
// Figured out the proper syntax for translation of the Python line below into Stan
// pop <- 0.5*(pop[:-1, :-1] + pop[1:, 1:]) is python syntax from DFM notebook.
// Note that indices start with 1 in Stan
pop1[i,j] <- pop[i,j];
pop2[i,j] <- pop[i+1,j+1];
populationDensity[i,j] <- 0.5*(pop1[i,j]+pop2[i,j]);
// Integrate the planet counts over the phase space volume
sumExpectedCounts <- sumExpectedCounts + (populationDensity[i,j]*vol[j,i]);
}
// Population model evaluated over the list of detected planets
//
popModelInsol <- population_model_insolation(nKois, nKois, lnf0, alpha, beta, koi_x, koi_rps_true, planet_radius_rng, x_rng);
for (k in 1:nKois) {
logPop[k] <- log( popModelInsol[k,k] );
sumLogPop <- sumLogPop + logPop[k];
}
// Combine the two terms to form the log-likelihood function for the Inhomogeneous Poisson Process
lnLikelihood <- sumLogPop - sumExpectedCounts;
// Catch error: if lnLikelihood has bad value, return a very small number
lnLikelihood0 <- if_else( is_inf(lnLikelihood) || is_nan(lnLikelihood) , -1e15, lnLikelihood );
return lnLikelihood0;
}
}
// end function block
data {
// Declare precomputed variables as data
int<lower=1> nKois;
int<lower=1> nRadiusGrid;
int<lower=1> nxGrid;
real x_rng[2];
real planet_radius_rng[2];
real x_grid[nxGrid];
real planet_radius_grid[nRadiusGrid];
real vol[nxGrid-1,nRadiusGrid-1];
real koi_x[nKois];
real koi_rps_obs[nKois];
real koi_rps_err1[nKois];
real koi_rps_err2[nKois];
real koi_rps_obs_lower[nKois];
real koi_rps_obs_upper[nKois];
real e;
real new_completeness[nxGrid,nRadiusGrid];
}
parameters {
// For v1: only compute population level parameter posteriors.
real<lower=-10,upper=5> alpha;
real<lower=-5,upper=5> beta;
real<lower=-5,upper=5> lnf0;
# Ad hoc lower bound constraint on planet radius
real<lower=0.5> koi_rps_true[nKois];
# Mixing parameter
real<lower=0,upper=1> mixingParam;
}
model {
// Hyperpriors
alpha ~ uniform(-10,5);
beta ~ uniform(-5,5);
lnf0 ~ uniform(-5,5);
mixingParam ~ uniform(0,1);
// Sample the planet radius errors
for (n in 1:nKois)
// Draw from either left-side Gaussian with prob f1 or right-side Gaussian with prob f2
// Where mixing weights are
// f1 = koi_rps_err2[n]/(koi_rps_err1[n] + koi_rps_err2[n])
// f2 = koi_rps_err1[n]/(koi_rps_err1[n] + koi_rps_err2[n])
if(mixingParam < koi_rps_err2[n]/(koi_rps_err1[n] + koi_rps_err2[n]) )
// Draw from left-side Gaussian, of width koi_rps_err2
koi_rps_true[n] ~ normal(koi_rps_obs[n],koi_rps_err2[n]) T[,koi_rps_obs[n]];
else
// Draw from right-side Gaussian, of width koi_rps_err1
koi_rps_true[n] ~ normal(koi_rps_obs[n],koi_rps_err1[n]) T[koi_rps_obs[n],];
//koi_rps_true[n] ~ normal(koi_rps_obs[n], koi_rps_err2[n]);
// Sample using lnLikelihood function
increment_log_prob(lnlike(alpha, beta, lnf0, nKois, planet_radius_rng, x_rng, vol, x_grid, planet_radius_grid, koi_x, koi_rps_true, nRadiusGrid, nxGrid, e, new_completeness));
}
"""
# Compiled Stan Model
sm = StanModel(model_code=stan_model_occ)
data = {'new_completeness':new_completeness, 'nKois':nKois,'nRadiusGrid':nRadiusGrid, 'nxGrid':nxGrid,
'x_rng':x_rng, 'planet_radius_rng':planet_radius_rng,'x_grid':x_grid,
'planet_radius_grid':planet_radius_grid,
'vol':vol,'koi_x':koi_x, 'koi_rps_obs':koi_rps, 'koi_rps_err1':koi_rps_err1, 'koi_rps_err2':abs(koi_rps_err2),'e':e,
'koi_rps_obs_lower':koi_rps_obs_lower,'koi_rps_obs_upper':koi_rps_obs_upper}
init = [{'lnf0':-0.1,'alpha':-2.2,'beta':-0.5}]
# Set total number of samples per chain and number of chains
# number of production samples is half of the total number of samples;
# the other half are warmup samples.
nTotalPerChain = 500
nSamplesPerChain = int(round(nTotalPerChain/2))
nChains = 5
fit = sm.sampling(data=data, iter=nTotalPerChain, chains=nChains, n_jobs=-1)
# Return a dictionary of arrays of posterior samples
la = fit.extract(permuted=True)
alpha = la['alpha']
beta = la['beta']
lnf0 = la['lnf0']
a = fit.extract(permuted=False)
print(fit)