Threading evaluation of deterministic

[Kyle Foreman]

Hello all

There was a post on here a while ago about threading deterministics to
speed them up ( http://groups.google.com/group/pymc/browse_thread/thread/a2a6f8ecf9e76d10/
). I've gotten it to work (using Anand's Brownian covariance function
as a template: https://github.com/pymc-devs/pymc/blob/master/pymc/gp/cov_funs/brownian.py
).

In my case the overhead is high enough that it actually slows things
down when using 8 threads, but I thought I'd put the code snippet here
in case others find it useful:

# set up a list of MVNs
pi_samples = [mc.MvNormalCov('pi_%s'%r,
np.zeros(sample_points.shape[0]), C,
value=np.zeros(sample_points.shape[0])) for r in regions]

# this function takes the MVNs and then evaluates and
transforms them
def find_pi_grid(mvn, r, pi):
interpolator = interpolate.bisplrep(x=sample_points[:,0],
y=sample_points[:,1], z=mvn, xb=ages[0], xe=ages[-1], yb=years[0],
ye=years[-1], kx=kx, ky=ky)
pi_grid = interpolate.bisplev(x=ages, y=years, tck=interpolator)
t = [t_index[data.year[j]] for j in r_index[r][0]]
a = [a_index[data.age[j]] for j in r_index[r][0]]
pi[r_index[r][0]] = pi_grid[a,t]

# the deterministic uses map_noreturn to evaluate pi for each
r
@mc.deterministic
def pi(pi_samples=pi_samples):
pi = np.zeros(data.shape[0])
threadpool.map_noreturn(find_pi_grid, [(pi_samples[r], r, pi) for r
in range(len(regions))])
return pi

In this way evaluating the set of MVNs is parallelized across threads
(vs looping through each to build up the deterministic sequentially).
I imagine if you were dealing with more process intensive
subcomponents (e.g. much larger covariance matrices in the MVN or a
more complex transformation?) this might help speed things up.