14 views

Skip to first unread message

Nov 15, 2022, 4:25:57 PM11/15/22

to TensorFlow Probability

I am attempting to fit a model with VI. Here's the model:

```

def gen_lognorm_model(y):

"""Based on Heirarchical 8 Schools model.

This is a pooling model with a learned weight parameter, tau, that weights mean y and y.

See TFP eyample https://www.tensorflow.org/probability/eyamples/Eight_Schools"""

y, = tuple(tf.cast(y, tf.float32) for y in (y, )) # somewhat complicated, but ready to be expanded for further data.

mean_loc = tf.math.log(tf.math.reduce_mean(y))

mean_scale = tf.math.reduce_std(y)

return tfd.JointDistributionSequential([

tfd.LogNormal(loc=mean_loc, scale=1.), # y_mean

tfd.LogNormal(loc=tf.math.log(.5), scale=.2), # tau

tfd.Independent( # lognorm_y

tfd.LogNormal(loc=tf.math.log(y), scale=1.),

reinterpreted_batch_ndims=1),

tfd.Independent(

# tfd.LogNormal(loc=tf.ones_like(y), scale=.1),

tfd.Deterministic(tf.ones_like(y)),

reinterpreted_batch_ndims=1), # est_scale

# tfd.Deterministic(tf.ones_like(y)),

lambda est_scale, lognorm_y, tau, y_mean: ( # est_y

tfd.Independent(

tfd.LogNormal(

loc=((y_mean[..., tf.newaxis] + lognorm_y * tau[..., tf.newaxis])/(tau[..., tf.newaxis] + 1.)),

scale=est_scale

),

reinterpreted_batch_ndims=1

))

])

"""Based on Heirarchical 8 Schools model.

This is a pooling model with a learned weight parameter, tau, that weights mean y and y.

See TFP eyample https://www.tensorflow.org/probability/eyamples/Eight_Schools"""

y, = tuple(tf.cast(y, tf.float32) for y in (y, )) # somewhat complicated, but ready to be expanded for further data.

mean_loc = tf.math.log(tf.math.reduce_mean(y))

mean_scale = tf.math.reduce_std(y)

return tfd.JointDistributionSequential([

tfd.LogNormal(loc=mean_loc, scale=1.), # y_mean

tfd.LogNormal(loc=tf.math.log(.5), scale=.2), # tau

tfd.Independent( # lognorm_y

tfd.LogNormal(loc=tf.math.log(y), scale=1.),

reinterpreted_batch_ndims=1),

tfd.Independent(

# tfd.LogNormal(loc=tf.ones_like(y), scale=.1),

tfd.Deterministic(tf.ones_like(y)),

reinterpreted_batch_ndims=1), # est_scale

# tfd.Deterministic(tf.ones_like(y)),

lambda est_scale, lognorm_y, tau, y_mean: ( # est_y

tfd.Independent(

tfd.LogNormal(

loc=((y_mean[..., tf.newaxis] + lognorm_y * tau[..., tf.newaxis])/(tau[..., tf.newaxis] + 1.)),

scale=est_scale

),

reinterpreted_batch_ndims=1

))

])

```

In the final distribution, `est_scale`, the commented out LogNormal distribution was resulting in a posterior that seemed pretty off. I set the scale parameter of the final functional distribution(`est_y`) to `1.` and ran again, which gave different results, though not better. In interest of exploring possible ways to model this problem, I decided to set `est_scale` with the tfd.Deterministic layer that is present now. Howevver, this (contrary to my expectation) gives results that are different from setting the scale in `est_y` to 1. I'm unclear on why this is. Given that the overall behavior of this model is not meeting my expectations and that changing from `scale=1.` to `scale=tfd.Determnisitc` is also giving (to me) unexpected results, I'm concerted that I've misunderstood something important. Why would using a deterministic distribution give results different from setting the scale to the same value as a float?

Thanks,

Lenhart

Nov 22, 2022, 11:10:29 AM11/22/22

to TensorFlow Probability, lenhart...@gmail.com

Hey -- sorry for the late response.

Can you be more specific about the unexpected behavior? The model looks ok to me, but with two lines making me nervous:

- For the `lognorm_y` parameter, you take the log of the passed in values `y`. Is that intentional? In the classic eight schools dataset, `y = [28, 8, -3, 7, -1, 1, 18, 12]`, so your `lognorm_y` would contain `nan`'s once you take the logarithm.

- For the likelihood (`est_y`), do you want to take a log of the location? Otherwise this could be fairly unstable.

-C

Reply all

Reply to author

Forward

0 new messages

Search

Clear search

Close search

Google apps

Main menu