Re: Using MVN as surrogate posterior fails

[Emily Fertig]

Hi Jalal,

`build_asvi_surrogate_posterior` currently only works for surrogate posteriors defined as TFP JointDistributions (for ASVI examples, see the TFP release notes). To fit an MVN surrogate posterior, you're better off defining a trainable MVN directly, like this:

```

initial_loc = [100.1, 100.1]

initial_scale_tril = 4.*tf.eye(2)

trainable_loc = tf.Variable(initial_loc)

trainable_scale = tfp.util.TransformedVariable(

initial_value=initial_scale_tril,

bijector=tfb.FillScaleTriL()

)

surrogate_posterior = tfd.MultivariateNormalTriL(

loc=trainable_loc, scale_tril=trainable_scale

)

```

In your example, you'd be fitting the MVNTriL surrogate directly to the MVNTriL prior. For a more standard VI example (where you have a prior over a latent variable, and some noisy observations, and you want to use VI to approximate the posterior of the latent conditioned on the observations) see the notes/changes I made in your code below. This blog post has more info on VI in TFP.

Hope this helps!

Emily

```

tfb = tfp.bijectors

tfd = tfp.distributions

####################################

prior123=tfd.MultivariateNormalTriL(loc=[110.5,105.6],scale_tril=tf.linalg.cholesky([[20.,5.],[5.,20.]]))

# Say this is the true value of `z`...

ground_truth = prior123.sample()

# ... and you have 20 noisy observations of it, and you want to use VI to

# approximate the posterior distribution of `z` conditioned on the observations.

obs = tfd.MultivariateNormalDiag(ground_truth, [1., 1.]).sample(20)

# The likelihood expresses the (log) probability of your observations,

# conditioned on `z`.

likelihood = lambda z: tf.reduce_sum(

tfd.MultivariateNormalDiag(z, [1., 1.]).log_prob(obs))

# The target model is log p(z, obs) = log p(z) + log p(obs|z)

def func123(z):

return prior123.log_prob(z) + likelihood(z)

####################################

# Define a trainable location and scale for the MultivariateNormalTriL

# surrogate posterior.

trainable_loc = tf.Variable([100.1,100.1])

trainable_scale = tfp.util.TransformedVariable(

initial_value=4.*tf.eye(2),

bijector=tfb.FillScaleTriL()

)

surrogate_posterior = tfd.MultivariateNormalTriL(

loc=trainable_loc, scale_tril=trainable_scale

)

```

On Fri, May 7, 2021 at 6:54 AM jalal haghigh <jalal.ha...@gmail.com> wrote:

Dear TFP users,

I am trying use variational inference in case of MVN distribution, but seems missing something and it doesn't work  just initial values are returned. If I use batch of Normal distribution (for variational distribution) the code works. Can anybody please guide me to be able to use MVN with full covariance as surrogate posterior. Thanks.

https://colab.research.google.com/drive/1Yv5ye7YT322d9F-Umz9SjT3h88HmB6iu?usp=sharing

--
You received this message because you are subscribed to the Google Groups "TensorFlow Probability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfprobabilit...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfprobability/b694b1ae-5375-448f-b8db-433bad3ec4a6n%40tensorflow.org.

[jalal haghigh]

Very helpful. Thank you. Thanks for detailed and step by step answer. Thanks again.