HW5 prob1

[Dav Clark]

Problem 1 seems a little underconstrained... are we supposed to assume
some distributional form on p and/or q? It seems like as stated, the
problem is trivial - we just sample from the p(x_i | x_-i), which is
given, and combine those samples to make our q_i's...

Am I missing something?

DC

[Joseph Austerweil]

hey

when is the hw due? i haven't started it yet. is it hard?

=joe

[Joseph Austerweil]

sorry, i meant to send that to dav.
=joe

[Alekh Agarwal]

It'll be useful to all though I guess if Dav replies to the list :)

[Percy Liang]

> Problem 1 seems a little underconstrained... are we supposed to assume
> some distributional form on p and/or q? It seems like as stated, the

You don't need to, but you can think about p and q being exponential
families if you'd like since that's what was done in class.

> problem is trivial - we just sample from the p(x_i | x_-i), which is
> given, and combine those samples to make our q_i's...

But what distribution would you take the x_-i's from? The optimal
q_i should have a relatively simple expression which you should
arrive at with some algebraic manipulation of the global objective
function (min KL).

[Dav Clark]

On Nov 23, 11:11 am, Percy Liang <percyli...@gmail.com> wrote:

> But what distribution would you take the x_-i's from? The optimal
> q_i should have a relatively simple expression which you should
> arrive at with some algebraic manipulation of the global objective
> function (min KL).

I guess that gets at the heart of my question... from the reading in
the MCMC paper, as well as the presentation in class, it seems that
*proper* Gibb's sampling simply maintains a state vector of the most
recenty sampled values for each x_j. And these are the x_-i that
would get used in each sampling step.

But given what you said, I take this to mean that at each step I use
the total knowledge I have about the already sampled x_-i values to
generate my new x_i sample.

Thanks for the answer! Hope you're enjoying the long weekend.

DC

[Percy Liang]

> I guess that gets at the heart of my question... from the reading in
> the MCMC paper, as well as the presentation in class, it seems that
> *proper* Gibb's sampling simply maintains a state vector of the most
> recenty sampled values for each x_j. And these are the x_-i that
> would get used in each sampling step.
>
> But given what you said, I take this to mean that at each step I use
> the total knowledge I have about the already sampled x_-i values to
> generate my new x_i sample.

For sampling, you use your last sampled values of x_-i.

But for mean-field, you need to use q_-i(x_-i) somehow.

-Percy