Re: hw 4 questions
Percy Liang
> * In the formula for p(y = 0 | x), should \xi have "j" as a
> subscript? Otherwise, it seems that each coordinate of x
> exponentiates the entire vector \xi.
Yes, it's a typo. It should be \xi_j.
> * The hint suggests introducing variables latent variables, and it
> denotes these variables with two subscripts. Does this mean that we
> are to introduce m * n latent variables in total? Wouldn't it make
> sense to reuse the same latent variable to train the same latent
> variable on every example, rather than to have one latent variable
> per example per binary input?
Mathematically, it's cleaner to think of there being m*n variables
total (think about all the random variables in the model).
In code, you can reuse the same m variables for each example.
> * If each iteration of the EM algorithm is to run in O(m) time,
> does that mean that n can be considered fixed? Or, should each
> iteration of the EM algorithm only process a single example rather
> that make one pass through all of the examples?
The algorithm should spend O(m) time on each example, which implies O
(mn) time per EM iteration.
> Problem #2:
> * I attempted to compute f*(u) by maximizing ux + log(x) with
> respect to x, per the second formula in the "dual relationship".
> However, when I substitute this equation for f*(u) into the first
> formula of the dual relationship and maximize with respect to u,
> that equation is not -log(x). Where have I gone wrong? Is some sort
> of iterative maximization process necessary? Does sup_{x} mean
> "maximum value of x, but subject to the constraints in the other
> equation"?
You should get f(x) back. The sup is unconditional.
>
> Problem #3:
> * Is m the number of *pairs* of exclusively male twins, or the
> number of persons that are male and are twins? If it's the former,
> then would m + f + b be the total number of pairs of twins in the
> sample? If it's the latter, then does the participation of one twin
> imply that his/her twin is also included in the study?
The former. Always consider pairs (of twins), never single persons.
-Percy