jdm
unread,Jan 1, 2012, 6:36:33 PM1/1/12You do not have permission to delete messages in this group
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to
The following claim featured in a research paper I've been studying -
however, no proof accompanied it beyond a statement that the
approximation could be obtained using Taylor series at order 2 - and
it wasn't clear what the variable was supposed to be or around which
point.
Let p and q be discrete probability distributions of random variables
taking on values from a set with M+1 elements;
p=(p_0, ..., p_M)
(p_i = P(random variable with distribution p is equal to i))
Likewise, q=(q_0, ..., q_M)
Where p and q are close - defined as |p_{i} - q_{i}| << q_{i} \forall
i - let e_i denote the value (p_i - q_i).
Then, according to the paper, D(p||q) \approx D(q||p) \approx sum_{i=0}
^{M}(e_{i}^{2}/q_{i})/2.
I haven't been able to verify this approximation for myself, as I
stated, and if anyone reading this can help (even by arguing that the
approximation isn't in fact valid) it would be much appreciated!
Many thanks,
James McLaughlin.