singular covariance matrix Gaussian pdf question
zl2k
I searched the group and found some message dealing with the singular
covariance matrix problem. One of which is using pseudo inverse based
on svd to generate pseudo inversed covariance matrix for the singular
matrix. My question, in order to calculate the Gaussian pdf, how
should I deal with the determinant of the singular matrix which is 0
and will give inf in gaussian pdf? Thanks for comments.
zl2k
Ray Koopman
Consider first the bivariate normal case in which the variables are
perfectly correlated. Then the density is zero everywhere except on
some line, where it is univariate normal.
The n-variate case is analogous. If the covariance matrix has rank k
then the density is zero everywhere except in some k-dimensional
hyperplane, where it is k-variate normal.
Let x be such an n-vector, with mean vector m and covariance matrix S.
Let P be the n by k matrix of unit-length eigenvectors corresponding
to the nonzero eigenvalues of S, and let Q be any n by (n-k) matrix
whose columns span the null space of S. Let y = P'(x-m), and let z =
Q'(x-m). If z'z > 0 then x is not in the hyperplane, and the density
is zero. If z = 0 then x is in the hyperplane, where its conditional
density is that of y, which is k-variate normal with mean vector 0
and covariance matrix P'SP (which is diagonal; the variances are the
nonzero eigenvalues of S).
vontr...@cs.com
It's the product of the non-zero singular values. Ray gave a good
geometric explanation of why this works. It's the determinant of the
full support basis vectors of the covariance matrix.
Greg Heath
Go back to that thread and find the sentence
Now replace inv(C) with C# and det(C) with the product s1*s2*...sr.
Hope this helps.
Greg