Problems in high dimensional GMM

[paul stridence]

Any one interested in high dimensional Gaussian Mixture Model? I'm
confused about the computation of the prior probabilty P(x | \theta),
where '\theat' denotes parameters here.

In high dimensional spaces the Gaussian covariance matrix is singular,
so i have to use the nonzero eigenvalues and corresponding
eigenvectors to approximat the determinant and the inverse of the
matrix repectively. But, P(x | \theat) is unstable in this case that
it may be either zero or infinity since the denominator of the
Gaussian distribution varies drastically. Therefore the clustering
precesion is often unaccessibly low.

So is there any better solutions for this? I have actually read the
papers 'EM in the High-Dimensional spaces ' and 'Probabilistic Visual
Learning for Object Representation' but got completely poor results
contrary to that declared in both of them, so i got mistakes anywhere?

Any suggestion is grateful.

Best,

Stridence

[hulijo]

Dear Paul,
GMMs are tricky to use in high-dimensional spaces. So first of all,
are you using diagonal covariance matrices? Training of full
covariance matrices requires a lot of data. Second of all, what about
your initialization - are you using KNN or LBG initialization? I would
suggest to start with a small 2d problem where you can actually
visualize your results. Use only diagonal covariance matrices and LBG
initialization. Then you'll see what happens.

Regarding your question, I think you have a problem with
singularities. So, you should use some variance flooring.

Regards,

Vito