COVARIANCE IN MULTIVARIATE NORMAL DISTRIBUTION FUNCTION

[tarun]

Covariance of muti-dimensional normal distribution can be negative , but when i want to use mvnpdf it is giving an error ' covariance elements must be ositive values'.Can u tell me whether i went wrong somewhere

[Peter Perkins]

If we're talking here about the MVNPDF function from the Statistics
Toolbox, then I'm not aware that it's ever had an error message that
says that. It does require the cov matrix to be positive definite, e.g.,

>> mu = [1 -1]; Sigma = [-1 -.4; -.4 .3];
>> mvnpdf([0 0], mu, Sigma)
??? Error using ==> mvnpdf at 136
SIGMA must be symmetric and positive definite.

but that's not the same as requiring all the covariances (the
off-diagonal elements) to be positive, e.g.,

>> mu = [1 -1]; Sigma = [1 -.4; -.4 .3];
>> mvnpdf([0 0], mu, Sigma)
ans =
0.071323

MVNPDF requires the cov matrix to be positive definite, i.e., its
eigenvalues must all be positive. There is no such thing as a
covariance matrix with negative eigenvalues, and the density does not
exist in the usual sense if some of the eigenvalues are zero.

Hope this helps.

[tarun]

I have calculated a covariance of a set of 13-dimension vectors. Now when i have passed this matrix as a parameter into the mvnpdf it is giving an error message that 'covariance must be a positive-definite matrix.

Can you suggest me how to create a mvnpdf for a given 13-dimensional vectors.

[Peter Perkins]

The PDF for the a multivariate normal distribution with a semi-defintie
cov matrix does not exist in the usual sense. It's analogous to asking
for the PDF of a normal distribution with mean 1 and variance 0.

You can do one of two things:

1) remove some of your variables

2) recognize that your cov matrix is only an estimate, and that the real
cov matrix is not semi-definite, and find some better way of estimating
it. But with 13 variables, you're estimating 13*14/2 covariances, and
you need a lot of data to do that accurately.

[tarun]

Peter Perkins <Peter....@MathRemoveThisWorks.com> wrote in message <ieag2p$bsq$1...@fred.mathworks.com>...

Thanks a lot Peters for your suggestions,

But I didn't get your second point . Can you please elaborate.

I have to deal with sample of size 25 which are of 13-dimensions and had to create a multivariate normal distribution with those 25 values in matlab and get the probability for those values.So I used mvnpdf.

[Peter Perkins]

On 12/15/2010 9:03 AM, tarun wrote:
> I have to deal with sample of size 25 which are of 13-dimensions and had
> to create a multivariate normal distribution with those 25 values in
> matlab and get the probability for those values.So I used mvnpdf.

1) Q: How many observations do you have? A: 25
2) Q: How many covariances do you need to estimate? A: 91

You can't possibly expect to get a reasonable estimate of the covariance
matrix. By way of anaolgy, how would you estimate the covariance
between these two vectors:

x = [1]
y = [2]

It can't be done, unless you somehow bring other information into th4e
problem.

[Greg Heath]

If the covariance is not positive definite,
then the vectors actually lie in a lower
dimensional subspace, i.e., the data is not
really 13 dimensional.

The true dimensionality is given by

covX = cov(X);
truedim = rank(covX)

You can recover a posdef covariance by projecting
the data on the orthogonal basis spanned by
the eigenvectors of covX corresponding to
positive eigenvalues.

Hope this helps.

Greg