Multivariate UNIFORM distribution
John Williams
Dejanews indicates that someone asked this question about a year ago on
sci.math.research, but no "satisfactory" responses were recorded. So
here it comes again.
I'm looking for a way to generate samples from a multivariate uniform
distribution with a specified covariance matrix.
For the univariate case it's easy, the variance is equal to (a-b)^2/12.
That would be fine if I wanted independent samples, however in general I
have a 3x3 positive definite covariance matrix, with non zero
off-diagonal terms.
For the multivariate normal it's easy, just multiply a vector of
independent samples drawn from N(0,1) by the cholesky decomposition of
the covariance matrix. Is there a similar trick for the uniform
distribution?
Thanks,
John W.
--
,---------------------------------------.
| John Williams |
| Space Centre for Satellite Navigation |
| QUT, Brisbane, Australia |
| Email: ja.wi...@student.qut.edu.au |
| Ph : (+61 7) 3864 2458 |
| Fax : (+61 7) 3864 1361 |
`---------------------------------------'
Paige Miller
> I'm looking for a way to generate samples from a multivariate uniform
> distribution with a specified covariance matrix.
>
> For the univariate case it's easy, the variance is equal to (a-b)^2/12.
> That would be fine if I wanted independent samples, however in general I
> have a 3x3 positive definite covariance matrix, with non zero
> off-diagonal terms.
>
> For the multivariate normal it's easy, just multiply a vector of
> independent samples drawn from N(0,1) by the cholesky decomposition of
> the covariance matrix. Is there a similar trick for the uniform
> distribution?
I suggest you also post to sci.stat.math
Here's my opinion...generate independent uniform RVs with univariate
uniform distribution and variance 1, and call this matrix X. Then, if
your desired covariance matrix is called S, the following transformation
Y = X * (S**(1/2))
creates matrix Y which has the desired variance-covariance matrix S.
Note: (S**(1/2)) is the cholesky decomposition of S. Thus
variance(Y) = variance (X * S**(1/2)) = variance (X) *
variance(S**(1/2))
= I * S = S
I have used this trick many times to generate multivariate normal
distributions with the proper covariance matrix, I see no mathematical
reason why it would not work because you have a uniform distribution.
--
+---------------------------------+------------------------------------+
| Paige Miller | "It's nothing until I call it" |
| Eastman Kodak Company | Bill Klem |
| Pai...@kodak.com | NL Umpire |
+---------------------------------+------------------------------------+
| The opinions expressed herein do not necessarily reflect the |
| views of the author nor the views of the Eastman Kodak Company. |
+----------------------------------------------------------------------+
Michael Cohen
: John Williams wrote:
:
: > I'm looking for a way to generate samples from a multivariate uniform
: > distribution with a specified covariance matrix.
:
: Here's my opinion...generate independent uniform RVs with univariate
: uniform distribution and variance 1, and call this matrix X. Then, if
: your desired covariance matrix is called S, the following transformation
:
: Y = X * (S**(1/2))
:
: creates matrix Y which has the desired variance-covariance matrix S.
: Note: (S**(1/2)) is the cholesky decomposition of S. Thus
:
: = I * S = S
But Y need not be uniform. (I don't think there is an easy answer to this
one.)
:
: I have used this trick many times to generate multivariate normal
: distributions with the proper covariance matrix, I see no mathematical
: reason why it would not work because you have a uniform distribution.
:
Normality is preserved by linear transformations; uniformity is not.
--
Michael P. Cohen home phone 202-232-4651
1615 Q Street NW #T-1 office phone 202-219-1917
Washington, DC 20009-6310 office fax 202-219-2061
mco...@cpcug.org
David desJardins
> I'm looking for a way to generate samples from a multivariate uniform
> distribution with a specified covariance matrix.
What do you mean by "multivariate uniform distribution"? If you just
mean a multivariate distribution projects to the uniform distribution in
each variable, then your problem seems badly underdetermined.
For example, here are some very different two-variable distributions all
with cov(x,y)=1/24:
a) Generate x uniform in [0,1], then set y = 2x - [2x].
b) Generate y uniform in [0,1], then set x = 2y - [2y].
c) Generate x uniform in [0,1], then:
if 0 <= x <= 4^{-1/3}, set y = 4^{-1/3} - x,
if 4^{-1/3} < x <= 1, set y = x.
It doesn't seem hard to see that there will be an uncountable number of
distributions all with the same covariance matrix. You have a complete
multivariate distribution to play with, and only a few constraints.
David desJardins
Paul Fackler
below. The Cholesky idea is not a good one as it does not preserve
uniformity and indeed may result in variate outside the desired support.
Hope this helps
Paul
I wrote a paper on theis topic that appeared a number of years ago in the
American Journal of Agricultural Economics. The method, however, uses the
fractile correlation or other non-partmatric correlation measures rather
than the product moment correlation. The reason is that the product moment
correlation is not invariant to changes in marginals but non-paramteric
correlation measures are. The fractile correlation is a natural choice
(it's sample analog is the Spearman rank correlation) because it can be
easily estimated and the the estimator has good sampling properties (for
reasons discussed in the paper I used a different non-parametric correlation
measure).
If one can use a non-parameteric correlation measure then the simple
solution to generating random numbers is to generate normals with the right
correlation, transform these to variates with uniform marginals (using the
standard normal CDF) and then use an inverse probility transform with your
given marginals. Details are in the paper and I have computer code (MATLAB)
to implement the approach. Note that the method is not limited to bivariate
distributions but is unchanged for any number of dimensions (there is a
subtle existence problem that does not arise very often in 3 or more
dimensions; the paper discusses this).
There was a paper by Li and Hammond that solved the problem you state with
the product moment correlation (assuming you met product moment
correlation). Essentially it also uses a multivariate normal and then
performs the above transformation but it requires a search procedure to
determine the product moment correlation of the normal that gives the
correct product moment correlation in the transformed variates.
My paper is:
Paul L. Fackler, Modeling Interdependence, AJAE 1991, pp1091-1097.
S.T. Li and J.L. Hammond Generation of Psuedo-random Numbers etc, IEEE
Transactions on Systems Man and Cybernetics, Sept. 1975, pp. 557-61.
Hope this helps.
Paul
Robert Baur wrote:
> Is anybody out there who could point me answers to the following
> problem?
>
> We have 2 marginal distributions, with given probability density
> function, say f(x) and g(y). Both of them are continous. A 2x2
> correlation matrix sigma is given as well . Our task is to generated a
> correlated sample (x,y) such that x follows f(x) and the second variable
> follows g(y). The correlation of the sample should equal the given
> correlation matrix sigma. If the 2 marginal distribution are normal
> distributions, we have manged to do it. Unfortunately we have not manged
> to solve the general case.
>
> Is it possible to solve the general case?
> Are there distributions with which a solution is obtainable?
John Williams wrote:
> Hello,
> Dejanews indicates that someone asked this question about a year ago on
> sci.math.research, but no "satisfactory" responses were recorded. So
> here it comes again.
>
> I'm looking for a way to generate samples from a multivariate uniform
> distribution with a specified covariance matrix.
>
Herman Rubin
John Williams <ja.wi...@student.qut.edu.au> wrote:
>Hello,
>Dejanews indicates that someone asked this question about a year ago on
>sci.math.research, but no "satisfactory" responses were recorded. So
>here it comes again.
>I'm looking for a way to generate samples from a multivariate uniform
>distribution with a specified covariance matrix.
>For the univariate case it's easy, the variance is equal to (a-b)^2/12.
>That would be fine if I wanted independent samples, however in general I
>have a 3x3 positive definite covariance matrix, with non zero
>off-diagonal terms.
>For the multivariate normal it's easy, just multiply a vector of
>independent samples drawn from N(0,1) by the cholesky decomposition of
>the covariance matrix. Is there a similar trick for the uniform
>distribution?
The problem with this is that there is no clear idea of what
constitutes a multivariate uniform distribution. The methods
used for a multivariate normal do not work; that it can be done
for the normal distribution is based on any of several properties
of the normal distribution. Even for two random variables, the
possibility of doing it for the normal, and having uncorrelated
variables being independent, extends to nothing else.
One CAN attempt to have all linear combinations uniform. This
can be done in two or three dimensions only. In two dimensions,
the distribution is on an ellipse, with the density being of the
form C(1-|x|^2)^{-1/2} if the ellipse is a circle. In three
dimensions, it is on the surface of an ellipsoid, with the
uniform distribution if the ellipsoid is a sphere. For
covariance matrices not proportional to the identity, use
appropriate affine transformations. These project down to lower
dimensions. But three is the limit.
For two dimensions, one can use the density, on the square
whose vertices are +- 1, of (1 + c*x*y)/4. The resulting
correlations are limited to 1/3 in magnitude. Possibly more
terms of an expansion in Legendre polynomials could be used,
but the non-negativity of the density remains a problem.
The normal is the normal, and there is none else like it.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
hru...@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558
Robert Israel
|> I'm looking for a way to generate samples from a multivariate uniform
|> distribution with a specified covariance matrix.
I'm a bit puzzled. What do you mean by "multivariate uniform distribution
with a specified covariance matrix"? The word "uniform" would seem to indicate
that the probability measure is normalized Lebesgue (n-dimensional) measure
on a certain set S. Anywhere I've seen a multivariate uniform distribution
used, that set would be a Cartesian product of intervals. But this would
always give you independent random variables, and you explicitly say
you don't want this. So S must be something else, and you're not telling
us what.
Well, if you're leaving the choice up to me I'll take something
easy to work with. Since the covariance matrix C is positive definite,
you can choose an orthonormal basis {v_i} of its eigenvectors,
Let C v_i = r_i v_i. Then take
S = { sum_i t_i sqrt(3 r_i) v_i, -1 <= t_i <= 1 }.
Thus to generate a sample point from this distribution, let T_i be independent
uniform random variables in [-1, 1] and let X = sum_i T_i sqrt(3 r_i) v_i.
Robert Israel isr...@math.ubc.ca
Department of Mathematics http://www.math.ubc.ca/~israel
University of British Columbia
Vancouver, BC, Canada V6T 1Z2
Volker W. Elling
> I'm looking for a way to generate samples from a multivariate uniform
> distribution with a specified covariance matrix.
For two variables, you can do it like this: let X,Y be uniformly
distributed
on [0,1] and independent. Dfine
Y and Z=f(Y+aX)
where
f(i+b) = b for i integer, b in [0,1[
Y and Z are uncorrelated for a=1 and maximally correlated for a=0
(because
Z=Y, and Var(Y,Y) is the maximum covariance). The covariance is a
polynomial
in a. By continuity, varying "a" should produce any covariance in
between.
Z is uniformly distributed.
I am not sure how to generalize to more variables. Maybe you
can succesively define
Z_{n+1}=f(b_1Y_1+...+b_nY_n+aX_{n+1})
where Y_1...Y_n are variables with known covariances and X_{n+1}
a fresh independent variable. b_i=1, rest 0 gives complete dependence on
Y_i,
while a=1 gives complete independence (for all b_i). Can all
theoretically
possible covariance values in between be achieved?
By the way, the following question is interesting: Given two fixed
variables
X and Y with known correlation [ =cov(X,Y)/sqrt(var(X)var(Y)) ],
which values for Cor(X,Z) and Cor(Y,Z) can be achieved? For example, if
|Cor(X,Y)| < 1, Cor(X,Z)=1 and Cor(Y,Z)=1 simultaneously is not
possible.
-- Volker Elling
-- Informatik/Mathematik, RWTH Aachen -
http://lem.stud.fh-heilbronn.de/~elling
Paige Miller
>
> Paige Miller (pai...@kodak.com) wrote:
> : I have used this trick many times to generate multivariate normal
> : distributions with the proper covariance matrix, I see no mathematical
> : reason why it would not work because you have a uniform distribution.
> :
>
> Normality is preserved by linear transformations; uniformity is not.
Aha! Good point, I had a sneaking suspicion in the back of my mind that
I was overlooking something important...
Thanks!
Gus Gassmann
>
> In article <36D34A...@student.qut.edu.au>, John Williams <ja.wi...@student.qut.edu.au> writes:
>
> |> distribution with a specified covariance matrix.
>
> with a specified covariance matrix"? The word "uniform" would seem to indicate
> that the probability measure is normalized Lebesgue (n-dimensional) measure
> on a certain set S. Anywhere I've seen a multivariate uniform distribution
> used, that set would be a Cartesian product of intervals. But this would
> always give you independent random variables, and you explicitly say
> you don't want this. So S must be something else, and you're not telling
> us what.
I have no idea what John Williams means by "specified covariance
matrix",
either. The thing to do seems to be an acceptance-rejection method,
assuming
that the set S is available: S must obviously be bounded, so embed it in
a
suficiently large rectangle. Generate a point uniformly in this
rectangle.
If the point is in the set S, return it, otherwise continue generating
points
in the rectangle until you find one that lies in S. Clearly the
efficiency
of this method depends on the "size" of the set S relative to the
rectangle.
For special sets S, other methods may be available. For example, you may
want
to generate points uniformly on a hypersphere. One application I have
seen for
that is that it allows you to decompose a multivariate normal
distribution into
two independent parts, a direction (uniformly distributed on the unit
sphere)
and a length (distributed like a chi distribution with n degrees of
freedom
- where n is the dimension of S).
-------------------------------------------------------
gus gassmann (Horand....@dal.ca)
Paul Fackler
only possible but very simple to generate random variables with uniform
marginals and a specified correlation matrix. The trick is to recognize that
the correlation, r, of x=\Phi(X) and y=\Phi(Y), where X and Y are bivariate
normal and \Phi is the (univariate) normal CDF, is related to the
(product-moment) correlation, c, of X and Y by c=2sin(\pi/6 r). So one can
generate X and Y as N(0,[1 c; c 1]) and compute x and y as above. By
construction x and y have uniform marginals and they will have the desired
correlation. The method generalizes without difficulty to higher dimensions
although the possability exits that the transform from r to c will result in a
non-positive definite matrix even though r is p.d. This is unlikely to occur
often in practice because the function 2sin(\pi/6 r) is so close to an
identity mapping on [-1,1]. Obviously this approach is not unique but it is
easy and produces the desired result of producing uniform variates with a
specied second moment matrix. This is a big improvement over simulations that
assume independence.
Incidently the foirmula for c was (apparently) derived first by Karl Pearson
and appeared in a 1936 piece by Hotelling and Pabst in the Annals of Math
Stat.
MATLAB code to perform this operation is:
function x=randcorr(n,R)
% RANDCORR Generates corremlated random variables
% Generates n vector valued variates with uniform marginals and correlation
matrix R.
% Returns an nxk matrix, where k is the order of R.
k=size(R,1);
R=2*sin((pi/6)*R);
x=normcdf(randn(n,k)*chol(R));
Cheers,
Paul
Herman Rubin
John Hudson <jedh...@cix.compulink.co.uk> wrote:
>In article <yL_A2.6371$Ww4.41...@dca1-nnrp1.news.digex.net>,
>mco...@cpcug.org (Michael Cohen) wrote:
>> Paige Miller (pai...@kodak.com) wrote:
>> : John Williams wrote:
>> :
>> : > I'm looking for a way to generate samples from a multivariate
>> > uniform
>> : > distribution with a specified covariance matrix.
>> : >[snip]
>Surely the problem is there is only ONE way to correlate M.V. Gaussian: by
>linear operations, eg. the Cholesky method. There is no such uniqueness
>for non-Gaussian distributions. Perhaps you can specify the support to be
>uniform in which case it is only necessary to dream up some arbitrary M.V.
>pdf inside a hypercube and normalise it to unity volume.
There are lots of ways to have correlated multivariate normals, but there
is one natural way, which has lots of properties. But none of the useful
properties has been used in a natural manner for uniform marginals.
One can do better with multivariate Cauchy (no mean or variance, but
this is not that much of a problem), but here there are too many
alternatives available.
Nedialko
function X=randcorr(n,r)
% RANDCORR Generates correlated uniform random variables
% X=randcorr(n,r) returns an nx2 matrix of n bivariates
% with uniform marginals and correlation r.
% see also: chol, normcdf
c = 2*sin(pi/6 * r);
R = [1 c; c 1];
X=normcdf(randn(n,2)*chol(R));
and it does work as Paul Fackler wrote.
---------
WRT other posts, recall mostly that (see the Michael Cohen post)
while normality is preserved by linear transformations, uniformity is not.
e.g. Volker Elling's claims hold for a=0 (only!),
for a 'general' <a>, neither the related <r>, nor uniformity
(as with any general linear transform) will be preserved.
Nedialko
I find this one more elegant, and seems to work out a bit better in the numbers.
Note that both of my Matlab codes are for 2nd order (simplicity first!), but are straightforward to generalize.
function X=randcor2(n,r)
% RANDCOR2 Generates correlated uniform random variables
% X=randcor2(n,r) returns an nx2 matrix of n bivariates
% with uniform marginals and correlation r.
U = -1 + 2*rand(2,n);
C = [1 r; r 1]; [V,D] = eig(C); % lambdas = diag(D);
W = V*sqrt(3 * D);
X = (W*U)';