Joint Probability Distribution in Matlab
Sumit Maheshwari
I want to find the joint probability distribution of two independent random variables. I used the function hist3 to implement that.
What I actually want is that the joint distribution should provide the multiplied values of probabilities (i.e. joint probabilities for independent variables) and the pair of variables.
Please help.
Think two, count blue.
Let F1 and F2 be the probability functions, and let X1 and X2 be the two
random variables. Then,
[t1,t2] = ndgrid(X1, X2];
Joint = cat(3,F1(t1).*F2(t2), t1, t2);
then Joint(I,J,1) is the joint distribution, Joint(I,J,2) is the first random
variable, and Joint(I,J,3) is the second random variable.
Sumit Maheshwari
Thanks for your response.
Let X1 is my first data series and X2 is second data series, both of the same length. Then how is the criterion for independent joint distribution is fulfilled.
When inputing X1 and X2 with 10000 variables, Matlab says "Out of memory. Type HELP MEMORY for your options". Is there any way to resolve this.
Does F1(t1) gives the probability density directly? How?
Think two, count blue.
> Let X1 is my first data series and X2 is second data series, both of the same
> length. Then how is the criterion for independent joint distribution is
> fulfilled.
>
> When inputing X1 and X2 with 10000 variables, Matlab says "Out of memory. Type
> HELP MEMORY for your options". Is there any way to resolve this.
>
> Does F1(t1) gives the probability density directly? How?
Are you working with continuous or discrete random variables? If you are
working with discrete random variables then F1(x) should simply be the
probability of occurrence of x, except that F1 should accept a complete matrix
of x and evaluate the probability at each point. (There are, by the way, more
efficient code variations than what I showed earlier.)
If, though, you are working with discrete random variables, then you would not
be speaking of probability *density*. The reference to probability *density*
suggests you are working with continuous functions. But what then are your X1
and X2 such that they can have "10000 variables" ? Are X1 and X2 your
_distributions_, or are they points at which you wish to sample? If they are
points at which you wish to sample, then do you want the joint distribution to
be the two density functions multiplied together, or do you want the joint
distribution value to represent the cdf of the first distribution over X1(K)
to X1(K+1), multiplied by the cdf of the second distribution over X2(J) to
X2(J+1) ? Your mention of hist3 earlier suggests you are looking for per-
rectangle probabilities, but your other wording does not support that very well.
Please be clearer as to what you want.
Sumit Maheshwari
> On 11-01-14 02:13 PM, Sumit Maheshwari wrote:
>
> > Let X1 is my first data series and X2 is second data series, both of the same
> > length. Then how is the criterion for independent joint distribution is
> > fulfilled.
> >
> > When inputing X1 and X2 with 10000 variables, Matlab says "Out of memory. Type
> > HELP MEMORY for your options". Is there any way to resolve this.
> >
> > Does F1(t1) gives the probability density directly? How?
>
> Are you working with continuous or discrete random variables? If you are
> working with discrete random variables then F1(x) should simply be the
> probability of occurrence of x, except that F1 should accept a complete matrix
> of x and evaluate the probability at each point. (There are, by the way, more
> efficient code variations than what I showed earlier.)
Sorry for all the confusion created. I have two series of data and I want to find the joint distribution of as a conditional random variable describing both the data. The data series are conditionally independent and hence I guess we can multiply the PDFs of both the series.
>
> If, though, you are working with discrete random variables, then you would not
> be speaking of probability *density*. The reference to probability *density*
> suggests you are working with continuous functions. But what then are your X1
> and X2 such that they can have "10000 variables" ? Are X1 and X2 your
> _distributions_, or are they points at which you wish to sample? If they are
> points at which you wish to sample, then do you want the joint distribution to
> be the two density functions multiplied together, or do you want the joint
> distribution value to represent the cdf of the first distribution over X1(K)
> to X1(K+1), multiplied by the cdf of the second distribution over X2(J) to
> X2(J+1) ? Your mention of hist3 earlier suggests you are looking for per-
> rectangle probabilities, but your other wording does not support that very well.
>
> Please be clearer as to what you want.
I have taken a certain step size and plotted the continuous probability density functions for both the data. Although the data are discrete, I want to work as continuous PDFs which is required for the problem I am addressing. Now, my question drops down to, would it be technically correct just to multiply the two PDFs to get a joint PDF. If so, then, the Nx2 matrix will represent the values of the joint model and their corresponding multiplied probability values will represent their Joint PDF.
Please correct me if I am wrong.
Think two, count blue.
> I have taken a certain step size and plotted the continuous probability
> density functions for both the data. Although the data are discrete, I want to
> work as continuous PDFs which is required for the problem I am addressing.
> Now, my question drops down to, would it be technically correct just to
> multiply the two PDFs to get a joint PDF. If so, then, the Nx2 matrix will
> represent the values of the joint model and their corresponding multiplied
> probability values will represent their Joint PDF.
If you have independent variables and boundary points, then the joint pdf
int(int(F1(x1) * F2(x2), x2=c..d), x1=a..b)
would (but only on the condition of independence!) be the same as
int(F1(x1) * int(F2(x2), x2=c..d), x1=a..b)
which would (still requiring independence!) be the same as
int(F1(x1), x=a..b) * int(F2(x2), x2=c..d)
which is what you had hoped it to be, the cdf of the first function over the
first range, multiplied by the cdf of the second function over the second
range. Thus, yes, on the condition of independence, you can multiply the
discrete cdf's.
This does not depend upon the number or size or position of the intervals
being the same for the two functions.
It would not, however, give you an Nx2 matrix: it would give you an NxM
matrix, where N is the number of intervals for the first variable and M is the
number of variables for the second value. With your 10,000 values, that's
going to create an array of 100 million elements, needing about 800 megabytes.
Which would be too large for a 32 bit version of Matlab.
If you are working with a joint probability and discrete variables, you are
going to be generating a grid of data of all the combinations, not a vector.
There is, I suppose, a possibility that the only cases that matter to you are
the cases where the two independent variables happen to fall within the same
discrete interval. But if so, that would be a single vector (a diagonal of the
above-mentioned grid), and you wouldn't have asked earlier for the joint
distribution to include the probability and the variables, as knowing the
value of one variable would, in that case, automatically imply the range of
the other, so copying both would be redundant. Thus I doubt you are only
interested in one of the diagonals... and thus you are going to have memory
problems if you try to compute the joint probability all at one time.
If you want to histogram the joint probability, there are much more efficient
ways to do it than to calculate all the pairs ahead of time.
> Although the data are discrete, I want to work as continuous PDFs which is
> required for the problem I am addressing.
Now *that* could prove to be a real problem. There are some mathematical
procedures that apply to continuous variables that just don't apply to
discrete variables and for which there is no real discrete analogue. Sometimes
there _are_ decent discrete analogues to functions involving continuous variables.
If you wanted to do something like a minimization, you would have difficulty
using the majority of the minimizers, as most of them assume your function is
at least twice differentiable whereas discrete samples are not differentiable
at all.
We would need to know more about what you intend to do, I think.
(Though if the PDFs you refer to are the F1 and F2 mentioned before, then it
is fine that they are defined over continuous intervals, as long as you
recognize that your X1 and X2 are going to have to take the CDF over each of
the intervals in order for you do produce your discretized joint distribution.)
Sumit Maheshwari
Hi. Hope you doing well.
The variables I am using have equal lengths and thus, lets say, it would be a matrix of 10000*10000 elements.
>
> If you are working with a joint probability and discrete variables, you are
> going to be generating a grid of data of all the combinations, not a vector.
>
> There is, I suppose, a possibility that the only cases that matter to you are
> the cases where the two independent variables happen to fall within the same
> discrete interval. But if so, that would be a single vector (a diagonal of the
> above-mentioned grid), and you wouldn't have asked earlier for the joint
> distribution to include the probability and the variables, as knowing the
> value of one variable would, in that case, automatically imply the range of
> the other, so copying both would be redundant. Thus I doubt you are only
> interested in one of the diagonals... and thus you are going to have memory
> problems if you try to compute the joint probability all at one time.
I think you are quite right when you say 'I need only the diagonal elements'. The range of values for both the variables is not the same though. X has its values lying from 22 to 23 and Y has its values from 0.2 to 0.5.
Now, assuming that these two variables are independent, I want to find
f(Z)=f(X)*f(Y)
Where Z is a continuous random variable describing X and Y as a mixture of conditionally independent Weibull distribution. (X and Y are Weibull distributed random variables).
f(X) is PDF of X and f(Y) is PDF of Y.
>
> If you want to histogram the joint probability, there are much more efficient
> ways to do it than to calculate all the pairs ahead of time.
>
>
> > Although the data are discrete, I want to work as continuous PDFs which is
> > required for the problem I am addressing.
>
> Now *that* could prove to be a real problem. There are some mathematical
> procedures that apply to continuous variables that just don't apply to
> discrete variables and for which there is no real discrete analogue. Sometimes
> there _are_ decent discrete analogues to functions involving continuous variables.
>
> If you wanted to do something like a minimization, you would have difficulty
> using the majority of the minimizers, as most of them assume your function is
> at least twice differentiable whereas discrete samples are not differentiable
> at all.
>
> We would need to know more about what you intend to do, I think.
Now that you have mentioned the 'minimization' , I want to tell you the problem. I want to find the forward and backward recursions using the joint probability distribution. That will be using forward (alpha) and backward (beta) variables which in turn requires the PDF of Z. Z is the joint model for X and Y (to describe both together, jointly).
The forward backward variables are used in the EM (Expectation Maximization), may be Baum Welch as an algorithm.
>
> (Though if the PDFs you refer to are the F1 and F2 mentioned before, then it
> is fine that they are defined over continuous intervals, as long as you
> recognize that your X1 and X2 are going to have to take the CDF over each of
> the intervals in order for you do produce your discretized joint distribution.)
Thank you for your support. Kindly help me with a little conceptual understanding of joint PDF as it is now confusing me too. :)