What's the robust way of finding percentile in Matlab? Re: Is it possible for the mean to be larger than 90 percentile in data?
Mike
I am seeking a Matlab function for finding percentile in a dataset.
I hope that function is robust.
Because when I use my own function for finding percentile, it gives some
chaotic results looking counter-intuitive.
Please see below for two problems that I've found.
What might be wrong?
-----------
My code: where W is a vector of my Monte Carlo outcomes, CI_up = 0.95,
CI_lo=0.05, N_MC=5000 the number runs in the Monte Carlo simulation:
W_sorted=sort(W)
W_CI_up=0.5*(W_sorted(floor(N_MC*CI_up))+W_sorted(ceil(N_MC*CI_up)));
W_CI_lo=0.5*(W_sorted(floor(N_MC*CI_lo))+W_sorted(ceil(N_MC*CI_lo)));
"Mike" <housi...@gmail.com> wrote in message
news:ev8rtb$oei$1...@news.Stanford.EDU...
> Moreover, in another run of my Monte Carlo, I found the mean, 90%, 10% to
> be all the same...
>
> Could this be correct?
>
> Thanks
>
> "Mike" <housi...@gmail.com> wrote in message
> news:ev8rcs$nvb$1...@news.Stanford.EDU...
>> Hi all,
>>
>> I am playing around with some Monte Carlo generated random outcome data.
>> I found that the mean is lrger than the 90% percentile and the 90%
>> percentile and 10% percentile are the same.
>>
>> Can this happen? Or is there something wrong with my data?
>>
>> Thanks
>>
>
>
Mike
% compute 25th percentile (first quartile)
Q(1) = median(y(find(y<median(y))));
% compute 50th percentile (second quartile)
Q(2) = median(y);
% compute 75th percentile (third quartile)
Q(3) = median(y(find(y>median(y))));
---------------
Is it more correct than mine?
It is from this webpage:
http://www.tech.plym.ac.uk/spmc/links/matlab/matlab_statistics.html
"Mike" <housi...@gmail.com> wrote in message
news:ev8tfn$pfr$1...@news.Stanford.EDU...
Roger Stafford
<housi...@gmail.com> wrote:
------------------
Your calculations for percentile look perfectly valid to me. However,
it is easy to create artificial W's that will give you those odd results.
For example, suppose 91% of them are equal to 0 and the remaining 9% are
1's. You will get 0 for the 10-percentile and 90-percentile values, and
yet the mean is .09. Or, if all are 0's, you get all three equal.
However, these are very unusual distributions. If you got them in a Monte
Carlo procedure, you had best take a good look at the source of your
random numbers. I think you'll find they aren't really random. For
example, you may be resetting the 'state' quantity in 'rand' each time in
a loop and thereby getting constant values or constants until the clock
happens to click during the loop. You should set 'state' only once for an
entire run.
Roger Stafford
richar...@comcast.net
wrote:
>Hi all,
>
>I am seeking a Matlab function for finding percentile in a dataset.
>
snip
If you have the Statistics toolbox, check out prctile
-Dick Startz
Francis Burton
>
>I hope that function is robust.
>
>Because when I use my own function for finding percentile, it gives some
>chaotic results looking counter-intuitive.
Apparently there's more than one definition of percentile:
http://www.itl.nist.gov/div898/handbook/prc/section2/prc252.htm
In MATLAB, you would sort the data vector and then use one of
the formulae given in the link above.
Francis
fethi
aslak grinsted
serxval
comes from Ospray toolbox and works fine):
=======================================
function y = percentile(x, pct)
% y = PERCENTILE(x, pct)
% Example: percentile(x, 0.10) is the value that is higher than 10%
% of the elements of x, and less than the remaining 90%.
% If the length of x is such that there is no element of x exactly
% corresponding to the 'pct' position, a weighted average of the two
% adjacent values is used. pct must be between 0 and 1 inclusive.
%
% percentile(x, 1) is a slow way to get max(x).
% percentile(x, 0.5) is the median of x.
% percentile(x, 0) is a slow way to get min(x).
%
% If x is a matrix, percentile operates on columns, returning
%multiple columns.
% If pct is a vector, multiple rows are returned, one per element of
%pct.
%
% See also median, sort, max, min.
%--------------------------------------------------------------------
% NOTA: Proviene del toolbox OSPREY.zip. Como este toolbox me daba
% problemas al usar otros programas (nunca supe pq, daba errores al
% Matlab) solo cogi la funcion "percentile.m" de el pero NO las
demas.
%--------------------------------------------------------------------
x=x(:);
I=find(x>=0);
x=x(I);
[mm,nn] = size(x);
if (mm == 1), x = x.'; end
x = sort(x);
n = ((mm - 1) * pct(:) + 1);
r = rem(n, 1);
r1 = r * ones(1, nn);
y = (1-r1) .* x(n-r,:);
ix = find(r); % when n=mm, x(n+1,:) doesn't exist
y(ix,:) = y(ix,:) + r1(ix,:) .* x(n(ix)-r(ix)+1,:);
=======================================
Bye...
Serval.
..........................................
Ben Mottinger
Note that this is a common method for percentile, but not the same as
prctile from the Statistics Toolbox. Matlab doesn't interpolate the
extremities. For example, in a small data set, the 99th percentile
might be beyond the range of where prctile interpolates. So an
interpolated value (using the method above) could be much lower than
the result from prctile, which would be the max of the set.
Just a bit of caution--there's more than one way to skin this cat.
StephenLL
>
>
> serxval wrote:
>>
>>
>> Hi Mike. What you are looking for is this "percentile.m"
function
Question for the group:
I took the code from R's quantile function and translated to matlab
for my own personal use. It has 9 different versions of taking
percentiles. It also work's along any dimension.
I would be happy to add it to the file exchange but I am not sure
what I need to do from a licensing perspective.
I will also attempt to contact someone from the r-project to see what
they have to say.
Any thoughts?
Stephen
Per S
cumulative distribution and the use pchip to find 0.05 and 0.95
n=1000;
p=0.10; %percentile
x=randn(1,n);%some random distribution
x1=pchip((1:n)/n,sort(x),p/2) %lower bound
x2=pchip((1:n)/n,sort(x),1-p/2) %upper bound
xm=pchip((1:n)/n,sort(x),0.5) %median etc...
/Per