Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
i.i.d order statistics and extreme value theory - advice on where asymptotic normal ends and nonnormal limiting distributions begin
12 views
Skip to first unread message
jdm
Dec 30, 2011, 3:30:03 PM12/30/11
to
I've been studying some cryptographic research in which the asymptotic
normal distribution of the empirical sample quartile of order q is
used to construct statistical models of the amount of data required
for a successful cryptanalysis.
The main issue I have is that, while I'm pretty sure that such models
have continued to be used for order statistics X_i (with i near to n)
where the asymptotic normal distribution is inaccurate and where
something based on extreme-value theory for the mth extremes would
have been better, I don't have any idea as to how to compute an
estimate for the value of i (or indeed q) above which the asymptotic
normal might be considered suspect.
As an example, I'm currently dealing with the situation X_1 <=
X_2 ...<= X_n, where n = 2^{41}-1 = 2,199,023,255,551. In particular,
I'm trying to work out whether the asymptotic normal is likely to be
adequate when drawing conclusions about the top 2^{17} = 131,072
values or not - and while this seems a high m for m-th-extreme, it's
not so high in relation to n, and this would mean I was dealing with
the top 0.000006388% of values.
Can anyone give me some advice here?
Many thanks,
James McLaughlin.
normal distribution of the empirical sample quartile of order q is
used to construct statistical models of the amount of data required
for a successful cryptanalysis.
The main issue I have is that, while I'm pretty sure that such models
have continued to be used for order statistics X_i (with i near to n)
where the asymptotic normal distribution is inaccurate and where
something based on extreme-value theory for the mth extremes would
have been better, I don't have any idea as to how to compute an
estimate for the value of i (or indeed q) above which the asymptotic
normal might be considered suspect.
As an example, I'm currently dealing with the situation X_1 <=
X_2 ...<= X_n, where n = 2^{41}-1 = 2,199,023,255,551. In particular,
I'm trying to work out whether the asymptotic normal is likely to be
adequate when drawing conclusions about the top 2^{17} = 131,072
values or not - and while this seems a high m for m-th-extreme, it's
not so high in relation to n, and this would mean I was dealing with
the top 0.000006388% of values.
Can anyone give me some advice here?
Many thanks,
James McLaughlin.
Herman Rubin
Dec 31, 2011, 12:00:03 PM12/31/11
to
needs to avoid the extremes. This is the case for other
reasonable distributions as well; the rate of convergence
to the normal distribution, as found by calculus, in general
needs only that there are large numbers on both sides, and
not on anything else.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hru...@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558
jdm
Dec 31, 2011, 6:30:04 PM12/31/11
to
> If one is sampling from the uniform distribution, one only
> needs to avoid the extremes. This is the case for other
> reasonable distributions as well;
Does the chi-squared distribution count as a "reasonable distribution"
> needs to avoid the extremes. This is the case for other
> reasonable distributions as well;
for these purposes?
Moreover, if "avoid the extremes" means avoiding only the asymptotic
normal distribution for X_1 and X_N, but not for instance X_2 and
X_{N-1}, this would appear to contradict H.A. David stating in "Order
Statistics" that the extremes and mth extremes X_m, X_{N-m+1} are non-
Normally distributed. (albeit with little information on how, given N,
to identify the values of m for which this is the case, except the
statement
"If r/n -> \lambda as n-> infinity, fundamentally different results
(regarding the distribution of order statistic X_r) are obtained
according as
(a) 0 < \lambda < 1, or
(b) \lambda = 0 or 1
with r or N-r fixed ... The latter case includes the extremes X_1,
X_N, and corresponds to the mth extremes X_m, X_{N-m+1} with m fixed.
These have nonnormal limiting distributions.")
(taken from the opening paragraphs of Section 9.1)
0 new messages