Herman Rubin writes:Really? I can't speak for those I don't know, but as far as my department
>In article <1993Dec8.022208.9...@EE.Stanford.EDU> cl...@tukey.Stanford.EDU ( ) writes:
>>Herman Rubin writes:
>>And what do armed nuclear weapons have to do with density estimation?
>The intent was to indicate both the magnitude of the danger and the
at Bell Labs is concerned, this statement is plainly wrong, and there are
several of us here who have worked in the field of local fitting. Our work
has been almost entirely motivated by real engineering and manufacturing
problems for which other existing methods have been found to be unsatisfactory.
>>Sorry, but these are just cheap cough outs. It is YOUR responsibility,So, someone walks into your office with a dataset, and says `I want to
>>as the one promoting this parametric approach, to provide the tools and
>>methods to convert the vague ideas one has in practice into formal loss
>>functions, sensible models and prior distributions for the parameters.
>These are not copouts, any more than it is a copout if a drug company
estimate the density'. You say `Go away, come up with a large parametric
model, an estimate (from no data) of the prior density on the parameter
space, and a global loss function, then come back'.
For the sake of the story, assume the client comes back with the requested
Is this really the complete role you see for a statistician?
>>As I've said in previous posts, coming up with sensible choices isA question, and a reasonable one at that! A welcome change from your usual
>>unreasonable in many applications; no single loss function can adequately
>>capture the performance of a function estimate. Local smoothness of an
>>unknown density is often a reasonable assumption to make. Converting this
>>into a global parametric model and prior distribution is difficult. If you
>>wish to promote a large scale parametric approach, this is YOUR problem.
>What is local smoothness? There are hundreds of ways of expressing it.
attempts to portray posters as stupid and describing others' work as `just
By local smoothness, I mean existence of a good polynomial approximation
There is of course no reason to restrict ourselves to polynomial approximations;
Under the most widely studied asymptotics (bandwidth -> 0) the analagous
>In most "real world" density situations, the density is infinitelyQuestionable, and I wouldn't place too much faith in estimators that rely
>differentiable, usually even analytic.
on this assumption. Probably not very relevant to our discussion.
> This by itself is useless forI have read hundreds of papers in density estimation and related fields
>inference. If you would read the literature, you would know that I
>have suggested classes of infinte-parametric models--the prior distribution
>problem is another matter indeed, but there are some negative results.
over the past few years, and would be glad to add yours if only you would
provide the references. I have been asking for specific details of your
methods throughout this thread, and requested you to send (p)reprints over
two weeks ago.
Until you provide references, you can hardly direct others to read the
>These negative results are of some importance, in that they point out`they should'. Good scientific argument. I also claim my procedures beat
>that certain types of simplifications do not work. I also have some
>preliminary results on robust procedures here; they should beat any
>of the kernel procedures.
kernel procedures, particularly with regard to moderate tail performance.
>>In the real world, we don't get to brush aside the difficult parts asPrior distributions, and (at least for some problems) models and loss
>>`someone else's problem'; `will be done in future work' and all the similar
>>excuses used all too frequently by some in academics.
>As I said before, the pharmaceutical company cannot solve the problem of
functions are mathematical abstractions of little direct interest to
anyone other than the statistician. It is the job of the statistician
to become sufficiently involved and knowledgeable about the problem, so
that informed decisions can be made about such items as needed, and if
informed decisions cannot be made, then to try other approaches. I have
no idea as to how to construct meaningful prior densities in possibly
hundreds or thousands of dimensions; until you can provide some guidance
as to how I should do so, your approach (or any other that requires me
to do so) can at best be considered of passing theoretical interest.
Leaving a biologist or engineer to make these decisions would in my view
>>Using function+derivative at 3 or 4 points in each dimension gives asIn numerical analysis one is usually interested in far higher degrees
>>much or more information than just the function on 5 or 6 points. Poor
>>specification? Maybe, but you will need at least as many parameters as
>>I need points to get a reasonable specification.
>You might be able to get one decimal place that way, but I doubt it.
of accuracy than in statistics. Since 9*4^8=589824, the original 100000
data points can't tell us much more. And your trying to model the situation
with "a 1000-parameter model, or even 10000", will provide about as much
information as I get from a 2^8 grid of points. Less, if I use local
I will also point out that I have never claimed my procedures (or anyone
>>If your global fitting involves global optimization (as do all globalIf the problems are `not that difficult', why are you having so much
>>procedures I'm aware of; you have provided no evidence to suggest otherwise)
>>then a large number of parameters makes the computational burden absolutely
>This is exactly what I dispute. Numerical methods are not that difficult
difficulty producing anything specific? What special features of your
procedure enable significant speedup over standard methods of global
Local fitting is linear in the number of points fitted directly, and therefore
>>Finally, if the user wishes to visualize the density estimate, one canOh. You're resorting to calling other people's work `crude methods' again.
>>only look at lower dimensional cross sections and/or projections. With
>>a local approach, one just computes the views of most interest. With a
>>global approach, you get nothing until you've fitted everything.
>It depends on what one wants to visualize. Possibly you want those; I
Say something intelligent. Show me precisely in what ways your methods are
better than mine in the moderate tails. To obtain good estimates in the tails,
it is surely obvious one needs some form of localization to prevent bias
being induced by incorrect specification elsewhere?
>>These issues have been thrashed out in detail in the literature, as youThe best place to start computing local fits is the paper by Cleveland
>>would quickly realize if you bothered to read any of the references I've
>Engineering literature or numerical analysis literature?
and Grosse. This was in the statistics literature. Do I sense a problem?
>>>I do not have enough information about the original problem to produceSo what are you saying, apart from spouting off as to how you think you are
>>>an algorithm. It is questionable whether there even IS enough information
>>>to produce a simple algorithm.
>>Then what is the basis for your claims of computational superiority?
>The knowledge of problems for which plugging into computer packages does
so much smarter than everyone else? Anyone who wants to estimate a density
has to write their own code from scratch? Are we allowed to use high level
languages, or does this have to be done in machine code?
And how does this answer the question:
You have still not provided a scrap of evidence that you have ever implemented
>When someone asks for an 8-dimensional object, the assumptions are alreadyAs opposed to failing to visualize the fit at all, which will miss just about
>of great importance. Judging from the little stated in the problem, the
>low-dimensional sections are likely to miss the important points.
You included early in your first post in this topic:
>In my opinion, this is not even the case for densityNow, stop trying to run for cover behind the 8 dimensional problem, and
>estimation in *1* dimension; there are ad hoc procedures, and for some of
>them their properties have been studied, but none from a good analysis of
start trying to justify your claims. You certainly haven't shown your
approach offers anything special for the 8-d problem, or any other problem.
>>Statistics in fact has very little to do with choosing between models.Hey! We agree on something.
>>In industry, the problems are to improve quality and productivity.
>>Complicated problems, no simple model and considerable advances have
>>been brought about through data analysis; often with no more than some simple
>>data visualization. Saying this is more likely to do damage than to bring
>>understanding simply falls flat in the face of evidence.
>If one is doing quality control, or operating a factory, that is the case.
>If one is carrying out a scientific investigation, this is not the case.Local fitting does not restrict one to using local polynomials. In some
>In a moderately well-tuned production facility, one may be looking at
>small perturbations, and low degree polynomials can be good fits, or
>it may be the case that most variables do not matter, or one of a
>number of other low-dimensional alternatives.
of my applications, I don't use local polynomials. There are procedures
for both global and local variable selection; procedures such as MARS do
the latter in the regression problem. There are semiparametric approaches and
conditionally parametric approaches for use where some variables are more
important than others; one can also smooth with different bandwidths in
different directions. You are hardly raising anything original here.
> In higher dimensionalAnd others propose using "a 1000-parameter model, or even 10000", which is
>problems, linear approximations are often used over ranges where linearity
>is overly restrictive, or other methods claimed to be robust, but which are
>only robust under massive assumptions, usually unstated, are produced.
inadequate except under massive assumptions. Often in high dimensions
there's insufficient data for anything more than a coarse fit.
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.