Is Racial Profiling a Type of Bayesian Inference?
aruzinsky
statistics, but I am uncertain of the kind of statistics.
Russell
> I'm certain that racial profiling is an informal application of
> statistics, but I am uncertain of the kind of statistics.
I shouldn't touch this topic with a 10 foot pole, but...
My motto is all too often the study of data requires cares.
A corollary of that might be there is no such thing as
informal statistics. Either the methodology is applied
carefully or it isn't really statistics, just quackery.
I'm sure many officers try hard, I doubt if the people in
law enforcement, especially on patrol, have the time for
careful analysis, even if they have the desire and skills.
I'm more inclined to believe that cognitive illusions tend
to take over, driven by the heuristics hardwired into our
brains by millions of years of evolution (see the book
_Inevitable Illusions: How Mistakes of Reason Rule Our
Minds_, or several others that deal with these ideas).
Good training helps condition officers to control their
reactions to these hueristics, and in some cases the
hueristics may even invoke the correct response and
should be followed. But in any case, it's not statistics,
at least as I use the word. Just my opinion...
Cheers,
Russell
Herman Rubin
Russell <russell...@wdn.com> wrote:
>On Jul 23, 8:41=A0pm, aruzinsky <aruzin...@general-cathexis.com> wrote:
>> I'm certain that racial profiling is an informal application of
>> statistics, but I am uncertain of the kind of statistics.
>I shouldn't touch this topic with a 10 foot pole, but...
>My motto is all too often the study of data requires cares.
>A corollary of that might be there is no such thing as
>informal statistics. Either the methodology is applied
>carefully or it isn't really statistics, just quackery.
Alas, this cannot always be the case. There are too
many situations in which decisions have to be made, and
there is no clearly available likelihood function.
Statistics is NOT methodology. Rather, one needs to
consider all the consequences of the proposed action
in all the states of nature, and unfortunately many,
if not most, uses fail miserably at this. I include
most of the current alpha-level testing.
Prior Bayesian inference, which I endorse and which
often can be approximated when likelihood functions
and posteriors are not computable, often requires such
choices. Everything can be abused, including racial
profiling, but in many cases it must be used.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hru...@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558
Russell
> In article <99fb319d-4f76-458c-b8c5-1be00c4d7...@k30g2000yqf.googlegroups.com>,
Herman,
I largely agree with you except that I would classify the
need "to consider all the consequences of the proposed
action in all the states of nature" as the field of risk
assessment, an application of (among other things)
statistics, rather than statistics alone. Perhaps that's
just a semantic quibble...
Cheers,
Russell
Herman Rubin
Russell <russell...@wdn.com> wrote:
>On Jul 24, 1:42=A0pm, hru...@odds.stat.purdue.edu (Herman Rubin) wrote:
>> In article <99fb319d-4f76-458c-b8c5-1be00c4d7...@k30g2000yqf.googlegroups=
>.com>,
>> Russell =A0<russell.mar...@wdn.com> wrote:
>> >On Jul 23, 8:41=3DA0pm, aruzinsky <aruzin...@general-cathexis.com> wrote=
<> >> I'm certain that racial profiling is an informal application of
<> >> statistics, but I am uncertain of the kind of statistics.
<> >I shouldn't touch this topic with a 10 foot pole, but...
<> >My motto is all too often the study of data requires cares.
<> >A corollary of that might be there is no such thing as
<> >informal statistics. =A0Either the methodology is applied
<> >carefully or it isn't really statistics, just quackery.
<> Alas, this cannot always be the case. =A0There are too
<> many situations in which decisions have to be made, and
<> there is no clearly available likelihood function.
<> Statistics is NOT methodology. =A0Rather, one needs to
<> consider all the consequences of the proposed action
<> in all the states of nature, and unfortunately many,
<> if not most, uses fail miserably at this. =A0I include
<> most of the current alpha-level testing.
<> Prior Bayesian inference, which I endorse and which
<> often can be approximated when likelihood functions
<> and posteriors are not computable, often requires such
<> choices. =A0Everything can be abused, including racial
<> profiling, but in many cases it must be used.
>Herman,
>I largely agree with you except that I would classify the
>need "to consider all the consequences of the proposed
>action in all the states of nature" as the field of risk
>assessment, an application of (among other things)
>statistics, rather than statistics alone. Perhaps that's
>just a semantic quibble...
>Cheers,
>Russell
I do not think this is a semantic quibble. The risk
assessment, unfortunately, has to be different for
different people, and this is not the job of statistics,
although many have placed it on the statisticians.
The statistician needs to advise the person using the
data as to how to optimize, or come close to it, his
own risk. But the full risk assessment has to be
taken into consideration, not parts of it which miss
major consequences.
aruzinsky
> On Jul 23, 8:41 pm, aruzinsky <aruzin...@general-cathexis.com> wrote:
>
> > I'm certain that racial profiling is an informal application of
> > statistics, but I am uncertain of the kind of statistics.
>
> I shouldn't touch this topic with a 10 foot pole, but...
>
It's a Yes or No question!
> My motto is all too often the study of data requires cares.
> A corollary of that might be there is no such thing as
> informal statistics. Either the methodology is applied
> carefully or it isn't really statistics, just quackery.
A euphemism for "fudge" is "model" as in "Gaussian white noise
model." Name one area of applied statistics without fudge.
> I'm sure many officers try hard, I doubt if the people in
> law enforcement, especially on patrol, have the time for
> careful analysis, even if they have the desire and skills.
> I'm more inclined to believe that cognitive illusions tend
> to take over, driven by the heuristics hardwired into our
> brains by millions of years of evolution (see the book
> _Inevitable Illusions: How Mistakes of Reason Rule Our
> Minds_, or several others that deal with these ideas).
> Good training helps condition officers to control their
> reactions to these hueristics, and in some cases the
> hueristics may even invoke the correct response and
> should be followed.
> But in any case, it's not statistics,
> at least as I use the word. Just my opinion...
>
> Cheers,
> Russell
I am leaning toward informal statistical decision theory. You do not
have to know the exact number of bullet chambers in a gun to quickly
decide not to play Russian roulette.
Rich Ulrich
Yes, it invokes Bayesian inference. I've read of
experiments on "profiling" where my own quick
reaction was to disagree with the experimenters.
OFF TOPIC.
In the instance of the black Harvard professor arrested
at home, which must have inspired the comment --
I've seen the cop on TV and he looks like a straightshooter,
who believes that he was suitably polite.
I've heard it alleged that he has had other complaints.
I can easily imagine that he did not identify himself
clearly as a police officer at the start; and that it
might take an hour of intensive group role-playing
to convince him that his own initial approach is *not*
entirely, suitably polite (if he could be convinced at all).
Or, maybe all fault is the other way. I've known nice,
soft-spoken blacks who surprised me by taking offense
at someone.
If the officer is at fault, it probably had nothing to do
with race. Cops tend to act harsh with anyone in
suspicious circumstances. Blacks tend to take it
personally.
--
Rich Ulrich
Russell
> On Jul 24, 10:44 am, Russell <russell.mar...@wdn.com> wrote:
>
> > On Jul 23, 8:41 pm, aruzinsky <aruzin...@general-cathexis.com> wrote:
>
> > > I'm certain that racial profiling is an informal application of
> > > statistics, but I am uncertain of the kind of statistics.
>
> > I shouldn't touch this topic with a 10 foot pole, but...
>
> It's a Yes or No question!
Please note that the body of your post, which was what I
addressed, was "I'm certain that racial profiling is an
informal application of statistics, but I am uncertain
of the kind of statistics.", which in fact is not a
question at all.
>
> > My motto is all too often the study of data requires cares.
> > A corollary of that might be there is no such thing as
> > informal statistics. Either the methodology is applied
> > carefully or it isn't really statistics, just quackery.
>
> A euphemism for "fudge" is "model" as in "Gaussian white noise
> model." Name one area of applied statistics without fudge.
A famous statistician once said, "All models are wrong. Some
are useful."
> > I'm sure many officers try hard, I doubt if the people in
> > law enforcement, especially on patrol, have the time for
> > careful analysis, even if they have the desire and skills.
> > I'm more inclined to believe that cognitive illusions tend
> > to take over, driven by the heuristics hardwired into our
> > brains by millions of years of evolution (see the book
> > _Inevitable Illusions: How Mistakes of Reason Rule Our
> > Minds_, or several others that deal with these ideas).
> > Good training helps condition officers to control their
> > reactions to these hueristics, and in some cases the
> > hueristics may even invoke the correct response and
> > should be followed.
> > But in any case, it's not statistics,
> > at least as I use the word. Just my opinion...
>
> > Cheers,
> > Russell
>
> I am leaning toward informal statistical decision theory. You do not
> have to know the exact number of bullet chambers in a gun to quickly
> decide not to play Russian roulette.
That's a classic a risk analysis.
Cheers,
Russell
aruzinsky
>
> - Show quoted text -
My point was that racial profiling typically attempts to optimize
costs (maximizing expected utility) whereas Bayesian Inference does
not therefore calling it an application of statistical decision theory
is more appropriate.
See
http://www.infra.kth.se/~soh/decisiontheory.pdf
As statistical decision theory, racial profiling can be done more
rigorously to set police policy.
It is possible to get crime statistics, e.g., here
http://www.informaworld.com/smpp/1820777723-94262195/content~db=all~content=a903875794
and guesstimate the utility costs. An advantage of this procedure is
that points of contention (probably utility costs) can easily be
recognized compared to recognizing points of contention in endless
hours of tradition debate using connotative words and denotatively
ambiguous homonyms such as "racist."
Russell
>
> and guesstimate the utility costs. An advantage of this procedure is
> that points of contention (probably utility costs) can easily be
> recognized compared to recognizing points of contention in endless
> hours of tradition debate using connotative words and denotatively
>
> - Show quoted text -
My point is that the results don't have to be perfect,
the assumptions of the models will never be perfectly
met in real life data (if they were, I'd think the data
suspect), but these things need to be done with care
and interpreted with care to be truly useful. One can
download all the data one can find, but how good is the
data? What else are potential sources of statistical
bias in the data? What covariates are highly colinear
with race and crime? Correlation is not causation.
Such things need to be studied before one even decides
whether to use technique A or B for the analysis. (You
might be right in your choice of technique. I don't
know.) One will almost certainly find the data wanting
in some respect. What might that do to the results?
I agree that endless hours of debate might be illuminated
by actual facts. But I don't want armed police making
decisions based on a guesstimate of the utility costs or
guesstimates produced in whatever statistical framework
one thinks is most applicable. It's not that such
results are necessarily fundamentally wrong in all
cases, but they might be wrong in too many cases and they
might provide a cloak of scientific respectability for
decisions that is not justified.
Cheers,
Russell
Nick
> and guesstimate the utility costs. An advantage of this procedure is
> that points of contention (probably utility costs) can easily be
> recognized compared to recognizing points of contention in endless
> hours of tradition debate using connotative words and denotatively
> ambiguous homonyms such as "racist."
I don't believe these probability utility costs can be easily
recognised. How would you evaluate the utility of alienating an ethnic
group, over a generation. Like it or not law and order is mainly an act
or sham designed to discourage criminal behaviour. It is not about
capturing the odd extra offender.
The social dynamics of Race/Crime are much harder to model than say
finance and we know where that took us.
Ambiguous as it may be racist is a valuable word in that it helps remind
us of problems with historical over simplifications.
aruzinsky
But, you are already a voluntary user of grossly inaccurate
guesstimation. You are making an important decision based on "might",
i.e., nonzero probability, instead of more accurate probability
estimates.
Russell
The use of the word "might" in this case is rhetorical
rather than statistical, and even if it wasn't, I'm not
going to draw a gun based on the resulting probabilities.
Cheers,
Russell
aruzinsky
Which "rhetorical"?:
1. Of or relating to rhetoric.
2. Characterized by overelaborate or bombastic rhetoric.
1. Use is not exclusive of statistics.
2. Use is inappropriate for mathematicians, scientists and engineers.
In addition to communication, they shouldn't even think in type 2
rhetoric.
Russell
Rhetorical as in the use of the would "might" means
I am not claiming I have proof of the statement I
made therefore I'm allowing that the claim could be
wrong, an opinion that I think more people should
hold about more of their beliefs than it appears
they do. ;-)
Cheers,
Russell
Herman Rubin
aruzinsky <aruz...@general-cathexis.com> wrote:
>On Jul 24, 6:12=A0pm, Russell <russell.mar...@wdn.com> wrote:
>> On Jul 24, 6:51=A0pm, aruzinsky <aruzin...@general-cathexis.com> wrote:
>> > On Jul 24, 10:44=A0am, Russell <russell.mar...@wdn.com> wrote:
>> > > On Jul 23, 8:41=A0pm, aruzinsky <aruzin...@general-cathexis.com> wrot=
>e:
...................
>My point was that racial profiling typically attempts to optimize
>costs (maximizing expected utility) whereas Bayesian Inference does
>not therefore calling it an application of statistical decision theory
>is more appropriate.
The logical approach is to maximize expected utility.
It is then a theorem that the optimal method to do this
is Bayesian Inference.
Often, one cannot do the posterior optimization, but
can do a good job on the prior optimization, and even
consider restrictions. This is the prior Bayes
approach, form which when full optimization is
possible and computable becomes the "ordinary"
Bayesian approach. This is what should be done more
often; one can get good results even if the ordinary
posterior approach is difficult or even impossible.
aruzinsky
>
> I don't believe these probability utility costs can be easily
> recognised. How would you evaluate the utility of alienating an ethnic
> group, over a generation.
One cost of false positives in collecting suspects is property damage
caused by by riots.
> Like it or not law and order is mainly an act
> or sham designed to discourage criminal behaviour.
> It is not about
> capturing the odd extra offender.
>
That is poorly worded because laws define criminal behavior. It is
wrong because, if a murderer is executed, that person is forever
prevented, not "discouraged," from further crime. Also, law
enforcement substantially stops crimes while in progress, e.g., Ponzi
schemes. Maybe, you think Bernie Madoff has been "discouraged"
enough?
> The social dynamics of Race/Crime are much harder to model than say
> finance and we know where that took us.
>
I do not believe that statistical decision theory is responsible for
the current state of the economy.
>
> Ambiguous as it may be racist is a valuable word in that it helps remind
> us of problems with historical over simplifications.
Name one historical over simplification.
aruzinsky
I suggest that next time you follow "might," with "for all I know."