Laplace and the Monty Hall paradox?
Stan Brown
it there, but cancellation are not effective for many servers.)
In Laplace's /Philosophical Essay on Probabilities/, I've read a
passage that troubles me, and I hope someone can shed some light
on it. It's on pages 15-18 of the 1951 Dover paperback containing
the translation into English by Truscott and Emory.
The troublesome passage refers back to Laplace's "Sixth Principle",
which I quote here:
L: Sixth Principle.-- Each of the causes to which an
L: observed event may be attributed is indicated with just
L: as much likelihood as there is probability that the event
L: will take place, supposing the event to be constant. The
L: probability of the existence of any one of these causes
L: is then a fraction whose numerator is the probability of
L: the event resulting from this cause and whose denominator
L: is the sum of the similar probabilities relative to all
L: the causes; if these various causes, considered a priori,
L: are unequally probable, it is necessary , in place of the
L: probability of the event resulting from each cause, to
L: employ the product of this probability by the possibility
L: of the cause itself.
This troubled me on its own, the idea of reasoning back from
observed events to the probability of causes, but I forged ahead
to his Seventh Principle. I won't quote it here because the
example illustrates my difficulty better than the general
principle:
L: Let us imagine an urn which contains only two balls, each
L: of which may be either white or black. One of these balls
L: is drawn and is put back into the urn before proceeding
L: to a new draw. Suppose that in the first two draws white
L: balls have been drawn; the probability of again drawing a
L: white ball at the third draw is required.
At this point I stopped to think. Try as I might, I couldn't see
a _logical_ procedure for computing the desired probability on
the information given.
L: Only two hypotheses can be made here: either one of the
L: balls is white and the other black, or both are white. In
L: the first hypothesis the probability of the event
L: observed is 1/4; it is unity or certainty in the second.
So far I have no difficulties: the above four lines could be
duplicated by any student at the end of a first lecture on
probability. But now it gets tricky:
L: Thus in regarding these hypotheses as so many causes, we
L: shall have for the sixth principle 1/5 and 4/5 for their
L: respective probabilities.
I see how he combined the numbers: 1/4 + 1 = 5/4; (1/4)÷(5/4) =
1/5 and 1÷(5/4) = 4/5. The result is plausible to this extent:
the urn must indeed contain either two white balls or one white
and one black, and the sum of the probabilities he gives is 1/5 +
4/5 = 1. I can follow his calculation, but my problem is with
this sort of ex post facto reasoning, using a single experiment
with known outcome to assign probabilities to unknown causes.
Laplace continues:
L: But if the first hypothesis occurs, the probability of
L: drawing a white ball at the third draw is 1/2; it is
L: equal to certainty in the second hypothesis; multiplying
L: then the last probabilities by those of the corresponding
L: hypotheses, the sum of the products, or 9/10, will be the
L: probability of drawing a white ball at the third draw.
My problem is with reasoning about the _probability_ of an event
(two white balls being in the urn, or one of each color being in
the urn) after the fact. This seems like looking at a golf ball on
a particular tuft of grass and asking the probability it would
land there; or it seems like the Monty Hall paradox.
It seems to me that an assumption is necessary to solve this
problem. For instance, we can assume that the person who
originally filled the urn did so at random. Then there would be a
1/4 probability it received two white balls, 1/2 for one of each
color, and 1/4 for two black balls. Our experiment has ruled out
the last case, so we are left with (1/4)÷(1/4+1/2) = 1/3 as the
probability of two white balls and (1/2)÷(1/4+1/2) = 2/3 as the
probability of one white and one black. In this case the
probability of a third white ball is (1/3)x1 + (2/3)*(1/2) = 2/3.
But other assumptions are possible. For instance, the person who
filled the urn might have deliberately chosen two white balls. In
this case the probability of a third white ball is obviously 1.
Or the person might have chosen one white and one black ball, in
which case the probability of a third white ball is 1/2.
So Laplace says the probability of a white ball on the third draw
is 9/10, and he does not mention making any assumptions. I say it
could be 2/3 or 1/2 or 1 (or something else), depending on
assumptions, but that on the facts as stated, without _some_
additional assumption, it cannot be computed at all.
Laplace was a great mathematician, but he lived two hundred years
ago and the mathematics of probability have advanced since then.
Even so, I suspect it's more likely that I'm wrong than that he
is. Can anyone put your finger on the logical flaw here, where
mine or that of the Marquis?
To avoid confusion, please don't send e-mail copies of public
responses. Thanks!
--
Stan Brown, Oak Road Systems, Cortland County, New York, USA
http://OakRoadSystems.com/
"My theory was a perfectly good one. The facts were misleading."
-- /The Lady Vanishes/ (1938)
John Uebersax
paradox concerning Laplace's "principle of indifference."
The principle of indifference states "given no reason to think
otherwise, consider all alternatives equally likely." The rub is
that the set of alternatives can be structured in different ways.
To illustrate, suppose you're given an urn and told:
1. The urn contains 2 marbles
2. Each marble is either white or black
3. There are not 2 black marbles in the urn
and then asked to estimate the probability that the urn has 2 white marbles.
One could reason two different ways:
View A
There are two alternative states:
i. the urn contains two white marbles
ii. the urn contains one white and one black marble
By this view, one estimates the probability of two white marbles is 1/2.
View B
There are three alternative states:
i. The urn contains two white marbles
ii. Marble 1 (by some arbitrary numbering) is white and Marble 2 is black
iii. Marble 1 is black and Marble 2 is white
By this view, one estimates the probability of two white marbles as 1/3.
Either View A or View B might be considered simpler, depending on whom
you ask. I suspect for Laplace, and for most nonstatisticians, View A
is simpler. But if I knew that a statistician had filled the urn, View
B might seem more obvious, as one would suppose a statistician would
apply random sampling.
I see parallels between this paradox and something philosophers call
"Goodman's paradox," as well as parallels to Occams razor:
"Choose from a set of otherwise equivalent models of a given phenomenon
the simplest one" [but who decides when models are 'otherwise equivalent'
and who decides which is 'simplest'?]
and Kant's categorical imperative:
"Act as if the maxim of your action were to become through your
will a universal law of nature."
[but the same action can define multiple maxims. For example, fighting
a war in self-defense is both killing (immoral) and acting in self-
defense (moral)].
--------------------------------------------------------------------------------
John Uebersax, PhD (858) 597-5571
La Jolla, California (858) 625-0155 (fax)
email: jsueb...@yahoo.com
Statistics: http://ourworld.compuserve.com/homepages/jsuebersax/agree.htm
Psychology: http://members.aol.com/spiritualpsych
--------------------------------------------------------------------------------
David Heiser
"John Uebersax" <jsueb...@yahoo.com> wrote in message
news:f6f3f967.03022...@posting.google.com...
> Stan's problem is very interesting. I believe it points to a more basic
> paradox concerning Laplace's "principle of indifference."
> "Choose from a set of otherwise equivalent models of a given phenomenon
> the simplest one" [but who decides when models are 'otherwise
equivalent'
> and who decides which is 'simplest'?]
These ideas that John points out here are absolutely fundamental.
Along the same line... Fisher's equiprobable sample, which is the basis for
using sampling/population distributions to come to some conclusions.
How do we determine "equi-probable"? By opinion of the most learned expert?
If the p value suggests a rejection of the simplest model, is it because of
the logical weakness of the hypothesis, or is because a more complicated
model is a better approximation to reality? This is the fundamental problem
in SEMNET, on determining whether a given Structural Equation fits or does
not fit a set of data and theory.
David Heiser
Aleks Jakulin
> The principle of indifference states "given no reason to think
> otherwise, consider all alternatives equally likely." The rub is
> that the set of alternatives can be structured in different ways.
The crucial issue is how to estimate the probability when the frequencies are
low. Laplace suggested that instead of computing the probability as n/N, one
should do something like n_a+1/(N+A), where A is the number of outcomes, n_a is
the count for outcome a, and N is the total outcome count.
Effectively, Laplace suggested a uniform prior in the Bayesian context. This
isn't "logical", but we use these things very often in machine learning, and we
obtain improved predictive accuracy. This is what we normally cite:
@inproceedings{,
author={Cestnik, B.},
title={Estimating probabilities: A crucial task in machine learning},
booktitle={Proc. 9th European Conference on Artificial Intelligence},
year={1990},
pages={147--149}
}
Aleks