Draft for Technical Introduction to Probability for Stan

[Michael Betancourt]

One of the common problems that we keep running into is that
everyone has different, and usually incomplete, understandings
of probability. Of course probability theory is relatively abstract
and we’re not going to be able to teach it to those without
reasonable math backgrounds: in that case the approaches
of McElreath and Kruschke are probably best. What doesn’t
exist in the literature is an introductory treatment of probability
theory and inference that goes into all of the technical details
for those who do have a reasonable math background. I’m
talking about concepts like concentration of measure, the
difference between probability, probability mass functions,
and probability density function.

Anyways, I think we’re going to need a treatment like that
in the Stan manual going forward, and this is my best
attempt at an initial draft, https://github.com/betanalpha/stan_intro/blob/master/stan_intro.pdf.
Conditioned on a familiarity with calculus everything should,
hopefully, be self-contained and well-defined.

The inference section is still a little bit in flux, but I’m really
happy about the general organization of the probability sections.
That said, I’m sure there are many topics that unclear or
too hastily introduced or could use more examples to build
intuition, so let me have it. Either here on the list or via an
issue/PR. Otherwise let me know what you think.

[Andrew Gelman]

It’s been said that when writing textbooks we each write what we wish we had been given to read when we were learning the subject. I took a look at what Micheal wrote and I think it’s excellent, with the target audience being Michael Betancourt circa 2010. It’s hard for me to judge how others would find it. I think that for some readers it would be just excellent, and for others it would be super-scary. To put ti another way, Michael refers to “reasonable math background” which covers a very wide range.
See you
A

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Michael Betancourt]

I’d say c.2006, but, yeah, definitely. The problem with going any simpler is that
we can’t explain why things like Jacobians arise, or why you have to be careful
with parameterizations. This was the lowest level I could find that was sufficiently
self-consistent — anything simpler will have to involve various “because I told you
so”s which often end up being very unfulfilling.

[Dustin Tran]

I looked it as well and think it’s excellent. It organizes a lot of ideas that I wish were presented this way more often (e.g., I really like section 5’s presentation on MAP estimators as dirac point mass approximations, Laplace estimators as the previous+covariance, variational estimators as optimization given a divergence function).

That said, I can’t tell how feasible this is for a person just getting into probability, even assuming the person has all the necessary mathematical background. It all seems too abstract to get a good handle of these objects as a first exposure. Joe Blitzstein and Carl Morris have an ongoing book manuscript on probability theory for statistical sciences that caters to the applied realm very well; they have several organizational techniques that might be useful here.

I like the separation of estimators into deterministic and stochastic; I also took a similar route as I’m writing this Python package that does statistical inference, and the internals use two base classes which correspond to these two paradigms. I wanted to mention one caveat, which is that the distinction can be quite blurry and it could be important to be mindful of these complications. As some examples which I’m sure you’re familiar with: a MAP optimization procedure with constant learning rate can converge with the target distribution as the stationary distribution; stochastic approximation theory uses Monte Carlo estimators within the steps to reach a stationary point and is not deterministic; and the expressiveness of the approximating family can grow unboundedly to indeed include the posterior as one such member in the family.

Dustin

[Andrew Gelman]

D
All fine but please don’t use the phrase “MAP estimator” in my presence. Thank you.
A

[Andrew Gelman]

I prefer to just call it the posterior mode.  I don’t like “MAP estimator” (or “Map estimator”) because to me it mystifies the whole process.  It’s not a new “estimator,” it’s just a particular summary of the posterior distribution.

On Mar 19, 2016, at 10:13 PM, Jiqiang Guo <guo...@gmail.com> wrote:

Andrew:

Do you mind telling me why?

Thanks,

Jiqiang 

[Bob Carpenter]

For what it's worth, I found the posterior mode as an
estimator much easier to understand (less mystical) than
MCMC (which I only later understood to compute expectations
using a particular estimator).

- Bob

[Dustin Tran]

I think this has varying levels of mystifying-ness. To others "posterior" is already mystifying enough. I think “a posteriori” is okay and so my okay-ness with MLE transfers over to M “a posteriori”. What about “mode of a joint density”, which is what it really is, and this has applications beyond finding modes of conditional distributions whose joint has an analytic form.

(I hope we can all agree it’s not as ambiguous as random effects or empirical Bayes!)

Dustin

[Jonah Gabry]

I've started a pull request and several issues over at Michael's repository (https://github.com/betanalpha/stan_intro) in case anyone wants to add anything.

[Bob Carpenter]

I'm about halfway through reading it pretty closely.
I think I get everything --- the math's at just the right level
for me now, and I like how it's finessing some of the more
complex definitions (a la Baez's awesome intro to manifolds).

What it needs is some more concrete examples (ones involving
trig are tough---if someone doesn't understand change of
variables, like I didn't, all those trig based examples are
complicated --- I needed to work through lognormal to "get
it").

It's all slightly unusual because rather than introducing
abstract sigma-algebras and random variables, everything's
baked into the space, so all the random variables are
projections (although the whole issue of random variables
is sidestepped). This makes it easy to generate sample spaces
by construction, whereas the usual level of indirection could be
confusing. I don't think I understood sample spaces until I
went through this construction myself. This got a little confusing
in the mass function setting, because in general, I can have a
continuous sample space (like a dartboard) and discrete
outcomes (like which wedge it falls in), to produce a mass
function. And I think what Michael's doing amounts to the same
thing in that there'd be three variables, x, y, and discrete
outcome, which are tightly coupled. If I understood what Michael
was up to, that is.

- Bob

> On Mar 20, 2016, at 5:19 PM, Jonah Gabry <jga...@gmail.com> wrote:
>
> I've started a pull request and several issues over at Michael's repository (https://github.com/betanalpha/stan_intro) in case anyone wants to add anything.
>

[Avraham Adler]

On Monday, March 21, 2016 at 11:35:52 AM UTC-4, Bob Carpenter wrote:
> I'm about halfway through reading it pretty closely.
> I think I get everything --- the math's at just the right level
> for me now, and I like how it's finessing some of the more
> complex definitions (a la Baez's awesome intro to manifolds).

Hi, Bob. You are saying that it's at the right level for you now. Just noting that most, if not all y'all, are Ph.D's (or at least post-grads) in math or computer science. Many of the users, such as myself, will not have such a rigorous grounding in statistics. We'll probably have more than nothing (Masters, actuarial exams, self-taught/online courses, etc.)

While most of what I have read so far (and I have not finished it yet) is rather accessible to the educated layperson, I'd put in a request for a little more explanation of some of the more esoteric concepts. For example, I'm very happy that Michael takes the time to explain probability spaces (something with which most practitioners will have little experience) but would be very appreciative if a little more exposition was made about conditional probability and what exactly the auxiliary space is and how it interacts with the unconditional probability space.

I've noticed a few typos/grammar errors. Would you want an issue for those or a fork/pull request on the LaTeX file?

Thanks,

Avi

[Bob Carpenter]

I agree. We'll need some more gentle introductions, even
for people with good math skills (I took topology and a year of
analysis as an undergrad math major, and I knew the basics
of measure theory coming in).

The best intro I've ever read to measure theory is in
the appendix to Anderson and Moore's book on Optimal Filtering.
It just lays out all the definitions very concisely and
consistently in a way very few other sources do. But no examples,
so it's more of a good way to consolidate understanding than to
learn from.

- Bob

[Michael Betancourt]

> What it needs is some more concrete examples (ones involving
> trig are tough---if someone doesn't understand change of
> variables, like I didn't, all those trig based examples are
> complicated --- I needed to work through lognormal to "get
> it”).

I’m happy to add more examples if people can suggest
useful ones, but I do have the warning about knowing
calculus specifically for the reason that things like
Jacobians and change of variables won’t make sense
without it.

> It's all slightly unusual because rather than introducing
> abstract sigma-algebras and random variables, everything's
> baked into the space, so all the random variables are
> projections (although the whole issue of random variables
> is sidestepped). This makes it easy to generate sample spaces
> by construction, whereas the usual level of indirection could be
> confusing.

Typically the material is presented as an abstract probability
triplet (sample space, sigma algebra, and probability distribution)
that doesn’t have any meaningful sense of value. Then random
variables map that system onto R or some R^{n} which does
have value, basically defining a “measurement” of the abstract
system. Then you go through all of the rigamarole of pushing
the sigma algebra and probability distribution forward onto
the image of the random variable to give a new probability
system you can actually calculate with using calculus. But
if you ignore all of the intermediate steps then all you’ve
done is changed representations (and maybe marginalized).

I never liked any of that as it’s never how things are done
operationally, hence my focus on equivalent sample spaces
as just being a representation degeneracy, which is ultimately
all it is.

> I don't think I understood sample spaces until I
> went through this construction myself. This got a little confusing
> in the mass function setting, because in general, I can have a
> continuous sample space (like a dartboard) and discrete
> outcomes (like which wedge it falls in), to produce a mass
> function. And I think what Michael's doing amounts to the same
> thing in that there'd be three variables, x, y, and discrete
> outcome, which are tightly coupled. If I understood what Michael
> was up to, that is.

This is a slightly particular example — if you can partition a
sample space into sets then you can always construct a discrete
probabilistic system implicitly by the mass function with takes

p(x_{1}) = \mathbb{P} [ A_{1} ].

Kind of like a singular marginalization from a continuous space
into a discrete one.

[Michael Betancourt]

As I noted in the original email, this is not meant as a gentle introduction
to people who want to learn about probability but don’t like math. In fact,
I _really_ don’t like trying to teach probability without some mathematical
foundation, because all it leads to is paradoxes and confusion!

What always happens is that people try to introduce probability theory
with probability densities and then inevitably everyone gets confused
with what are probabilities and what are densities and intuiting probability
as density and then becoming frustrated.

Or you could stick to probabilities over discrete spaces, but then you
don’t get to do anything on continuous spaces, which means no Gaussians
and other members of the exponential family.

This introduction is meant as the minimal material needed to really understand
statistical computation so that there are no “gotchya!”s that will bite the reader
later when they try to apply any lessons to real problems. Note that I went
to great effort to focus on concepts and relationships between concepts and
not on unhelpful calculation or proofs.

I am happy to add more examples and spend more time expositing on concepts
(suggestions, please!) At the same time, however, we have to be careful that
the reader doesn’t take the examples too seriously and lose some of the
necessary generality/abstraction.

[Bob Carpenter]

> On Mar 21, 2016, at 5:31 PM, Michael Betancourt <betan...@gmail.com> wrote:
>
>> What it needs is some more concrete examples (ones involving
>> trig are tough---if someone doesn't understand change of
>> variables, like I didn't, all those trig based examples are
>> complicated --- I needed to work through lognormal to "get
>> it”).
>
> I’m happy to add more examples if people can suggest
> useful ones, but I do have the warning about knowing
> calculus specifically for the reason that things like
> Jacobians and change of variables won’t make sense
> without it.

I wasn't saying don't do calc, I was just saying that you
could do simpler examples that aren't crazily behaved
trig functions, such as normal -> lognormal, or gamma to
inverse gamma.

It's one of those things that if they aren't already fluent
with changes of variables, your multivariate example is a
difficult place to start.

But I already see we're going to need a more gentle intro to
get readers to the point where they can read what you're
writing, so you can probably get away sticking to abstractions.

>> It's all slightly unusual because rather than introducing
>> abstract sigma-algebras and random variables, everything's
>> baked into the space, so all the random variables are
>> projections (although the whole issue of random variables
>> is sidestepped). This makes it easy to generate sample spaces
>> by construction, whereas the usual level of indirection could be
>> confusing.
>
> Typically the material is presented as an abstract probability
> triplet (sample space, sigma algebra, and probability distribution)
> that doesn’t have any meaningful sense of value. Then random
> variables map that system onto R or some R^{n} which does
> have value, basically defining a “measurement” of the abstract
> system. Then you go through all of the rigamarole of pushing
> the sigma algebra and probability distribution forward onto
> the image of the random variable to give a new probability
> system you can actually calculate with using calculus. But
> if you ignore all of the intermediate steps then all you’ve
> done is changed representations (and maybe marginalized).
>
> I never liked any of that as it’s never how things are done
> operationally, hence my focus on equivalent sample spaces
> as just being a representation degeneracy, which is ultimately
> all it is.

I understand. It's how Michael Collins (CS prof at Columbia
and at Google) explained it to me when I was trying to learn
probability theory because it's how he and Tommi Jakkola taught
it at MIT.

In the setup you have, random variables are just projections.
But if you define random variables, I haven't seen it yet.
Nor do you really talk about the required projections, but
rather do it implicitly by imposing structure on your sample
space (specificially product structure with dimensions correspodning
to [components of] random variables of interest).

>> I don't think I understood sample spaces until I
>> went through this construction myself. This got a little confusing
>> in the mass function setting, because in general, I can have a
>> continuous sample space (like a dartboard) and discrete
>> outcomes (like which wedge it falls in), to produce a mass
>> function. And I think what Michael's doing amounts to the same
>> thing in that there'd be three variables, x, y, and discrete
>> outcome, which are tightly coupled. If I understood what Michael
>> was up to, that is.
>
> This is a slightly particular example — if you can partition a
> sample space into sets then you can always construct a discrete
> probabilistic system implicitly by the mass function with takes
>
> p(x_{1}) = \mathbb{P} [ A_{1} ].
>
> Kind of like a singular marginalization from a continuous space
> into a discrete one.

Right. I really do get all this now, I think. What I was poining
out is that you are implicitly doing this when you construct the
sample space as a cross-prodcut of a discrete and continuous space.
Each discrete outcome corresponds to uncountably many points in the
sample space (though only one when you project down to its dimension).

- Bob

[Bob Carpenter]

> On Mar 21, 2016, at 5:40 PM, Michael Betancourt <betan...@gmail.com> wrote:
>
> As I noted in the original email, this is not meant as a gentle introduction
> to people who want to learn about probability but don’t like math. In fact,
> I _really_ don’t like trying to teach probability without some mathematical
> foundation, because all it leads to is paradoxes and confusion!

I completely agree. I wasn't objecting to the level of math. Just that
you need some concrete examples to help understand the nuances or even
basic concepts of the definitions.

> What always happens is that people try to introduce probability theory
> with probability densities and then inevitably everyone gets confused
> with what are probabilities and what are densities and intuiting probability
> as density and then becoming frustrated.

Exactly my path. :-) I was soooo confused after the basic math
stats books and BDA (which only muddled matters further for me with
its fast and loose approach to overloading random and bound variables
and density function names). It was only once I sat down and read
about measure theory and got a proper definition of random variables
that it all made sense. And it's not really any more complicated going
that route, so I'm really confused why everyone doesn't do what you're
doing in the intros---it's exactly how the MIT CS course on stats
was organized (build up "natural" product spaces).

> Or you could stick to probabilities over discrete spaces, but then you
> don’t get to do anything on continuous spaces, which means no Gaussians
> and other members of the exponential family.

That was Feller's approach, and it's how most people in natural language
try to go, but it makes the jump to continuous spaces really complicated
because they look like a whole new beast (not to mention everyone in
the applied world confuses the event probability Pr[] with the
density/mass p()).

> This introduction is meant as the minimal material needed to really understand
> statistical computation so that there are no “gotchya!”s that will bite the reader
> later when they try to apply any lessons to real problems. Note that I went
> to great effort to focus on concepts and relationships between concepts and
> not on unhelpful calculation or proofs.

Understood.

> I am happy to add more examples and spend more time expositing on concepts
> (suggestions, please!) At the same time, however, we have to be careful that
> the reader doesn’t take the examples too seriously and lose some of the
> necessary generality/abstraction.

I think Baez did a great job of this in his intro going back and
forth between general definitions and the sphere so you had something
that wasn't R^n to hang some intuitions off of.

- Bob

[Michael Betancourt]

I wasn't saying don't do calc, I was just saying that you
could do simpler examples that aren't crazily behaved
trig functions, such as normal -> lognormal, or gamma to
inverse gamma.

It's one of those things that if they aren't already fluent
with changes of variables, your multivariate example is a
difficult place to start.

Yeah, but the others don’t lead to much difference visually

and they introduce the additional complication of an added

constraint.  I had to carefully construct this one to be

visually striking in the figure.

In the setup you have, random variables are just projections.
But if you define random variables, I haven't seen it yet.
Nor do you really talk about the required projections, but
rather do it implicitly by imposing structure on your sample
space (specificially product structure with dimensions correspodning
to [components of] random variables of interest).

But in the end random variables are just maps between

probability triples and they don’t actually end up being

all that useful concepts in practice, hence I don’t introduce

them explicitly at all.  I just don’t see the need to do such

and never have.

[Bob Carpenter]

> On Mar 22, 2016, at 5:03 PM, Michael Betancourt <betan...@gmail.com> wrote:
>
> ...

> But in the end random variables are just maps between
> probability triples and they don’t actually end up being
> all that useful concepts in practice, hence I don’t introduce
> them explicitly at all. I just don’t see the need to do such
> and never have.

I'm not saying your intro has to be complete, just that it'll be
missing a widely used concept in applied stats if you don't cover
random variables. But that's OK, your intro doesn't have to be
complete.

- Bob

[Michael Betancourt]

A widely used concept in applied problems or in “applied stats” classes?
Again, I avoided it because I didn’t find it to be all that useful a concept
in practice. Let me know if I’m wrong.

[Bob Carpenter]

The language of random variables is widely used in applied
stats papers and textbooks, so yes, it's important for classes.
Do you need it to do applied stats? No! Will it be confusing
to read the literature if you don't know what it is? Yes!
It's what I found most confusing in the applied literature,
especially Bayesian stuff that conflates a ton of notation and
uses the language of random variables, like BDA.

- Bob

[Michael Betancourt]

That’s why I aimed to make this self-contained! It would be
impossible to explain enough background to make all textbooks
understandable. I’m instead trying to provide a single reference
that explains what Stan does and leave it to others to try to
connect that to other texts and variable notations.

[Jonah Sol Gabry]

What about a sentence or footnote explicitly pointing out the absence of the "random variable" from this treatment? 

You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/jug6OX20c8g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Aki Vehtari]

I really enjoyed reading the text, although I'm biased as I'm familiar
with the topic. I guess many of my master level students would have
difficulties but those same have have difficulties with BDA, too. I'll
give this to some of my more advanced students to read, too. Overall
it's an excellent overview (or beginning of such overview) and answers
many FAQs in the Stan users list.

One specific comment. You write
"Consequently the small world is equivalent to a probability distribution over the measurements space conditioned on the small world,
...
This conditional probability distribution is also known as the likelihood."

p(d|theta) as a function of d is conditional probability distribution.

p(d|theta) as a function of theta is the likelihood L(d|theta) which
is not a probability distribution. Sometimes it's proportional to a
probability distribution (posterior) and sometimes prior is required
to make it possible to normalise the posterior.

I think it is important to make this distinction to have a different
name for p(d|theta) and L(d|theta), but also because the models are
often described as generative models using conditional probability
distributions (observation models), but in Stan we often also write
some terms of log likelihood directly (e.g. for models with
censoring). I think that the new notation is also making it more
explicit that we are making statements about parameters theta.

Aki

[Bob Carpenter]

I think it might help to add a discussion of the relation between
quantiles and (inverse) CDFs. You can compute the marginal CDF
of any parameter theta at point x conditioned on some data y as

F_theta(x) = Pr[theta > x | y] = E[I[theta > x] | y]

But I don't know how to compute quantiles directly as expectations.

And what can I do if one of my variables has a marginal posterior
that's Cauchy? I don't think I can even properly marginalize it out,
or will there be cases where

INT f(alpha) * Cauchy(alpha | 0, 1) d.alpha

converges, even though

INT alpha * Cauchy(alpha | 0, 1) d.alpha

doesn't? That must be the case if in the posterior some of the
other variables are not Cauchy. (I realize these are going to
be problematic for NUTS, but that's a different issue.)

- Bob

[Michael Betancourt]

p(d|theta) as a function of d is conditional probability distribution.

p(d|theta) as a function of theta is the likelihood L(d|theta) which
is not a probability distribution. Sometimes it's proportional to a
probability distribution (posterior) and sometimes prior is required
to make it possible to normalise the posterior.

This is not entirely true — any conditional probability distribution,

p[ E | theta ], is a probability distribution for any theta in the

second argument and a measurable function for any E in the

first argument.  They are mathematically the same object.

The distinction I think you are trying to get at is more subtle,

and involves not just the likelihood but also the marginal

likelihood.  When we update from the prior to the posterior

measure theoretically we need a function,

P^{post} [E_{Theta} | d ] = (dP^{post} / dP^{prior})(theta)  P^{prior} [ E_{Theta} ]

P^{post} [E_{Theta} | d ] \equiv f(theta) P^{prior} [ E_{Theta} ].

f is the Radon-Nikodym derivative of the posterior with respect

to the prior, but is also the Radon-Nikodym derivative of the 

likelihood with respect to the marginal likelihood (this is basically

the measure theoretical version of Bayes’ Theorem).  

So the ambiguity that can arise is between the likelihood 

(a conditional probability distribution) and this updating

function (a Radon-Nikodym derivative).  But because you

need measure theory to make all of this make sense, and

it doesn’t really affect any practical use of Bayes, I don’t

think it’s all that helpful to discuss.

[Michael Betancourt]

The manuscript has been significantly updated and expanded,
https://github.com/betanalpha/stan_intro/blob/master/stan_intro.pdf.

[Aki Vehtari]

My thoughts about this have been influenced, e,g, by A. P. Dawid who has wrote the following

By A.P. Dawid
-------------------------------------------------------------------------

In a (doubtless ill-fated) attempt to explain more clearly the
essential distinction between a STATISTICAL MODEL and a LIKELIHOOD
FUNCTION --- one that until recently was widely understood but now
seems to engender great confusion --- I will try and make the point
as concrete as possible.

I am fishing. Everybody agrees that fish will bite according to a
Poisson process (hence the name!) of some rate \lambda bites per
hour. \lambda is unknown and we wish to make some inference about
\lambda from observing my performance.

Observer O1 plans to turn up after 3 hours and find out how many
fish, N, I have caught.

The STATISTICAL MODEL for N is the Poisson family Po(\lambda), with
discrete probability mass function:

(1) p_1( n | \lambda) = \exp{- 3 \lambda} \lambda^n /n!
(n = 0, 1, ...; \lambda >0 )

Observer O2 asks me to call her mobile as soon as I have caught 12
fishes. She will note the time T of this event.

The STATISTICAL MODEL for T is a Gamma family, Ga(12, \lambda),
with continuous probability density function:

(2) p_2( t | \lambda) = (\lambda^{12}/ 11!) t^{11} \exp{ -\lambda t}
(t > 0; \lambda > 0)

Now it so happens that I catch my 12th fish after exactly 3 hours.
Thus O1 observes N=12, while O2 observes T=3. Even though they
have ** utterly different ** STATISTICAL MODELS, involving even
different variables and different sample spaces, for this
particular outcome both O1 and O2 agree that the LIKELIHOOD
FUNCTION for \lambda (i.e. p_1( 12 | \lambda) for O1, p_2( 3 |
\lambda) for O2) is proportional to

(3) L(\lambda) = \lambda^{12} \exp{ -3 \lambda}
(\lambda > 0)

Indeed, I too Would obtain this, using as my STATISTICAL MODEL the
full Poisson process, and as my data the complete record of my
performance (i.e. exactly when each fish was caught) in the first 3
hours.

In no sense is (3) a PROBABILITY DISTRIBUTION for anything (or even
a probability for anything).

This example can be used to make various points about the
likelihood principle, optional stopping, sufficiency, etc. --- but
I will leave it there for now.

I surmise that one reason for the current confusion is the
(correct) statement that, in order to calculate a posterior
distribution we need both a prior distribution and a likelihood.
In fact it is only the ``observed'' likelihood function (e.g. (3)
in the story above) that is required. But typically we will have
started from a fully specified STATISTICAL MODEL, using this to
derive the relevant observed LIKELIHOOD FUNCTION. Because of this
link the distinction between the two has been elided.

Phil

-------------------------------------------------------------------------
A. P. Dawid, Professor of Statistics, Department of Statistical Science,
University College London, Gower Street, London WC1E 6BT, United Kingdom.
Tel:+44 (0)20 7679 1861 Fax: +44 (0)20 7383 4703
E-mail: da...@stats.ucl.ac.uk
Personal web page: http://tinyurl.com/2675a
Department web page: http://www.ucl.ac.uk/Stats/
-------------------------------------------------------------------------

[Michael Betancourt]

Phil is making the same mistake, conflating the likelihood
(a collection of probability distributions over data from a
frequentist perspective of a conditional probability distribution
from a Bayesian perspective) with the Radon-Nikodym
derivative that gets used in updating (what he calls the
“observed” likelihood).

The likelihood is a conditional probability distribution, so
if we plug in any data event then we get a function from
parameters to probabilities,

L[E_{D} | theta ] : theta \mapsto [0, 1].

But that’s not all that useful because we’ll mostly be
considering point events which have zero measure,
so this map degenerates to mapping everything to zero.

We could look at the corresponding conditional probability
density function l(d | theta), but then this depends on the
sample space. It’s why you get different density functions
if you’re looking at counts or time in Phil’s example.

When we’re performing a Bayesian update we care
about neither of these. We care only about the the

Radon-Nikodym derivative of the likelihood with respect

to the marginal likelihood,

f(d, theta) = dL_{conditional} / dL_{marginal} = l (d | theta} / l(d).

This is now a map from a point in data space and a parameter
to a real number, and we can plug in the observed data to
update our prior to our posterior,

P_{post} = f(d, theta) P_{prior}.

Now here’s the cool thing — f(d, theta) is invariant under
measurable maps. You can see this from the density
representation — the Jacobians cancel in the ratio. So
the actual learning is invariant to the sample space we
use to express the measurements, as it should be! This
is why in Phil’s example the “observed” likelihood was
invariant. But it’s really the observed likelihood/marginal
likelihood ratio that’s true invariant.

[Aki Vehtari]

Thanks for the additional explanation. I now accept that likelihood is a conditional probability distribution.

Aki

[Avraham Adler]

On Friday, April 1, 2016 at 2:26:48 PM UTC-4, Michael Betancourt wrote:
> The manuscript has been significantly updated and expanded,
> https://github.com/betanalpha/stan_intro/blob/master/stan_intro.pdf.

Speaking as a practitioner without a firm grounding in measure theory, this is significantly better as an introduction, not only than the original, but from other texts I have come across. Thank you.

Avi

[Michael Betancourt]

Okay, hopefully the last big change has been pushed: http://github.com/betanalpha/stan_intro.

The biggest change was an expanded discussion of propositional logic that completely
motivates measure and probability theory. Ultimately this is an introduction to measure/
probability theory, only without the usually mathematical pedantry and a much stronger
motivation from logic as a means of describing arbitrary systems. I also set up the
measure-theoretic version of Bayes’ Theorem from the beginning, again motivated
by logic, which makes the construction of Bayesian inference super easy.

I’m sure some of the notation can be cleaned up, and I’m happy to add more exposition
(or even exercises at some point?) if anyone finds sections particularly esoteric or
hard to read, but overall I think this is the level of detail that is necessary to really
understand what’s going on.

Once again, any and all comments are welcome.

[Charles Margossian]

Thank you creating this technical introduction: very stimulating and enlightening!

I find section 4 challenging, so I'm sharing my thoughts in the hope of getting some clarification, and pointing out a concept other readers might struggle with. I am having a hard time with Concentration of Measure, particularly the fact the neighborhood we want to explore does not include the mode in the hyper-sphere example.

So here's what I get: the volume element is proportional to r^(D-1), so dV increases as we move away from the origin, which is where the mode is. Consequently, the PMF is NOT concentrated at the origin but at a certain shell. This is clearly illustrated in figure 4.2 (I would consider rewriting the equations for dV and PDF in the figure's description). From a "markov chain point of view" this is telling me that the vast majority of the parameters I want to include in my sample are located in the "typical set shell". If I only stick around the mode, I will get the parameters with the highest PDF, but most of the mass will be lost, and my final sample will be unrepresentative.

Ok -- so if I were to pick the most "massive" shell, it would certainly not be located at the origin. But why am I constrained to pick a shell? If instead of picking a shell, I were picking a finite volume to do my exploration in, wouldn't the most massive volume include the origin of the sphere (and maybe part of the typical set shell)? So the key question is why do I need to pick a shell, instead of a volume, which to me is what only exploring the typical set seems to imply.

My tentative answer is that it takes computational power for a markov chain to move around the parameter space. But I am then unsure how exploring the entire typical set shell ends up being less costly than moving to the center of the sphere and out of it again.

Any clarification would be welcome!

~ Charles

[Michael Betancourt]

You’re not picking a single shell, you’re trying to understand where
in parameter space you should be evaluating the target function.
Any evaluations near the mode don’t contribute to any expectation
and so are wasted. You are welcome to include them, but will
pay a significant computational cost to do so.

[Jonah Sol Gabry]

Thanks for sharing Michael. The updates since the last version I read are really great. I'm about halfway through the latest version and will read the rest tonight on the train back to NYC from Boston. I'll let you know if I have any comments. 

You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/jug6OX20c8g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Michael Betancourt]

Just a warning that the first chapter should be taken

with a giant grain of salt as Bob very kindly informed

me of just how misinformed I was about logic!  Turns

out the connection between logic and probability is

not quite well-formed yet.

[Jonah Sol Gabry]

On Friday, May 13, 2016, Michael Betancourt <betan...@gmail.com> wrote:

Just a warning that the first chapter should be taken

with a giant grain of salt as Bob very kindly informed

me of just how misinformed I was about logic!  Turns

out the connection between logic and probability is

not quite well-formed yet.

Ok I'll be very interested to see what Bob's perspective changes about the first chapter. I thought it was solid so I'm going to need some of Bob's wisdom on that as well.