Stan 3 density notation: final stages

[Bob Carpenter]

I updated with respect to the Stan meeting discussion on Tuesday:

https://github.com/stan-dev/stan/wiki/Stan-3-Density-Notation-and-Increments

I think we're nearing agreement. To summarize, the unnormalized
form is:

target += normal_lpdf(y | mu, sigma);

and the normalized form is

target += normal_lpdf<norm>(y | mu, sigma);

We add = for assignment and deprecate <- and support += generally, but
only allow target to be used on LHS with +=.

We change suffix _lp to _target and _log to _lpdf in user-defined functions.

We do not allow vectorization control (at least for now).

We do not add a special link=logit, etc. form of control.

All existing functionality still supported, but deprecated. If we get
it in before Stan 3, we could just pull it out from under people in
Stan 3, or we could wait a year. The downside is all the material written
using our existing notation; the risk is that people will keep using
the existing notation, even if we deprecate it.

There are still a couple issues to resolve:

* vertical bar with lcdfs, lccdfs, and ldiff_cdfs?

* allow access to target as a value?

* allow vectors on RHS of += the way we do with increment_log_prob()?

Anyway, none of this will be at all challenging to build, though there's
going to have be a ton of tests around it to make sure it all works
and the deprecation warnings are right.

- Bob

[Michael Betancourt]

* vertical bar with lcdfs, lccdfs, and ldiff_cdfs?

I vote yes.

* allow access to target as a value?

I vote only in print statements, although I’d be fine with no.

[Ben Goodrich]

On Thursday, January 21, 2016 at 7:06:46 PM UTC-5, Michael Betancourt wrote:

* allow access to target as a value?

I vote only in print statements, although I’d be fine with no.

I am strongly for yes to allowing access to target as something like the get_lp() function now. This is important for verifying that user-defined Stan functions which are exposed to R are returning the expected result.

There was also the @ normal_lpdf(y | mu, sigma) notation, which sort of combined the concepts of incrementing and target. I'm not strongly advocating for it and I would be okay with what Bob summarized. But I think the @ syntax should get a full hearing.

Ben

[Andrew Gelman]

Any chance we could just have _lp instead of _lpdf or is the general judgment that _lp would confuse people?

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Krzysztof Sakrejda]

On Thursday, January 21, 2016 at 3:22:31 PM UTC-5, Bob Carpenter wrote:
> I updated with respect to the Stan meeting discussion on Tuesday:
>
> https://github.com/stan-dev/stan/wiki/Stan-3-Density-Notation-and-Increments
>
> I think we're nearing agreement. To summarize, the unnormalized
> form is:
>
> target += normal_lpdf(y | mu, sigma);
>
> and the normalized form is
>
> target += normal_lpdf<norm>(y | mu, sigma);
>
> We add = for assignment and deprecate <- and support += generally, but
> only allow target to be used on LHS with +=.
>
> We change suffix _lp to _target and _log to _lpdf in user-defined functions.
>
> We do not allow vectorization control (at least for now).
>
> We do not add a special link=logit, etc. form of control.

After reading the summary this makes sense to me.

> All existing functionality still supported, but deprecated. If we get
> it in before Stan 3, we could just pull it out from under people in
> Stan 3, or we could wait a year. The downside is all the material written
> using our existing notation; the risk is that people will keep using
> the existing notation, even if we deprecate it.
>
> There are still a couple issues to resolve:
>
> * vertical bar with lcdfs, lccdfs, and ldiff_cdfs?

Yes

> * allow access to target as a value?

Yes for print statements, no for anything else? The whole benefit of only allowing += is that
it's something you build up.

> * allow vectors on RHS of += the way we do with increment_log_prob()?

Seems confusing to me, I'd prefer no unless it would be a performance consideration for some reason?

[Krzysztof Sakrejda]

On Thursday, January 21, 2016 at 11:54:51 PM UTC-5, Andrew Gelman wrote:
> Any chance we could just have _lp instead of _lpdf or is the general judgment that _lp would confuse people?

Don't know about general judgement but I use enough different tools that having _lp but _lcdf or some other similar combination would force me to go to docs every time I need to write Stan code. Whatever it is I'd prefer consistent.

[Bob Carpenter]

> On Jan 22, 2016, at 9:18 AM, Krzysztof Sakrejda <krzysztof...@gmail.com> wrote:
>
> On Thursday, January 21, 2016 at 11:54:51 PM UTC-5, Andrew Gelman wrote:
>> Any chance we could just have _lp instead of _lpdf or is the general judgment that _lp would confuse people?
>
> Don't know about general judgement but I use enough different tools that having _lp but _lcdf or some other similar combination would force me to go to docs every time I need to write Stan code. Whatever it is I'd prefer consistent.

That's one of the two reasons the proposal stands as it is.
The other is that we want the code to be easily readable even
by people who aren't familiar with Stan and "_lp" is too opaque.

On that score, "_log_pdf" would be even clearer than "_lpdf",
but there seems to be a strong aversion to the full "log" --- and
it's more verbose. And I'm guessing people will figure out the
"l" is log once they see the "pdf" --- but that remains to be
seen!

- Bob

[Jonah Gabry]

I'm curious to see what users' reactions will be when we change assignment from "<-" to "=" given that the majority of users seem to use the RStan interface and are used to "<-". Personally, I'm fine with making that change, as it brings Stan more in line with the overwhelming majority of programming languages, but I do think there is something nice about using "<-" for assignment, as "=" already has other uses unrelated to assignment.

[Ben Goodrich]

On Friday, January 22, 2016 at 1:06:32 PM UTC-5, Jonah Gabry wrote:

I'm curious to see what users' reactions will be when we change assignment from "<-" to "=" given that the majority of users seem to use the RStan interface and are used to "<-". Personally, I'm fine with making that change, as it brings Stan more in line with the overwhelming majority of programming languages, but I do think there is something nice about using "<-" for assignment, as "=" already has other uses unrelated to assignment.

If we do += for incrementing target, then it makes sense to use = for assignment. And it would be pretty easy for the parser to catch a <- and suggest changing it to =. And it is pretty easy for people to do a find-and-replace in their existing Stan programs. But it is a change that affects almost every line without providing a whole lot of benefit. Perhaps if there were more people in favor of the @ notation, we could keep <- for assignment.

Ben 

[Jonah Sol Gabry]

I didn't like the @ notation at first, but the more I think about it the less I dislike it. 

--
You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/325D3xsaI08/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Krzysztof Sakrejda]

On Friday, January 22, 2016 at 1:35:39 PM UTC-5, Ben Goodrich wrote:
> On Friday, January 22, 2016 at 1:06:32 PM UTC-5, Jonah Gabry wrote:I'm curious to see what users' reactions will be when we change assignment from "<-" to "=" given that the majority of users seem to use the RStan interface and are used to "<-". Personally, I'm fine with making that change, as it brings Stan more in line with the overwhelming majority of programming languages, but I do think there is something nice about using "<-" for assignment, as "=" already has other uses unrelated to assignment.
>
>
> If we do += for incrementing target, then it makes sense to use = for assignment. And it would be pretty easy for the parser to catch a <- and suggest changing it to =. And it is pretty easy for people to do a find-and-replace in their existing Stan programs. But it is a change that affects almost every line without providing a whole lot of benefit. Perhaps if there were more people in favor of the @ notation, we could keep <- for assignment.

Would we also have an @increment_log_prob(...) or similar, or would the "@" notation restrict incrementing the target density to only certain functions? I have often used the functionality of defining arbitrary densities. with increment_log_prob...


>
> Ben 

[Ben Goodrich]

I'm not sure that question has been discussed. My intuition is that if you do

@ 42;

it would be conceptually equivalent to

target += 42;

so maybe an explicit increment_log_prob() function would not be needed.

Ben

[Krzysztof Sakrejda]

Yikes, that would be very obscure. Krzysztof

>
> Ben

[Jonah Sol Gabry]

Yeah the @ in conjunction with one of the densities looks ok but I agree that @42 would be strange.

--
You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/325D3xsaI08/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Bob Carpenter]

Abs

Whatever we do, normal_lpdf(y | mu, sigma) is just going to be an
expression evaluating to a real number (or maybe a vector, but I'm leaning
against it). So I'm envisioning a statement

target += <expression>;

or

@ <expression>;

Either way, it increments the log density accumulator with the value of the expression,
so any valid expression would work.

I tend to agree that symbols like @ tend to make code look like a mess
and that the "target +=" form seems much less obscure for someone
who doesn't know the language. That's why I dropped it.

- Bob

[Jonah Sol Gabry]

What about something in between those two, e.g.

@ += <expression>;

where @ is just shorthand for target since it looks a bit like a target. Just using target is probably better, but thought I'd throw it out there.

--
You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/325D3xsaI08/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Joshua N Pritikin]

On Fri, Jan 22, 2016 at 03:11:24PM -0500, Jonah Sol Gabry wrote:
> What about something in between those two, e.g.
>
> @ += <expression>;
>
> where @ is just shorthand for target since it looks a bit like a
> target. Just using target is probably better, but thought I'd throw it
> out there.

Have we considered whitespace indent level (like python)? It might make
more sense here than it does even in python because users almost always
want to increment the log probability. Since users are generally in the
habit of indenting to the appropriate level already, that might be a
least effort and least visually noisy way to indicate it.

$0.02

[Daniel Lee]

I'm vetoing using whitespace indent levels.

some reasons:

1) not backwards compatible

2) users are still jamming everything in strings without any care to indentation

3) ... no. I can't deal with it.

(I don't really have veto power, but I'm very strongly opposed to using whitespace in a "clever" way)

Daniel

 

[Jonah Sol Gabry]

On Fri, Jan 22, 2016 at 4:01 PM, Daniel Lee <bea...@alum.mit.edu> wrote:

2) users are still jamming everything in strings without any care to indentation

I still don't understand why people continue to do this. Well, I never understood why anyone ever did it in the first place either.

[Bob Carpenter]

Old habits die hard. I don't even like ending lines
of R without a semicolon!

Only Python users (and makefile experts) are used to whitespace (tab)
indent, and even with nicely formatting text editors like emacs, it
can lead to errors.

The nice part about paren balancing is that editors highlight
the matching parens and do the indentation so you get
something like a checksum. That's why I feel safe using no-paren
indents in our C++ code.

The downside of bracketsq is a bit of redundancy and wasted
vertical space for the close brackets. And all in, I think
well-formatted Python is about the easiest code to read out
there.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Bob Carpenter]

This is getting off topic, but I think they put things
in strings because that's how the getting-started docs
got started.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Michael Betancourt]

I support Dan’s imaginary veto.

-- 

You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Daniel Lee]

There's been a lot of discussion. Here's where I stand:

Proposed suffixes:

- I'd prefer if the pdfs on the log scale were left without a suffix, but I think I'm the only one. I was thinking about treating lpdfs / lpmfs as first class objects that are always on the log scale. Yes, it doesn't run parallel with cdfs, but I was thinking about the most common use case and how Stan programs look. I'm assuming this idea won't be supported by anyone else and this will be the last I bring it up, unless someone else supports it.

- log pdf / pmf: I prefer _lp, but would be ok with _lpdf. Although I understand Michael's point about forcing the user to think about pdf vs pmf, I think it's going to cause confusion and slow down the process.

Vertical bar notation:

- yes. I don't know what to do with user-defined multivariate distributions, but we can figure that out later.

Normalization control:

- I prefer having "<norm>" and "". It saves space. I see the point about not being the best decision for adding other control, but it's either normalized or not.

- default: unnormalized. For speed.

Increment log density statement:

- one more vote for "target +="

- one downvote for @. Looks too much like an attribute to me.

- I want something on the left size. target seems right. Second choice: model.

Link functions:

- I'm ok with the proposed which is the same as what we have now.

- If we went with <>, would it be too confusing to drop the "link=" so we could do something like: poisson<log>... nm, I don't think this.

Misc:

- +1 for switching assignment from <- to =.

- +1 for += everywhere

[Bob Carpenter]

> On Jan 22, 2016, at 4:20 PM, Daniel Lee <bea...@alum.mit.edu> wrote:
>
> There's been a lot of discussion. Here's where I stand:
>
> Proposed suffixes:
> - I'd prefer if the pdfs on the log scale were left without a suffix, but I think I'm the only one. I was thinking about treating lpdfs / lpmfs as first class objects that are always on the log scale. Yes, it doesn't run parallel with cdfs, but I was thinking about the most common use case and how Stan programs look. I'm assuming this idea won't be supported by anyone else and this will be the last I bring it up, unless someone else supports it.

I'm pretty sure that's also what Andrew wants. I'm leaning
toward just being more explicit and consistent.

It's easier to write and easier to read as normal() when you
understand Stan, but harder to learn in a way that
supports what Stan actually does.

> - log pdf / pmf: I prefer _lp, but would be ok with _lpdf. Although I understand Michael's point about forcing the user to think about pdf vs pmf, I think it's going to cause confusion and slow down the process.

This is the biggest drawback to "_pdf" --- I don't
like having to distinguish. Especially since the two
options don't even partition the probability functions.
What we're really defnining as a model with a Stan
program is often the joint density of a combination of
discrete and continuous variables.

I prefer nothing to "_lp". I don't think "_lp" conveys
enough and it lacks parallelism with "_cdf" and "_rng".

> Vertical bar notation:
> - yes. I don't know what to do with user-defined multivariate distributions, but we can figure that out later.

I'm also on the fence on this one. I don't think user-defined
or developer-defined multivariates will be a major. We can
change the way the function's defined to match, for example.

real my_distribution_pdf(y | foo, bar) {
...

}

> Normalization control:
> - I prefer having "<norm>" and "". It saves space. I see the point about not being the best decision for adding other control, but it's either normalized or not.
> - default: unnormalized. For speed.

I like this, too. I think the idea is that <norm> will force
normalization and leaving it blank will leave in the template
parameter. So we could control from the outside and still norm
everything. I don't think there's ever a case where the user
needs the unnormalized form.

> Increment log density statement:
> - one more vote for "target +="
> - one downvote for @. Looks too much like an attribute to me.
> - I want something on the left size. target seems right. Second choice: model.

Models are joint densities in my mind, so using "model" here
seems like a type error.

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Daniel Lee]

Thanks, Bob!

On Fri, Jan 22, 2016 at 4:30 PM, Bob Carpenter <ca...@alias-i.com> wrote:

> On Jan 22, 2016, at 4:20 PM, Daniel Lee <bea...@alum.mit.edu> wrote:
>
> There's been a lot of discussion. Here's where I stand:
>
> Proposed suffixes:
> - I'd prefer if the pdfs on the log scale were left without a suffix, but I think I'm the only one. I was thinking about treating lpdfs / lpmfs as first class objects that are always on the log scale. Yes, it doesn't run parallel with cdfs, but I was thinking about the most common use case and how Stan programs look. I'm assuming this idea won't be supported by anyone else and this will be the last I bring it up, unless someone else supports it.

I'm pretty sure that's also what Andrew wants.  I'm leaning
toward just being more explicit and consistent.

It's easier to write and easier to read as normal() when you
understand Stan, but harder to learn in a way that
supports what Stan actually does.

> - log pdf / pmf: I prefer _lp, but would be ok with _lpdf. Although I understand Michael's point about forcing the user to think about pdf vs pmf, I think it's going to cause confusion and slow down the process.

This is the biggest drawback to "_pdf" --- I don't
like having to distinguish.  Especially since the two
options don't even partition the probability functions.
What we're really defnining as a model with a Stan
program is often the joint density of a combination of
discrete and continuous variables.

I prefer nothing to "_lp".  I don't think "_lp" conveys
enough and it lacks parallelism with "_cdf" and "_rng".

Ok. My new ranking is:

1) _lpdf

2) _lpdf / lpmf

3) (nothing)

4) _lp

 

> Vertical bar notation:
> - yes. I don't know what to do with user-defined multivariate distributions, but we can figure that out later.

I'm also on the fence on this one.  I don't think user-defined
or developer-defined multivariates will be a major.  We can
change the way the function's defined to match, for example.

  real my_distribution_pdf(y | foo, bar) {
    ...
  }

+1. I didn't think that one through.

 

> Normalization control:
> - I prefer having "<norm>" and "". It saves space. I see the point about not being the best decision for adding other control, but it's either normalized or not.
> - default: unnormalized. For speed.

I like this, too.  I think the idea is that <norm> will force
normalization and leaving it blank will leave in the template
parameter.  So we could control from the outside and still norm
everything.  I don't think there's ever a case where the user
needs the unnormalized form.

> Increment log density statement:
> - one more vote for "target +="
> - one downvote for @. Looks too much like an attribute to me.
> - I want something on the left size. target seems right. Second choice: model.

Models are joint densities in my mind, so using "model" here
seems like a type error.

+1. I'm now just for having "target" on the lhs. I do not want "" (nothing) on the lhs. I also do not want "model" on the lhs.

[Michael Betancourt]

What we're really defnining as a model with a Stan
program is often the joint density of a combination of
discrete and continuous variables.

No no no no no no no.  I know Andrew likes thinking about

models this way due to the prior/likelihood ambiguities in  

hierarchical models, but that’s not what we do in Stan.

Stan defines a conditional probability density function,

p(alpha | beta) where alpha are defined in the parameters

block (or functions there of) and beta are defined in data,

transformed data, or sometimes as local constants.  The

importance of this distinction, besides where the variables

are defined, is that alpha must be continuous while beta 

can be continuous or discrete.

Sloppy equivalences like

p(alpha | beta) \propto p(alpha, beta)

make sense only when we keep this type distinction in

mind and can actually drop the intermediate constant.  

If we were just defining a joint distribution then we could 

write

p(alpha, beta) = p(alpha | beta) p(beta) \propto p(beta | alpha)

p(alpha, beta) = p(beta | alpha) p(alpha) \propto p(alpha | beta)

hence p(beta | alpha) and p(alpha | beta) would be

equivalent!  But Stan can define the only the latter

when any of the beta are discrete.

To be more formal, what we are defining is a disintegration of

the joint distribution p(alpha, beta) which requires more

information than the joint distribution itself.

[Ben Goodrich]

On Friday, January 22, 2016 at 4:41:45 PM UTC-5, Daniel Lee wrote:

+1. I'm now just for having "target" on the lhs. I do not want "" (nothing) on the lhs. I also do not want "model" on the lhs.

I don't hate the word "target" on the lhs, but I don't think it is going to be evident to users what is being targeted or how the rhs pertains to that. The somewhat longer word "objective" seems like the accurate term if we were doing optimization but not so much if we were doing MCMC. I suppose you can say MCMC is targeting the posterior distribution not the thing that is actually being incremented.

Ben

[Michael Betancourt]

“target distribution” is the MCMC equivalent of “objective function” in optimization.

In Bayesian inference the target distribution is the posterior, although Stan is

technically more general than that.  I’m not sure what you mean by the posterior

not being the thing that is being incremented — at any point if we stopped then

the current state of the LHS would be the target distribution.

[Ben Goodrich]

On Friday, January 22, 2016 at 4:59:18 PM UTC-5, Michael Betancourt wrote:

“target distribution” is the MCMC equivalent of “objective function” in optimization.

In Bayesian inference the target distribution is the posterior, although Stan is

technically more general than that.  I’m not sure what you mean by the posterior

not being the thing that is being incremented — at any point if we stopped then

the current state of the LHS would be the target distribution.

Yes, it's the posterior distribution being targeted by MCMC but if

normal_lpdf(y | mu, sigma);

evaluates to some number like 0.12345 it is not as if

target += normal_lpdf(y | mu, sigma);

means something like "increase the posterior distribution by 0.12345". It means "increase the value of the log of the numerator in Bayes' theorem by 0.12345".

When maximizing, it is reasonably clear that the goal is to choose the values of the parameters to make the objective function as large as possible. I think "target" would be perceived by users as implying "I am adding 0.12345 to something in order to make it closer to its target".

Ben

[Bob Carpenter]

Right --- there's an asymmetry between the data x and
parameters theta to put it back in Andrew's notation.
And what we care about is that the model returns
something that's proportional to

target = log p(theta | x) + Z(x)

where we don't care about Z(x) [assuming the unconstraining
transforms and Jacobians].

All I'm saying is that if you code the model as usually
written and include the normalization terms, what you
get for lp__ (target) is log p(theta, x), not
log p(theta | x).

But what's cool about Stan is that you don't have to
code the model that way. As long as whatever you code
is proportional to p(theta | x), you're good to go.

- Bob

[Michael Betancourt]

Well “target distribution” is standard MCMC terminology and

we can emphasize that in the doc.  Besides, we never actually

add a number of the LHS — we’re always aggregating a function.

[Ben Goodrich]

On Friday, January 22, 2016 at 5:27:20 PM UTC-5, Michael Betancourt wrote:

Well “target distribution” is standard MCMC terminology and

we can emphasize that in the doc.  Besides, we never actually

add a number of the LHS — we’re always aggregating a function.

I guess that is what I anticipate users finding confusing. The way the += syntax is used in non-Stan examples all over the internet suggests numbers rather than defining a function piece by piece that is within a constant of the log density of the posterior distribution being targeted by MCMC.

Ben
 

On Jan 22, 2016, at 10:18 PM, Ben Goodrich <goodri...@gmail.com> wrote:

On Friday, January 22, 2016 at 4:59:18 PM UTC-5, Michael Betancourt wrote:

“target distribution” is the MCMC equivalent of “objective function” in optimization.

In Bayesian inference the target distribution is the posterior, although Stan is

technically more general than that.  I’m not sure what you mean by the posterior

not being the thing that is being incremented — at any point if we stopped then

the current state of the LHS would be the target distribution.

Yes, it's the posterior distribution being targeted by MCMC but if

normal_lpdf(y | mu, sigma);


evaluates to some number like 0.12345 it is not as if

target += normal_lpdf(y | mu, sigma);


means something like "increase the posterior distribution by 0.12345". It means "increase the value of the log of the numerator in Bayes' theorem by 0.12345".

When maximizing, it is reasonably clear that the goal is to choose the values of the parameters to make the objective function as large as possible. I think "target" would be perceived by users as implying "I am adding 0.12345 to something in order to make it closer to its target".

Ben

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

[Michael Betancourt]

Fair point — in fact I think that’s why I originally suggested a stream.

But I don’t think there’s any way to discriminate between adding

functions and adding values so it may have to be something that

we tackle with detailed doc.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Ben Goodrich]

On Friday, January 22, 2016 at 5:59:16 PM UTC-5, Michael Betancourt wrote:

Fair point — in fact I think that’s why I originally suggested a stream.

But I don’t think there’s any way to discriminate between adding

functions and adding values so it may have to be something that

we tackle with detailed doc.

I guess I would slightly prefer a keyword like t_lpdf to target but if others like target better, we can go with that.

[Michael Betancourt]

I would also prefer t_lpdf to emphasize that we’re specifying a log PDF

but I believe others thought that “target” was more readable and I can

see that argument.

[Jonah Gabry]

One last try with @ and then I'll get off the bandwagon: what about @lpdf?

[Michael Betancourt]

I'm way off the @ bandwagon.

[Jonah Sol Gabry]

Bandwagon officially dismantled.

You received this message because you are subscribed to a topic in the Google Groups "stan development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-dev/325D3xsaI08/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-dev+u...@googlegroups.com.

[Andrew Gelman]

Hi, I’d actually use lp always, thus normal_lp or normal_cdf_lp or whatever. I guess the alternative would be to use normal_lpdf or normal_lcdf.

> On Jan 22, 2016, at 9:18 AM, Krzysztof Sakrejda <krzysztof...@gmail.com> wrote:
>
> On Thursday, January 21, 2016 at 11:54:51 PM UTC-5, Andrew Gelman wrote:
>> Any chance we could just have _lp instead of _lpdf or is the general judgment that _lp would confuse people?
>
> Don't know about general judgement but I use enough different tools that having _lp but _lcdf or some other similar combination would force me to go to docs every time I need to write Stan code. Whatever it is I'd prefer consistent.
>
>
>>
>>> On Jan 21, 2016, at 3:21 PM, Bob Carpenter <@alias-i.com> wrote:
>>>
>>> I updated with respect to the Stan meeting discussion on Tuesday:
>>>
>>> https://github.com/stan-dev/stan/wiki/Stan-3-Density-Notation-and-Increments
>>>
>>> I think we're nearing agreement. To summarize, the unnormalized
>>> form is:

>>>
>>> target += normal_lpdf(y | mu, sigma);
>>>

>>> and the normalized form is
>>>
>>> target += normal_lpdf<norm>(y | mu, sigma);
>>>
>>> We add = for assignment and deprecate <- and support += generally, but
>>> only allow target to be used on LHS with +=.
>>>
>>> We change suffix _lp to _target and _log to _lpdf in user-defined functions.
>>>
>>> We do not allow vectorization control (at least for now).
>>>
>>> We do not add a special link=logit, etc. form of control.
>>>
>>> All existing functionality still supported, but deprecated. If we get
>>> it in before Stan 3, we could just pull it out from under people in
>>> Stan 3, or we could wait a year. The downside is all the material written
>>> using our existing notation; the risk is that people will keep using
>>> the existing notation, even if we deprecate it.
>>>
>>> There are still a couple issues to resolve:
>>>
>>> * vertical bar with lcdfs, lccdfs, and ldiff_cdfs?
>>>
>>> * allow access to target as a value?
>>>
>>> * allow vectors on RHS of += the way we do with increment_log_prob()?
>>>
>>> Anyway, none of this will be at all challenging to build, though there's
>>> going to have be a ton of tests around it to make sure it all works
>>> and the deprecation warnings are right.
>>>
>>> - Bob

[Andrew Gelman]

But I’d prefer no suffix at all for pdfs…

But in any case I defer to the consensus judgments.

A

[Dustin Tran]

(As a minor person sitting in the corner inspecting all this, and I certainly should very little say, I also prefer _lp on everything. _lpdf and _lcdf look like a jumble of letters.)

[Bob Carpenter]

It'll work just like any other program, like

double f(double x, sigma) {
double target = 0;
target += -(x * x) / (sigma * sigma * 2);
target -= log(sigma);
return target;
}

A variable is part of the function definition. Given
an execution context and state of execution (for
instance, called with x = -1.2, sigma = 0.5,
done executing first two statements), it will have
a value. So in a progamming language context, you
usually think of a variable as a mapping from
execution contexts (often modeled as some kind of
continuation) and a value. The value will have a type
determined by the variable's type.

The big difference is that the target value in Stan
will underlying be an accumulator collecting a sequence
of values (usually autodiff variables) and will itself
have to be evaluated to get a result (the evaluation is
just summing, but we collect them up for efficiency to
reduce the size of the resulting expression graph). Another
difference is that it's implicitly declared and initialized
(optionally) to the Jacobian of the constraining transform.

- Bob

>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Bob Carpenter]

Nobody's minor --- please jump in if you have something to say.

Are you agreeing with Andrew, normal_lp() for log pdf/pmf and
normal_cdf_lp() for log CDF?

That's back to where we started with "lp" in place of "log",
which just feels more obscure. But the argument against "log"
for those keeping score is that it conflicts with the
link function for Poisson and nobody likes poisson_log_log().
And I think we've prett much tabled the <link=log> business
because we think users will overgeneralize.

- Bob

[Ben Goodrich]

On Friday, January 22, 2016 at 11:22:33 PM UTC-5, Bob Carpenter wrote:

Are you agreeing with Andrew, normal_lp() for log pdf/pmf and
normal_cdf_lp() for log CDF?  

That's back to where we started with "lp" in place of "log",
which just feels more obscure.  

I would rather have a _log suffix than have an _lp suffix where the "p" refers to different concepts depending on what prefixes it.

Ben

[Dustin Tran]

Yes I think _lp should appended, so normal_lp() and normal_cdf_lp(). Levels of detail:

_lpdf / _lpmf / _lcdf
_lpdf / _lcdf
_log / _cdf_log (including _log_log)
_lp / _cdf_lp
(nothing) / _cdf

Most of us are okay with dropping the distinction between _lpdf and _lpmf due to its added confusion for users. I don’t think _log makes sense, one reason being the requirement of _log_log, and more generally that it’s doesn’t convey anything regarding density (“lognormal_log” is also strange). I think I’m okay with _lp because we’ve already been using log prob for so many things both code wise and user-wise, and so I/stan users have already internalized it. _log_pdf/log_cdf would be too verbose but any jumble that requires enunciating at least 4 characters seems a bit much both for typing and internalizing.

Dustin

[Bob Carpenter]

I originally derived "lp__" from "log pdf" and "log pmf"
because Andrew calls pdfs and pmfs "probability functions"
in BDA and "lp__" was the internal accumulator for the log
probability function. In retrospect, I regret that decision
because it's too easy to take it to mean "log probability", and
it's a density, not a probability.

The reason I don't like the unmarked case "normal(y | mu, sigma)"
for the log density is that I want the expression to just be a function,
and the function normal( | , ) is pretty much universally understood
in papers and textbooks to mean the linear scale.

I don't mind "lognormal_log", but "poisson_log_log" is pushing
things. I also don't like neg_binomial_2, becuase like the link=log,
it's just another parameterization (though I don't think it's
considered a link function when you switch normal or gamma or negative
binomial parameterizations).

- Bob

P.S. Why is "pdf" lower-cased? It's not pronounced like
a word, so even Andrew wouldn't lower-case it on
first principles. Maybe the early probability theory
authors thought "PDF" everywhere looked tacky :-)


- Bob

[Ben Goodrich]

On Friday, January 22, 2016 at 11:48:53 PM UTC-5, Dustin Tran wrote:

Yes I think _lp should appended, so normal_lp() and normal_cdf_lp(). Levels of detail:

I could live with _lp referring either to the log of a PDF or a PMF but normal_cdf_lp() seems really weird to me. A CDF is already returning a probability and even if you put in on the log scale, what additional information is the trailing p supposed to convey? I get that poisson_log_log() is bad but poisson_loglink_log() is not so bad (just long). Although I don't mind typing more characters at all if we can convey the meanings to users more clearly.

Ben

[Michael Betancourt]

This is exactly why being explicit is important — when everything is properly

labeled the Stan program itself can put errors into context.  You can’t have

pmfs with parameters left of the bar, or CDFs incremented into the target,

etc, etc.  Also the names give users something to look up when they’re

running into trouble.  All of this needs to be supported in the doc, but it

leads to a much cleaner user experience, especially for new users (it’s

always better to have a giant pile of material to read than not know what

your problem is).

Again, I’m not arguing for what is easier for _me_ to read/use, but rather

new users.

[Aki Vehtari]

Could you add also couple full model examples with the proposed notation? For example some mixture model with unnormalized and normalized densities, and some censored observation model for _cdf or for _ccdf. It would be easier to see how nice and clear it would look like (and others proposing something else could first test with these same examples how they would look).

Aki

[Bob Carpenter]

I think that's an excellent idea. Will do.

- Bob