Function documentation in the manual; feedback requested

[Daniel Lee]

Hi all,

Here are some options for the documentation. It's on the feature/issue-843-test-nan-Phi branch, if you want to edit manually.

I've put together 3 options.

1. the main use first, infinite cases not explicitly mentioned.

2. ordered by range of x, inf explicitly mentioned

3. every case broken out, including inf, nan.

Other thoughts:

- should we use a different notation for ranges?

- Phi(x) on the left side -- should it be in the code font?

- location of the definition of \Phi(x)? use of \equiv?

[Ben Goodrich]

On Friday, August 22, 2014 10:01:20 AM UTC-4, Daniel Lee wrote:

Here are some options for the documentation. It's on the feature/issue-843-test-nan-Phi branch, if you want to edit manually.

I've put together 3 options.

1. the main use first, infinite cases not explicitly mentioned.

Yes
 

Other thoughts:

- should we use a different notation for ranges?

I think this is fine. If you thinking about the alternative notation, $x \in \left(-37.5,8.25\right)$, that can be confusing in some countries.

- Phi(x) on the left side -- should it be in the code font?

Yeah, I think that would emphasize that Phi(x) is the computational implementation and $\Phi\left(x\right)$ is the mathematical function.

- location of the definition of \Phi(x)? use of \equiv?

Nah, if we are emphasizing that Phi(x) is the computational implementation, then $\equiv$ would seem counter to that.

Ben

P.S. We also need to implement Phi_inv(p) soon, such that Phi_inv(Phi(x)) = x. But should Phi_inv(0) return -37.5 + epsilon or -inf? I guess input to Phi_inv() that was less than 0 or greater than 1 would throw an exception nowdays, rather than NaN or an infinite value.

[Daniel Lee]

Thanks.

On Aug 22, 2014 11:08 AM, "Ben Goodrich" <goodri...@gmail.com> wrote:
>
> On Friday, August 22, 2014 10:01:20 AM UTC-4, Daniel Lee wrote:
>>
>> Here are some options for the documentation. It's on the feature/issue-843-test-nan-Phi branch, if you want to edit manually.
>>
>> I've put together 3 options.
>> 1. the main use first, infinite cases not explicitly mentioned.
>
>
> Yes

And this order is preferred? Main, then special cases?

>  
>>
>> Other thoughts:
>> - should we use a different notation for ranges?
>
>
> I think this is fine. If you thinking about the alternative notation, $x \in \left(-37.5,8.25\right)$, that can be confusing in some countries.
>

Exactly. Thanks, I read most of my stuff in English.

>> - Phi(x) on the left side -- should it be in the code font?
>
>
> Yeah, I think that would emphasize that Phi(x) is the computational implementation and $\Phi\left(x\right)$ is the mathematical function.

Great. Will figure it out.

>
>> - location of the definition of \Phi(x)? use of \equiv?
>
>
> Nah, if we are emphasizing that Phi(x) is the computational implementation, then $\equiv$ would seem counter to that.

Sorry, I wasn't clear. I meant within the paragraph. An other option is to put a \Phi(x) = ... within the first line of the conditional equation. I thought it would have been too busy and it was clear defined within the paragraph.

>
> Ben
>
> P.S. We also need to implement Phi_inv(p) soon, such that Phi_inv(Phi(x)) = x. But should Phi_inv(0) return -37.5 + epsilon or -inf? I guess input to Phi_inv() that was less than 0 or greater than 1 would throw an exception nowdays, rather than NaN or an infinite value.
>

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ben Goodrich]

On Friday, August 22, 2014 1:48:16 PM UTC-4, Daniel Lee wrote:

Sorry, I wasn't clear. I meant within the paragraph. An other option is to put a \Phi(x) = ... within the first line of the conditional equation. I thought it would have been too busy and it was clear defined within the paragraph.

I see. I think it is fine as is.

> P.S. We also need to implement Phi_inv(p) soon, such that Phi_inv(Phi(x)) = x. But should Phi_inv(0) return -37.5 + epsilon or -inf? I guess input to Phi_inv() that was less than 0 or greater than 1 would throw an exception nowdays, rather than NaN or an infinite value.

Still need to decide on this.

Also, why does the derivative of Phi(x) overflow to 1 for x > 27.5? Can't we catch that and set the derivative equal to 0 if x > 27.5?

Ben

[Bob Carpenter]

I agree that we want Phi in the definition.

If all the distributions always raise exceptions on NaN, then
it doesn't need to be explicitly mentioned in each one. But then
it wouldn't hurt. I'd be tempted to group the two infinite cases
together followed by NaN with some space in between each group

- Bob

[Bob Carpenter]

[change subject]

more below

> On Aug 22, 2014 11:08 AM, "Ben Goodrich" <goodri...@gmail.com> wrote:

> ...

> > P.S. We also need to implement Phi_inv(p) soon, such that Phi_inv(Phi(x)) = x. But should Phi_inv(0) return -37.5 + epsilon or -inf? I guess input to Phi_inv() that was less than 0 or greater than 1 would throw an exception nowdays, rather than NaN or an infinite value.

I'm not sure what inv_Phi(0) or inv_Phi(1) should do. On the
one hand, I'm tempted to go with the behavior of log and exp,
with Phi(-inf) = 0, inv_Phi(0) = -inf, and similarly for +inf and 1.
On the other, that's going to be an automatic rejection. What
do the gradients look like out at the edges?

I believe we've been somewhat inconsistent about this in that
I think we make sure inv_logit doesn't underflow to 0 or overflow to
1 no matter what the input. I'm not sure that's right either.

There's no way to make the thing properly invertible outside
of (-38,38) or thereabouts (if that's where things overflow or
underflow with double).

- Bob

[Bob Carpenter]

On Aug 22, 2014, at 2:25 PM, Ben Goodrich <goodri...@gmail.com> wrote:

> ...

> Also, why does the derivative of Phi(x) overflow to 1 for x > 27.5? Can't we catch that and set the derivative equal to 0 if x > 27.5?

Coder error (that may have been me) --- it should be 0.

I think this can go on Daniel and Peter's pass of cleaning
up boundary behavior.

- Bob

[Ben Goodrich]

On Friday, August 22, 2014 10:29:04 PM UTC-4, Bob Carpenter wrote:

[change subject]

more below

> On Aug 22, 2014 11:08 AM, "Ben Goodrich" <goodri...@gmail.com> wrote:

> ...

> > P.S. We also need to implement Phi_inv(p) soon, such that Phi_inv(Phi(x)) = x. But should Phi_inv(0) return -37.5 + epsilon or -inf? I guess input to Phi_inv() that was less than 0 or greater than 1 would throw an exception nowdays, rather than NaN or an infinite value.

I'm not sure what inv_Phi(0) or inv_Phi(1) should do.  On the
one hand, I'm tempted to go with the behavior of log and exp,
with Phi(-inf) = 0, inv_Phi(0) = -inf, and similarly for +inf and 1.
On the other, that's going to be an automatic rejection.   What
do the gradients look like out at the edges?

+inf at both ends, since the derivative wrt p is the reciprocal of the standard normal PDF at x = Phi_inv(p).

R returns + or - infinity. We at least ought to do something coherent for all implemented CDFs, CCDFs, and inverse CDFs.

Ben
 

[Daniel Lee]

Alright, time to nitpick. This is what it currently looks like on the branch I'm working on.

Here are specific things questions I have. If no one has anything, I'll pick something and move on. See annotated image for numbering.

1. Should "Phi" be bold? Should "x" be italics? Should "x" match the math "x"?

2. Does anyone care that this "x" doesn't look like the other "x"? If they should match, which one should it be?

3. Order of the equations ok? I could see an argument for x < -37.5, -37.5 <= x <= 8.25, x > 8.25

4. Space between groups is 6pt. Look about right?

5. I'm using "inf" for infinity. That ok? I could use \infinity. Also, should we keep this group if it's obvious from the first group? I would think x = inf is included in x > 8.25, but I'm happy to include it for consistency.

6. I'm separating out NaN as its own group. I tried putting it within the same group as inf and it looked a little odd, but could be convinced to go that way.

7. "throws exception"? Is that clear enough? Not all functions throw; some return NaN, others return booleans or integers, so it'll probably be worth documenting behavior for each function for NaN. For the infinity cases, I could see us not explicitly stating what happens on the bounds if it's obvious. For Phi, if x = inf case isn't written out, I think it's still clear.

8. between ":" and "x" I've added a large space ("\;"). Does that look ok?

Thanks.

[Ben Goodrich]

On Saturday, August 23, 2014 12:05:58 AM UTC-4, Daniel Lee wrote:

Here are specific things questions I have. If no one has anything, I'll pick something and move on. See annotated image for numbering.

1. Should "Phi" be bold? Should "x" be italics? Should "x" match the math "x"?

No, I think unbold and monospaced is fine for Phi(x).
 

3. Order of the equations ok? I could see an argument for x < -37.5, -37.5 <= x <= 8.25, x > 8.25

I would order it starting with x < -37.5 and increasing, except for NaN last.
 

5. I'm using "inf" for infinity. That ok? I could use \infinity. Also, should we keep this group if it's obvious from the first group? I would think x = inf is included in x > 8.25, but I'm happy to include it for consistency.

Implied by x > 8.25

Ben

[Bob Carpenter]

On Aug 23, 2014, at 12:05 AM, Daniel Lee <bea...@alum.mit.edu> wrote:

> Alright, time to nitpick.

Please don't encourage me.

> 1. Should "Phi" be bold?

No. The bold part's just supposed to be like syntax highlighting
to help you scan the doc and find the definition.

> Should "x" be italics?

No, it should be in math.

> Should "x" match the math "x"?

Yes.

So that means $\mbox{Phi}(x)$ on the left. That's using
the roman script for mathematical functions in math.

I'm also OK with $\Phi(x)$ to put it all in math notation.

> 2. Does anyone care that this "x" doesn't look like the other "x"? If they should match, which one should it be?

See above.

I'd prefer the whole thing to be math.

> 3. Order of the equations ok? I could see an argument for x < -37.5, -37.5 <= x <= 8.25, x > 8.25
> 4. Space between groups is 6pt. Look about right?
> 5. I'm using "inf" for infinity. That ok? I could use \infinity.

Yes, \infty please. This follows from the whole thing being math.

For distributions, I was using \mbox{\sf Poisson}, which I
commonly see in books. That's a bit different than \mbox{\tt Poisson}
or \code{Poisson}, which renders in "typewriter" code font.

> Also, should we keep this group if it's obvious from the first group? I would think x = inf is included in x > 8.25, but I'm happy to include it for consistency.

I was about to say that. When it follows from comparisons, no
reason to include it. I'm OK including it if we want to be
super specific -- that way the functions would match that aren't
defined with conditionals.

Are you going to define exp() the same way, with

exp(x) = e^x if x \in [-800,800]
= 0 if x < -800
= inf if x > 800

or whatever the bound is?

This might be useful to our users to give them ideas of the scales at
which arithmetic is even remotely feasible.

> 6. I'm separating out NaN as its own group. I tried putting it within the same group as inf and it looked a little odd, but could be convinced to go that way.

I like it on its own.

> 7. "throws exception"? Is that clear enough? Not all functions throw; some return NaN, others return booleans or integers, so it'll probably be worth documenting behavior for each function for NaN. For the infinity cases, I could see us not explicitly stating what happens on the bounds if it's obvious. For Phi, if x = inf case isn't written out, I think it's still clear.

I'm ok with just "error" or "exception". It might be even better just
to say on the outside that NaN raises an exception --- it doesn't
return the value "throw exception" as the equality sign seems to imply now
that I look at it.

> 8. between ":" and "x" I've added a large space ("\;"). Does that look ok?

I prefer \mbox{if }. With a single trailing space.

- Bob

[Jiqiang Guo]

Correct a typo: $Phi$ is integrated from -inf.

Jiqiang

[Michael Betancourt]

I think you’re trying to do too much with too few elements.  There are four key pieces that you’re trying to present:

1) The function signature

2) The function description in words

3) The mathematical definition

4) The implementation

At the very least I think you should have something like

signature (as is)

Description (as is once you remove the equation)

Mathematical definition displayed on new line

Implementation (as is)

But I disagree on the level of implementation detail.  The user shouldn’t need to know when we transition from

erf to 0 or 1, it should just work.  What is useful to the user is not the implementation but rather

1) the mathematical domain of the function

2) the practical domain of the function

(2) is the only part of the implementation that is useful to the user.  For Phi it’s all double values, but for exp

we’re limited to log(max_double) and for something like phi_inv we can quote the threshold values where

everything becomes indistinguishable from the boundary.

So, for example,

\code{ real exp(real x) }

    The natural exponential function

        exp(x) = e^{x}

    Domain: -infty < x < infty

    Range: 0 < exp(x) < infty

    

    Implementation Domain: -max_double < x < log(max_double)

    Range: 0 < exp(x) < max_double

Lastly, if boundary values are really considered important then we should include a plot of the function

as that will be far more informative anyways.

On Aug 23, 2014, at 8:32 AM, Jiqiang Guo <guo...@gmail.com> wrote:

Correct a typo: $Phi$ is integrated from -inf.

Jiqiang

On Aug 23, 2014 12:05 AM, "Daniel Lee" <bea...@alum.mit.edu> wrote:

Alright, time to nitpick. This is what it currently looks like on the branch I'm working on.

<Screenshot 2014-08-22 23.39.46.png>

Here are specific things questions I have. If no one has anything, I'll pick something and move on. See annotated image for numbering.

1. Should "Phi" be bold? Should "x" be italics? Should "x" match the math "x"?

2. Does anyone care that this "x" doesn't look like the other "x"? If they should match, which one should it be?

3. Order of the equations ok? I could see an argument for x < -37.5, -37.5 <= x <= 8.25, x > 8.25

4. Space between groups is 6pt. Look about right?

5. I'm using "inf" for infinity. That ok? I could use \infinity. Also, should we keep this group if it's obvious from the first group? I would think x = inf is included in x > 8.25, but I'm happy to include it for consistency.

6. I'm separating out NaN as its own group. I tried putting it within the same group as inf and it looked a little odd, but could be convinced to go that way.

7. "throws exception"? Is that clear enough? Not all functions throw; some return NaN, others return booleans or integers, so it'll probably be worth documenting behavior for each function for NaN. For the infinity cases, I could see us not explicitly stating what happens on the bounds if it's obvious. For Phi, if x = inf case isn't written out, I think it's still clear.

8. between ":" and "x" I've added a large space ("\;"). Does that look ok?

<Screenshot 2014-08-22 23.37.18.png>

Thanks.

[Bob Carpenter]

On Aug 23, 2014, at 5:53 AM, Michael Betancourt <betan...@gmail.com> wrote:

> I think you’re trying to do too much with too few elements.


> There are four key pieces that you’re trying to present:
>
> 1) The function signature
> 2) The function description in words
> 3) The mathematical definition
> 4) The implementation

Good breakdown.

> At the very least I think you should have something like
>
> signature (as is)
> Description (as is once you remove the equation)
> Mathematical definition displayed on new line

I take it from the example you mean like you'd find it in
Wikipedia or in an algebra book?

> Implementation (as is)
>
> But I disagree on the level of implementation detail. The user shouldn’t need to know when we transition from
> erf to 0 or 1, it should just work.

I'm not sure what you mean here. It's only going to work as
far as floating point and our implementation let it.

> What is useful to the user is not the implementation but rather
>
> 1) the mathematical domain of the function
> 2) the practical domain of the function

I agree that they don't need to know about and should not be told
about implementation details. So they don't need to know when
log1p(x) transfers from the built-in definition to a Taylor series
at first or second order.

I also agree that the practical domain's what's important.

>
> (2) is the only part of the implementation that is useful to the user. For Phi it’s all double values, but for exp
> we’re limited to log(max_double) and for something like phi_inv we can quote the threshold values where
> everything becomes indistinguishable from the boundary.
>
> So, for example,
>
> \code{ real exp(real x) }
> The natural exponential function
> exp(x) = e^{x}
>
> Domain: -infty < x < infty
> Range: 0 < exp(x) < infty
>
> Implementation Domain: -max_double < x < log(max_double)

This brings up the issue of how to code this.

What you have is the implementation domain that'll produce
a finite result.

If you extend the implementation domain above log(max_double)
you get +inf. That includes +inf itself.

This to me seems the same kind of issue as underflowing to 0.


> Range: 0 < exp(x) < max_double

This is the finite range of output, but the full range will also
include +inf and NaN. But it can't produce negative numbers
or -inf.

> Lastly, if boundary values are really considered important then we should include a plot of the function
> as that will be far more informative anyways.

That would be great. I think most things
would benefit from both a linear and log-scale
plot. But that's probably asking too much to do in
this pass.

- Bob

[Daniel Lee]

So... what should I do to get us started down this path? Anyone want to edit it to some place where enough of us are happy and I'll start submitting pull requests?

The implementation detail can go into the C++ documentation, but I think if it were to actually get into the user facing documentation, it might get people to think about the numeric issues involved with trying to implement models in software. But maybe that's not what we need to expose to users anyway.

[Peter Li]

I'll do the same for the reverse mode functions -- most of them now return NaN for gradients now using the unlikely() compiler hint that Bob suggested.

[Jeffrey Arnold]

Is there a way to make the function signatures in the manual more machine readable? Right now for stan-mode and the pygments lexer, I have a json file with function signatures and documentation: https://github.com/stan-dev/stan-mode/blob/master/stan_lang.json.  However, I generate in an embarrassingly manual way from a text file https://github.com/stan-dev/stan-mode/blob/master/scripts/stan-reference.txt based on the output of pdftotext of the pdf version of the manual that has been manually edited.  I use this file to generate lists of functions and snippets.  I had tried to find a good parser for LaTeX but never had much luck.   I don't have a great suggestion; using templates to generate the manual adds another dependency and complications.   

--

[Bob Carpenter]

I'm not sure what kind of machine you're talking about.
If the machine is C++, the name and signature of each function
is represented programatically in the singleton function_signatures
defined in stan/gm/ast.hpp. If you want something machine readable,
we can write some C++ to dump it out.

This handles everything but the operators.

- Bob

On Aug 26, 2014, at 12:48 AM, Jeffrey Arnold <jeffrey...@gmail.com> wrote:

> Is there a way to make the function signatures in the manual more machine readable? Right now for stan-mode and the pygments lexer, I have a json file with function signatures and documentation: https://github.com/stan-dev/stan-mode/blob/master/stan_lang.json. However, I generate in an embarrassingly manual way from a text file https://github.com/stan-dev/stan-mode/blob/master/scripts/stan-reference.txt based on the output of pdftotext of the pdf version of the manual that has been manually edited. I use this file to generate lists of functions and snippets. I had tried to find a good parser for LaTeX but never had much luck. I don't have a great suggestion; using templates to generate the manual adds another dependency and complications.
>
>
>
> On Fri, Aug 22, 2014 at 7:01 AM, Daniel Lee <bea...@alum.mit.edu> wrote:
> Hi all,
>
> Here are some options for the documentation. It's on the feature/issue-843-test-nan-Phi branch, if you want to edit manually.
>
> I've put together 3 options.
> 1. the main use first, infinite cases not explicitly mentioned.
> 2. ordered by range of x, inf explicitly mentioned
> 3. every case broken out, including inf, nan.
>
> Other thoughts:
> - should we use a different notation for ranges?
> - Phi(x) on the left side -- should it be in the code font?
> - location of the definition of \Phi(x)? use of \equiv?
>
>

> <Screenshot 2014-08-22 09.52.16.png>