cdf, ccdf, log scale, and truncation patterns

[Bob Carpenter]

What are we going to implement for the CDFs?

Just literally implementing the cdfs is dangerous
in terms of numerical precision.

I see four functions that make sense on their own:

foo_cdf(y,...)

foo_cdf_log(y,...) = log(foo_cdf(y,...))

foo_ccdf(y,...) = 1 - foo_cdf(y,...)

foo_ccdf_log(y,...) = log(1 - foo_cdf(y,...))

There are numerical stability issues with trying
to calculate the log values directly or calculate
a ccdf by subtracting a cdf.

In truncated distributions, we also need to evaluate
distributions bounded above and below in (L,U) with

foo_truncate_lu(U,L,...) = - log(foo_cdf(U,...) - foo_cdf(L,...))

= - log_sum_exp(foo_cdf_log(U,...),
0,
foo_ccdf_log(L,...))

The one-sided bounds both reduce to one of the above.
To bound above by U,

foo_truncate_u(U,...) = - log(foo_cdf(U,...) - 0)

= - foo_cdf_log(U,...)

and to bound below by L,

foo_truncate_l(L,...) = - log(1 - foo_cdf(L,...))

= - foo_ccdf_log(L,...)

Of course, we also need gradients of all these functions,
which means either calculating them directly or implementing
in terms of functions that can be auto-diffed.

- Bob

[Michael Betancourt]

I'm working with Dan on analytic derivatives -- until autodiff can
also auto-integrate I think we're going to have to do it by hand. :-)

> --
>
>

[Ben Goodrich]

On Friday, September 14, 2012 4:51:41 PM UTC-4, Bob Carpenter wrote:

What are we going to implement for the CDFs?

Just literally implementing the cdfs is dangerous
in terms of numerical precision.

I see four functions that make sense on their own:

   foo_cdf(y,...)

   foo_cdf_log(y,...)  = log(foo_cdf(y,...))

   foo_ccdf(y,...)     = 1 - foo_cdf(y,...)

   foo_ccdf_log(y,...) = log(1 - foo_cdf(y,...))

There are numerical stability issues with trying
to calculate the log values directly or calculate
a ccdf by subtracting a cdf.

In case you didn't know, inside the R source code under

src/nmath/p*.c

there are implementations of all these things that I presume handle the numeric stability issues as well as can be expected. So we might look at those for inspiration.

Ben