prob propto 1/sigma

[Bob Carpenter]

From Andrew:

> ... is there a way we can include that prior p(sigma) proportional to 1/sigma more transparently, instead of increment_log_prob? Or maybe increment_log_prob is the clearest approach.


I assume

increment_log_prob(log(1/sigma));

is not what you meant :-)

It's not a proper density is it? Should we add improper densities?
We could also add an improper:

y ~ flat();

If so, you can make a feature request on GitHub, and preferably
again let us know about priority? Or I can do it if you think we
should.

You can almost get there with a Pareto with very small
shape and lower bound, as in:

sigma ~ pareto(1e-30, 1e-30);

which will make sigma proportional to (1/y)^(1 + 1e-30), and the
1 + 1e-30 will underflow to 1. But it also requires that pesky
lower bound.

- Bob

[Michael Betancourt]

If sigma is constrained, i.e. sigma<lower=0>, then the Jacobian essentially
gives the 1/sigma prior. Remember, 1/sigma implies uniform for log(sigma).

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Aki Vehtari]

Now I'm confused. Is the default prior for the constrained parameter flat for the original parameter sigma or flat for transformed parameter log(sigma)?

[Michael Betancourt]

It’s “flat” on the unconstrained space, so with the Jacobian it becomes 1/sigma on the constrained space.

Remember that by default we just don’t add anything to the log posterior except for the Jacobian.

[Bob Carpenter]

The parameters are all uniform by default on their
constrained values.

So if you declare:

real<lower=0> sigma;

then sigma is uniform on (0,inf).

What happens under the hood is that there is an unconstrained
parameter sigma_raw with support on (-inf,inf), and then
sigma is defined by the inverse-log transform:

sigma <- exp(sigma_raw);

and the total log probability is incremented by the log Jacobian,
here log(abs(exp'(sigma_raw))) = sigma_raw, so that the effect is

lp__ <- lp__ + sigma_raw;

The unconstrained representantations and Jacobians are all
detailed in a chapter near the end of the manual.

So when Andrew takes sigma propto 1/sigma, the effect is:

lp__ <- lp__ + log(1/sigma);

which reduces to

lp__ <- lp__ - sigma;

so it cancels the Jacobian and leaves the transform of
sigma uniform, i.e., log(sigma) propto constant.

And before anyone asks, let me say that Stan isn't smart
enough to avoid adding sigma and subtracting sigma --- it
just does both operations and calculates gradients with respect
to both and it all cancels arithmetically, not symbolically.

- Bob

[Marco Inacio]

On 14-03-04 09:17 AM, Bob Carpenter wrote:
> The parameters are all uniform by default on their
> constrained values.
>
> So if you declare:
>
> real<lower=0> sigma;
>
> then sigma is uniform on (0,inf).
>
> What happens under the hood is that there is an unconstrained
> parameter sigma_raw with support on (-inf,inf), and then
> sigma is defined by the inverse-log transform:
>
> sigma <- exp(sigma_raw);

Nice explanation, I'm just curious on what transformation stan uses for,
say <lower=0.5> or <lower=-12, upper=15>

[Daniel Lee]

There's a great section on how transformations happen towards the end of the manual. I don't have the section number of hand. Let me know if you can't find it.

[Daniel Lee]

For transformations, Section 51 in the manual v2.2.0. "Transformation of Variables."