Jacobian adjustment not caught by autodiff, how does Stan do it?

[andre.p...@googlemail.com]

Dear all,

suppose an arbitrary model like in the manual as example of the jacobian adjustment:

data {
real<lower=0> y;
}
// ...
model {
log(y) ~ normal(mu, sigma);
}

and later

target += - log(fabs(y));

How does Stan know this is part of a of adjustment, and do not add the 2nd derivative of log(y),

precisely I assumed that everything added to target is handled by Stans auto-derivation. Consequently

is d/dy - log(fabs(y)) is also added to the likelihood?

Andre

[Ben Goodrich]

On Sunday, April 16, 2017 at 2:21:07 AM UTC-4, andre.p...@googlemail.com wrote:

data {
real<lower=0> y;
}
// ...
model {
log(y) ~ normal(mu, sigma);
}

and later

target += - log(fabs(y));

How does Stan know this is part of a of adjustment, and do not add the 2nd derivative of log(y),

Stan knows that y is a constant array, so the derivatives are constants, so adding them to log posterior kernel would have no effect on the sampling, so Stan doesn't bother. You would be better off constructing log_y in the transformed data block and feeding log_y to the normal log-likelihood function.

[Ben B]

@BGoodrich

Since y is data here and isn't being sampled over, wouldn't it make sense to change this example?

In that model,

log(y) ~ normal(mu, sigma)

and

y ~ lognormal(y | mu, sigma)

*Are*, from a sampling perspective, equivalent. The log probabilities are a bit off, but whatever. I was looking at this transformed parameter stuff a couple days ago and that was bugging me.

I think it'd just make more sense to show a manually constrained sigma parameter. Here's a gist comparing the two lognormal examples: https://gist.github.com/bbbales2/2f68c4a9111284c0df8ad2f56505b495 . Here's a gist showing a manually constrained sigma variable: https://gist.github.com/bbbales2/34602cacc6cddeae68c68228670882f3 .

Also, and I don't know if this is what Andre was asking, but if this were a parameter and we wanted to enforce a lower bound constraint:

real<lower = 0.0> sigma;

Then if we wrote that in the model, Stan would already apply that transformation, and so manually adding an adjustment would be like a 2nd adjustment (dunno if that's what he's asking). In this case, the <lower=0.0> specification is in the data block so it's only used to verify the input data and there's no transform, but again, in this example it doesn't really matter :D.

As a side note, it might be worth referencing chapter 20 (especially section 20.3, Change of Variables) as well as 33.1 in this section. It's got a good discussion about the transformations where you need the Jacobian and the ones where you don't. (Also, @Andre, check out Section 20.3)

Ben

[Ben Goodrich]

On Sunday, April 16, 2017 at 1:07:11 PM UTC-4, Ben B wrote:

@BGoodrich

Since y is data here and isn't being sampled over, wouldn't it make sense to change this example?

In that model,

log(y) ~ normal(mu, sigma)

and

y ~ lognormal(y | mu, sigma)

*Are*, from a sampling perspective, equivalent. The log probabilities are a bit off, but whatever. I was looking at this transformed parameter stuff a couple days ago and that was bugging me.

Using the lognormal function is less numerically stable than using the normal function on the logarithm of the outcome.
 

Also, and I don't know if this is what Andre was asking, but if this were a parameter and we wanted to enforce a lower bound constraint:

real<lower = 0.0> sigma;

Then if we wrote that in the model, Stan would already apply that transformation, and so manually adding an adjustment would be like a 2nd adjustment (dunno if that's what he's asking). In this case, the <lower=0.0> specification is in the data block so it's only used to verify the input data and there's no transform, but again, in this example it doesn't really matter :D.

For any constraints declared in the parameters block, Stan already does the Jacobian adjustment, in which case doing a second Jacobian adjustment would be wrong. For transformations in data or transformed data, no Jacobian adjustment is needed. If you use sampling statements, then Stan will warn you more than 100% of the time when a Jacobian adjustment is needed by the user. If you use target += statements, then Stan will not always warn you when a Jacobian adjustment is needed by the user.

Ben

[andre.p...@googlemail.com]

Stan knows that y is a constant array, so the derivatives are constants, so adding them to log posterior kernel would have no effect on the sampling, so Stan doesn't bother. You would be better off constructing log_y in the transformed data block and feeding log_y to the normal log-likelihood function.

Ben,

So the only way to get it wrong is, when y is a parameter, in for example in a latent approach.

As you say, in the case, where y is data it makes sense to use a data transformation. This

doesn't require the Jacobian adjustment, as it is a constant.

In the fist case, where y is parameter, however, do we have to correct the log-likelihood by

subtracting the derivative of the Jacobian transform of our parameter?

Andre

 

[Bob Carpenter]

For explicit Jacobian adjustments, it's the log of the absolute
value of the determinant of the Jacobian matrix.

You need the adjustment whenever you have a parameter theta
and you have a function f and want to define your distribution
on f(theta) and then pull it back so it does the right thing
on theta. Whereas if you have a variable theta and define the
density on theta, then f(theta) will be distributed correctly
without a Jacobian adjustment.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[andre.p...@googlemail.com]

For explicit Jacobian adjustments, it's the log of the absolute
value of the determinant of the Jacobian matrix.

You need the adjustment whenever you have a parameter theta
and you have a function f and want to define your distribution
on f(theta) and then pull it back so it does the right thing
on theta.  Whereas if you have a variable theta and define the
density on theta, then f(theta) will be distributed correctly
without a Jacobian adjustment.

- Bob

Bob,

absolut correct.

If I have f(theta) ~ ... then I add " the absolute 

value of the determinant of the Jacobian matrix" in the target 

statement. But every expression added this "target" is handled by

autodiff. Is this true? Thus my added "absolute 

value of the determinant of the Jacobian matrix" is handle through

autodiff, we add also the derivative of the  "absolute 

value of the determinant of the Jacobian matrix".

Andre

[Bob Carpenter]

Yes. You just need to add the log absolute determinant;
Stan will autodiff the result just like everything else.

- Bob