Re: Higher-Order Autodiff Interface

[Bob Carpenter]

On 2/5/13 3:54 PM, Michael Betancourt wrote:> Happy to chat anytime, but for future reference the methods I need are
>
> scaler V = - log_p
> matrix H = - d_{i} d_{j} log_p
> vector traceMH = -d_{r} ( Trace[ M_{ij} d_{j} d_{k} log_p ] )
> vector traceMHplusV = - d_{r} ( 0.5 * Trace[ M_{ij} d_{j} d_{k} log_p ] + log_p )
>
> -Betancourt

Peter:

After releasing Stan 1.1.1 today, I'm going to work
full time on the fvar branch to support what Michael
needs to implement Riemann Manifold HMC.

It's fantastic that you've gotten so far already.
Definitely far enough for us to start testing models
that we implement directly. Between the two of us,
we should be able to finish off the rest of the functions
we need.


How about I provide the following, where theta is
the current set of parameters and log p(theta) is
the log probability up to a proportion.


1. Potential (i.e., negative log probabilty)

scalar, V = - log(p(theta))

2. Gradient of Potential

vector, d_i V

3. Hessian of potential:

matrix, H = d_i d_j V

4. Gradient of matrix-Hessian product:

vector, d_i -Tr[M H]

5. Gradient of half matrix-Hessian plus potential,
which for a fixed double matrix M, gives

vector, d_i (V - Tr[M H])


It's easy enough to build 5 out of 4 and 2, so I might
do it that way. The work done for the graident of the potential
is dwarfed by the gradient of the matrix-Hessian product.

I propose that we define two additional classes
in stan/model:

prob_hessian : public prob_grad

--- allow user-definitions of above

prob_hessian_ad : public prob_grad_ad

--- compute above with auto-dif

Alternatively, I could just replace the prob_grad
and prob_grad_ad classes.

- Bob

[Peter Li]

Alright. I'm still a little bit confused about what I should do now. Should I continue to try and figure out Phi and lmgamma? If not or after that, what should I move on to?

Thanks,

Peter

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
Peter W Li
Columbia College Class of 2016

BA Candidate in Mathematics & Statistics

5193 Lerner Hall, 2920 Broadway | New York, NY 10027

Mobile: (321) 426-8314 | E-mail: pwl...@columbia.edu