Sub-Hessians with STAN math autodiff library

[Ryan Giordano]

I'm using the STAN math autodiff library for a modeling problem with a large, sparse Hessian.  I know the sparsity structure in advance.  In order to efficiently compute the Hessian, I will need to compute sub-matrices of the Hessian.  AFAIK, this isn't supported in the current STAN libraries.  There are two ways I can think of to do this without substantially modifying my objective function, and a minimal example of both is attached.  I'd like to ask for feedback or other suggestions.

The easiest is to compute sub-rows of the Hessian by simply changing the range of the outer loop in hessian.hpp.  This is what I've done in the attached sub_hessian.hpp.

In order to compute sub-matrices on the diagonal, I created a functor that takes a smaller input vector than the full input vector, copies the values of the smaller vector into their appropriate places of the larger vector, and evaluates the objective.  Code is worth a thousand words, so please take a look at the SubObjective functor in the attached code.  This doesn't seem to go any faster than sub-rows of the full Hessian.  Presumablyt this is because it needs to evaluate the full objective with fvar datatypes in both cases?  Because it's not faster than getting the whole row, this doesn't seem to be a good way to go.

I can't think of a way to quickly get arbitrary off-diagonal strict sub-matrices without redefining the objective function to take as arguments an arbitrary combination of double and fvar types.  I can't think of a tidy way to do this in full generality, and I think to use Eigen matrix operations I'd have to promote everything to fvar types anyway.  Any other ideas?

I'm new to both C++ and STAN, so apologies in advance if I'm confused or need extra clarification for responses.  :)

Ryan

[Bob Carpenter]

You're right that we don't have a sparse Hessian operation.

And you're right as to why --- we don't have a way to call
a mixed double/var vector function.

I don't know of a better way of doing this than promoting,
which is tantamount to just computing the whole dense
Hessian.

I played around with trying to have two implementations of
our var and fvar types, one that's a "real" autodiff variable
and one that just wraps doubles to get around the type problem,
but it just pushes it down one level, and the run-time costs
of doing it with virtual functions or conditionals is too high.

I still have the feeling that it must be possible to engineer
such a thing with a clever use of something like the CRTP, but
I could never figure out how to do it. I don't know if other
autodiff libs have reasonable solutions. If you find out, I'd
like to hear more about what they do.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <basic.cpp><sub_hessian.hpp>