Using ceres from python

[Dustin Lang]

Hi,

I have been checking out Ceres as an option for solving the large non-linear
optimization problem that arises in "the Tractor", our project to measure
astronomical sources through generative image modeling. The code is in python,
so I'm trying to figure out our options for using Ceres.

Currently we are doing our own naive optimization using scipy's sparse lsqr for
the linear solver, and then doing line search. Most of the derivatives are
computed numerically (finite differences).

Ceres (like most solvers and samplers) seems to take the view that you pass it
everything and then it goes off and does its work and eventually gives back
your answer. I often find that I actually want to be able to feed it one step
at a time, maybe plot some outputs or compute some diagnostics or write a
checkpoint during development, or provide some "grass" for the user to watch
during production. I want the design turned inside out, so that my code drives
the outer (LevMar) loop and calls Ceres to advance one step at a time. In the
case of using it from python, that might more easily allow the derivatives to
be computed in python as usual and then "pushed" to Ceres, rather than having
Ceres "pull" them. What do you think, is this a situation you've encountered
in your work before? Think there's any chance such a refactor would be
feasible?

I guess another option for me is to write a python extension module that
provides the interface class Ceres needs and does its work by calling the
appropriate python methods from C++. Sounds a little hairy, but it could work.


cheers,
dustin

[Sameer Agarwal]

Hi Dustin,

Have a look at include/ceres/iteration_callback.h which allows you to define instances of the IterationCallback interface which  are executed every iteration. Using the return value from this object you can control the execution of the solver.

You will also need to look at settings

Solver::Options::callbacks

Solver::Options::update_state_every_iteration

I think it will allow you to do what you need.

Sameer

--
----------------------------------------
Ceres Solver Google Group
http://groups.google.com/group/ceres-solver?hl=en?hl=en

[Keir Mierle]

Also, if you happen to develop a python wrapper, we'd love to integrate it.

[Dustin Lang]

Hi Sameer,

I don't see how callbacks are going to help with the core issue, which is
that my "CostFunction" is not callable from C++. I was asking about
"turning the interface inside out". I want Ceres to run until it needs me
to compute something, describing what it needs computed and where it wants
the results to go, and letting me do it (using python code, in my case).
In other words, I want to wrap Ceres, I don't want Ceres to wrap me.

--dstn

On Fri, 27 Apr 2012, Sameer Agarwal wrote:

> Hi Dustin,
> Have a look at include/ceres/iteration_callback.h which allows you to define instances of the IterationCallback

> interface which ï¿œare executed every iteration. Using the return value from this object you can control the execution

[Sameer Agarwal]

Hi Dustin,

That puts us in the land of user hand assembling matrices. We explicitly wanted to avoid that with Ceres. 

In part because we want to deal with things like holding parameter constant, parameter block reorderings, local parameterizations etc. 

Since the driving factor here is the callability of cost functions defined in python, an extension is a better and perhaps easier route than inverting the flow of control in Ceres. 

We have talked about this for a while, but never had the time to write one. As Keir said,  we would love the ability to do so added to Ceres :)

Sameer

On Fri Apr 27 09:36:50 GMT-700 2012, Dustin Lang <ds...@astro.princeton.edu> wrote:

Hi Sameer,

I don't see how callbacks are going to help with the core issue, which is
that my "CostFunction" is not callable from C++. I was asking about
"turning the interface inside out". I want Ceres to run until it needs me
to compute something, describing what it needs computed and where it wants
the results to go, and letting me do it (using python code, in my case).
In other words, I want to wrap Ceres, I don't want Ceres to wrap me.

--dstn

On Fri, 27 Apr 2012, Sameer Agarwal wrote:

> Hi Dustin,
> Have a look at include/ceres/iteration_callback.h which allows you to define instances of the IterationCallback

> interface which  are executed every iteration. Using the return value from this object you can control the execution

[Keir Mierle]

Hi Dustin,

Unfortunately inverting control in Ceres is a deep change that I don't expect to happen, likely ever. However, it's unclear why you can't implement what you need from a callback. If you want to compute the entire analytic Jacobian from Python, well... we don't support that. Arguably, the purpose of Ceres is to provide the user a modelling layer so they don't have to construct the Jacobian.

Take a look at levenberg_marquardt.cc and evaluator.h. You can see that we hide the evaluation of the cost and Jacobian behind a class which doesn't know anything about the individual cost functions; all the LM loop sees is giant parameter vectors, and an opaque SparseMatrix base type that it passes to the linear solvers.

However, to view Ceres as just another LM loop is missing the real benefit of Ceres. The magic comes from our CostFunction structure and the primary Evaluator implementation found in program_evaluator.h. By providing Ceres with the ability to evaluate tiny pieces of your cost function, Ceres is able to exploit the sparsity (e.g. for Schur-type solving) and provide threaded Jacobian evaluation for free. At the same time, you are able to construct your cost function in a natural way: by describing each residual one at a time.

If you look at the CostFunction class, you can see that it's mostly virtual and should be easy to implement from Python. The callback should be similarly easy. You can include a thunk if you need in both callbacks, so you can access all program state. One approach would be to create an Optimizer class in Python, which constructs the program, and then the iteration callback is simply a method on the Optimizer class which has access to all state.

Keir

On Fri, Apr 27, 2012 at 9:36 AM, Dustin Lang <ds...@astro.princeton.edu> wrote:

Hi Sameer,

I don't see how callbacks are going to help with the core issue, which is that my "CostFunction" is not callable from C++.  I was asking about "turning the interface inside out".  I want Ceres to run until it needs me to compute something, describing what it needs computed and where it wants the results to go, and letting me do it (using python code, in my case). In other words, I want to wrap Ceres, I don't want Ceres to wrap me.

--dstn

On Fri, 27 Apr 2012, Sameer Agarwal wrote:

Hi Dustin,
Have a look at include/ceres/iteration_callback.h which allows you to define instances of the IterationCallback

interface which  are executed every iteration. Using the return value from this object you can control the execution