Weighted cost function wrapper

[Paul-Edouard Sarlin]

Hello everyone,

In COLMAP, we have a helper that can wrap any cost functor to multiply its residuals by a diagonal weight matrix, using ConditionedCostFunction (source here). We would like to extend this to non-diagonal weight matrices. I know that there was a CR for a WeightedCostFunction, which was eventually killed (relevant discussion). This was 11 years ago - in 2024 would this be somehow easier to implement?

I gave this a try assuming an auto-diff cost functor (code here) but somehow the template deduction in operator() fails when interfacing with Ceres' internals (in variadic_evaluate.h). Do you see how I could make this work?

Many thanks.

Best,

Paul-Edouard Sarlin

[Sameer Agarwal]

Hi Paul,

We are fans of colmap :)

The basic problem with the WeightedCostFunction was that the temporaries needed to implement were not thread safe. Since we were trying to do this at the level of CostFunction, we did not know the size of these matrices at compile time, so we could not assume that the required temporaries could be allocated on the stack. 

If you are trying to do this at the level of AutoDiffCostfunction, really SizedCostFunction then things should be simpler. That said, if your aim is to only do this at the level of AutoDiffCostFunction, why not do this at the level of the functor -- as in, have a Weighted functor, which applies the weight matrix to the output of the actual costfunctor and pass that object to AutoDiffCostFunction. That way you do not have to worry about jacobians etc. Also since everything is static, the compiler should be able to inline everything.

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/abf18ac8-de7f-4d9d-b541-61166fd9b5ffn%40googlegroups.com.

[Paul-Edouard Sarlin]

Hi Sameer,

Thanks for your reply. We are definitely fans of Ceres :) What you suggest makes sense, I have managed to make it work. Here is the code in case anyone is interested: https://github.com/colmap/colmap/pull/2804/files/0723ea4d214097104a07e4fd1833658401c41e68 The only catch is that the wrapper functor needs to know about the residual and parameter shapes of the parent functor at compile time so we are forced to expose them through some standardized interface.

Best,

Paul-Edouard

On 24 Sep 2024, at 17:56, 'Sameer Agarwal' via Ceres Solver <ceres-...@googlegroups.com> wrote:

Hi Paul,

We are fans of colmap :)

The basic problem with the WeightedCostFunction was that the temporaries needed to implement were not thread safe. Since we were trying to do this at the level of CostFunction, we did not know the size of these matrices at compile time, so we could not assume that the required temporaries could be allocated on the stack. 

If you are trying to do this at the level of AutoDiffCostfunction, really SizedCostFunction then things should be simpler. That said, if your aim is to only do this at the level of AutoDiffCostFunction, why not do this at the level of the functor -- as in, have a Weighted functor, which applies the weight matrix to the output of the actual costfunctor and pass that object to AutoDiffCostFunction. That way you do not have to worry about jacobians etc. Also since everything is static, the compiler should be able to inline everything.

Sameer

On Tue, Sep 24, 2024 at 12:58 AM Paul-Edouard Sarlin <paul.edou...@gmail.com> wrote:

Hello everyone,

In COLMAP, we have a helper that can wrap any cost functor to multiply its residuals by a diagonal weight matrix, using ConditionedCostFunction (source here). We would like to extend this to non-diagonal weight matrices. I know that there was a CR for a WeightedCostFunction, which was eventually killed (relevant discussion). This was 11 years ago - in 2024 would this be somehow easier to implement?

I gave this a try assuming an auto-diff cost functor (code here) but somehow the template deduction in operator()fails when interfacing with Ceres' internals (in variadic_evaluate.h). Do you see how I could make this work?

Many thanks.

Best,

Paul-Edouard Sarlin

-- 
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/abf18ac8-de7f-4d9d-b541-61166fd9b5ffn%40googlegroups.com.

-- 
You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/R08n1xkjvVY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUDyzf%2BkV0kgq6Nn2X5j%2BaYLn3YW740QW3tob5xXYEp68w%40mail.gmail.com.

[Sameer Agarwal]

Nice bit of template meta programming. 

Two comments.

1. WeightedCostFunction is really WeightedCostFunctor

2. I do not think you need to cast the weights matrix to T, you can now just do the multiplication of scalars with the residual vector. It is not clear to me what you gain by using Eigen to do the matmul there though, well perhaps in the scalar case you will get a little bit of performance but in the Jet case you are just creating work for the compiler. I doubt your residuals are huge, and you want things to be inlined anyways, so it may be worth thinking about writing a simple matrix-vector multiplication loop there.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/4EFCA6E4-F4AC-486F-A313-E68B5B3C8E06%40gmail.com.