Conditional evaluation in auto-diff cost function (e.g., if-statement) possible?

[Arne Jacobs]

Hi All!

I do not have much background knowledge about optimization in general, but ceres-solver has still "just worked" for me so far, and that's why I love it. So, thanks for that great piece of software.

However I now have a problem for which I cannot find any advice...

In particular, I am trying to fit a set of 3D points to a cube, or at least to the three visible faces of a cube. The size of the cube is known and an initial estimate of its position and orientation (pose), and now I want to optimize the pose parameters.

Naively I would compute the residual of a given point by finding the cube face/plane where it fits best, and computing the residual to that face/plane. This would involve computing the point-plane distance and some conditional statements.

Using an auto-diff cost function, I guess computing the point-plane distance already is not feasible, is it? And then I did not find anything about if-statements inside cost functions. I could imagine these would make the cost function non-differentiable. Is that correct?

Would the only option be to use numeric differentiation? Or is there another way?

I did not find anything related to cuboid fitting with ceres-solver, either. Maybe because a cube is also not differentiable at the edges?

Thanks for any comments, even if it is just you agreeing that ceres-solver is a great piece of software ;-).

Cheers,

Jarne

[Sameer Agarwal]

Jarne,

Are you trying to do, iterated closed point like solution? where you assign points to nearest faces, and then optimize the pose of the cube, and then re-assign points to the nearest face?

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/0a32cb39-3456-4bb8-a549-b695f7eda45cn%40googlegroups.com.

[Arne Jacobs]

Yes that’s exactly what I am aiming for. Iterated closest point is similar in the point to plane version, with the difference that I only have three (perpendicular) planes. 

Instead of directly doing EM, I wanted to use Ceres-solver to be able to use its robust loss functions, and to integrate the cube fitting into a bigger problem (cube observed at different times by multiple sensors).

You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/oWIvPcBEDrs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUDnmzRqhLSpfUTLvPMww3FRDYgTc-qa76pU862yNJwUow%40mail.gmail.com.

[Sameer Agarwal]

Jarne,

You can use if statements inside the autodiff function, because the derivative is evaluated at a point. However your objective function is not smooth and its possible that you may have trouble optimizing it. 
Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAPjtwkeq_NsvAVkVA9zTA7aCuhPNX1zetKrjiSJKXXi8TJ0CBw%40mail.gmail.com.

[Arne Jacobs]

Thanks for the advice!

With this new information I was also able to find some older discussions on similar topics. From that I got that if doing some kind of expectation maximization, it is not a good idea to do point membership updates during the Ceres optimization, but to determine point memberships between successive calls of the Ceres solver. Is that correct?

Furthermore, if you use some if-statements inside a cost function, as long as its result is constant (this might sound strange first to have such an if-statement, but it still might make sense for simplicity of implementation), then this will not affect the smoothness of the function, right?

Thanks a lot!

Cheers,

Arne

[Alex Stewart]

You can update point associations during a solve and that can work well in some situations.  It really depends on how these changes affect the shape of your overall cost surface - i.e. are they small and smooth or large and discontinuous.  Functionally, changing the associations within a solve vs between concurrent solves is trading off some violation of the expectations of the internal state of the solver for speed.  However, if the changes are relatively smooth, this is not dissimilar from the linearisation of the true nonlinear problem that happens at each iteration.

In the case of conditionals, you might want to look at the switchable constraints for robust map optimisation from Niko Sunderhauf and Ed Olson.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/23ec8e24-39e7-4770-aaec-586ec49d3994n%40googlegroups.com.

[Arne Jacobs]

Sorry, another question regarding this: as I have only 3 perpendicular planes, would it be feasible to compute a point's residual to each of the three planes in the cost function, and weight it (e.g., with something like a Gaussian) depending on the residual magnitude?

As the planes are orthogonal, also the derivatives of the residuals to the three planes should be orthogonal, shouldn't they?

Weighting the residuals by some measure of its magnitude seems somehow stupid to me, but also inexplicably tempting... don't know why.

[Sameer Agarwal]

Let me add a few things to what Alex said.

There are multiple things to consider.

1. Sparsity

2. Descent property.

3. Smoothness

The first is that when you are doing high dimensional ICP, you want to solve a sparse nonlinear least squares problem since the dense jacobian is too large to handle. But ICP Jacobians have dynamic sparsity which requires special handling both in your cost function and inside ceres. But this is not an issue for you since you are only optimizing a small number of parameters, so a dense Jacobian, per point (or more) is fine.

The second issue is that it is important that the objective function does not increase as a result of these conditionals. Which is also not an issue for you since you will only assign to a different plane if the cost of the last assignment is higher.

The principal issue you are going to face is the non-smoothness of your objective, but that one is neither here nor there since as Alex points out the linearized model is already an approximation. So as long as you have sufficient number of points you will likely be okay.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CADGyVzjjQmQ6u8G4a5cF%2BSbqZXoCGi4mbXsFO8JmjAu%3D61knVA%40mail.gmail.com.

[Sameer Agarwal]

At the risk of being snarky, let me say don't do things you do not understand :)

if you can write down a cost function which makes geometric/probabilistic sense then do it, otherwise not.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/12ad4d12-f92e-41ab-a00e-263d26bc2f33n%40googlegroups.com.

[Arne Jacobs]

Thanks a lot!

Regarding things you don't understand: I had to look up "snarky" ;-). But I completely agree, and as it seems I can go with the conditional approach (I have about 300-1000 points for the three planes together), I won't start experimenting.

I have one last question, though: as the condition would be based on the distance of a point to the planes (i.e., the magnitude of the residual), will that work for the Jet-templated cost function?

Will the result of the comparison be the same for both the real and the Jet-templated evaluation? I.e., will the smallest of the three residual magnitudes be the same? I would guess that in general this is not true for the derivatives of the residual magnitudes, but I don't know about the Jets and how they relate to the actual derivatives...

For the numerical differentiation this wouldn't matter, but using that is the last resort I've been told.

Jarne

[Sameer Agarwal]

On Wed, Jan 5, 2022 at 10:44 AM 'Arne Jacobs' via Ceres Solver <ceres-...@googlegroups.com> wrote:

Thanks a lot!

Regarding things you don't understand: I had to look up "snarky" ;-). But I completely agree, and as it seems I can go with the conditional approach (I have about 300-1000 points for the three planes together), I won't start experimenting.

I have one last question, though: as the condition would be based on the distance of a point to the planes (i.e., the magnitude of the residual), will that work for the Jet-templated cost function?

yes since the comparison is only done on the scalar part of the jet and not the derivative part.

Also jets are relatively straightforward code, I encourage you to read jet.h.

Sameer

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/1b6ef94b-483c-4d34-b89f-e0cc9958e2d9n%40googlegroups.com.

[Arne Jacobs]

Thanks a lot! Will have a look into jet.h.

Jarne

[Rasmus]

Hello,

did you end up solving this problem successfully? I am about to implement the same use case, and I am wondering if my "which face is visible" and raycasting logic can be simplified.

[Arne Jacobs]

Hi,

I ended up with a cost function that converged nicely with a good enough initial guess. I would have to look it up, but it was not really complicated. If I would have to do it again I would probably not go with a cube in the first place, but with something that had some nicer analytical function which is also steady everywhere, like a sphere.

I went for a cube because I could put fiducial markers on the faces to easily detect them in camera images, but now work Segment anything I reckon a sphere would do nicely also for cameras.

Cheers,

Jarne 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/e08e5d81-4c06-454e-bb3e-7eb584953c7fn%40googlegroups.com.

[Rasmus]

Yeah actually I'm adapting a calibration solution utilizing spheres to work with boxes, because those are way easier to construct at a sufficient size in reality.

If you have any code or notes still lying around, I'd love to take a peek, otherwise I will just plug along and see if my approach actually turns into a differentiable problem.

Best,

Rasmus