Costfunction evaluation to "false"

[Manuel Hofer]

Hi

I have a question regarding the effect of an AutoDiffCostFunction evaluating to false. As a short explanation about the problem that I am trying to solve using CERES, it is similar to the Dynamic Fusion algorithm. That means I basically have a depth frame which I want to align to previously computed 3D model, by unprojecting depth pixels from the frame, then projecting then into a rendered depth frame at the previous camera pose which holds references to 3D vertices and normals, and computing a point-to-plane error for all depth pixels that have a reference value. In contrast to a rigid fusion algorithm (e.g. kinect fusion) I do not only have one 6 DoF transformation (R and t), but multiple ones.

So far so good... One problem is, that it can happen that unprojected pixels can fall onto pixels in the reference frame which do not hold any information (no associated 3D vertex and/or normal), or might also fall outside the image borders of the reference frame. Hence, I cannot evaluate a residual at this point, and therefore return false in my error function.

This can easily happen for quite a lot of pixels, and when I start the optimization it says that evaluation of the residuals and the jacobian failed. When I change the code such that I never return false, but a constant penalty for these cases where I do not have any reference data, it works.

My question now is:

Is there some sort of threshold, e.g. which says if more than a certain number of residuals can not be evaluated it just doesn't work?

Or is it the case that some mathematical term cannot be computed because there are not enough constraints on the unknowns?

For clarification: I made sure that each unknown (= the 6 transformation parameters for one specific transformation) has at least one costfunction associated to it which does not return false.

At the moment I am not quite sure if I made a mistake in preparing my data, or if it is just a "thresholding" problem, as mentioned above.

I would be thankful for any hint!

Thanks!

Best, Manuel

[Sameer Agarwal]

Manuel,
My replies are inline.

On Sun, Nov 20, 2016 at 9:22 AM Manuel Hofer <man....@gmail.com> wrote:

Hi

I have a question regarding the effect of an AutoDiffCostFunction evaluating to false. As a short explanation about the problem that I am trying to solve using CERES, it is similar to the Dynamic Fusion algorithm. That means I basically have a depth frame which I want to align to previously computed 3D model, by unprojecting depth pixels from the frame, then projecting then into a rendered depth frame at the previous camera pose which holds references to 3D vertices and normals, and computing a point-to-plane error for all depth pixels that have a reference value. In contrast to a rigid fusion algorithm (e.g. kinect fusion) I do not only have one 6 DoF transformation (R and t), but multiple ones.

So far so good... One problem is, that it can happen that unprojected pixels can fall onto pixels in the reference frame which do not hold any information (no associated 3D vertex and/or normal), or might also fall outside the image borders of the reference frame. Hence, I cannot evaluate a residual at this point, and therefore return false in my error function.

This can easily happen for quite a lot of pixels, and when I start the optimization it says that evaluation of the residuals and the jacobian failed. When I change the code such that I never return false, but a constant penalty for these cases where I do not have any reference data, it works.

My question now is:

Is there some sort of threshold, e.g. which says if more than a certain number of residuals can not be evaluated it just doesn't work?

Or is it the case that some mathematical term cannot be computed because there are not enough constraints on the unknowns?

No there is no threshold, or rather the threshold is 1. If a residual cannot be evaluated, it is not clear what the value of the jacobian/objective function is at that point. If you want to fake/invent a value (which is what you are doing with a penalty below), it is the user's prerogative to do so. 

This is not about the number of constraints, it is that if we start ignoring terms like that, it destroys the monotonicity of the objective function.

Sameer

 

For clarification: I made sure that each unknown (= the 6 transformation parameters for one specific transformation) has at least one costfunction associated to it which does not return false.

At the moment I am not quite sure if I made a mistake in preparing my data, or if it is just a "thresholding" problem, as mentioned above.

I would be thankful for any hint!

Thanks!

Best, Manuel

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/94bf2507-f544-4b19-9709-6ba9c7f48e0b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Manuel Hofer]

Thanks for your quick reply :-)