Zero iterations count when fitting plane to images

[Веркеенко Михаил]

Hello.

I'm trying to use ceres for plane (Ax + By + Cz + D = 0) fitting on stereopair images.

For example, I have coarse estimation of plane in object space and appropriated wall on stereopair images.

Cost function looks like

struct DepthPlaneFittingCostFunction {
 DepthPlaneFittingCostFunction(const cv::Mat &img_1, const cv::Mat &img_2,
 const SphericalCamera &cam_1, const SphericalCamera &cam_2, const cv::Point &initial_pt, int meas_type_ = CV_TM_SQDIFF_NORMED)
 : p_g_image_1(&img_1), p_g_image_2(&img_2), p_cam_1(&cam_1), p_cam_2(&cam_2), meas_type(meas_type_), image_pt(initial_pt)
 {
 cam_1_ray = cam_1.GetRay(openMVG::Vec2(image_pt.x, image_pt.y));


 image_rect = cv::Rect(0, 0, img_1.cols, img_1.rows);
 }


 bool operator()(const double* const plane, double* residuals) const
 {
 auto dist = abs(plane[3]/(cam_1_ray.x()*plane[0] + cam_1_ray.y()*plane[1] + cam_1_ray.z()*plane[2]));
 
 openMVG::Vec3 obj_pt = p_cam_1->_C + dist*cam_1_ray;


 openMVG::Vec2 prj_pt = p_cam_2->Project(obj_pt);


 cv::Rect template_1(image_pt.x - template_size, image_pt.y - template_size, 2*template_size + 1, 2*template_size + 1);
 cv::Rect template_2(cvRound(prj_pt.x()) - template_size, cvRound(prj_pt.y()) - template_size, 2*template_size + 1, 2*template_size + 1);


 if((template_1 & image_rect) != template_1 || (template_2 & image_rect) != template_2)
 {
 residuals[0] = 1.;
 return true;
 }


 cv::Mat res;


 cv::matchTemplate((*p_g_image_1)(template_1), (*p_g_image_2)(template_2), res, meas_type);


 residuals[0] = res.at<float>(0);


 return true;
 }


 // Factory to hide the construction of the CostFunction object from
 // the client code.
 static ceres::CostFunction* Create(const cv::Mat &img_1, const cv::Mat &img_2,
 const SphericalCamera &cam_1, const SphericalCamera &cam_2, const cv::Point &initial_pt, int meas_type_ = CV_TM_SQDIFF_NORMED) {
 return (new ceres::NumericDiffCostFunction<DepthPlaneFittingCostFunction, ceres::CENTRAL, 1, 4>(
 new DepthPlaneFittingCostFunction(img_1, img_2, cam_1, cam_2, initial_pt, meas_type_)));
 }


 const cv::Mat *p_g_image_1, *p_g_image_2;


 int meas_type;


 const SphericalCamera *p_cam_1, *p_cam_2;


 openMVG::Vec3 cam_1_ray;


 cv::Point image_pt;


 static size_t template_size;


 cv::Rect image_rect;
};

Solver initialization

ceres::Solver::Options options;
options.preconditioner_type = ceres::JACOBI;

options.linear_solver_type = ceres::DENSE_SCHUR;
options.minimizer_progress_to_stdout = true;
options.max_num_iterations = 100;


ceres::Solver::Summary summary;
ceres::Solve(options, &problem, &summary);

So I'm minimizing SSD between plane points projections on images.

In some cases it works but sometimes I've got zero iterations for solving problem, so nothing was done

ceres\block_sparse_matrix.cc:78 Allocating values array with 41088 bytes.

ceres\trust_region_minimizer.cc:226 Terminating: Gradient tolerance reached. Gradient max norm: 0.000000e+000 <= 1.000000e-010

Solver Summary (v 1.10.0-no_lapack-eigensparse-openmp)

                                     Original                  Reduced

Parameter blocks                            1                        1

Parameters                                  4                        4

Residual blocks                          1284                     1284

Residual                                 1284                     1284

Minimizer                        TRUST_REGION

Dense linear algebra library            EIGEN

Trust region strategy     LEVENBERG_MARQUARDT

                                        Given                     Used

Linear solver                     DENSE_SCHUR              DENSE_SCHUR

Threads                                     1                        1

Linear solver threads                       1                        1

Linear solver ordering              AUTOMATIC                        1

Cost:

Initial                         1.899640e+001

Final                           1.899640e+001

Change                          0.000000e+000

Minimizer iterations                        0

Successful steps                            0

Unsuccessful steps                          0

Time (in seconds):

Preprocessor                           0.0056

  Residual evaluation                  0.0000

  Jacobian evaluation                  0.9493

  Linear solver                        0.0000

Minimizer                              0.9552

Postprocessor                          0.0001

Total                                  0.9609

Termination:                      CONVERGENCE (Gradient tolerance reached. Gradient max norm: 0.000000e+000 <= 1.000000e-010)

plane before

0.998471 -0.0551461 0.0038819 6.91695

plane after

0.998471 -0.0551461 0.0038819 6.91695

Any help?

[Sameer Agarwal]

This means that you are not filling out the jacobians or the jacobians are zero in the very first iteration and the solver assumes that you have reached a minimum.

The conditional statements in your functor seem to be setting the residual to a constant value, and in another to a float. The former depending on the underlying conditions maybe generating a constant function, or the latter maybe generating a constant jacobian because the perturbations generated by the numeric diff cost function are too small.

please work in doubles. 

Also why are you using a schur based solver? this just required DENSE_QR and you are good to go.

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/adb89c6e-1003-4ef5-8a06-3050743f3665%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Веркеенко Михаил]

You're right. Relative changes in plane params are too small to change SSD (not subpixel in opencv).

I've made some changes to solver options

ceres::Solver::Options options;

options.linear_solver_type = ceres::DENSE_QR;

options.preconditioner_type = ceres::JACOBI;

options.minimizer_progress_to_stdout = true;

options.max_num_iterations = 100;

options.numeric_derivative_relative_step_size = 1e-2;

options.initial_trust_region_radius = 1e1;

options.min_relative_decrease = 1e-1;

Also I've chaged cost function to work only with plane distance.

Unfortunately relative change is still too small.

Which option should I change to increase this value?

here is cost functions call for one point

id = 351

plane dist = -2.82787

SSD = 0.0406463

id = 351

plane dist = -2.82786

SSD = 0.0406463

id = 351

plane dist = -2.82787

SSD = 0.0406463

[Sameer Agarwal]

Have a look at the options struct for dynamicautodiffcostfunction

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/ea8c4769-b69a-4750-b69b-c7b36f8626ac%40googlegroups.com.