Bundle adjustment does not work :(

[Anton Zhilin]

I'm trying to get Ceres bundle adjustment example working. Here is my simplified SnavelyReprojectionError for two cameras:

template <typename T> bool FPCostFunctor::operator()(const T* const camera, const T* const realCoords, T* const residual) const { const T* angles = &camera[0]; const T& b = camera[3]; const T& x = realCoords[0]; const T& y = realCoords[1]; const T& z = realCoords[2]; T transformed[3]; const T& xTransformed = transformed[0]; const T& yTransformed = transformed[1]; const T& zTransformed = transformed[2]; ceres::AngleAxisRotatePoint(angles, realCoords, transformed); transformed[1] += b; residual[0] = x / z - T(xl); residual[1] = y / z - T(yl); residual[2] = xTransformed / zTransformed - T(xr); residual[3] = yTransformed / zTransformed - T(yr); return true; }

Here is Solver.Options:

ceres::Solver::Options options{}; options.minimizer_type = ceres::MinimizerType::TRUST_REGION; options.linear_solver_type = ceres::SPARSE_SCHUR; options.preconditioner_type = ceres::PreconditionerType::JACOBI; options.visibility_clustering_type = ceres::VisibilityClusteringType::CANONICAL_VIEWS; options.sparse_linear_algebra_library_type = ceres::SparseLinearAlgebraLibraryType::SUITE_SPARSE; options.dense_linear_algebra_library_type = ceres::DenseLinearAlgebraLibraryType::LAPACK; options.num_linear_solver_threads = 4; options.use_explicit_schur_complement = false; options.max_num_iterations = 200; options.minimizer_progress_to_stdout = true; options.num_threads = 4; options.eta = 1e-5; options.max_solver_time_in_seconds = 1e5; options.use_nonmonotonic_steps = false; options.trust_region_strategy_type = ceres::TrustRegionStrategyType::LEVENBERG_MARQUARDT; options.use_inner_iterations = false; options.logging_type = ceres::LoggingType::SILENT; options.minimizer_progress_to_stdout = false; options.gradient_tolerance = 1e-16; options.function_tolerance = 1e-16; options.parameter_tolerance = 1e-10; options.inner_iteration_ordering = std::make_shared<ceres::ParameterBlockOrdering>(); for (size_t i = 0; i < nPoints; ++i) { options.inner_iteration_ordering->AddElementToGroup(positions[i].data(), 0); }

Problem is, I keep getting large, wrong results from the solver. They are theoretically possible, but obviously far from truth for a specific dataset.

What can the problem be? I assume that I shouldn't have to do much extra work to get this example working for me.

[Keir Mierle]

Hi Anton,

Are you able to run the examples in the Ceres source base? Can you please post the result from FullReport()? Is your initialization good?

Thanks,

Keir

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/01dd5ef1-253b-4cb7-a7e5-2251b34a8dae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Vitus]

Hi Anton,

Can you give a little more information on your formulation? 

I have a couple of questions that might be causing your trouble, but my assumptions could be wrong. In order to compute the residual, you need to compute the position of the landmark in the camera frame though the following code looks like you are using the point in the global frame for computing the residual for the left camera.

residual[0] = x / z - T(xl);
residual[1] = y / z - T(yl);

When you are computing the point in the camera frame you are only applying an offset in the y-coordinate but it should have a full 3d offset as in here.

transformed[1] += b;

I hope this helps.

cheers,

Mike

[Anton Zhilin]

xl, yl, xr, yr are not actual pixel positions. I transform them as follows before getting started:

```cpp

xl = (xl - width/2) / focus;

yl = (yl - height/2) / focus;

xr = (xr - width/2) / focus;

yr = (yr - height/2) / focus;

```

So I hope that I can use global coordinates in this way. But I get residuals scaled down by `focus`, which is also OK, I guess?

Tranform actually contained an error. It should be as follows:

```cpp

transformed[0] += T(1.0);

transformed[1] += b;

```

It means that I use horizontal shift as global metric unit, I allow vertical shift, but not shift in depth-axis.

Unfortunately, this change did not affect the results in any way.

воскресенье, 21 августа 2016 г., 22:40:02 UTC+3 пользователь Mike Vitus написал:

[Anton Zhilin]

I have not tried to run the example exactly. I’m essentially trying to load my feature points into the solver, which is configured, as in the example.

FullReport() output:

Solver Summary (v 1.12.0-eigen-(3.2.9)-lapack-suitesparse-(4.5.0)-openmp)

                                     Original                  Reduced
Parameter blocks                         4496                     4496
Parameters                              13489                    13489
Residual blocks                          4495                     4495
Residual                                17980                    17980

Minimizer                        TRUST_REGION

Sparse linear algebra library    SUITE_SPARSE
Trust region strategy     LEVENBERG_MARQUARDT

                                        Given                     Used
Linear solver                    SPARSE_SCHUR             SPARSE_SCHUR
Threads                                     4                        4
Linear solver threads                       4                        4
Linear solver ordering              AUTOMATIC                  4495, 1
Use inner iterations                     True                     True
Inner iteration ordering              4495, 1                  4495, 1

Cost:
Initial                         2.179317e+002
Final                           3.356981e+001
Change                          1.843619e+002

Minimizer iterations                       17
Successful steps                            9
Unsuccessful steps                          8
Steps with inner iterations                11
Line search steps                         108

Time (in seconds):
Preprocessor                           1.1566

  Residual evaluation                  0.1248
    Line search cost evaluation        0.0000
  Jacobian evaluation                  0.3045
    Line search gradient evaluation    0.2811
  Linear solver                        1.2105
  Inner iterations                    17.0584
  Line search polynomial minimization  0.0173
Minimizer                             18.6622

Postprocessor                          0.0005
Total                                 19.8194

Termination:                      CONVERGENCE (Parameter tolerance reached. Relative step_norm: 4.554931e-011 <= 1.000000e-010.)

Initialization is 0 for all 4 camera params that I use, and initial coordinates are (0, 0, 10), which should be in front of the camera.
Limits are: -0.1 to 0.1 for camera params, and z > 1.0

суббота, 20 августа 2016 г., 23:22:40 UTC+3 пользователь Keir Mierle написал:

[Mike Vitus]

Do you only have one camera block for the entire ceres problem?

[Anton Zhilin]

Yes, I have two cameras, first is at origin, and second is slightly rotated and shifted to the right by 1.

вторник, 23 августа 2016 г., 0:41:38 UTC+3 пользователь Mike Vitus написал: