Adding Inner Constraints

[Ghasem Abdi]

Hi folks,
I have developed a ceres solver for doing photogrammetric bundle adjustment to optimize camera params including translation, rotation, intrinsic, and distortion params as well as ground coordinates of points. In this context, for each observation, I have added a residual block related to camera params and ground coordinates. Now, I am going to add a positional inner constraints saying the average of all ground coordinates before and after adjustment should be same. I have a problem here, how can I define another cost function along the original one which has access to intermediate unknown parameters results for estimating the average of ground coordinates. Please advise
Regards,

[Sameer Agarwal]

Ghasem,

I do not understand the problem you are trying to solve. Can you state it mathematically?

Sameer 

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/de3608a9-383f-4544-8225-4753cd1c1a3dn%40googlegroups.com.

[Ghasem Abdi]

The following is a general overview of the common implementation for bundle adjustment that adds camera params and tie points (world coordinates) to the problem followed by an adjustment. Now, I am going to add a constraints saying "the average of world coordinates of tie points before and after adjustment should be same". In this context, I should estimate the average of world coordinates of tie points before adjustment and create another cost function to estimate them every iteration and try to keep them as same as the ones before adjustment. Please advise?

for (auto& track : tracks) {
     for (const auto& obs : track) {
        auto& camera_model = photos.at(photo_id).camera_model;
        std::vector<std::pair<double*, int>> parameters_list;
        camera_model.MutableCameraParameters(parameters_list);
        auto cost_func = new ceres::AutoDiffCostFunction<ReprojectionError, 2, 12, 3>(new ReprojectionError(obs, camera_model));
        std::vector<double*> param_ptrs;
        for (const auto& ptr_and_size : parameters_list) {
          param_ptrs.emplace_back(ptr_and_size.first);
        }
        param_ptrs.emplace_back(&track.mutable_world_points()[0]);
        problem.AddResidualBlock(cost_func, null_ptr, param_ptrs);
     }
}
ceres::Solver::Options options;
Solver::Summary summary;
Solve(options, &problem, &summary);

[Ghasem Abdi]

for (auto& track : tracks) {
     for (const auto& obs : track) {
        auto& camera_model = photos.at(photo_id).camera_model;
        std::vector<std::pair<double*, int>> parameters_list;
        camera_model.MutableCameraParameters(parameters_list);
        auto cost_func = new ceres::AutoDiffCostFunction<ReprojectionError, 2, 12, 3>(new ReprojectionError(obs, camera_model));
        std::vector<double*> param_ptrs;
        for (const auto& ptr_and_size : parameters_list) {
          param_ptrs.emplace_back(ptr_and_size.first);
        }
        param_ptrs.emplace_back(&track.mutable_world_points()[0]);
        problem.AddResidualBlock(cost_func, null_ptr, param_ptrs);
     }
}

Here, I am gonna add a constraint saying "Average of all param_ptrs.at(4) (the quantity is the number of tracks) should be a constant values"

ceres::Solver::Options options;
Solver::Summary summary;
Solve(options, &problem, &summary);

[Dmitriy Korchemkin]

Ghasem,

As far as I understand, you're trying to avoid global shift ambiguity by adding additional residual that depends on all 3d points. I would suggest avoiding this approach due to the following reasons:
a. Residual that depends on all world points ruins hessian structure favorable for Schur complement, thus limiting performance of linear solver
b. Under assumptions of gaussian noise in feature point detections and optimization of l2 re-projection errors world points (optimal tirangulated world points seen as random variables) might not have finite moments.

Global shift can be adjusted post-optimization via a simple coordinate transform applied to world points and positions of cameras.

If your main concern is shift ambiguity in BA - I would suggest tackling that problem by fixing position of one camera (and, optionally, constraining scale of translation of another camera to avoid scale ambiguity), followed by imposing your invariant of choice after optimization.
This way you'll avoid evaluation costs of new residual as well as preserve hessian structure that allows faster solving of linear sub-problems via Schur complement.

[Ghasem Abdi]

Thanks Dmitriy, it makes a lot of sense. The one that you offered is the minimum constraints bundle adjustment and the one that I mentioned is the inner constraints bundle adjustment. Theoretically, I do not have any ground control point to fix the entire network during bundle adjustment and I was thinking adding these inner constraints to enrich the network. Based on your suggestion, I am trying to fix one of the camera and I hope someday we can have proper infrastructure in this valuable library for inner constraints bundle adjustment.