Covariance estimation in bundle adjustent problem with fixed 3d coordinates (openMVG)

[Niko Haaraniemi]

Dear Ceres wizards!

I've been trying to compute covariance matrices from OpenMVG's  Ceres BA solution that also includes known world coordinates as fixed parameters in the problem. ( for fixing oMVG uses this method: problem.SetParameterBlockConstant(gcp_landmark_it.second.X.data())).

Still I end up in  rank deficient Jacobian and get the error: "Jacobian matrix is rank deficient Number of columns: 82605 rank: 23294."

I've been living in understanding that I should not have the gauge ambiguity problem if I provide the at least 3 fixed 3d points in the adjustment. Is my understanding wrong?

Here is my code how I compute covariance. Just in case I do something wrong there:
ceres::Covariance::Options options_cov;
options_cov.sparse_linear_algebra_library_type = ceres::SUITE_SPARSE;
ceres::Covariance covariance(options_cov);
std::vector<std::pair<const double *, const double*> > covariance_blocks;

auto structure = sfm_data.structure.begin();
covariance_blocks.push_back(std::make_pair(structure->second.X.data(),structure->second.X.data()));
auto intr = map_intrinsics.begin();

std::cout << "computation started" << std::endl;
if (covariance.Compute(covariance_blocks,&problem))
{
std::cout << "computation finished" << std::endl;
double covariance00[6*6];

covariance.GetCovarianceBlock(structure->second.X.data(),structure->second.X.data(), covariance00);
std::cout << *covariance00 << std::endl;
}

Any advice is appreciated
-Niko

[Sameer Agarwal]

Hi Niko,

There are one of two things going on here.

1. The gauge ambiguity is not resolved by fixing the 3d points. This can happen for example when you really have two disconnected reconstructions that you are treating as one.

2. The more likely cause here is that the underlying Jacobian is numerically poorly conditioned, which prevents us from computing its inverse reliably. 

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/a128556e-50e1-430e-b795-1260d2119d2dn%40googlegroups.com.

[Niko Haaraniemi]

Hi Sameer,

Thank you for your response.

1. I think the reconstruction should be ok.

2.  Is there a way to improve the condition of the Jacobian? For example defining the adjustment model differently with my points for Ceres? Or should I go and seek another tool/method that can do the inverse for poorly conditioned Jacobian (if it even make sense to estimate covariance for ill conditioned Jacobian)?

Thank you for the guidance,
Niko

[Sameer Agarwal]

On Tue, Sep 14, 2021 at 11:05 PM Niko Haaraniemi <niko.ha...@gmail.com> wrote:

Hi Sameer,

Thank you for your response.

1. I think the reconstruction should be ok.

2.  Is there a way to improve the condition of the Jacobian? For example defining the adjustment model differently with my points for Ceres? Or should I go and seek another tool/method that can do the inverse for poorly conditioned Jacobian (if it even make sense to estimate covariance for ill conditioned Jacobian)?

You can try re-parameterizing your system to improve its conditioning, but I suspect that you have some very poorly conditioned 3d points, which is why you have such a large numerical rank deficiency.  

Sameer

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/8500da3e-a09e-4d97-afaf-b0bb489f473bn%40googlegroups.com.

[Niko Haaraniemi]

Hey,

Thanks for the tips Sameer. I think I need to ensure my 3d points at first and then consider re-parameterizing.

Niko

[Dmitriy Korchemkin]

With east coordinate being 2.4e7 almost half of resolution of double precision floating point number is already taken by that coordinate offset.

It is quite typical for sfm engines to center (or center and scale) reconstruction before bundle adjustment for this reason.

On Wed, Sep 29, 2021, 10:48 AM Niko Haaraniemi <niko.ha...@gmail.com> wrote:

Hey Sameer,

I finally figured out my problem. It was too long long coordinate values. So, yes numerically poor conditioned Jacobian beacause of rounding my locations at some point of the code inside ceres. Covariance computation seem to work fine after shortening my ETRS coordinates that normally looks like this north: 6815363.763 east:24480950.385 z:103.033.

Is this known issue? It was not clear for me as the tool still uses double data type.

-Niko

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/3765c815-2e23-49c1-9967-ee117cf9ddd3n%40googlegroups.com.

[Sameer Agarwal]

Niko,

Points like this are a definite source of Ill conditioning of the Jacobian.

Also if a camera only sees distant points, it constrains it's rotating but not the translation. That will also show up as ill-conditioning.

Sameer

On Wed, Sep 29, 2021, 12:48 AM Niko Haaraniemi <niko.ha...@gmail.com> wrote:

Hey Sameer,

I finally figured out my problem. It was too long long coordinate values. So, yes numerically poor conditioned Jacobian beacause of rounding my locations at some point of the code inside ceres. Covariance computation seem to work fine after shortening my ETRS coordinates that normally looks like this north: 6815363.763 east:24480950.385 z:103.033.

Is this known issue? It was not clear for me as the tool still uses double data type.

-Niko

maanantai 20. syyskuuta 2021 klo 8.19.45 UTC+3 Niko Haaraniemi kirjoitti:

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/3765c815-2e23-49c1-9967-ee117cf9ddd3n%40googlegroups.com.

[Niko Haaraniemi]

Ok. Thanks again Dmitriy and Sameer!

-Niko