Homogeneous Vector Parameterization

[Chris Sweeney]

I was toying around with the new homogeneous vector parameterization that was just added (using it for 3D points in BA) and am getting some mixed results. It my opinion that this is indeed the "right" way to optimize 3D points but I find that, for instance, ceres takes longer to optimize for BA with this parameterization and that mean/median reprojection error is not necessarily lower. These metrics don't necessarily mean that one way is better or worse, just noting the observed differences.

I am wondering if there is any measurable data to show the trade-offs (benefits?) of using this parameterization for BA. Perhaps the Bundle Adjustment in the Large datasets would be a nice candidate?

-Chris

[Markus Moll]

Hi

Are you solving a Euclidean reconstruction problem? Homogeneous coordinates
are particularly important only if your reconstruction is in a projective
frame (see also http://en.wikipedia.org/wiki/Homogeneous_coordinates). In that
case, the behavior near or at infinity becomes a lot more important (these
points could actually be very close), and homogeneous coordinates are a better
parametrization.

Markus

[Chris Sweeney]

Yes, indeed I am solving Euclidean problems. Again, I do agree that this parameterization is the correct thing to be doing. I am just wondering if there are any numbers readily available that can help quantify the performance differences.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/4046283.lLbu7m16AZ%40x2.
For more options, visit https://groups.google.com/d/optout.

[Sameer Agarwal]

Chris,

Could you share some comparison numbers with us? Speed and reconstruction quality.

And if possible the v2 log.

Sameer

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O49k3M-5RBOg_d_832bYNY1en36w%2BGq2rherxknBBXQVw%40mail.gmail.com.

[Chris Sweeney]

Sorry for the delay. Here are some results. I am estimating camera poses with Global SfM first, then performing N-View triangulation (midpoint method + using Ceres for BA on the track while holding cameras constant), then running full BA to optimize camera poses and points. When performing triangulation I only keep points where the reprojection errors of the initial triangulation is above a certain threshold. After BA, I remove 3D points that have high reprojection errors.

I ran each method about 10 times and the stats below were representative of what I was seeing across all runs. Given my method for triangulating and filtering points, this should be a decent indication of the performance difference since the camera poses are approximately equivalent for both scenarios.

Here are some results for reconstructing Notre Dame from flickr images:

Without homogeneous parameterization (531 cameras, 233800 points)

Triangulation time = 19.8069s

BA time = 316.035s

Mean reprojection error = 1.17591

Median reprojection_error = 0.968659

With homogeneous parameterization (525 camera, 231647 points)

Triangulation time = 25.2355s

BA time = 527.916s

Mean reprojection error = 1.18038

Median reprojection error = 0.971605

I think the visual quality is very similar and the reprojection error and number of points/cameras is similar (though the homogeneous one is almost always just slightly lower). the increased time is what stands out to me the most. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O49k3M-5RBOg_d_832bYNY1en36w%2BGq2rherxknBBXQVw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUAKXuNDmxgLUj4m3-S2H2tJHBeh85fs760OYiUrOhxC_Q%40mail.gmail.com.

[Chris Sweeney]

To clarify: "though the homogeneous one is almost always just slightly lower" by "lower" I meant to say that the homogeneous one is almost always a bit worse.

[Sameer Agarwal]

Thanks Chris. The increased time does not make sense to me.

Can you post the Summary::FullReport with and without homogeneous parameterization. Is it more iterations in one case than the other? or is it more time per iteration?

Sameer

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O5%3DetKrMqkRuL_gTJDiAKxK9YvJn3ASkU0_K8ndUcU%3DCw%40mail.gmail.com.

[Michael Vitus]

Hi Chris,

How are you computing your cost term for the homogeneous parameterization?

Mike

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/4046283.lLbu7m16AZ%40x2.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O49k3M-5RBOg_d_832bYNY1en36w%2BGq2rherxknBBXQVw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUAKXuNDmxgLUj4m3-S2H2tJHBeh85fs760OYiUrOhxC_Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Sweeney]

The cost is standard reprojection error for a pinhole camera (for optimizing a track and for BA).

Sameer -- I can try to get you specific numbers on the iterations and time per iterations tomorrow. If I remember correctly the homogeneous parameterization was using more iterations and not necessarily more time per iterations. But I need to double check that.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/75ae8e96-d555-4b70-8de4-e690c20db54e%40googlegroups.com.

[Michael Vitus]

Sorry, I should have been more explicit on why I was asking.  There are many different ways to formulate the cost function for the reprojection error.  For instance, are you converting the homogeneous point to a Euclidean point and then computing the reprojection error?  Or are you doing the transformation in the perspective space and then computing the reprojection error?  These should have different numerical properties which would affect the convergence.

[Sameer Agarwal]

Good point, to elaborate.

if you have a homogenous point X you could dehomogenize first and then apply the extrinsics.

Y = X / X[3];

U = R Y + t

image_point = U/U[2];

or you could do apply the camera intrinsics directly.

U = R * X[0:3] + t * X[3];

image_point = U/U[2];

You should be doing the latter and not the former. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/37947c84-abd1-4bb8-b9bb-e565bc2f49cc%40googlegroups.com.

[Chris Sweeney]

I am indeed doing the latter.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUASQJo2oKhj%3Dr-EJWZ8Nk4FwjWDYaHUqpOVVsK1-seQTQ%40mail.gmail.com.

[Sameer Agarwal]

okay then I would definitely like to see v2 execution logs.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O6WX2S5VbzmPcs9DYmVBdun9O6xkdjNKxPu79wd77ibbg%40mail.gmail.com.

[Chris Sweeney]

I perform 2 rounds of BA after triangulation, removing outlier points after the first round. A similar number of outliers are removed with both methods.

No homogeneous parameterization: 100 its (15 inner), 150s

With homogeneous parameterization: 215 its (22 inner), 358s

Attached are the v2 logs using homogeneous and not using homogeneous. I only include the BA logging. The residual and jacobian evaluation are so much slower for the homogeneous vector parameterization. Both ways have similar intial and final costs.

Chris

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/37947c84-abd1-4bb8-b9bb-e565bc2f49cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUASQJo2oKhj%3Dr-EJWZ8Nk4FwjWDYaHUqpOVVsK1-seQTQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O6WX2S5VbzmPcs9DYmVBdun9O6xkdjNKxPu79wd77ibbg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUBMAkdakUTj4QbdACiZeasz0StsJYU1Y%2BoYhEyD4HPpew%40mail.gmail.com.

[Chris Sweeney]

Oops sorry, for this data I only performed 1 round of BA after triangulation.

[Sameer Agarwal]

Chris, thanks for the logs. So I see two things going on here.

1. The cost per iteration has gone up.

The cost of a pure LM iteration has gone up by about .1 second.

The cost of a LM iteration + inner iteration has gone up by 1 second.

The inner iterations are more expensive. We are paying an extra cost for doing the jacobian eval now, since we need to compute the point local parameterization and apply it.  We are doing 100s of thousands of these calls -- probably millions every inner iteration so I am not surprised by the increase in time.

2. The solve with the homogeneous parameterization is taking 

You will also notice that the linear solver time has improved from .4 seconds per solve to .3 seconds per solve.

For 1) we can look into reducing the cost of applying the local parameterization jacobian. But 2) is more concerning/interesting. I noticed that the problems in the two cases are slightly different. So it is hard to tell if the increased number of iterations has to do with the problem or the parameterization.

Is it possible to do an apples to apples comparison here?

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O7JSyh9n%2Bxg%2B3oZmvaKn7oZrnFMH89FPW6zwtvhJ4vC%2Bg%40mail.gmail.com.

[Chris Sweeney]

I think you left out some words in 2) but I assume you meant to say homogeneous is taking more iterations?

I don't have time now to do an apples to apples comparison (sorry) but I have run this test (homogeneous vs no homogeneous) at least 20 times now and am consistently having more iterations when using the homogeneous parameterization. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUCLpZj3-CEDWuftWPo5Pat3V%2BPah9FcVRbiLC39YzS7DQ%40mail.gmail.com.

[Sameer Agarwal]

Thanks Chris. I will take a look.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAB_h4O46Cow60kg1tw1zY9GXhsggwbBbLA20HZABZ2YY6m1s-Q%40mail.gmail.com.