contribution: trifocal initialization

[Ricardo Fabbri]

Dear Theia team,

I have recently worked with several colleagues on writing new solvers for calibrated trifocal geometry estimation, and we are in the process of incorporating this into widely used software such as Theia. I would like to know if there is interest in this community on the ability to bootstrap the SfM pipelines from three views instead of two. If so, I can work on a pull request as the C++ code is ready for Theia, only depends on Eigen, has unit tests, is easy to maintain, and so on.

The idea is that the usual technique of starting SfM from an essential matrix that requires 5 points between two views, then using P3P, sometimes fails from lack of reliable point features or camera configuration degeneracies.  Specially in indoor and man-made scenes (also cars, mechanical parts, etc), it might be desirable to use our technique that allows only 3-4 correspondences or mixed point-line correspondences across 3 views, as points, lines and curves are widely available in these common scenarios.

What the code provides is

* A solver for reconstructing 3-view camera pose (calibrated) from three corresponding points and a corresponding line, which can be used inside RANSAC.

* A solver for reconstructing 3-view camera pose from three corresponding points, where at least two of the points are attributed with a tangential direction (or incident line). This occurs when observing curves (points in curves which have tangents), vector fields such as fluid particle velocities, and also corners, which have multiple directions attributed to a point.

Please let me know if in your opinion this code can be useful for Theia. The present version of the code runs in the order of 1s per solve, in many cases a lot faster. However, this can be useful even for offline bootstrapping a hard low-texture indoor scene, and also when RANSAC is not used. In some aplications, such as in indoor metrology, the user actually clicks corresponding features, and the object (such as an indoor fluid tank) has not enough point features, but plenty of lines. The above techniques actually uses the available points, if any, since usually even hard scenes provide a couple of reliable points.

The code is generic and can be further extended as a general-purpose solver for other feature combinations or the uncalibrated case as well. I will be happy to contribute the code to Theia if desired.

If you are interested, here is our paper preprint https://arxiv.org/abs/1903.09755

Best regards,

Ricardo.

[Chris Sweeney]

Hi Ricardo,

Thanks for the detailed description of your contributions! It sounds like this would be very interesting to add to Theia -- indeed 2-view estimation is a clear weak point of any SfM system, and adding a 3rd view dramatically improves the local geometry accuracy in my experience. I would love to see this added to Theia! At the very least, the core minimal solver would be valuable and certainly anything additional to integrate it would be really appreciated. Let me know how I can help, if I can give refactor hints, etc.

Chris

--
You received this message because you are subscribed to the Google Groups "Theia Vision Library" group.
To unsubscribe from this group and stop receiving emails from it, send an email to theia-vision-lib...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/theia-vision-library/71483207-4c14-4f99-b297-dab2d15399ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.