Bundle_adjust | min-distortion option

[Jean Baptiste Barré]

hi,

With ASP3.6.0-alpha-2025-09-18.

the bundle_adjust has now this min-distortion option that allow you to optimize small distortion parameters. I'm running bundle_adjust with it to calibrate the camera (tsai parms) as the command fails without it  (not enough IPs after ransac filtering). 
In the end, all parameters (k1,k2,..p1,p2,...) are optimized, even if some distortion parameters are forced to 0 before the bundle_adjust. I am having difficulty calibrating the KFA-1000 camera. Residuals are small enough, but, by keeping all distortion params, I'm afraid of ending up with an over-adjusted camera model, which seems to be the case when looking at the DEM. I'd like to keep some parameters at 0 (tangential distortion) which seems to be impossible.

I don't fully understand the effect of minimal distortion on calibration and wonder if we can still fix some params at 0. 

What do you suggest regarding this min-distortion?

Here is an example of a typical bundle_adjust, I'm running to solve the intrinsics: 

bundle_adjust -t nadirpinhole --datum WGS84 --camera-weight 0 --skip-rough-homography --overlap-limit 2 --ip-per-tile 100 --max-pairwise-matches 20000 --ip-num-ransac-iterations 500000 --solve-intrinsics --intrinsics-to-float other_intrinsics --min-distortion 1e-16 --heights-from-dem /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/data/arcticdem_0269_20237_20239_crop_seed.tif --heights-from-dem-uncertainty 10.0 --remove-outliers-params "75 3 5 8" --inline-adjustments /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/data/0269-20237_sub_high.tif /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/data/0269-20238_sub_high.tif /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/data/0269-20239_sub_high.tif /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/camera_models_init/bc/run-0269-20237_sub_high_cam_init.tsai /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/camera_models_init/bc/run-0269-20238_sub_high_cam_init.tsai /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/camera_models_init/bc/run-0269-20239_sub_high_cam_init.tsai -o /mnt/data-desk/air/DATA_SERVER/0269_20237_20239_proc/camera_models_init/ba_intr/run --threads 12

[Oleg Alexandrov]

The issue you brought up has been on my to-do list for a while. I now added the option --fixed-distortion-indices to bundle adjustment to control which lens distortion parameters are kept fixed. These distortion values will not be changed with the --min-distortion value. This will be in the 2025-10-11 build at https://github.com/NeoGeographyToolkit/StereoPipeline/releases. The documentation of this program has more details.

> I'm running bundle_adjust with it to calibrate the camera (tsai parms) as the command fails without it  (not enough IPs after ransac filtering).

This is a bit concerning. Lens distortion is tricky. It is suggested to use dense matches from disparity, ensure that you have a good stereo baseline (images have different perspectives), and there are lots of overlaps between the images, especially in corners. Careful evaluation of resulting terrain models is suggested. 

We have a lengthy writeup of many aspects of bundle adjustment at https://stereopipeline.readthedocs.io/en/latest/bundle_adjustment.html. Happy to hear how things are working for you.

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/5555ff8d-5609-471b-ac5b-e96c688e0a23n%40googlegroups.com.

[Jean Baptiste Barré]

thank you Oleg. I'll test with fixed-distotion-indices and keep you informed.
JB

[Jean Baptiste Barré]

Hi Oleg, 

Here is my feedback after running a series of tests.

The new option --fixed-distortion-indices works perfectly. Thank you for implementing it so quickly!

Regarding the calibration by itself, I'm afraid my strategy isn't robust enough for KFA1000 images. It performs well on a few strips but fails on many others. It consists of a succession of 3 Bundle_adjust (BA) : 1- extrinsics only, 2 - intrinsics, 3- extrinsics refinement.  When the calibration fails, the principal point offsets (xp, yp) end up far from the optical center, sometimes by several tens of millimeters (even >100 mm).  One potential issue I noticed is that ASP finds a limited number of matching points (see the screenshot in the document below). I’m unsure what the best approach would be to generate more reliable matches: Should I precompute matches using ipfind/ipmatch beforehand, or would it be better to fine-tune the filtering parameters within bundle_adjust? Another possible limitation is the small convergence angles between image pairs (typically 3–6°). Would you consider that a critical issue for calibration?

For clarity, I’ve shared a short document with an example of the problem: https://docs.google.com/document/d/1Uvg-TwWSQ1A2tn-cK_WBLlg1yGxaFyKazgM_y6rIo48/edit?usp=sharing

Many thanks in advance for your time and suggestions.
JB

[Oleg Alexandrov]

The key issue seems to be that a huge number of your interest points got removed as outliers. Maybe it is because this control is too tight:  --remove-outliers-params '50.0 3.0 0.1 2.0'. For now, given that you have lens distortion, it is suggested to let as many features be in, so I would use here something like '75 3 50 50'. So, residuals of 50 pixels would be allowed. Bundle adjustment does well with high residuals, as internally it attenuates them, but it does not do well when too many matches are thrown out.

This one is likely excessive:  --ip-num-ransac-iterations 500000. The default of 1000 should be fine. This one --ip-per-tile 200 could likely be increased to even 500 I think.

This one:  --overlap-limit 2 could likely be increased. More images overlapping is better. Maybe having 5 or more is good.

This one: --heights-from-dem-uncertainty 10.0 can be quite tricky. It really depends on how good your DEM is, and your alignment. In your situation, may try to put this say to 100 or so. We don't want to tie too tightly to the DEM at this stage.  You can check to see if there is misalignment between mapprojected images and your DEM. Misalignment is not good if this option is used.

>  When the calibration fails, the principal point offsets (xp, yp) end up far from the optical center,

I will suggest keeping the principal point fixed for now. Calibration of the principal point is not well-posed, because it correlates strongly with camera position. See option --intrinsics-to-float. Also option --intrinsics-to-share.

>Another possible limitation is the small convergence angles between image pairs (typically 3–6°). Would you consider that a critical issue for calibration?

This is not great, but if you have enough images, it will likely work.

So, the most important thing you can do is to ensure you have a lot of "clean" interest point matches. These are the ones saved with suffix -clean.match, and are after outlier filtering. Any such clean matches can be reused at next stages, rather than creating them from scratch, with the option --clean-match-files-prefix.

It is suggested to mapproject the images first, and then do interest point matches from mapprojected images, especially in steep terrain as what you have. https://stereopipeline.readthedocs.io/en/latest/tools/stereo_gui.html#mapip

You should have lots of clean matches, especially in corners, otherwise nothing will work. 

Later, for fine-tuning, dense matches from disparity may work, but for now lots of clean matches all over the place will work. So, this is for later (https://stereopipeline.readthedocs.io/en/latest/bundle_adjustment.html#dense-ip).

It is also not clear to me how you evaluate if you need calibration to start with. Do you get a DEM with warping without that or with later triangulation error (ray intersection error)? Having good evaluation criteria is important. 

May also consider not optimizing focal length at least to start with. Calibration is a bit tricky. One may need to think carefully about which parameters matter most early on.

So, my near-term suggestion is to ensure matches are plenty. You can write privately as well if this public discussion gets too lengthy. I am optimistic one can do well in your case, but more work may be needed.

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/729ccacb-e313-4b4a-beca-8e285d9623a1n%40googlegroups.com.