High residuals and DEM artifacts with SPOT-5 stereo using bundle adjustment

[Francesco Ioli]

Dear Oleg and ASP Community,

I am using ASP to process SPOT-5 stereo images for glacier DEM generation. While the workflow succeeds for most pairs, about 25% produce high bundle adjustment residuals (hundreds of pixels) and the resulting DEMs in general have high triangulated error and show artefacts such as bands with high triangulation error (that are then removed by filtering), scattered points with extreme elevation values.

The affected images are of good quality—sometimes better than pairs yielding good results. Disparity maps look reasonable, but triangulated point clouds are problematic, suggesting issues with the SPOT-5 camera model or incorrect RPC computation.

this is the filtered DEM

This is a zoom of the hillshaded DEM before applying the filtering. The noise are negative values in the DEM that are then clearly filtered out.
However, this is the H disparity map computed from the stereo-F.tif and it looks definitely better than the resulting DEM. 

These are the residuals after BA:

IMAGERY_0.TIF, 302.49276505682542, 0.39772270783466424, 922
IMAGERY_1.TIF, 1064.4643429676958, 1.1861021192584178, 922

And I actually noticed that the cameras are almost not moved in the bundle. This is the adjusted version of camera 1 (here I was getting slightly better results decreasing the tri-weight in the bundle):

-2.9121780666644839e-12 -1.8474757597442705e-13 5.7778665628041269e-12
1 9.6721976158939355e-13 -2.6907099333228719e-12 -6.3774563585167884e-13 

This is my workflow (I omit the full file paths in the commands):

1. RPC Computation: Using add_spot_rpc with elevation limits 0–4500 m for the Aletsch Glacier area.
2. Bundle Adjustment with 
   bundle_adjust IMAGERY_0.TIF IMAGERY_1.TIF METADATA_0.DIM METADATA_1.DIM -o outputs/ba/ba -t spot5 --heights-from-dem COP-DEM.tif --heights-from-dem-uncertainty 30.0 --max-iterations 500 --ip-per-tile 200 --matches-per-tile 50 --datum WGS_1984 --tri-weight 0.1
   I also experimented with --tri-weight 0.05 and other values, but results remain inconsistent.
3. Map Projection: Using Copernicus 30m DEM (ellipsoidal heights) smoothed with a Gaussian filter (kernel size 3 or 5).
4. Stereo Correlation with asp_mgm on mapprojected images (with parameters similar to  “fine quality” parameters used by Shashank and Shean https://zenodo.org/records/4554647):
   
   parallel_stereo mapproject_0.tif mapproject_1.tif METADATA_0.DIM METADATA_1.DIM stereo/stereo COP-DEM.tif --stereo-algorithm asp_mgm -t spot5maprpc --cost-mode 3 --corr-kernel 7 7 --subpixel-mode 9 --subpixel-kernel 15 15
5. DEM and Point Cloud Generation using Tukey outlier removal.
6. Co-registration with ICP and Nuth & Kääb masking out glacier areas, vegetation, water bodies etc. 

Have others encountered similar issues with SPOT-5 data?

Could this be a jitter problem?

Do you have any recommended parameter adjustments or strategies to improve results?

I attach a zip file with one of the "problematic" stereo pair for replicating the issue. 

asp_issue.zip

Thank you for your insights!

Francesco Ioli

Department of Geography
University of Zurich

[Oleg Alexandrov]

I did a preliminary study. 

My best guess so far is that bundle adjustment is actually not a problem, as the resulting cameras look good even if the residuals are large. I think there are a lot of outliers here that result in high residuals.

What appears to happen is that something is not right with the filtering or other options in the stereo.default file. 

I did a run without that stereo.default file (it can be hidden so it does not get loaded). I also used the RPC cameras, for speed, since for SPOT5 loading cameras is very slow. I did not use bundle adjustment at all.

Here's the command I ran on a couple of clips chosen in stereo_gui:

parallel_stereo IMAGERY_0.TIF IMAGERY_1.TIF METADATA_0.DIM METADATA_1.DIM -t rpc --stereo-algorithm asp_mgm run_rpc/run --left-image-crop-win 8019 4146 2010 2426 --right-image-crop-win 7984 3488 2062 2530 --corr-seed-mode 1 --sgm-collar-size 256 --threads 16

 

On the left is the DEM you shared, for the small area, and on the right is the DEM I make. They are both noisy, but the one I make has a lot more coverage.

Note that I did not use mapprojection here, which would likely make the DEM on the right even better. This area is very steep, and I think that explains the  holes.

My suggestion would be to see if you can reproduce the result on the right, which should be quick. Then, trying to see if mapprojecting would improve things. One can either mapproject onto the hole-filled and smoothed version of the DEM on the right, or onto your prior DEM, which hopefully was adjusted to be relative to the ellipsoid, per the doc.

Using RPC cameras for all parts of the process is likely good to start with, as given your mapprojected images with RPC it appears that RPC is fine.

If happy enough with the results, one can try a bigger area, or switching back to the exact SPOT5 model, or use bundle adjustment again. 

Let me know how it goes. This is a tricky area, with very large contrast between snow and bare ground.

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/675efd37-cafb-410a-8b39-9583d2800d59n%40googlegroups.com.

[Francesco Ioli]

Dear Oleg,

Thank you so much for your hints. I was able to replicate your results, and, indeed, without the bundle adjustment, the resulting DEM is fine. 

 I therefore looked deeper into the bundle results, and found that most of the matched keypoints were rejected after the first run of the bundle (the left screenshot below shows the .match file, the right one shows the -clean.match file), and the only matches considered as valid have a very bad distribution. As a result, most of them clearly have large reprojection errors (see the ba-final_residuals_pointmap.csv). 

I guess that the matched points are rejected due to a bad camera geometry (at least for one image) that produces large epipolar errors in the matched points. I also tried to use SIFT as local feature descriptor, but the result was similar, so I guess the problem is not in the keypoints extraction and matching. 

The strange fact is that this happens on multiple image pairs, but not in all of them. 

Do you have any suggestions on this? 

Thanks in advance!

Cheers

Francesco

[Oleg Alexandrov]

I am getting a lot more clean matches with the RPC camera rather than the SPOT5 exact camera.  So, ASP thinks that the SPOT5 camera does not agree as well with the images as the RPC camera. I am not sure why. Many valid matches are filtered out as having high reprojection error.

My suggestion would be to avoid bundle adjustment with this camera. If it gives good enough results that way, maybe it is good enough. It is suggested to run point2dem with the option --errorimage and check not only the DEM but also the resulting IntersectionErr.tif file for problems. The point2dem has more on this.

We did not study SPOT5 for a long time, and I am not sure what issues it may have. 

On Tue, Jan 28, 2025 at 2:34 AM Francesco Ioli <francesc...@gmail.com> wrote:

Dear Oleg, 

Thank you so much for your hints. I was able to replicate your results, and, indeed, without the bundle adjustment, the resulting DEM is fine.  I therefore looked deeper into the bundle results, and found that most of the matched keypoints were rejected after the first run of the bundle (the left screenshot below shows the .match file, the right one shows the -clean.match file), and the only matches considered as valid have a very bad distribution. As a result, most of them clearly have large reprojection errors (see the ba-final_residuals_pointmap.csv). I guess that the matched points are rejected due to a bad camera geometry (at least for one image) that produces large epipolar errors in the matched points. I also tried to use SIFT as local feature descriptor, but the result was similar, so I guess the problem is not in the keypoints extraction and matching. The strange fact is that this happens on multiple image pairs, but not in all of them.  But do you have any suggestions on this?   Thanks in advance!

Cheers

Francesco

On Tuesday, January 21, 2025 at 8:53:03 PM UTC+1 oleg.al...@gmail.com wrote:

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/058b6155-1000-440a-ae21-b014c5569ff0n%40googlegroups.com.