Bundle-adjusted stereo produces lower-quality DEM

[Johanna Alen-Bella]

Hello!

I am running into an issue where my bundle-adjusted stereo produces lower-quality DEMs than the DEMs pre-adjustment. 

For this process, I have done the following:
1. Ran the bundle adjustment via the command:

bundle_adjust <mosaic images> <xml files> --mapprojected-data ‘<mpfinal.tif> <COPdem.tif>’ --min-matches 0 -o run/run

Here are the values from the final_residuals_stats.txt:

Mean and median norm of residual error and point count for cameras:
2013_09_23_01_mosaic.xml, 0.20325445852790747, 0.15932942112850354, 4791
2013_09_23_02_mosaic.xml, 0.26887238171911332, 0.18226643797647357, 5438
2014_08_25_01_mosaic.xml, 0.23543223564032964, 0.18611253699615665, 4354
2014_08_25_02_mosaic.xml, 0.32902476033063344, 0.22749145358867898, 3839
Camera weight position and orientation residual errors:
2013_09_23_01_mosaic.xml, 0.2733383998085806, 0.0050294865508786609
2013_09_23_02_mosaic.xml, 0.13105159473618685, 0.0040431678187474356
2014_08_25_01_mosaic.xml, 0.30900705441179216, 0.0019573341301389268
2014_08_25_02_mosaic.xml, 0.10435338494642199, 0.0022094353736378759

Per the documentation, this all looks good to me.

2. I then ran the stereo command which is the following:

stereo -t rpc <mpfinal.tif> run_stereo/run_stereo --bundle-adjust-prefix run/run --alignment-method none

3. Next, I ran the point2dem command to create the new, bundle-adjusted DEM

point2dem --dem-hole-fill-len 200 <*-PC.tif>

Unfortunately, when I look at the image in QGIS, I get the following result which is not desired. Shown here with a hillshade on it for visualization:

I ran this using 2 mosaics and their equivalent map projected versions and then with 4 to see if I got a better result. Unfortunately, the one with 4 was even worse.

Any idea as to what might be causing this? Is there another command I should be using to produce the bundle-adjusted DEMs?

Thanks!

[Oleg Alexandrov]

It appears that you used multiview stereo. This is discouraged, and pairwise stereo followed by merging of the results would work better. This is described in https://stereopipeline.readthedocs.io/en/latest/sfm.html#multiview-reconstruction.

For stereo, it is also suggested to use the asp_mgm algorithm (https://stereopipeline.readthedocs.io/en/latest/next_steps.html#stereo-pipeline-in-more-detail). If there are artifacts where the slopes are steep, the workflow with mapprojection is suggested.

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/76bf896c-7c81-4c7e-b229-eb4594fc1927n%40googlegroups.com.

[Oleg Alexandrov]

To add to the previous, maybe the issue is using  --alignment-method none. It is better to let stereo handle alignment itself, especially if the left and right images do have a notable difference of perspective. 

[Johanna Alen-Bella]

Thank you for your quick reply!

On a second look I noticed (in the 2-mosaic example) that landforms are still recognizable, but much distorted and warped in size. Apologies for the giant image:

On the left is the hillshade of the DEM, on the right is a Google Earth satellite layer. Both are in 1:10000 scale per QGIS, so the hillshade has been shrunken down a good deal. Here's the offset, as if someone dragged the bottom right corner to re-size the DEM, with corresponding areas circled:

Two things:

1. I ran it all again without the alignment-method none; didn't make a difference as far as I can tell.

2. You mentioned multi-view versus pairwise stereo. Could you elaborate on this? I'm pretty new to this and I'm not quite sure what command I should be running to properly bundle-adjust my mosaicked images.

Thank you!

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

>On a second look I noticed (in the 2-mosaic example) that landforms are still recognizable, but much distorted and warped in size. Apologies for the giant image:

You can try to create an orthoimage with point2dem by modifying a little your point2dem command (the doc for that is here: https://stereopipeline.readthedocs.io/en/latest/tools/point2dem.html#orthoimage-and-error-image).

Overlaying the orthoimage onto the base layer can make it clearer, maybe.

>On the left is the hillshade of the DEM, on the right is a Google Earth satellite layer. Both are in 1:10000 scale per QGIS, so the hillshade has been shrunken down a good deal

That looks like a huge change. Usually that suggests something is not right about the cameras. 

>2. You mentioned multi-view versus pairwise stereo. Could you elaborate on this? I'm pretty new to this and I'm not quite sure what command I should be running to properly bundle-adjust my mosaicked images.

I assume this means you don't use multiview stereo. So, your bundle-adjustment is done for all images, but stereo per pair, as it should be, per our doc. (I also have a suggested command below.)

I am not sure what you mean by "mosaicked" images. The images should be raw, as produced by the camera. 

No merge of images should take place. Also these should not be ortho products. 

The simplest thing to do could be to take it slow. Use only two images. One left image, and another one is the right image. 

No bundle adjustment, for now.

Do stereo with the left and right image, such as:

stereo left.tif right.tif left.xml right.xml run/run 

Check the stereo convergence angle as stereo runs. It is hopefully no less than 10 degrees.

Run point2dem as:

point2dem run/run-PC.tif --orthoimage run/run-L.tif 

and overlay run/run-DRG.tif onto your base layer and see what you get. 

Maybe this will clarify something.

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Johanna Alen-Bella <johanna....@gmail.com>
Sent: Wednesday, January 8, 2025 3:35 PM
To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>
Subject: [EXTERNAL] [BULK] Re: Bundle-adjusted stereo produces lower-quality DEM

 

CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/04d44eb8-9b8e-4739-a815-8c27b5c59046n%40googlegroups.com.

[Johanna Alen-Bella]

> I am not sure what you mean by "mosaicked" images. The images should be raw, as produced by the camera.

Per year, I have four images. For example, I have 2013_09_23_01_P001_PAN, 2013_09_23_01_P002_PA,  2013_09_23_02_P001_PAN, 2013_09_23_02_P002_PAN. On those I've used the following command:

wv_correct <original.tif> <original.xml> <wv_corr.tif>

cp <original.xml> <wv_corr.xml>

Then, I've mosaicked the two P001_PAN images together, and the two P002_PAN images together:

dg_mosaic --fix-seams *_wv_corr.tif

Should I be doing the bundle adjustment before the dg_mosaic step?

Thank you!

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

This appears to be Maxar (DigitalGlobe) data. It usually does not need bundle adjustment.

You can try avoiding wv_correct and mosaicking with dg_mosaic too, as these are auxiliary steps and not strictly necessary to get a first idea about how good the data is.

The simplest thing one can do, even before stereo, is to get a prior DEM of that area, ensure it is relative to WGS84 (https://stereopipeline.readthedocs.io/en/latest/next_steps.html#choice-of-initial-guess-terrain-model), and mapproject your input images with the corresponding cameras onto this DEM with our mapproject program (can crop the DEM to a small area first to reduce the time). Such a mapprojected image should be overlaid onto your existing layer as a sanity check. Can  do same thing for the other images too and overlay them all.

Then, you can run stereo without any of the above tools, just with one left image and one right image (assuming they form a stereo pair). For example, run 2013_09_23_01_P001_PAN with  2013_09_23_02_P001_PAN, if they overlap, and check what the stereo convergence angle is.

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Johanna Alen-Bella <johanna....@gmail.com>

Sent: Wednesday, January 8, 2025 7:57 PM

To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>

Subject: Re: [EXTERNAL] [BULK] Re: Bundle-adjusted stereo produces lower-quality DEM

 

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/be2091f5-7117-44e0-97f9-20dd862173a5n%40googlegroups.com.

[Johanna Alen-Bella]

Thank you so much for your fast replies!

I have a Copernicus DEM that I have transformed to WGS84 like so:

gdalwarp -t_srs EPSG:32611 -r cubic COP_DEM_initial.tif <proj_cop_dem.tif>

dem_geoid <proj_cop_dem.tif> --geoid egm96 --reverse-adjustment -o COP_DEM_final.tif

I've had good results doing parallel-stereo on this without the bundle adjustments, so I think my original images and cameras are fine. I was looking to do bundle adjustments to increase the accuracy as much as I could.

I'll think on it and see what my next step is. Thank you again!

[Oleg Alexandrov]

Bundle adjustment should work well unless some mixup happens. The following command should work:

bundle_adjust -t dg \

  left.tif right.tif \

  left.xml right.xml \

  --tri-weight 0.1 \

 -o run/run

One should check the input and output report files, per https://stereopipeline.readthedocs.io/en/latest/tools/bundle_adjust.html#output-files.

For DigitalGlobe data, these should change very little, both in the report files measuring in pixels and the ones that have meters. 

Then, a stereo run can be reused, this time with bundle-adjusted cameras, with the option --prev-run-prefix, or a new run can be created altogether. In either case, the produced DEM should be very similar to the one without bundle adjustment, at least for DigitalGlobe data.

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/e3ac9f33-7842-4c68-ac1f-ffeb7aa06ffcn%40googlegroups.com.