Correlation of multi-temporal images - shadow effect at different resolutions

[Ariane]

Hello,

I am using Ames and parallel_stereo in its function as a pure image correlator (--correlator-mode option) to track surface displacements. Here, I am matching images that were acquired at different times throughout the year with variable sun positions. The moving shadows are often picked up as displacement, however, I noticed that the magnitude and area affected differs with the resolution of the input images even though I am keeping the correlation and sub pixel kernels constant in terms of size (in meter, not pixel). I am not entirely sure how to explain this effect and was wondering if you have any insights into why this might be?

Here is the command I am running (for the 10m example):

parallel_stereo img1_10m.tif img2_10m.tif 20210520_20210927_10m --correlator-mode -t rpc --datum Earth --skip-rough-homography --stereo-algorithm asp_bm --subpixel-mode 2 --corr-kernel 9 9 --subpixel-kernel 19 19 --threads 0

I am using the following two input images (Sentinel-2 clips) from Mai and September of the same year over stable terrain (disparity should be 0, but is not due to the different illumination):

I have upsampled both input images to different spatial resolutions (using nearest neighbor interpolation): 10 m (original), 8 m, 6 m, and 4 m.

Here are the obtained displacement fields (band 1 (first row) & band 2 (second row) of the filtered disparity file *-F.tif) converted into units of meter for comparison. Correlation kernels are kept approx. constant in units of meter. The magnitude of the estimated displacement over the shaded area is decreasing, even though kernel should contain exactly the same features:

My best guess would be an improved subpixel matching due to the substantially higher number of pixels in a correlation kernel at higher resolution?

Thanks in advance!
Ariane

[Oleg Alexandrov]

It is a fundamental limitation of stereo correlation that it cannot distinguish between true changes on the ground and ones due to illumination. I am not sure if you would get better results with the asp_mgm algorithm instead of asp_bm while using say --subpixel-mode 9 instead of --subpixel-mode 2.

A robust approach for change detection with ASP is to create one terrain model with a stereo pair acquired with images close in time, then another terrain model with another set of stereo images that are similar to each other in illumination but may be different than the previous pair, then align the two terrain models, and take their difference.

Otherwise, it is suggested to use images with similar illumination. Many satellites on purpose have an orbit so that they always visit the same site at about the same time of day, to avoid such issues.

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/0be5fcbe-4376-4d61-b40e-893764a28668n%40googlegroups.com.

[Ariane]

Hi Oleg,

Thanks for the quick response.

I understand that stereo correlation will not be able to distinguish the shadow bias from true displacement. My question was more towards why the correlator is estimating different magnitudes of displacement depending on the spatial resolution of the input images? As I am keeping correlation kernel sizes constant (nr of pixels adjusted to the respective resolution), I would have expected that the estimated shadow bias remains constant as well.

[David Shean]

Hi Ariane,

This looks like the start of a nice correlator sensitivity study.

Is there a reason you are using nearest neighbor interpolation for the upsampling?  I would recommend bicubic or other methods better suited for upsampling.

Which Sentinel-2 products are you using?  Level-2A?  Identical tile and relative orbit?  Using 16-bit input images?

The 9x9 px mgm correlator might be useful for comparison here.  When I’ve used block matching for applications like this, I’ve always started with a larger integer corr_kernel (e.g., 31x31 px) and then smaller sub-pixel kernel (e.g., 11x11).  

This recently published study may also be relevant: https://doi.org/10.1029/2024EA003660.  Others on the mailing list have also done sensitivity studies with the ASP correlator, so they could chime in.

-David

--
David Shean
Civil and Environmental Engineering
University of Washington
https://www.ce.washington.edu/facultyfinder/david-shean

201 More Hall, Box 352700
3760 E. Stevens Way NE
Seattle, WA 98195-2700
Office: (206) 543-3105, Wilcox Hall 265
Pronouns: he, him, his

On May 15, 2025, at 11:06 AM, Oleg Alexandrov <oleg.al...@gmail.com> wrote:

It is a fundamental limitation of stereo correlation that it cannot distinguish between true changes on the ground and ones due to illumination. I am not sure if you would get better results with the asp_mgm algorithm instead of asp_bm while using say --subpixel-mode 9 instead of --subpixel-mode 2.

A robust approach for change detection with ASP is to create one terrain model with a stereo pair acquired with images close in time, then another terrain model with another set of stereo images that are similar to each other in illumination but may be different than the previous pair, then align the two terrain models, and take their difference.

Otherwise, it is suggested to use images with similar illumination. Many satellites on purpose have an orbit so that they always visit the same site at about the same time of day, to avoid such issues.

On Thu, May 15, 2025 at 10:46 AM Ariane <ariane....@gmail.com> wrote:

Hello,

I am using Ames and parallel_stereo in its function as a pure image correlator (--correlator-mode option) to track surface displacements. Here, I am matching images that were acquired at different times throughout the year with variable sun positions. The moving shadows are often picked up as displacement, however, I noticed that the magnitude and area affected differs with the resolution of the input images even though I am keeping the correlation and sub pixel kernels constant in terms of size (in meter, not pixel). I am not entirely sure how to explain this effect and was wondering if you have any insights into why this might be?

Here is the command I am running (for the 10m example):

parallel_stereo img1_10m.tif img2_10m.tif 20210520_20210927_10m --correlator-mode -t rpc --datum Earth --skip-rough-homography --stereo-algorithm asp_bm --subpixel-mode 2 --corr-kernel 9 9 --subpixel-kernel 19 19 --threads 0

I am using the following two input images (Sentinel-2 clips) from Mai and September of the same year over stable terrain (disparity should be 0, but is not due to the different illumination):

<input_imgs.png>

I have upsampled both input images to different spatial resolutions (using nearest neighbor interpolation): 10 m (original), 8 m, 6 m, and 4 m.

Here are the obtained displacement fields (band 1 (first row) & band 2 (second row) of the filtered disparity file *-F.tif) converted into units of meter for comparison. Correlation kernels are kept approx. constant in units of meter. The magnitude of the estimated displacement over the shaded area is decreasing, even though kernel should contain exactly the same features:

<example1.png>

My best guess would be an improved subpixel matching due to the substantially higher number of pixels in a correlation kernel at higher resolution?

Thanks in advance!
Ariane

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/0be5fcbe-4376-4d61-b40e-893764a28668n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/CADDkNmOcWT9kgHHfgCr-pw2JfF6ummZQ21r4JD08_cnDoNRThA%40mail.gmail.com.

[Oleg Alexandrov]

> I noticed that the magnitude and area affected differs with the resolution of the input images even though I am keeping the correlation and sub pixel kernels constant in terms of size (in meter, not pixel). 

When stereo correlation cannot precisely match the two images, it just tries to get away with nearest most plausible match. This really speaks more of the limitation of correlation with not great data. If you try a different stereo algorithm and different subpixel mode, you will likely get different kind of quirks, but maybe less pronounced.

Also, nearest neighbor interpolation is likely introducing its own artifacts. That is quite a rough method.

(Also note David's response.)

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/3e65231e-f1b3-493d-a6f7-a9a56fe27714n%40googlegroups.com.

[Ariane]

Thank you for the feedback and the study - I will have a look!

What are the benefits of using a smaller subpixel kernel together with a larger correlation kernel? If I remember correctly, the default is the other way around.

To answer your questions:

Which Sentinel-2 products are you using?  Level-2A? 

-> L1C

Identical tile and relative orbit? 

-> yes

Using 16-bit input images?

-> 32bit-float

The idea behind nearest neighbor was to not introduce any new pixel values. I just reran with cubic interpolation, but the differences are minor and the effect persists:

I have also tested mgm, however, here it is difficult to keep the correlation kernels fixed in actual (meter) size as 9 pixels is the maximum kernel size, so the results are not as comparable: