Subpixel-mode 9 (SGM Poly4) and kernel size

[Gianluca Chiarolanza]

Dear ASP developers,

I am investigating the effects of different configurations of parallel_stereo on DEM quality, focusing particularly on the correlation kernel (ck) and subpixel-refinement kernel (spk). I have been processing stereo DEMs of Ganymede using JunoCam data. Following the example provided in the ASP documentation, I have also been testing MGM with  subpixel-mode 9 (SGM Poly4).

I noticed an interesting difference between subpixel modes 2 and 9 that I would like to understand better.

With subpixel-mode 9, fixing ck and varying spk over a range of values (3–25 px) produced identical DEMs in my tests. However, fixing the other parameters and increasing ck from 3 to 9 px produced a visibly smoother DEM.

For example, the attached figure compares two MGM DEMs generated from the same JunoCam stereopair and with the same subpixel mode (9) but different ck (3 and 9 px, respectively). The DEMs are gridded at 3x the image GSD.

The difference is visible both in the shaded relief (top panels) and in the profile across a small impact crater (red transect). The ck=9 solution appears smoother and the crater morphology is flatter.

This led me to wonder about how the subpixel interpolation is implemented in modes 7–12.

The ASP documentation states: "The subpixel-mode If not set, or set to values 7-12, SGM will perform subpixel interpolation during the stereo correlation step and will not do additional work in the stereo refinement step".

My questions are therefore:

1) Is --subpixel-kernel ignored when using subpixel modes 7–12?
This would be consistent with my observation that changing --subpixel-kernel has no effect when using mode 9.
2) What determines the effective subpixel-interpolation kernel width used by SGM Poly4 (subpixel mode 9)? Is it a fixed internal value, or does it depend on the correlation kernel size? For example, does increasing ck also increase the spk used by the SGM subpixel interpolation?
3) More generally, would it be possible to have some additional details about how the subpixel interpolation in modes 7–12 is implemented, particularly for SGM Poly4 (mode 9)? Understanding this could be useful for my research.

As a control experiment, I also tested subpixel-mode 2, where --subpixel-kernel is explicitly configurable, using both ASP BM and ASP MGM. In all cases, changing ck while keeping spk fixed produced much smaller (if any) differences, whereas changing spk at fixed ck produced a clear change in smoothing.

I would be very grateful for any clarification on these points.

Best Regards,

Gianluca

[Oleg Alexandrov]

Gianluca,

> With subpixel-mode 9, fixing ck and varying spk over a range of values (3-25 px) produced identical DEMs in my tests. However, fixing the other parameters and increasing ck from 3 to 9 px produced a visibly smoother DEM.
> 1) Is --subpixel-kernel ignored when using subpixel modes 7-12?

Yes, it is ignored. All this is expected and I plan to document this better. In subpixel-modes 7-12 the sub-pixel step is done inside the SGM/MGM correlator, and the separate refinement stage is skipped. The --subpixel-kernel (spk) option only feeds the refinement modes (1-6), so it has no effect in mode 9. That is why varying it gave identical DEMs. The smoothing you see comes entirely from the correlation kernel ck. A larger ck averages the matching cost over a wider window, which smooths the disparity and flattens the crater profile.

> 2) What determines the effective subpixel-interpolation kernel width used by SGM Poly4 (subpixel mode 9)? Is it a fixed internal value, or does it depend on the correlation kernel size?

There is no sub-pixel kernel width in these modes. The offset is computed from a fixed 3-point stencil. For the winning integer disparity d, SGM takes the aggregated cost at C(d-1), C(d), C(d+1) along each axis and fits a curve to the normalized cost differences, giving an offset between -0.5 and +0.5 pixels. This is a fixed internal method and does not depend on ck. So ck does not change the sub-pixel width. It changes the matching window, and that is what smooths the result.

In our eperience these subpixel methods are smoother but likely not as accurate as the subpixel modes that use an actual kernel.

> 3) More generally, would it be possible to have some additional details about how the subpixel interpolation in modes 7-12 is implemented, particularly for SGM Poly4 (mode 9)?

Briefly: SGM/MGM aggregates matching costs along several paths into an accumulated cost volume. The integer disparity is the argmin of that volume. The sub-pixel offset is then computed from the three aggregated cost values around the minimum, per axis (mode 11, parabola, uses the full 3x3 neighborhood). The mode mapping is 7=none, 8=linear, 9=poly4, 10=cosine, 11=parabola, 12=lc_blend. Poly4 fits a fourth-order curve to the ratio of the left and right cost differences.

The source is the clearest reference:

VW, the 3-point stencil and the curve fits (poly4Fit is mode 9), src/vw/Stereo/SGM.cc:
https://www.google.com/url?q=https://github.com/visionworkbench/visionworkbench/blob/97e3743f9ab8f9d7b7fa995aedf128115bb7c2ba/src/vw/Stereo/SGM.cc%23L1445&source=gmail&ust=1786379016734000&sa=E
(poly4Fit is at line 1417; the per-pixel loop create_disparity_view_subpixel is just below.)

ASP, the mode 7-12 to SGM mapping, src/asp/Tools/stereo_corr.cc:
https://www.google.com/url?q=https://github.com/NeoGeographyToolkit/StereoPipeline/blob/89801868104b3447467477e259045b047269b959/src/asp/Tools/stereo_corr.cc%23L67&source=gmail&ust=1786379016734000&sa=E

> As a control experiment, I also tested subpixel-mode 2 ... changing spk at fixed ck produced a clear change in smoothing.

Right, and that is the expected behavior. Modes 1-6 run in the refinement stage where --subpixel-kernel is active, so there spk controls the smoothing. If you want a tunable sub-pixel smoothing knob on top of MGM, use a refinement mode (2, or 3 for Bayes EM) with a small ck, rather than mode 9.

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/aee6dfe9-95ff-4eb8-bd01-f30a76268819n%40googlegroups.com.

[Oleg Alexandrov]

To add to previous, stereo with mapprojected images (so, as a 2nd pass, so a DEM mosaic is produced, it is blurred a little, the images are mapprojected onto it at native resolution, and stereo is redone) tends to minimize quirks and artifacts of some subpixel methods.