mapproject getting stuck with no output

[Leith Bade]

Hi,

I have been following the documentation page on processing Chandrayaan-2 TMC camera data, but I ran into issue with mapproject that I can't figure out.

If I use mapproject with my TMC imagery and using the ISRO's matching derived DTM TIFF file for the stereo image pair I have picked then mapproject runs and finishes reasonably quickly with screeds on console output.

However I then tried to use the SLDEM2015 JP2 file for the same region, the process seems to hang with no console output at all. I can see the CPU maxes out over all cores for several seconds followed by a few seconds of idle. This continues at least for over an hour so I think it's getting stuck somewhere early in the program.

Does anyone know if I am doing something wrong or if this is a bug?

I am still learning a lot about working with moon imagery in ISIS and AMS so I apologise if I have overlooked something.

In case it helps the files I was using are:
ch2_tmc_nra_20250207T1102039417_d_img_d18.img ch2_tmc_nrf_20250207T1102039385_d_img_d18.img ch2_tmc_nrn_20250207T1102039385_d_img_d18.img
ch2_tmc_ndn_20250207T1102039385_d_dtm_d18.tif

The command that works:
 mapproject --processes 1 --tr 5 ~/data/pds4/isda/ch2/cho/nop/tmc/data/derived/20250207/ch2_tmc_ndn_20250207T1102039385_d_dtm_d18.tif ch2_tmc_nra_20250207T1102039417_d_img_d18.cub ba/run-ch2_tmc_nra_20250207T1102039417_d_img_d18.adjusted_state.json aft-map-proj.tif

The command that hangs:

mapproject --processes 1 --tr 5 ~/data/moon/sldem/SLDEM2015_512_60S_60N_000_360.JP2 ch2_tmc_nra_20250207T1102039417_d_img_d18.cub ba/run-ch2_tmc
_nra_20250207T1102039417_d_img_d18.adjusted_state.json aft-map-proj.tif

The version:
NASA Ames Stereo Pipeline 3.7.0
  Build ID: 3d1953916
  Build date: 2026-06-09

Built against:
  NASA Vision Workbench 3.7.0
    Build ID: db76af6f
  USGS ISIS 10.0.0
  Boost C++ Libraries 1.88.0
  GDAL 3.12.2 | 20260203

Running on Debian 13 with x86-64 CPU

[Oleg Alexandrov]

It looks that mapproject does not like decoding jp2 on the fly. We assume in ASP that a DEM is a nicely tiled tif. I will have a warning in our program. Not sure what else we can do internally. 

The suggested solution is to convert the jp2 to tiled tif, as then things seem to work, as you see for your other example. The command is something like:

gdal_translate input.jp2 output.tif \
    -co TILED=YES -co BIGTIFF=IF_SAFER --config GDAL_NUM_THREADS ALL_CPUS

It is suggested to inspect the resulting DEM stats and maybe hillshade it. JP2 can be lossy. I hope those folks built it correctly.

Chandrayaan-2 TMC is rather new for us. Also new for ISRO, I guess, so I think may have some rough edges, with jitter, registration, etc. The closest comparable camera is Kaguya, at 10 m /pix vs 5 m/pix for this one, and that has nearly global coverage, if needed as a reference and for DEM bulding. So let us know if anything unusual turns up.

Oleg

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/7904c4d0-70b5-4abe-9710-c8334568d886n%40googlegroups.com.

[Leith Bade]

Hi Oleg,

I appreciate Chandrayaan-2 is still rather new but so far I am impressed with the data. I mapped the coverage of all their files and they are not far off seamless global coverage so I hope the satellite continues to operate for a few more years to fill in the remaining gaps.

Even ISRO's precomputed DTM files are reasonable, horizontal alignment vs a NAC ROI mosaic was matching to with 4-5 degree decimal points (I didn't grab the absolute value in metres). For vertical the geodiff of their DTM file vs SLDEM after accounting for the units had a near constant ~50m offset.

Certainly no scientific comparison but it seems the main limitation is the accuracy of the SPICE as my results before pc_align do show a large horizontal offset. It will be interesting after I get time to experiment with pc_align and the budle_adjustment alignment options to see if I can do better than the ISRO DTM file.

Thanks for the hint about mapproject not liking JP2 which is rather unfortunate. I discovered geodiff seems to handle the JP2 fine, but then that revealed I needed to check the height units for SLDEM which are integers with 0.5m scale, which geodiff didn't account for.

Is there a way to tell the ASP tools what scale/offset to apply to a DEM file to get back to floating point of metres relative to the ellipsoid?

I can convert the JP2 to a floating point TIFF (along with covering the height units) but the storage space goes from 5GB to 45GB which is unfortunate but I guess will have to do for now.

[Oleg Alexandrov]

Leith,

I appreciate Chandrayaan-2 is still rather new but so far I am impressed with the data. I mapped the coverage of all their files and they are not far off seamless global coverage so I hope the satellite continues to operate for a few more years to fill in the remaining gaps.

That would be very nice.

Is there a way to tell the ASP tools what scale/offset to apply to a DEM file to get back to floating point of metres relative to the ellipsoid?

There is no way because I don't think there is a standard for what people put in such products and how they set the values. Often one should also watch for whether the data is relative to some geoid or ellipsoid (the former more for Mars though). So, one has to read the product documentation and adapt on a case-by-case basis.

 

I can convert the JP2 to a floating point TIFF (along with covering the height units) but the storage space goes from 5GB to 45GB which is unfortunate but I guess will have to do for now.

That's a lot. Maybe you can crop with gdal_translate to your desired extent as you convert this. 

The root problem with jp2 is that it seems optimized for efficiency and not quick random access, which is what our tools expect.

 

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/601d16d0-ecb1-4b4b-9818-afb583f2f1a4n%40googlegroups.com.

[Leith Bade]

Thanks for the idea about cropping the converted DEM that actually works reasonably well since I only need to decompress the region I am working on.

I have a new question around the parallel_stereo command with the Chandrayaan-2 file I am testing.

For some reason with the default algorithm (asp_bm) and the default subpixel mode (1) I get a better result when looking at the GoodPixelMap image then I do with asp_mgm and both subpixel mode 9 and 3.

The documentation suggests that asp_mgm should be better, and it certainly takes a lot longer to process then asp_bm. However near the large crater in my image I get large areas of red instead of grey, though in the flat areas it is pretty much entirely grey pixels.

The asp_bm image there is more red pixels in the flat areas, but the grey pixels are well distributed and cover more of the crater area.

The input was using the bundle adjusted cameras, and mapprojected images (though currently using the ISRO DEM file for this image set, I will try it with SLDEM mapprojected images tomorrow).

Is there something I have missed or done wrong?

I have attached screenshot showing the comparison, bm on left and mgm on right.

[Oleg Alexandrov]

What you can do is overlay your left and right mapproj images onto the DEM mapproj onto and see if there is some notable shift. Any such shift and warping makes it hard for the algorithms and likely you will need to ensure to do some alignment first. 

Second, likely itis good to look at hillshaded DEMs you make in addition to good pixel map. During correlation watch for the corr search range being printed. Anything wider than 100 means correlation is having a hard time because of mismatch in left and right images.

Given that this is near a crater, likely steep misrgistered terrain is problematic.

So a sensible approach could be stereo without mapproj images (but with bundle adjustment), then align that one to your ref. with pc align, inspect results, and apply the transform to cameras as in the pc align document. When when you sure all is registered, do stereo with mapproj images onto the ref (which we suggest should be blurred a bit and inspected).

Regretfully, misregistration is a source of a lot of problems and for this vendor initial misregistration is quite high. 

An alternative is to map-project onto the stereo dem obtained without map-projection, so not with external DEM, at least not right away. (Use bundle adjusted cameas at alls tages.) We have a two pass approach in https://stereopipeline.readthedocs.io/en/latest/next_steps.html#stereo-with-mapprojected-images

I hope registration is the problem. Or else let me know.

--

You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/ff42d61f-21be-47ce-bed8-13dfecedb174n%40googlegroups.com.