image_mosaic error while using on Corona historical image

[Geospatial]

Hi,

I am using Corona images acquired during 1967 (DS1038-2102DA series) where each image is stored in four parts. I am using image_mosaic to join them and everything is fine with most such images.

However, for three images, I am getting the following error message

VW Error: DiskImageResourceGDAL: "" is an unsupported file extension.

While running image_mosaic, I am also getting a warning message

Warning: Cached a new object (126436 B) and now we are larger than the requested maximum cache size (805 MB). Current size = 805 MB.
I checked the tif files and they are fine.

Not sure if this is related to cache size or not, if so, how to increase cache size?

Cheers,

Sanjeev

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

Greetings,

I am using Corona images acquired during 1967 (DS1038-2102DA series) where each image is stored in four parts. I am using image_mosaic to join them and everything is fine with most such images.

However, for three images, I am getting the following error message

VW Error: DiskImageResourceGDAL: "" is an unsupported file extension.

While running image_mosaic, I am also getting a warning message

This is usually a sign of trying to open an invalid file. I am not sure what is going on. Maybe you can examine the successful set you have and the unsuccessful one and see if there are any differences among them.  Or you can try to first use the tool with just two images, then with three. Something makes it confused and I don't know what it is.

Warning: Cached a new object (126436 B) and now we are larger than the requested maximum cache size (805 MB). Current size = 805 MB.

This can help with the cache:

https://stereopipeline.readthedocs.io/en/latest/installation.html?highlight=cache#settings-optimization

You can also examine your input images with gdalinfo -stats. It should print the block size. ASP complains if the images are stored in blocks which are as wide as the image but only a few pixels tall. They don't fit well in memory when they are large. 

Such images can be written into more manageable storage before being fed to ASP using the command:

gdal_translate -co compress=lzw -co TILED=yes -co INTERLEAVE=BAND -co BLOCKXSIZE=256 -co BLOCKYSIZE=256 input.tif output.tif

This will not change any actual pixel values.

[Geospatial]

Hi Oleg,

Many thanks for a quick reply and apologies for a long post. I tried many different things but still getting the error (VW Error: DiskImageResourceGDAL: "" is an unsupported file extension.).

I tried the following:

1.       Restarted the computer and redownloaded the software.

2.       Increased the cache to 4GB that improved processing speed and I did not get the message about low cache but did not affect the main error.

3.       I tried combinations of two sub-images but get the same error.

4.       I tried the following command for all sub-images and re-run image_mosaic with them but no luck

/home/sanjeev/Ames02/bin/gdal_translate -co compress=lzw -co TILED=yes -co INTERLEAVE=BAND -co BLOCKXSIZE=256 -co BLOCKYSIZE=256 /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_a.tif /home/sanjeev/DS1038-2102DA191_aa.tif
Input file size is 31609, 11057
0...10...20...30...40...50...60...70...80...90...100 - done.

5.       Following is the comparison of image sets

19670121/DS1038-2102DF191_a.tif (getting error message)

Driver: GTiff/GeoTIFF

Files: /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/19670121/DS1038-2102DF191_a.tif

Size is 35513, 10892

Coordinate System is `'

Metadata:

   TIFFTAG_DOCUMENTNAME={56D216C3-F91E-4A1D-88E7-F107B91FA9EF}

  TIFFTAG_MAXSAMPLEVALUE=255

  TIFFTAG_MINSAMPLEVALUE=0

  TIFFTAG_RESOLUTIONUNIT=3 (pixels/cm)

  TIFFTAG_XRESOLUTION=1432.1355

  TIFFTAG_YRESOLUTION=1432.072

Image Structure Metadata:

   INTERLEAVE=BAND

Corner Coordinates:

Upper Left  (    0.0,    0.0)

Lower Left  (    0.0,10892.0)

Upper Right (35513.0,    0.0)

Lower Right (35513.0,10892.0)

Center      (17756.5, 5446.0)

Band 1 Block=35513x1 Type=Byte, ColorInterp=Gray

 

DS1038-2102DA191_cc.tif (created with gdal_translate and getting error message with this one).

Driver: GTiff/GeoTIFF

Files: /home/sanjeev/DS1038-2102DA191_cc.tif

Size is 31609, 11057

Coordinate System is `'

Metadata:

  TIFFTAG_DATETIME=2013:03:06 11:47:04

  TIFFTAG_RESOLUTIONUNIT=3 (pixels/cm)

  TIFFTAG_SOFTWARE=Adobe Photoshop CS4 Windows

  TIFFTAG_XRESOLUTION=1436.749

  TIFFTAG_YRESOLUTION=1436.749

Image Structure Metadata:

  COMPRESSION=LZW

  INTERLEAVE=BAND

Corner Coordinates:

Upper Left  (    0.0,    0.0)

Lower Left  (    0.0,11057.0)

Upper Right (31609.0,    0.0)

Lower Right (31609.0,11057.0)

Center      (15804.5, 5528.5)

Band 1 Block=256x256 Type=Byte, ColorInterp=Gray

DS1038-2102DF189_a.tif (Image_mosaic worked)

Driver: GTiff/GeoTIFF

Files: /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/19670121/DS1038-2102DF189_a.tif

Size is 31425, 10992

Coordinate System is `'

Metadata:

  TIFFTAG_MAXSAMPLEVALUE=255

  TIFFTAG_MINSAMPLEVALUE=0

  TIFFTAG_RESOLUTIONUNIT=3 (pixels/cm)

  TIFFTAG_XRESOLUTION=1428.3368

  TIFFTAG_YRESOLUTION=1428.3591

Image Structure Metadata:

  INTERLEAVE=BAND

Corner Coordinates:

Upper Left  (    0.0,    0.0)

Lower Left  (    0.0,10992.0)

Upper Right (31425.0,    0.0)

Lower Right (31425.0,10992.0)

Center      (15712.5, 5496.0)

Band 1 Block=31425x1 Type=Byte, ColorInterp=Gray

6.       Following is the details of the error

:~$ /home/sanjeev/SteroPipeline/bin/image_mosaic  /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_d.tif /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download//DS1038-2102DA191_c.tif /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_b.tif /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_a.tif -o /home/sanjeev/DS1038-2102DA191f.tif --ot byte --overlap-width 7000 --blend-radius 2000 –rotate
--> Setting number of processing threads to: 16
Matching interest points between: /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_d.tif and /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download//DS1038-2102DA191_c.tif
   Looking for IP in left image...
   Using 67 interest points per tile (1024^2 px).
   Detecting IP
   Building descriptors
   Found interest points: 4590
   Looking for IP in right image...
   Using 67 interest points per tile (1024^2 px).
   Detecting IP
   Building descriptors
   Found interest points: 4590
--> Matching interest points using homography.
--> Adjusted uniqueness threshold: 0.8
           Matching: [***************************************************.] 99%
   Matched points: 660
relative_transform: Matrix3x3((0.999993,0.00384818,28543.6)(-0.00384818,0.999993,51.4513)(0,0,1))
absolute_transform: Matrix3x3((0.999993,0.00384818,28543.6)(-0.00384818,0.999993,51.4513)(0,0,1))
Matching interest points between: /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download//DS1038-2102DA191_c.tif and /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_b.tif
   Looking for IP in left image...
   Using 67 interest points per tile (1024^2 px).
   Detecting IP
   Building descriptors
   Found interest points: 4590
   Looking for IP in right image...
   Using 67 interest points per tile (1024^2 px).
   Detecting IP
   Building descriptors
   Found interest points: 4399
--> Matching interest points using homography.
--> Adjusted uniqueness threshold: 0.8
           Matching: [***************************************************.] 99%
   Matched points: 750
relative_transform: Matrix3x3((1,0.000163293,28907.5)(-0.000163293,1,184.426)(0,0,1))
absolute_transform: Matrix3x3((0.999992,0.00401147,57451.6)(-0.00401147,0.999992,124.634)(0,0,1))
Matching interest points between: /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_b.tif and /home/sanjeev/gisdata/Sundarbans/Declassified1996/v2/USGS_Order_Download/DS1038-2102DA191_a.tif
   Looking for IP in left image...
   Using 67 interest points per tile (1024^2 px).
   Detecting IP
   Building descriptors
   Found interest points: 4399
   Looking for IP in right image...
   Using 67 interest points per tile (1024^2 px).
   Detecting IP
   Building descriptors
   Found interest points: 4590
--> Matching interest points using homography.
--> Adjusted uniqueness threshold: 0.8
           Matching: [***************************************************.] 99%
   Matched points: 793
relative_transform: Matrix3x3((0.999996,0.00282516,27565.7)(-0.00282516,0.999996,143.429)(0,0,1))
absolute_transform: Matrix3x3((0.999977,0.00683659,85017.6)(-0.00683659,0.999977,157.484)(0,0,1))

VW Error: DiskImageResourceGDAL: "" is an unsupported file extension.

 

With regards,

Sanjeev

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

It looks that your "d" and "c" images are fine, but you hit errors later. Can you try to do a mosaic of just these two images? 

Your images are also quite big. One can try to see if things work with smaller versions, such as resized with gdal_translate -r average -co COMPRESS=LZW -outsize 25% 25%.

There are also many absolute paths. Those should be fine. I am a bit concerned though if there is perhaps some stray character somewhere which results in confusing the tool. You could try to type again the command by hand while using two input images and one output image in the same directory and making sure there are no spaces anywhere. 

If no luck at all, not even with one single pair of small images, you can share those images privately with me and also the exact command you used. 

Oh, and you can examine any two images you use and see if they overlap. Likely they do, as you get matches.

Anyhow, my best guess is that a user error somewhere but I cannot tell if I don't have the data.

Oleg

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Geospatial <srivasta...@gmail.com>
Sent: Wednesday, November 3, 2021 12:21 AM
To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>
Subject: Re: [EXTERNAL] image_mosaic error while using on Corona historical image

 

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/ea940709-06a7-4927-8cfa-17f6b23c8dc7n%40googlegroups.com.

[Geospatial]

Hi Oleg,

Thanks for additional suggestions. When I remove '--rotate' then the script works. I am working with two sets of images one set needing rotation while another do not. The errors were for 2 out of 6 files do not needing rotation and 1/6 file needing rotation. So all sorted out and now I need to fix just one mosaiced image which will be needing rotation.

With regards,

Sanjeev

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

So what did you do to make the problem go away with the other data? Did you run things differently or there was a user mistake? 

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Geospatial <srivasta...@gmail.com>

Sent: Friday, November 5, 2021 7:12 AM

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/d801c2fa-093e-4166-87eb-30238b327082n%40googlegroups.com.

[Geospatial]

Hi Oleg,

I did not do anything differently apart from remove --rotate. I am working on two series (DA and DF), the images acquired with forward camera don' need rotation while the one with backward (aft) camera need to be rotated. As some of the DF images were mosaiced, I noted unnecessary rotations in them so I tried image_mosaic without '--rotate' and it worked for all 6 DF images (I had trouble with three images). After this, I tried image-mosaic on one of the DA image without '--rotate' and it worked although I get rotation in that image. I tried the same command again after including '--rotate' but getting the error.

In summary, this issue is sorted out but don't exactly know how.

With regards,
Sanjeev

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

I see. Thanks.

Our developer Scott wrote this tool and now he is busy with other projects. 

In principle, these images can all be rotated outside this tool, then lined up horizontally, then mosaicked that way, then the image itself rotated again. That to avoid the --rotate error.

Or maybe the the tool itself can do the rotation for just one image, so it can be invoked several times as above.

Likely this is a bug, but for the moment we do not have allocated time for ASP. Hopefully one day.

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Geospatial <srivasta...@gmail.com>

Sent: Saturday, November 6, 2021 5:15 AM

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/bea69841-1f06-4c43-8a87-7ea300ae944cn%40googlegroups.com.