KeyError: 'ALOOKS'

[Lucy Goeddel Hackett]

Hello,

I am running MintPy on a stack of ALOS-1 acquisitions processed by stripmapStack in ISCE2. I first (successfully) run prep_isce as follows:

prep_isce.py -f "/volumes/sarmem/ALOS/ascending/Igrams/*/filt_*.unw" -m /Volumes/SARmem/ALOS/ascending/Igrams/20070211_20070814/referenceShelve/data.db -b /volumes/sarmem/ALOS/ascending/baselines/ -g /volumes/sarmem/ALOS/ascending/geom_reference/

though notice that the above takes a metafile from an individual interferogram, since I did not have a referenceShelve/data.dat directory for some reason (??).

After that, I ran:

smallbaselineApp.py alos_ascending.cfg

where alos_ascending.cfg contains the following text:

# vim: set filetype=cfg:
##------------------------------- ISCE/stripmapStack OPTIONS ------------------##
isce.processor          = stripmap  #[stripmapStack, topsStack]
isce.demSNWE            = 19 20 -100 -98  #[S, N, W, E] in degree
isce.demFile            = /volumes/sarmem/alos/ascending/dem/demLat_N19_N20_Lon_W100_W098.dem.wgs84
isce.azimuthLooks       = 10 # default in stripMapStack
isce.rangeLooks         = 10 # they have in template 8 - 10 is default
isce.maxTempBaseline    = 400 # ? 10000 is default
isce.maxPerpBaseline    = 200 # ? 5000 is default
isce.unwrapMethod       = snaphu
isce.filtStrength       = 0.8 # 0.8 is default - they have .5 in template
isce.applyWaterMask     = yes


##------------------------------- mintpy OPTIONS -------------------------------##
## load_data.py -H to check more details and example inputs.
mintpy.load.processor        = isce
##---------for ISCE only:
mintpy.load.metaFile         = /Volumes/SARmem/ALOS/ascending/Igrams/20070211_20070814/referenceShelve/data.rsc
mintpy.load.baselineDir      = /volumes/sarmem/alos/ascending/baselines
##---------interferogram datasets:
mintpy.load.unwFile          = /volumes/sarmem/alos/ascending/Igrams/*/filt_*.unw
mintpy.load.corFile          = /volumes/sarmem/alos/ascending/Igrams/*/filt_*.cor
mintpy.load.connCompFile     = /volumes/sarmem/alos/ascending/Igrams/*/filt_*.unw.conncomp
##---------geometry datasets:
mintpy.load.demFile          = /volumes/sarmem/alos/ascending/geom_reference/hgt.rdr
mintpy.load.lookupYFile      = /volumes/sarmem/alos/ascending/geom_reference/lat.rdr
mintpy.load.lookupXFile      = /volumes/sarmem/alos/ascending/geom_reference/lon.rdr
mintpy.load.incAngleFile     = /volumes/sarmem/alos/ascending/geom_reference/los.rdr
mintpy.load.azAngleFile      = /volumes/sarmem/alos/ascending/geom_reference/los.rdr
mintpy.load.shadowMaskFile   = /volumes/sarmem/alos/ascending/geom_reference/shadowMask.rdr
mintpy.load.waterMaskFile    = /volumes/sarmem/alos/ascending/geom_reference/waterMask.rdr
mintpy.load.bperpFile        = None

mintpy.reference.lalo     = 19.39, -99.21 # FIXME
mintpy.network.coherenceBased   = yes
mintpy.network.keepMinSpanTree  = no
mintpy.network.minCoherence     = 0.6
mintpy.troposphericDelay.weatherModel = ERA5
mintpy.deramp                 = linear

but I ran into the following error:

Traceback (most recent call last):
  File "/Users/hackettl/miniforge3/envs/iscesent/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/cli/smallbaselineApp.py", line 208, in main
    run_smallbaselineApp(inps)
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/smallbaselineApp.py", line 1117, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/smallbaselineApp.py", line 889, in run
    self.run_network_inversion(sname)
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/smallbaselineApp.py", line 384, in run_network_inversion
    mintpy.cli.ifgram_inversion.main(iargs)
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/cli/ifgram_inversion.py", line 257, in main
    run_ifgram_inversion(inps)
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/ifgram_inversion.py", line 1098, in run_ifgram_inversion
    ts, ts_cov, inv_quality, num_inv_obs = run_ifgram_inversion_patch(**data_kwargs)[:-1]
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/ifgram_inversion.py", line 654, in run_ifgram_inversion_patch
    weight_sqrt = calc_weight_sqrt(stack_obj, box,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hackettl/miniforge3/envs/iscesent/lib/python3.11/site-packages/mintpy/ifgram_inversion.py", line 520, in calc_weight_sqrt
    L = int(stack_obj.metadata['ALOOKS']) * int(stack_obj.metadata['RLOOKS'])
            ~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'ALOOKS'

In trying to diagnose this I have verified that the version of ISCE2 being used by mintpy is the same as the version used to create the input data. (ISCE version = 2.6.3).

Is the problem not having a stack-level referenceShelve/data.dat? If so, how can I generate this? Or if not, how can I avoid this error?

Thank you!

Lucy Hackett

[Lucy Goeddel Hackett]

I am working on MacOS and the Mintpy version is 1.5.3. Both this and ISCE were installed via mamba.

[Eric Fielding]

The fact that the `prep_isce.py` did not run correctly means that it probably did not put all the required information into the `data.rsc` file. It is normal to use the metadata from a single interferogram. Do you not have a `/volumes/sarmem/ALOS/ascending/Igrams/20070211_20070814/referenceShelve/data.dat` file? Note that the macOS sometimes ignores the case of the file paths and sometimes it does not, so you might be having trouble because you used `/Volumes/SARmem/ALOS/ascending/Igrams/20070211_20070814/referenceShelve/` instead of the lowercase path.

[Lucy Goeddel Hackett]

I do not have any data.dat files;  I see that they are in the example data structures, but I only have data.db files (See screenshot)- I'm not sure why ISCE (I'm guessing?) didn't generate them?

Thanks for the point on the file paths; I replaced all the file paths with correctly-cased ones but the same issue arises.

[Eric Fielding]

Oh, then the meta data must be in the `data.db` file instead of `data.dat`. Did you try to use that for the `prep_isce` input? 

[Lucy Goeddel Hackett]

It was already being used in the prep_isce.py call (see above). I just tried replacing:

mintpy.load.metaFile         = /Volumes/SARmem/ALOS/ascending/Igrams/20070211_20070814/referenceShelve/data.rsc

with:

mintpy.load.metaFile         = /Volumes/SARmem/ALOS/ascending/Igrams/20070211_20070814/referenceShelve/data.db

in the config file but it didn't make a difference...

[Eric Fielding]

Did you run the `prep_isce` with the lowercase path to the `data.db` file? MintPy does not know how to read the `data.db` or `data.dat` files. It depends on the `prep_isce` to translate it to a `data.rsc` file that it can read.

[Lucy Goeddel Hackett]

That's it, thank you! Had to delete the mintpy directory and start over, I think some things were not being re-run that needed to be.
Thank you so much Eric!!

[Sayyed Mohammad Javad Mirzadeh]

Hi Lucy,

I think you set the wrong directory for "mintpy.load.metaFile" option. Could you please let us have a screenshot from your main processing directory?

Recently, I finished processing the ALOS-1 stack with the ISCE and MintPy and may help you fix it. Could you please let me know where you got or downloaded the ALOS-1 dataset?

Sincerely,

Sayyed

[Lucy Goeddel Hackett]

Sayyed,

Thanks for the offer; it was the capitalization stuff that was off so it's fixed now! I downloaded the ALOS-1 data from ASF Vertex. If I run into more problems though I might ping you since you've recently done this; I'm currently waiting on the tropospheric correction to finish downloading from copernicus...

[Sayyed Mohammad Javad Mirzadeh]

Lucy,

Good to hear that it was fixed. That step will take a long time since the Copernicus server is going to be upgraded and the speed of downloading the ERA5 reanalysis data will be very slow. Another option can be the GACOS outputs for the tropospheric correction, which is supported by the MintPy.

BTW, I mentioned the paths to the variables in the MintPy template file as follows for the ALOS-1:

##---------for ISCE only:
mintpy.load.metaFile                 = ../merged/SLC/20080717/referenceShelve/data.dat
mintpy.load.baselineDir       = ../baselines
##---------interferogram datasets:
mintpy.load.unwFile                   = ../Igrams/*/filt_*.unw
mintpy.load.corFile                   = ../Igrams/*/filt_*.cor
mintpy.load.connCompFile     = ../Igrams/*/filt_*.unw.conncomp
#mintpy.load.intFile                   = ../Igrams/*/filt_*.int
mintpy.load.ionoFile                 = ../Ionosphere/*/iono.bil.unwCor.filt
##---------geometry datasets:
mintpy.load.demFile                   = ../geom_reference/hgt.rdr
mintpy.load.lookupYFile       = ../geom_reference/lat.rdr
mintpy.load.lookupXFile       = ../geom_reference/lon.rdr
mintpy.load.incAngleFile     = ../geom_reference/los.rdr
mintpy.load.azAngleFile       = ../geom_reference/los.rdr
mintpy.load.shadowMaskFile   = ../geom_reference/shadowMask.rdr

[Lucy Goeddel Hackett]

switching to GACOS was a lifesaver, I was getting stuck in an endless connection error stream with Copernicus. Thanks for the tip!