Prepare attributes file from Gamma products

[Sven]

Good morning,

first of all thanks for letting me join the PySAR group.

Given that I'm absolutely new to PySAR,  I'm a GAMMA user so my first necessity is to tranform the output of GAMMA processing into the ROI_PAC style readable by PySAR.

I made myself a first change of the names of some files into something similar to ROI_PAC, as explained in the "prep_gamma.py" script, e.g. the parameter file of the master multilook intensity (*.mli.par) into date1.amp.par, etc.

In the attached file there are the results of three different runs, to get the attributes file for Gamma product for PySAR, the processing for getting the DEM file and the processing for a single interferogram (5 files): in two of the three runs nothing happens.

As far as I can understand I should have set properly the PATH and PYTHONPATH into .bashrc: I'm working on Ubuntu 16.04,  Python 3.5.2, PySAR 1.1.0.

Thanks in advance if you can give me any help.

Sven.

[Yunjun Zhang]

Hi Sven,

The current released version is still in python2 (used to numbered as version 1.1.0), please try the development version in python3, you could use git clone to download it.

git clone https://github.com/yunjunz/PySAR.git

Note that you might need to modify your $PYSAR_HOME variable to the new path. Please try the Kuju volcano example to check if the installation is fine.

For the file naming, please make sure you have the "20180609_5rlks.amp.par" file also, which should be slightly different from "20180609.amp.par". 

After the two changes above, please try the following to see if it works. Note that the input of prep_gamma.py is data files, so you don't need to input the .par files as your 2nd and 3rd run.

prep_gamma.py diff_20170929-20180609_5rlks.unw

If this works, you should just need to add all the data files' path pattern (run "load_data.py -H" to check an example) into your template file, such as the one in Kuju example (https://github.com/yunjunz/PySAR/blob/master/examples/KujuAlosAT422F650.txt).

Yunjun

[Sven]

Good morning Yunjun,

I partially succeeded following your indications.

1) new release:

downloaded, no need to change anything into .bashrc, as I kept the same naming of my previous folders for the new release.

I gave a try to the Kuju example but I din't finish the processing probably for a missing package (!?!) which is not regularly imported, as you can see from the attached file.

I've therefore installed "python-statsmodels" and "python-scikits.statsmodels", but the same error still remains.

2) files naming

I succeeded with the naming indications you gave me for the files

I changed "date1.amp.par" and "date2.amp.par" into "date1_5rlks.amp.par" and "date2_5rlks.amp.par", then:

$ prep_gamma.py diff_date1-date2_5rlks.unw

here I got 3 new files:
- date1_5rlks.amp.corner
- date1_5rlks.amp.corner_full
- diff_date1-date2_5rlks.unw.rsc

Similarly:

$ prep_gamma.py filt_date1-date2_5rlks.cor

gave me the resource file "filt_date1-date2_5rlks.cor.rsc"

I was also able to get the *.rsc files for sim_date1-date2.hgt_sim, sim_date1-date2.utm.dem and sim_date1-date2.UTM_TO_RDC.

My concern is now about the "date1_date2_baseline.rsc", to be inserted in the IFG folders for processing, which looks very different compared to the one I got (date1-date2_5rlks.baseline)

Is there another script for getting this or should I do something different with "pysarApp.py"?

Thanks and sorry for disturbance.

Sven.

[Sven]

Sorry,

in the last row I wanted to say "prep_gamma.py" instead of "pysarApp.py"

Sven.

[Yunjun Zhang]

Hi Sven,

I have committed a fix to make the code independent from statsmodels module, please use git pull to update the code and try again.

The "date1_date2_baseline.rsc" file is generated by ROI_APC; for GAMMA, it's "date1-date2_5rlks.baseline" and "date1-date2_5rlks.base_perp". These two files can be generated using Gamma command base_orbit and base_perp, an example for them is attached below. prep_gamma.py in PySAR should be calling these two commands if the "date1_5rlks.amp.par", "date2_5rlks.amp.par" and "date1-date2_5rlks.off" files are available.

Here are the files I have for each interferogram for your reference:

# Before prep_gamma.py

071208-100220_4rlks.off

071208_4rlks.ramp.par

100220_4rlks.ramp.par

filt_071208-100220_4rlks.cor

diff_filt_071208-100220_4rlks.unw

# After prep_gamma.py

071208-100220_4rlks.base_perp

071208-100220_4rlks.baseline

071208-100220_4rlks.off

071208_4rlks.ramp.corner

071208_4rlks.ramp.corner_full

071208_4rlks.ramp.par

100220_4rlks.ramp.par

filt_071208-100220_4rlks.cor

filt_071208-100220_4rlks.cor.rsc

diff_filt_071208-100220_4rlks.unw

diff_filt_071208-100220_4rlks.unw.rsc

If you have all these files, you are all set to go.

Yunjun

[Sven]

Hi Yunjun,

now everything seems to be ok after your suggestions.

No problems at all for naming conversion from GAMMA to PySAR (prep_gamma.py), though I would like to know which baseline file I must put into the IFG folder: *.base_perp or *.baseline? In ROI_PAC you have just one.

I tried also to process the example and this was ok till the final step for the generation of the KMZ file.

The problem should be related to "pykml": I used both the package you put on the web site and pykml I got from pip (pip3 install pykml): the latter is the one I'm using now.

In the attached file you can see the output of the terminal; I'm using the very last release of PySAR I got through the "git pull" command as you suggested.

Sven.

[Yunjun Zhang]

Hi Sven,

For your question, it's the *.base_perp file. What prep_gamma.py did is to extract all the parameters from *.par, *.off, *.base_perp file and *.corner file, convert the internal variable names to ROIPAC convention and write all of them into the *.unw.rsc and *.cor.rsc files. Thus, after the first run, all those gamma parameter files are not needed anymore. However, since they are very small, I usually keep them there for backup. Similarly, prep_roipac.py merges the *.unw.rsc and *.baseline.rsc into *.unw.rsc.

The current released pykml is still in python2 and can not work with python3 code yet; the one I put on the website is an updated py2/3 compatible version, which you should use. After the version switch, you could run save_kml.py to test if it works, check save_kml.py -h for the detailed usage and example.

Cheers,

Yunjun

[Sven]

Hi Yunjun,

sorry for disturbing you again.

My problem with the "save_kml.py" script was very simple to fix: just changing the python version in the shabang of the script, from #!/usr/bin/env python to #!/usr/bin/env python3 (as far as I remember, there was a script in ROI_PAC to change the shabang of all non perl scripts into perl scripts, but I'm not sure; is there something similar for PySAR?)

Then processing of the example regularly goes till the end, though I had an error message (see the attached file) related to the attribute 'shape'.

I looked for information on this topic on Google and I realized the problem could be related to the OpenCV package missing, so I installed it (pip3 install opencv-python), though the problem still remains after the installation.

A missing image or wrong image path? More package(s) to install?

Thanks in advance and have a nice day.

Sven.

[Yunjun Zhang]

Hi Sven,

Thank you for pointing out the shebang, I have updated the save_kml.py to your suggestion.

I am not aware of the ROI_PAC script of changing shebang. It's not existed in PySAR.

As for the "shape" error, it's a bug while plotting DEM as background in shaded relief and topography contour, I just committed the fix. Please use git pull to update the code and re-run. No extra package is needed to install.

Thank you,

Yunjun

[Aoqing Guo]

Hi Yunjun,

I hope this email finds you well. I am currently working with MintPy for processing results obtained from the GAMMA processor, specifically for the step of unwrapping and correcting unwrapping errors. However, I am encountering an issue with the connected components file required for this step.

In my configuration file, I have set the parameter mintpy.load.connCompFile = ../interferograms/*/*.amp.corner, which points to the .corner files generated by prep_gamma. However, when I run the processing, I encounter the following error:

ValueError: unsupported input file extension: .corner supported file extensions: ['.unw', '.cor', '.int', '.dem', '.hgt_sim', '*to_rdc', '*2_rdc', '*2rdc']

It appears that MintPy does not recognize the .corner file extension. Additionally, when I attempt to include .corner in the supported file extensions, MintPy reports that the metadata file for the .corner file is missing, leading to another error.

Could you please advise on how I can resolve this issue or provide guidance on how to properly format or handle the .corner files in this context? Any suggestions or solutions would be greatly appreciated.

Thank you very much for your assistance.

Best regards,

Aoqing