Creation of IonStack in Mintpy

[Jasmine Hansen]

Hi Yunjun,

Thank you for helping with my mintpy 1.4.0 errors yesterday.

I wanted to start a different thread as I have few questions specifically about the new ion portion of mintpy as opposed to more general load_data.py issues.

Mintpy 1.4.0 failed near the end of running smallbaselineApp.py with the attached configuration file. The input files are from a recently executed topStack run including the ionospheric processing module.

This is the error:

view.py --dpi 150 --noverbose --nodisplay --update --memory 4.0 velocityERA5.h5 --mask no
view.py --dpi 150 --noverbose --nodisplay --update --memory 4.0 numInvIfgram.h5 --mask no
view.py --dpi 150 --noverbose --nodisplay --update --memory 4.0 /home/jasmine/JasmineShare/CURRENT_WORKING/25_365_ion/mintpy_ion/inputs/ionStack.h5 unwrapPhase- --zero-mask
view.py --dpi 150 --noverbose --nodisplay --update --memory 4.0 /home/jasmine/JasmineShare/CURRENT_WORKING/25_365_ion/mintpy_ion/inputs/ionStack.h5 coherence- --mask no -v 0 1
Traceback (most recent call last):
  File "/home/jasmine/miniconda3/envs/mintpy_ion/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/site-packages/mintpy/smallbaselineApp.py", line 1291, in main
    app.plot_result()
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/site-packages/mintpy/smallbaselineApp.py", line 1224, in plot_result
    Parallel(n_jobs=num_cores)(delayed(mintpy.view.main)(iargs) for iargs in iargs_list)
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/site-packages/joblib/parallel.py", line 1056, in __call__
    self.retrieve()
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/site-packages/joblib/parallel.py", line 935, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 542, in wrap_future_result
    return future.result(timeout=timeout)
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/home/jasmine/miniconda3/envs/mintpy_ion/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
Exception: No input dataset found!
available datasets:
['unwrapPhase-20180612_20180624', 'unwrapPhase-20180612_20180718', 'unwrapPhase-20180624_20180718', 'unwrapPhase-20180624_20180730', 'unwrapPhase-20180718_20180730', 'unwrapPhase-20180718_20180811', 'unwrapPhase-20180730_20180811', 'unwrapPhase-20180730_20190526', 'unwrapPhase-20180811_20190526', 'unwrapPhase-20180811_20190701', 'unwrapPhase-20190526_20190701', 'unwrapPhase-20190526_20190713', 'unwrapPhase-20190701_20190713', 'unwrapPhase-20190701_20190725', 'unwrapPhase-20190713_20190725', 'unwrapPhase-20190713_20190806', 'unwrapPhase-20190725_20190806', 'unwrapPhase-20190725_20190818', 'unwrapPhase-20190806_20190818', 'unwrapPhase-20190806_20200520', 'unwrapPhase-20190818_20200520', 'unwrapPhase-20190818_20200601', 'unwrapPhase-20200520_20200601', 'unwrapPhase-20200520_20200613', 'unwrapPhase-20200601_20200613', 'unwrapPhase-20200601_20200625', 'unwrapPhase-20200613_20200625', 'unwrapPhase-20200613_20200707', 'unwrapPhase-20200625_20200707', 'unwrapPhase-20200625_20200719', 'unwrapPhase-20200707_20200719', 'unwrapPhase-20200707_20200731', 'unwrapPhase-20200719_20200731', 'unwrapPhase-20200719_20200812', 'unwrapPhase-20200731_20200812', 'unwrapPhase-20200731_20200824', 'unwrapPhase-20200812_20200824', 'unwrapPhase-20200812_20210807', 'unwrapPhase-20200824_20210807', 'unwrapPhase-20200824_20210819', 'unwrapPhase-20210807_20210819']

When I use info.py on the ionStack.h5 file the only layers available are below (there is no coherence which I think is causing an issue).

HDF5 dataset "/bperp               ": shape (41,)               , dtype <float32>
HDF5 dataset "/date                ": shape (41, 2)             , dtype <|S8>
HDF5 dataset "/dropIfgram          ": shape (41,)               , dtype <bool>
HDF5 dataset "/unwrapPhase         ": shape (41, 3345, 7512)    , dtype <float32>

Do you think this is related to a set up issue on my side or something else? My intention is to produce two stacks (with and without the ion) as well as two velocity rate maps to compare ion and non ion runs.

thanks

jasmine

[Zhang Yunjun]

Hi Jasmine,

Sorry for getting back at this so late, we were occupied by the recent UNAVCO short course and NISAR workshop.

This error is because your mintpy.load.ionCorFile is not specified. Anyway, this is just a plotting error at the end, and does not affect any data file generation (this dataset is currently not used anywhere, except for plotting, if I remember it right). The code should be had been smart enough to skip the plotting. Could you open a issue on github for us keep track of it?

Yuan-Kai Liu (from Caltech) and I are integrating the ionospheric correction into mintpy, so we welcome feedback like this!

Cheers,

Yunjun

[Jasmine Hansen]

Hi Yunjun,

No problem at all!

Thanks for pointing out the missing ionCorFile

cheers

Jasmine

[Alejandro Oro]

Hello Jasmine! I wanted to ask you how do you get MintPy to process the ionoStack.h5 file in order to create a time series corrected by ionosphere? I modified the "ionosphere stack" section so it looks like this:

mintpy.load.ionUnwFile      = /mnt/g/iono_StackTest/ion/*_*/ion_cal/filt.ion  #[path pattern of unwrapped interferogram files]
mintpy.load.ionCorFile      = /mnt/g/iono_StackTest/merged/interferograms/*/filt_fine.cor  #[path pattern of spatial coherence       files]
mintpy.load.ionConnCompFile = /mnt/g/iono_StackTest/merged/interferograms/*/filt_fine.unw.conncomp  #[path pattern of connected components    files], optional but recommended

Then I run the smallbaselineApp.py script. Does this create a time series corrected by ionosphere?

Thanks in avance!

[Jasmine Hansen]

Hi Alejandro,

I would point you towards asking Yunjun for this question as I am still working out integrating these components into my work!

Cheers 

Jasmine Hansen
—————————————————————————————————

she/her

PhD Candidate in Dept. of Geological Sciences CU Boulder
Graduate Research Assistant at CIRES 

CIRES Room S271
University of Colorado Boulder
216 UCB
Boulder, CO 80309

-- 
You received this message because you are subscribed to a topic in the Google Groups "MintPy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mintpy/So8Ohn-EAPM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mintpy+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mintpy/84671057-cd0a-4589-a3ec-e3e4d1ef7747n%40googlegroups.com.

[Zhang Yunjun]

Hi Alejandro and Jasmine,

In the current smallbaselineApp, the ion files are loaded into inputs/ionStack.h5 and that’s it (correction are not implemented yet). However, one could use the individual scripts in mintpy to apply the corrections manually, step by step, using reference_point.py, ifgram_inversion.py, diff.py, etc. This requires a deeper familiarity of mintpy.

Yuan-Kai Liu from Caltech and I are on the process of integrating the iono correction into smallbaselineApp.py , and hopefully we will have it available via template options in the next release.

Yunjun

To view this discussion on the web visit https://groups.google.com/d/msgid/mintpy/4E8F1726-DCF0-439B-B408-32EF3E32CE49%40colorado.edu.

[Alejandro Oro]

Hi Jasmine! Thanks a lot for your answer!

Hello Yunjun! It is great news to know that this functionality will be implemented in the future. I am looking foward to use it since I am not enough familiar yet with MintPy to do it myself.

Best,

Alejandro

[Deha Umarhadi]

Hi Yunjun,

Are there any updates regarding the integration of iono correction into smallbaselineApp.py?

Best,

Deha

[Zhang Yunjun]

It's almost done. I just issued a PR on GitHub (https://github.com/insarlab/MintPy/pull/1172/) for this.

Cheers!

Yunjun