Stack of coherence not working

[Sven Borgstrom]

Hi Yunjun,

I'm asking for your support on a subject related to ISCE and not directly to MintPy but I asked on the ISCE Github forum 5 days ago, without any answer till now.

The reason why I'm asking you is explained later on.

I'm processing a S1 correlation stack in the run_files folder and, at "run_08_merge_burst_igram" I got the error:

topSwath = min(refSwaths, key = lambda x: x.sensingStart)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: min() arg is an empty sequence

(see also the attached file).

I obviously saw in the Github forum to check if there was a post on this subject, it was there, but this was closed without any fix.

About the merge burst step, I previously changed the "stackSentinel.py" script at row 731 into

"runObj.igram_mergeBurst(acquisitionDates, safe_dict, pairs)" otherwise I couldn't have generated the run_files folder.

I'm asking for your support because on November 26 there were some big debris flows in one of the islands of the Naples Gulf, Italy (at Ischia island, probably you heard that, with some casualities) and I was involved in a working group for the evaluation of the damages. I was thinking about a coherence change detection.

Sorry but otherwise I wouldn't have asked you for support.

Cheers,
Sven.

[Zhang Yunjun]

Hi Sven,

It seems to be a bug in the isce2/topsStack correlation workflow. I have replied you on the isce2 forum (https://github.com/isce-framework/isce2/discussions/630).

Cheers,

Yunjun

[Sven Borgstrom]

Hi Yunjun,

thanks for your indications.

Unfortunately, after the changes you suggested the issue still remains, with the same error message.

B.t.w. I don't want to disturb you anymore: I will use the interferogram workflow, which is the one I routinely use.

Just one question: I need to do a coherence change detection study with, let me say, the related Jupyter Notebook in the Tutorials web page: what should I do for this starting from the interferogram workflow? all the steps without unwrapping?

If I want to get a result without the Jupyter Notebook but only from the interferogram workflow + MintPy, is there the possibility to get a timeseries of coherence on single targets as I do for LoS ground deformations exploiting "save_kmz_timeseries.py"?

Sorry but also for me is the first time dealing with these aspects, as I usually exploit the phase for ground deformation studies.

Cheers,

Sven.

[Zhang Yunjun]

Hi Sven,

I assume you are referring to the notebook written by Marin Govorcin from JPL (https://nbviewer.org/github/insarlab/MintPy-tutorial/blob/main/applications/coherence_change_detection.ipynb). The inputs are ifgramStack.h5, geometryRadar.h5 and waterMask.h5 file, a.k.a., the result of `smallbaselineApp.py --dostep load_data`. As for the isce stack processing, I recommend running the entire workflow, including unwrapping, to avoid potential unexpected issues.

There are no coherence time-series like the phase time-series we are familiar with, because coherence is not a linear parameter in time and does not follow the closure relationship (coh_ab + coh_bc != coh_ac). However, the sequential coherence time-series (coh_12, coh_23, ..., coh_n-1,n) is frequently used in studies, such as in Stephenson et al. (2022). This is not available in mintpy, thus, can not be used via save_kmz_timeseries.py. I recommend you stick with the notebook from Marin.

O. L. Stephenson et al., "Deep Learning-Based Damage Mapping With InSAR Coherence Time Series," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-17, 2022, Art no. 5207917, doi: 10.1109/TGRS.2021.3084209.

Yunjun

[Sven Borgstrom]

Hi Yunjun,

just a couple of things:

- Your fix in the isce2/topsStack correlation workflow works fine: my issue was that before your fix, the run_generate_burst_igram was not created so it was not possible to run merge_burst_igram; so I removed everything and rerun again the correlation workflow, successfully.

B.t.w. I'm a bit confused now because the correlation workflow, according to https://github.com/isce-framework/isce2/blob/main/contrib/stack/topsStack/README.md doesn't unwrap the ifgs, whereas you told me to unwrap the workflow to avoid potential unexpected issues.

So which is the usefulness of the correlation stack?

To Author of the Notebook clearly referes in it to a "coherence_stack" (not ifgs stack), so which one should I use?

- I got an error running the first part of the Notebook, see the attached file.

In this regard, which are the dates I must insert into the Notebook?

My event was on Nov. 26, my last S1 acquisition on Nov. 30 and my stack starts from Jan. 10 (of course every 12 days).

As far as I can understand I should consider the last 3 dates i.e. Nov. 06, 18, 30, is it right?

Sorry but this is the first time for my to deal with coherence, as I usually exploit the phase for ground deformation studies.

Thanks,

Sven.

[Marin Govorcin]

Hi Sven,

Thanks for reporting this error, it looks like there is a bug in the plotting function "plot_ccd".

Please replace:

if orbit_direction is not None:

   ax.invert_yaxis() if orbit_direction.lower().startswith('a')  ax.invert_xaxis()

with:

if orbit_direction is not None:

    ax.invert_xaxis() if orbit_direction.lower().startswith('a') else ax.invert_xaxis()

    ax.invert_yaxis() if orbit_direction.lower().startswith('d') else ax.invert_xaxis()

As Yunjun already pointed out, it is safer to use stackSentinel.py interferogram workflow as the first part of the CCD Jupyter notebook is relying on the Mintpy to load and prepare the input data.

To clear out the confusion, the notebook is using coherence (layer) dataset from ifgramStack.h5.

I am also not familiar with the  stackSentinel.py correlation workflow, so would need to check it out to see how the data looks like after being loaded with Mintpy.

Another thing to consider is that Mintpy ingest coherence (filt_fine.cor) files calculated from filtered interferograms.

Best

Marin

[Sven Borgstrom]

Hi Marin,

thanks for reply.

I changed the first part of your Notebook following your indications and it works, thanks.

The problem comes from the second part, in the CCD calculation and this seems to be related to the "plot_ccd" function as you said and as you can see from the attached file (Part_1).

B.t.w. calculation for "difference" and "histogram matching" works fine, at least as far as I can understand, as I get the two figures, but not for "ratio" (see attached files Part_2 and 3).

I don't know if I was probably wrong somewhere, maybe in setting the dates: the event date is 20221126; for the starting date I set the first date of my dataset (20220110, 29 images, every 12 days);  the last is 20221212; the last pre-event is 20221118; the first post-event 20221130; S1 Descending data.

If you need more info from my side please let me know.

Cheers,

Sven.

[Marin Govorcin]

Hi Sven,

it is again plotting error, I issued PR with the fixes.

If you do not want to wait for the PR to get approved, you can replace these lines in plot_ccd:

for i in range(0, nimg):
        #Single plot
        if nimg == 1:
            method = method
            data = ccd_data
            ax = axs
        #Multiple plots  
        else:
            method = method[i]
            data = ccd_data[i]
            ax = axs[i]

with

for i in range(0, nimg):
        #Single plot
        if nimg == 1:
            plot_method = method
            data = ccd_data
            ax = axs
        #Multiple plots  
        else:
            plot_method = method[i]
            data = ccd_data[i]
            ax = axs[i]

and

        if title == True:
            if coherence_threshold is not None:
                txt = f'CCD on {event_date} \n {method} \n w/ thresh. {coherence_threshold}'
            else:
                txt = f'CCD on {event_date} \n {method}'
            ax.set_title(txt)

with

        if title == True:
            if coherence_threshold is not None:
                txt = f'CCD on {event_date} \n {plot_method} \n w/ thresh. {coherence_threshold}'
            else:
                txt = f'CCD on {event_date} \n {plot_method}'
            ax.set_title(txt)

Best

Marin

[Sven Borgstrom]

Hi Marin,

first of all thank you for the new Notebook of the MintPy page, you did a great job.

I had to install "folium" (conda install -c conda-forge folium) to get the last result, b.t.w. in this I had an issue (see the attached screenshot), which seems to be the coherence change map (see the attached *.kmz file) at a larger scale, not fixed on the island.

Your Notebook on the web page it's fine so maybe I have something missing (another conda package?); I checked also for "branca" but this is regularly installed.

In general terms, I was not able to see/discriminate the event (mudflow) we had on November 26 in the upper part of the island, probably due to the S1 resolution compared with the dimensions of the mudflow or maybe I was wrong in choosing the coherence value (0.7).

B.t.w. I don't want to disturb you now, we are close to Xmas, so please take your time to answer.

Merry Xmas to you, to Yunjun and to all who are reading the post.

Sven.

[Marin Govorcin]

Hi Sven,

I am glad to hear it worked. As for the web (folium map) it looks like something is wrong with the map bounds. The latitude and longitude (output area extent in (S, N, W, E) in degree) in the screen output of geocode.py should be the same as in map_bounds.

You can try to adjust these parameters manually:

map_bounds = [[attr["LAT_REF2"], attr["LON_REF1"]], 

                            [attr["LAT_REF3"], attr["LON_REF4"]]]

The S1 resolution could be one of the explanations, my suggestions would be to first visually inspect the pre_coherence and co_coherence  and see if you can see coherence change between these two where you expect to see the landslide. If you see there is something, you can try to refine the pre_coherence with start_date and coherence_threshold parameters. 

Best

Marin

[Sven Borgstrom]

Hi Marin,

thanks for reply during Christmas holidays.

I'll give a try and let you know, not only for the map parameters, but also to check the result inspecting pre_ and co_coherence.

Cheers,

Sven.