Clutter removal - Values of tr1, tr2 and n_p

101 views
Skip to first unread message

Kunal Rustagi

unread,
Apr 24, 2020, 1:20:36 AM4/24/20
to wradlib-users
Hello,

I have read the docs and it has been advised to perform clutter removal before gridding of data. While applying the gabella filter, please can someone explain me the significance of parameters passed to filter_gabella() function, namely tr1, tr2, n_p and thrsnorain.

I have referred this link https://docs.wradlib.org/en/stable/notebooks/classify/wradlib_clutter_gabella_example.html and have attached my output below.

Thank you,
Kunal
Screenshot from 2020-04-24 10-48-23.png

Kai Mühlbauer

unread,
Apr 24, 2020, 1:57:33 AM4/24/20
to wradli...@googlegroups.com
Hello Kunal,

there is a reference to the gabella paper in the wradlib-docs, but the
link is broken currently.

So here is the paper which explains the use of the filter and the
parameters.

https://www.copernicus.org/erad/online/erad-305.pdf

HTH,
Kai

P.S. The filter should be adapted to polar data (az, range) but it can
also work on cartesian/gridded data, AFAIR.


Am 24.04.20 um 07:20 schrieb Kunal Rustagi:
> **
>
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wradlib-user...@googlegroups.com
> <mailto:wradlib-user...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/wradlib-users/8dab70d3-8525-43e2-9868-0db6a9097700%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/8dab70d3-8525-43e2-9868-0db6a9097700%40googlegroups.com?utm_medium=email&utm_source=footer>.

Sambit Panda

unread,
Apr 24, 2020, 1:59:29 AM4/24/20
to wradli...@googlegroups.com
Please go through the paper mentioned (Gabella et al). All details are basically mentioned there. Basically tr1, is a threshold on the difference of dbz values of continuous pixels in order to check homogeneity! Similarly tr2 is a threshold for compactness for the Reflectivity structures.

Regards,
Sambit

--
You received this message because you are subscribed to the Google Groups "wradlib-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wradlib-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/wradlib-users/8dab70d3-8525-43e2-9868-0db6a9097700%40googlegroups.com.

Kunal Rustagi

unread,
Apr 24, 2020, 3:34:14 AM4/24/20
to wradlib-users
Hello Kai,

Thank you for the new link. As mentioned in your reply, is it safe then to carry out the clutter removal after interpolation?

Thanks,
Kunal

Kunal Rustagi

unread,
Apr 24, 2020, 3:34:38 AM4/24/20
to wradlib-users
Thank you so much Sambit Panda.


On Friday, April 24, 2020 at 11:29:29 AM UTC+5:30, Sambit Panda wrote:
Please go through the paper mentioned (Gabella et al). All details are basically mentioned there. Basically tr1, is a threshold on the difference of dbz values of continuous pixels in order to check homogeneity! Similarly tr2 is a threshold for compactness for the Reflectivity structures.

Regards,
Sambit

On Fri, Apr 24, 2020, 10:50 AM Kunal Rustagi <kunal....@gmail.com> wrote:
Hello,

I have read the docs and it has been advised to perform clutter removal before gridding of data. While applying the gabella filter, please can someone explain me the significance of parameters passed to filter_gabella() function, namely tr1, tr2, n_p and thrsnorain.

I have referred this link https://docs.wradlib.org/en/stable/notebooks/classify/wradlib_clutter_gabella_example.html and have attached my output below.

Thank you,
Kunal

--
You received this message because you are subscribed to the Google Groups "wradlib-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wradli...@googlegroups.com.

Kai Mühlbauer

unread,
Apr 24, 2020, 3:42:10 AM4/24/20
to wradli...@googlegroups.com
Hi Kunal,

I would suggest to do clutter removal on the polar data in any case, if
available.

If you interpolate, you might interpolate the clutter from one polar
pixel into multiple cartesian grid cells (depending on the resolution).

The other aspect is, that the clutter values might be reduced by
interpolation and these residuals might only be detected in
accumulations over long timeseries.

All depends also on your interpolation scheme.

You might just try for some case studies how the two different
approaches work out for your data and product.

Cheers,
Kai

Am 24.04.20 um 09:34 schrieb Kunal Rustagi:
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wradlib-user...@googlegroups.com
> <mailto:wradlib-user...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer>.

Kunal Rustagi

unread,
Apr 24, 2020, 3:57:04 AM4/24/20
to wradlib-users
Hello Kai,

I will try both the methods. I liked the way wradlib incorporates the average values for all the parameters mentioned in the paper (Gabella et Al) for the clutter detection. Great work.

Thanks,
 Kunal

On Friday, April 24, 2020 at 1:12:10 PM UTC+5:30, Kai Mühlbauer wrote:
Hi Kunal,

I would suggest to do clutter removal on the polar data in any case, if
available.

If you interpolate, you might interpolate the clutter from one polar
pixel into multiple cartesian grid cells (depending on the resolution).

The other aspect is, that the clutter values might be reduced by
interpolation and these residuals might only be detected in
accumulations over long timeseries.

All depends also on your interpolation scheme.

You might just try for some case studies how the two different
approaches work out for your data and product.

Cheers,
Kai

Am 24.04.20 um 09:34 schrieb Kunal Rustagi:
> Hello Kai,
>
> Thank you for the new link. As mentioned in your reply, is it safe then
> to carry out the clutter removal after interpolation?
>
> Thanks,
> Kunal
>
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send

Kunal Rustagi

unread,
Apr 24, 2020, 7:40:40 AM4/24/20
to wradlib-users
Hello Kai,

I tried removing clutter before converting polar to cartesian
and after converting to cartesian. Followed this link
for clutter removal.
But output, as seen from the screenshots, didnt turn out fine in either of the cases.

Can you please help me out here?

Thanks,
Kunal

On Friday, April 24, 2020 at 1:12:10 PM UTC+5:30, Kai Mühlbauer wrote:
Hi Kunal,

I would suggest to do clutter removal on the polar data in any case, if
available.

If you interpolate, you might interpolate the clutter from one polar
pixel into multiple cartesian grid cells (depending on the resolution).

The other aspect is, that the clutter values might be reduced by
interpolation and these residuals might only be detected in
accumulations over long timeseries.

All depends also on your interpolation scheme.

You might just try for some case studies how the two different
approaches work out for your data and product.

Cheers,
Kai

Am 24.04.20 um 09:34 schrieb Kunal Rustagi:
> Hello Kai,
>
> Thank you for the new link. As mentioned in your reply, is it safe then
> to carry out the clutter removal after interpolation?
>
> Thanks,
> Kunal
>
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
Screenshot from 2020-04-24 13-49-21.png
Screenshot from 2020-04-24 17-03-48.png

Kai Mühlbauer

unread,
Apr 24, 2020, 7:46:41 AM4/24/20
to wradli...@googlegroups.com
Hi Kunal,

first, make sure you use the keywords correctly to your data:

https://docs.wradlib.org/en/stable/generated/wradlib.clutter.filter_gabella.html#wradlib.clutter.filter_gabella

radial vs cartesian

It looks you treated your cartesian data as polar data.

Another thing is you would need to mask unnecessary data (no-precip) in
the output data.

It won't look so bad, if the two issues are resolved.

HTH,
Kai

Am 24.04.20 um 13:40 schrieb Kunal Rustagi:
> > an email to wradli...@googlegroups.com <javascript:>
> > <mailto:wradlib-user...@googlegroups.com <javascript:>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wradlib-user...@googlegroups.com
> <mailto:wradlib-user...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer>.

Kunal Rustagi

unread,
Apr 24, 2020, 8:01:27 AM4/24/20
to wradlib-users
Hi Kai,

Apologies for my mistake.
Would fillna() work in the case of no precipitation data?
If yes, what would be the suitable value to mask it?

Thanks,
Kunal
>      > <mailto:wradlib-users+unsub...@googlegroups.com <javascript:>>.

Kai Mühlbauer

unread,
Apr 24, 2020, 8:08:20 AM4/24/20
to wradli...@googlegroups.com
Hi Kunal,

fillna()? Are you using pandas/xarray?

Normally one should mask all non-precip values (make them nan or zero,
depending on the subsequent interpolation scheme) prior interpolation.

The notebook here shows a very basic workflow including your tasks:

https://docs.wradlib.org/en/stable/notebooks/basics/wradlib_workflow.html

I think we can surely improve on the documentation to better show the
details involved.

HTH,
Kai

Am 24.04.20 um 14:01 schrieb Kunal Rustagi:
> >      > <mailto:wradlib-user...@googlegroups.com
> <javascript:> <javascript:>>.
> >      > To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com>>
>
> >
> >      >
> >
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
>
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "wradlib-users" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to wradli...@googlegroups.com <javascript:>
> > <mailto:wradlib-user...@googlegroups.com <javascript:>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wradlib-user...@googlegroups.com
> <mailto:wradlib-user...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/wradlib-users/85778a33-5020-47c9-bda9-2440769e3f2f%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/85778a33-5020-47c9-bda9-2440769e3f2f%40googlegroups.com?utm_medium=email&utm_source=footer>.

Kunal Rustagi

unread,
Apr 24, 2020, 8:41:56 AM4/24/20
to wradlib-users
Hello Kai,

I thought numpy had a similar fillna().
z_arr[np.isnan(z_arr)]= -128
I filled all nan values with -128 using the above syntax. My two observations
for both the methods are attached. They look a lot cleaner now.

Would like to hear your comments. Maybe -128 is too extreme.

Thanks,
Kunal
>      >      > <mailto:wradlib-users+unsub...@googlegroups.com
>     <javascript:> <javascript:>>.
>      >      > To view this discussion on the web, visit
>      >      >
>      >
>     https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com>
>
>      >    
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com>>
>
>      >
>      >      >
>      >    
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer>
>
>      >    
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/wradlib-users/28fd385b-51ff-443e-8885-a031a237c246%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
>
>      >
>      >
>      > --
>      > You received this message because you are subscribed to the Google
>      > Groups "wradlib-users" group.
>      > To unsubscribe from this group and stop receiving emails from it,
>     send
>      > an email to wradli...@googlegroups.com <javascript:>
>      > <mailto:wradlib-users+unsub...@googlegroups.com <javascript:>>.
Screenshot from 2020-04-24 18-05-43.png
Screenshot from 2020-04-24 18-05-29.png

Kai Mühlbauer

unread,
Apr 24, 2020, 9:15:30 AM4/24/20
to wradli...@googlegroups.com
Hi Kunal,

looks much better, indeed.

You could also create a masked array with:

masked_z = np.ma.masked_invalid(z_arr)

and plot that.

If you need some values again:

filled_z = masked_z.filled(-128)

HTH,
Kai



Am 24.04.20 um 14:41 schrieb Kunal Rustagi:
> Hello Kai,
>
> >      >      > <mailto:wradlib-user...@googlegroups.com
> <javascript:>
> >      > <mailto:wradlib-user...@googlegroups.com
> <javascript:> <javascript:>>.
> >      > To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com>>
>
> >
> >      >
> >
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
>
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "wradlib-users" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to wradli...@googlegroups.com <javascript:>
> > <mailto:wradlib-user...@googlegroups.com <javascript:>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/wradlib-users/85778a33-5020-47c9-bda9-2440769e3f2f%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/85778a33-5020-47c9-bda9-2440769e3f2f%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/wradlib-users/85778a33-5020-47c9-bda9-2440769e3f2f%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/wradlib-users/85778a33-5020-47c9-bda9-2440769e3f2f%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "wradlib-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wradlib-user...@googlegroups.com
> <mailto:wradlib-user...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/wradlib-users/c740a0e3-1bc5-4869-955c-b971c6feaddf%40googlegroups.com
> <https://groups.google.com/d/msgid/wradlib-users/c740a0e3-1bc5-4869-955c-b971c6feaddf%40googlegroups.com?utm_medium=email&utm_source=footer>.

Kunal Rustagi

unread,
Apr 24, 2020, 11:03:50 AM4/24/20
to wradlib-users
Hello Kai,

Sorry, but I did not understand this post. Do you mean that after
replacing nan values, I should create a masked_z array before creating
a clutter map?

I have attached a screenshot of my clutter removal process. Please guide me.

Thanks,
Kunal
>      >      >      > <mailto:wradlib-users+unsub...@googlegroups.com
>      >      > <mailto:wradlib-users+unsub...@googlegroups.com
>     <javascript:> <javascript:>>.
>      >      > To view this discussion on the web, visit
>      >      >
>      >
>     https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com>
>
>      >    
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com>>
>
>      >
>      >      >
>      >    
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer>
>
>      >    
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/wradlib-users/2242406c-5acb-4948-ac36-3db3450698bb%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
>
>      >
>      >
>      > --
>      > You received this message because you are subscribed to the Google
>      > Groups "wradlib-users" group.
>      > To unsubscribe from this group and stop receiving emails from it,
>     send
>      > an email to wradli...@googlegroups.com <javascript:>
>      > <mailto:wradlib-users+unsub...@googlegroups.com <javascript:>>.
Screenshot from 2020-04-24 20-27-28.png
Reply all
Reply to author
Forward
0 new messages