How to remove "rings" around the radar in the exported NetCDF grid data file

[Eugenio Realini]

Hi all,

first of all thank you very much for this awesome library being released as open source software!

I'm both a Python newbie (I usually work with MATLAB) and a radar newbie (I usually work with GPS), so please be kind ;)

I've successfully used PyART to read Sigmet IRIS/RAW data files, plot the results and export the gridded data to a NetCDF file. My main objective was to read gridded reflectivity data in MATLAB, and I actually managed to use the MATLAB command "ncread" to read the NetCDF file produced by PyART.

My problem is that while the PyART PPI plot does not show the "rings" around the radar, when I plot the CAPPI data in MATLAB the rings are shown (screenshot here). So I've tried to remove the rings from the gridded data itself, with the masking procedure shown in one of the examples, but no luck so far...

I've set the same threshold on the minimum reflectivity value that I use for the plot_ppi function (14 dBZ) also in the MATLAB plots, but while that successfully removes most of the noise, the rings are unaffected.

Could anyone help me understanding how the remove those rings from the MATLAB plots? I mean, I applied some kind of image filtering to partially remove them "a posteriori", but I guess there should be a better way to avoid having them in the first place.

Thanks in advance!

This is the code I'm currently using:

import numpy as np

import matplotlib.pyplot as plt

import pyart

# read Sigmet IRIS/RAW data

filename = 'JEP100802152831.RAWJY9F'

radar = pyart.io.read(filename)

# mask out last 10 gates of each ray, this removes the "ring" around the radar.

radar.fields['reflectivity']['data'][:, -10:] = np.ma.masked

# perform Cartesian mapping, limit to the reflectivity field.

grid = pyart.map.grid_from_radars(

    (radar,),

    grid_shape=(600, 600, 2),

    grid_limits=((-100000.0, 100000.0), (-100000.0, 100000.0),

                 (2000, 5000)),

    weighting_function='Cressman',

    fields=['reflectivity'],

    refl_filter_flag=True)

    

# save grid to netCDF file

pyart.io.write_grid(filename + '.nc', grid)

# PPI plot

display = pyart.graph.RadarDisplay(radar)

fig = plt.figure()

ax = fig.add_subplot(111)

display.plot_ppi('reflectivity', 0, mask_tuple=['reflectivity', 14], vmin=0, vmax=72.)

display.set_limits(xlim=[-100, 100])

display.set_limits(ylim=[-100, 100])

display.plot_range_rings([25, 50, 75, 100])

display.plot_cross_hair(5.)

plt.show()

[Scott Collis]

Hey Eugenio!

Welcome! You will find we don’t bite :)

Could you post the data some where? We can then put an iPython Notebook together to help you out (Check out iPython here: http://ipython.org/)

What version of Py-ART are you using ?

print out pyart.__version__

Scott Collis

scolli...@gmail.com

--
You received this message because you are subscribed to the Google Groups "Py-ART Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyart-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Eugenio Realini]

Hi Scott,

thanks a lot!
I've put the data here:
https://www.dropbox.com/s/7s3mxn4ct0vmrvz/140123_PyART_test.zip

The version of Py-ART I'm currently using:
1.0.0.dev-Unknown

I just checked, I already have iPython, it was installed together with
Anaconda. Never used it, though.
Just let me know what I need to do.

Thanks again.
Eugenio


2014/1/23 Scott Collis <scolli...@gmail.com>:

> You received this message because you are subscribed to a topic in the

> Google Groups "Py-ART Users" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pyart-users/ayb7Ez2gk6E/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Kai Muehlbauer]

Hello Eugenio,

it seems, that you plot two different things.

In python you plot the raw data (polar data) while in matlab you plot
the gridded data. I checked the created nc file with ncview (also
python) and it seems that the problem arises within the gridding function.

Maybe this helps to find the problems source.

Cheers,
Kai

Am 22.01.2014 09:45, schrieb Eugenio Realini:
> Hi all,
>
> first of all thank you very much for this awesome library being released
> as open source software!
>
> I'm both a Python newbie (I usually work with MATLAB) and a radar newbie
> (I usually work with GPS), so please be kind ;)
>
> I've successfully used PyART to read Sigmet IRIS/RAW data files, plot
> the results and export the gridded data to a NetCDF file. My main
> objective was to read gridded reflectivity data in MATLAB, and I
> actually managed to use the MATLAB command "ncread" to read the NetCDF
> file produced by PyART.
>
> My problem is that while the PyART PPI plot does not show the "rings"
> around the radar, when I plot the CAPPI data in MATLAB the rings are
> shown (screenshot here

> <https://www.dropbox.com/s/knvjegxug5r45os/Screenshot%202014-01-22%2017.15.58.png>).

> --
> You received this message because you are subscribed to the Google
> Groups "Py-ART Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pyart-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

--

Kai Muehlbauer
Meteorological Institute University of Bonn
Auf dem Huegel 20 | +49 228 739083
D-53121 Bonn | kai.mue...@uni-bonn.de

[Kai Muehlbauer]

Hi all,

just another point here:

If I set the refl_filter_flag to false I get results similar to the
expected.

Cheers,
Kai

[Jonathan Helmus]

Eugenio,

The radar data you are looking at has some high dBZ artefacts near
the radar which are causing the "ring" around the radar. This is most
likely due to ground clutter. There are a number of fancy ways of
dealing with ground clutter which radar researchers like to play around
with. An easy method to remove the artefact in this case is to filter
out all gates above a certain reflectivity level when making the grid.
I made up a IPython notebook showing technique using your data which
might be helpful:

http://nbviewer.ipython.org/github/jjhelmus/ipython_notebooks/blob/master/Eugenio_Radar_Artifacts.ipynb

Cheers,

- Jonathan Helmus

[Eugenio Realini]

Hi Jonathan,

your solution works prefectly, thank you very much!
Here are my MATLAB plots, without rings:
https://www.dropbox.com/s/j0rdu7ng7if89uj/Screenshot%202014-01-24%2015.34.11.png

...and also today I learned something new :)

Cheers,
Eugenio

P.S. I would like to put a reference to Py-ART in a paper I'm writing.
I will surely cite the URL "http://arm-doe.github.io/pyart/", but if
there is a way to add also a bibliographic reference, just let me know


2014/1/24 Jonathan Helmus <jjhe...@gmail.com>:

[Eugenio Realini]

Hi Kai,

thanks! I tried setting that flag to false, but while the rings
disappear, the rest of the plot gets surrounded by unrealistically
high (I think) reflectivity values.
This is the plot I get with Jonathan's solution:
https://www.dropbox.com/s/j0rdu7ng7if89uj/Screenshot%202014-01-24%2015.34.11.png
This is the one I get by setting that flag to false:
https://www.dropbox.com/s/nwm54wc86a0k9hs/Screenshot%202014-01-24%2015.55.10.png

I'm not really sure what happens in the second case, but the first
plot seems more realistic to me.

Cheers,
Eugenio


2014/1/23 Kai Muehlbauer <kai.mue...@uni-bonn.de>:

[Kai Muehlbauer]

Hi Eugenio,

yes, Jonathan is all right. I did only look on the rings.

Maybe Jonathan can shed some light on the source of that border
artifacts, when not filtering. Must be somewhere in the interpolation
routine.

Cheers,
Kai

[Scott Collis]

Great stuff Guys,

This is a pin-up for how community software should work! 

Eugenio, if you don’t mind me asking what radar is this? Helps in my “Why we are doing this” talk 

For those that are going Jonathan will be giving a presentation on Py-ART.. We will be highlighting Kai’s contribution as “Why build modular community software”

Scott Collis

scolli...@gmail.com

[Eugenio Realini]

Hi Scott,

indeed this is how community software should work! I am developing open source scientific software myself (for processing GNSS observations, http://www.gogps-project.org/), but it's difficult for users to decide to "expose" themselves on the public mailing list. I am getting private emails quite often asking for support, but almost nothing on the mailing list...

Anyway, about the radar, it is a C-band Doppler Radar in Serpong, Indonesia. It's part of this project: http://www.jamstec.go.jp/iorgc/harimau/HARIMAU.html

I'm working on data from a GPS meteorology campaign in that area, and we are using the radar observations as a reference for evaluating the relation between precipitation and GPS-derived water vapor.

Eugenio

2014/1/24 Scott Collis <scolli...@gmail.com>

[Jonathan Helmus]

I think the original ring as well as the unrealistically high
reflectivity around the otherwise good looking plot when the
refl_filter_flag is set to False are both the results of a the clipping
reflectivity values greater than 60 dBZ when plotting in MATLAB. If you
examine the grid with the default filtering or with no reflectivity
filtering there is a region of very high reflectivity (100+ dBZ)
surrounding the good signals. The MATLAB script only plots locations
where the reflectivity is between 14 and 60 dBZ (any maybe something
with 37 dBZ, my MATLAB is not as good as Python). So the high
reflectivity sites are suppressed but the interpolation during gridding
gives a smooth signal into the real signals, hence at the edges of the
clipping we still see a rings or shadows around real signals.

I've not completely happy with this explanation as there are still a few
features that still puzzle me, but filtering the reflectivity at ~70 dBZ
during gridding gives a reasonable result that makes sense to me.

Also at this time there is not a better reference than the Py-ART
website (http://arm-doe.github.io/pyart/). We are working on a paper
covering a number of Open Source radar software packages but it will not
be out for a while. Scott or I will be sure to send an announcement to
the pyart-users list when a good paper on Py-ART is published.

Cheers,

- Jonathan Helmus

[Eugenio Realini]

Hi Jonathan and Scott,

as promised, here is the paper for which I used Py-ART, with a reference to the website, an endnote and an acknowledgement to Py-ART (and to the members of this mailing list).

I hope it will contribute to spread knowledge about this great library :)

https://www.researchgate.net/publication/265690181_An_observation_campaign_of_precipitable_water_vapor_with_multiple_GPS_receivers_in_western_Java_Indonesia

Cheers

Eugenio