Generic NetCDF reader

28 views
Skip to first unread message

Pierre Complex

unread,
Sep 13, 2021, 4:22:43 AM9/13/21
to pytroll
Hello,

I am interested in reading generic NetCDF files that comply
to the CF norm.

I generate such files as part of processing geostationary data
and output either full disk geostationary projection data or
gridded lat-lon data.

The satpy_cf_nc reader requires extra fields to be defined. Is
it correct that I cannot read a "plain" CF file without adding
extra information? If not, how should I proceed?

Regards,

Pierre

David Hoese

unread,
Sep 13, 2021, 10:17:00 PM9/13/21
to pytroll
HI Pierre,

We (satpy developers) would love Satpy to be able to read any CF file, but when it was attempted it was hard to get right for the use case the original developers needed it for. The `satpy_cf_nc` reader is meant to take advantage of the extra metadata that is added by the `cf` writer in Satpy. I've personally lost track of what the current CF reader does and requires in Satpy, but if you have the interest and the time to update it (or make a new reader based on it) that is more generic I will help you where I can.

Otherwise, you may have some options if you use xarray directly and do `ds = xr.open_dataset('your_file.nc')`. There are ways to make Satpy understand the data loaded this way, but it depends what you're really hoping to do with Satpy and your data. So what is your end goal?

Dave

Pierre Complex

unread,
Sep 30, 2021, 8:05:39 AM9/30/21
to pytroll
Hi Dave,

Sorry for the delay! I subscribed to the list using my gmail account which I rarely use apart for
google-hosted services.

Thank you for the reply. My goal is to load my CF files with geostationary grid mapping with
a proper geolocalisation in satpy to validate visually the file, using the coastlines superimposed
with the satellite data.

So, "xr.open_dataset" + generating the latitude and longitude for the GEOS projection using
the CF metadata entry.

In the meantime I used iris (sorry :-P ) that loads the files correctly but as it is about validation
I prefer to use several software pipelines if possible.

Regards,

Pierre

PS: if you have geolocalisation validation recipes I am interested.

David Hoese

unread,
Sep 30, 2021, 10:01:13 AM9/30/21
to pyt...@googlegroups.com
> PS: if you have geolocalisation validation recipes I am interested.

Forgive my ignorance, but what exactly does that type of validation mean?

Regardless, glad you got something to work for you even if it wasn't a
pytroll library.

Dave
> do `ds = xr.open_dataset('your_file.nc <http://your_file.nc>')`.
> There are ways to make Satpy understand the data loaded this way,
> but it depends what you're really hoping to do with Satpy and your
> data. So what is your end goal?
>
> Dave
>
> On Monday, September 13, 2021 at 3:22:43 AM UTC-5
> complex...@gmail.com wrote:
>
> Hello,
>
> I am interested in reading generic NetCDF files that comply
> to the CF norm.
>
> I generate such files as part of processing geostationary data
> and output either full disk geostationary projection data or
> gridded lat-lon data.
>
> The satpy_cf_nc reader requires extra fields to be defined. Is
> it correct that I cannot read a "plain" CF file without adding
> extra information? If not, how should I proceed?
>
> Regards,
>
> Pierre
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pytroll" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pytroll/UQZTmUeVMZg/unsubscribe
> <https://groups.google.com/d/topic/pytroll/UQZTmUeVMZg/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/98f6ae5a-f995-46c7-bd26-e5e39a343f89n%40googlegroups.com
> <https://groups.google.com/d/msgid/pytroll/98f6ae5a-f995-46c7-bd26-e5e39a343f89n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Pierre Complex

unread,
Oct 4, 2021, 9:25:37 AM10/4/21
to pytroll
On Thursday, September 30, 2021 at 4:01:13 PM UTC+2 dho...@gmail.com wrote:
> PS: if you have geolocalisation validation recipes I am interested.

Forgive my ignorance, but what exactly does that type of validation mean?
 
More likely my own (ignorance indeed). As of now, I plot the satellite data
with coastlines (from natural earth) to check the alignment of my data. I was
wondering how standard this is, that is all.

In any case, if I get to implement something to load NetCDF CF files in satpy
I'll make sure to share it!

Regards,

Pierre






David Hoese

unread,
Oct 4, 2021, 9:48:07 AM10/4/21
to pyt...@googlegroups.com
I know that cartopy can be used to plot NaturalEarth shapefiles onto
your data. I'm not sure I've heard of people doing validation with
cartopy or even NaturalEarth shapefiles. I suppose they are meant to be
pretty accurate and I'm not sure of what better datasets people use for
validation so it doesn't seem like a terrible idea.

Good luck.

Dave
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pytroll" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pytroll/UQZTmUeVMZg/unsubscribe
> <https://groups.google.com/d/topic/pytroll/UQZTmUeVMZg/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/95180d9c-edf6-44b4-8143-5a1d7be50430n%40googlegroups.com
> <https://groups.google.com/d/msgid/pytroll/95180d9c-edf6-44b4-8143-5a1d7be50430n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Pierre Complex

unread,
Oct 4, 2021, 9:58:19 AM10/4/21
to pytroll
Thanks!

BTW: I actually found the gist here https://gist.github.com/sfinkens/a07ac7982699ed8fc0fb433bb75818b7
and used it as inspiration (interesting region to look at + I had to check the typeOfEarthModel flag as well
for our archive).

Regards,

Pierre
Reply all
Reply to author
Forward
0 new messages