WRF model and ERDDAP

[Alessandro De Donno]

Dear all,

first post here!

I'd like to add some NetCDF files coming from WRF model as ERDDAP datasets.
Looking at NetCDF headers, the dimensions are:

dimensions:
        Time = UNLIMITED ; // (4 currently)
        south_north = 402 ;
        west_east = 288 ;
        bottom_top = 59 ;
        soil_layers_stag = 4 ;
        west_east_stag = 289 ;
        south_north_stag = 403 ;
        bottom_top_stag = 60 ;
        DateStrLen = 19 ;

Then there are some dimension-related variables:

char Times(Time, DateStrLen) ;
float XLAT(Time, south_north, west_east) ;
        XLAT:FieldType = 104 ;
        XLAT:MemoryOrder = "XY " ;
        XLAT:description = "LATITUDE, SOUTH IS NEGATIVE" ;
        XLAT:units = "degree_north" ;
        XLAT:stagger = "" ;
float XLAT_U(Time, south_north, west_east_stag) ;
        XLAT_U:FieldType = 104 ;
        XLAT_U:MemoryOrder = "XY " ;
        XLAT_U:description = "LATITUDE, SOUTH IS NEGATIVE" ;
        XLAT_U:units = "degree_north" ;
        XLAT_U:stagger = "X" ;
        XLAT_U:coordinates = "XLONG_U XLAT_U" ;
float XLAT_V(Time, south_north_stag, west_east) ;
        XLAT_V:FieldType = 104 ;
        XLAT_V:MemoryOrder = "XY " ;
        XLAT_V:description = "LATITUDE, SOUTH IS NEGATIVE" ;
        XLAT_V:units = "degree_north" ;
        XLAT_V:stagger = "Y" ;
        XLAT_V:coordinates = "XLONG_V XLAT_V" ;
float XLONG(Time, south_north, west_east) ;
        XLONG:FieldType = 104 ;
        XLONG:MemoryOrder = "XY " ;
        XLONG:description = "LONGITUDE, WEST IS NEGATIVE" ;
        XLONG:units = "degree_east" ;
        XLONG:stagger = "" ;
float XLONG_U(Time, south_north, west_east_stag) ;
        XLONG_U:FieldType = 104 ;
        XLONG_U:MemoryOrder = "XY " ;
        XLONG_U:description = "LONGITUDE, WEST IS NEGATIVE" ;
        XLONG_U:units = "degree_east" ;
        XLONG_U:stagger = "X" ;
        XLONG_U:coordinates = "XLONG_U XLAT_U" ;
float XLONG_V(Time, south_north_stag, west_east) ;
        XLONG_V:FieldType = 104 ;
        XLONG_V:MemoryOrder = "XY " ;
        XLONG_V:description = "LONGITUDE, WEST IS NEGATIVE" ;
        XLONG_V:units = "degree_east" ;
        XLONG_V:stagger = "Y" ;
        XLONG_V:coordinates = "XLONG_V XLAT_V" ;
float XTIME(Time) ;
        XTIME:FieldType = 104 ;
        XTIME:MemoryOrder = "0  " ;
        XTIME:description = "minutes since simulation start" ;
        XTIME:units = "" ;
        XTIME:stagger = "" ;

And the "actual" variables, most of them depending on Time, south_north, west_east. For example:

float ACGRDFLX(Time, south_north, west_east) ;
        ACGRDFLX:FieldType = 104 ;
        ACGRDFLX:MemoryOrder = "XY " ;
        ACGRDFLX:description = "ACCUMULATED GROUND HEAT FLUX" ;
        ACGRDFLX:units = "J m-2" ;
        ACGRDFLX:stagger = "" ;
        ACGRDFLX:coordinates = "XLONG XLAT" ;

Other variables depend on different dimensions:

float TSLB(Time, soil_layers_stag, south_north, west_east) ;
        TSLB:FieldType = 104 ;
        TSLB:MemoryOrder = "XYZ" ;
        TSLB:description = "SOIL TEMPERATURE" ;
        TSLB:units = "K" ;
        TSLB:stagger = "Z" ;
        TSLB:coordinates = "XLONG XLAT" ;

float U(Time, bottom_top, south_north, west_east_stag) ;
        U:FieldType = 104 ;
        U:MemoryOrder = "XYZ" ;
        U:description = "x-wind component" ;
        U:units = "m s-1" ;
        U:stagger = "X" ;
        U:coordinates = "XLONG_U XLAT_U" ;

I know I have to make different datasets for different set of dimensions.

A first problem is that Time, south_north and west_east values are indexes (i.e. Time = 0,1,2,3) and not "absolute" values.

There are some tools to post process WRF output to CF-friendly NetCDF files, let's say wrfout_to_cf (https://sundowner.colorado.edu/wrfout_to_cf/overview.html), but I don't know WRF model and I'd prefer to leave data files unchanged.

My question is: do you have some experience with WRF datasets in ERDDAP, some docs to read? Are NcML or a data transformation via-NCO necessary?

Thanks in advance,
  Alessandro

[Roy Mendelssohn - NOAA Federal]

Do you have a sample file you can point me to? I can tell you that you will need to have a different "dataset" for each group of coordinates.

Thanks,

-Roy

> --
> You received this message because you are subscribed to the Google Groups "ERDDAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to erddap+un...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/c61247b1-29b7-4869-a25d-05c6b6e0a7b4n%40googlegroups.com.

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Men...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[Alessandro De Donno]

Hi Roy,

thanks for your answer.

You can find the sample file here:

https://erddap-dev.cmcc-opa.eu/erddap/files/test_wrfout/2000/01/

It's the same file I used to extract the info in my previous post.

Thanks,

  A.

[Bob Simons - NOAA Federal]

Yes, you will have to make separate datasets for each group of multidimenisonal variables that share a common set of dimensions.

Another problem is that there is no Time[Time] (with absolute, not relative, numeric time values, following the netcdf standard of having a 1D variable with the same name as the dimension. Time[Time] is needed to include the absolute time values in the dataset. You can add Time[Time] with NCO or on-the-fly with .ncml.

But the main problem is that the XLAT and XLONG variables are multidimensional. This implies that dataset isn't using a cylindrical projection, which ERDDAP greatly prefers. It means that XLAT and XLONG need to be in a different dataset than the variables that use them. And the only thing that ties all the datasets together is the Time dimension (which lacks Time[Time]).

As is, this dataset is probably better handled by THREDDS, which simply passes through the data, as is.

If you want to get this dataset into ERDDAP in the way that is friendliest to users, I suggest making a new dataset with new data files, derived from this dataset, which:

* uses a cylindrical projection

* has time[time], lat[lat], lon[lon], soil_layers[soil_layers], altitude[altitude] (or similar for the vertical dimension).

I hope that helps.

--

[Roy Mendelssohn - NOAA Federal]

HI:

I looked at your file (thanks). I think you would be best served by putting this into a THREDDS server, and have ERDDAP pick up whatever options you choose. I say this because you could overcome lat-lon (it is a mercator projection so while lats or lons are not of a constant spacing, they do not vary by lat or lon). You could get around time by using ncml to read the base time from the file name. However, unless I am totally misunderstanding the file, each variables has 4 "times", which look like forecast times relative to the base time in the filename. This would cause problems when trying to aggregate in ERDDAP. THREDDS has a number of options for creating a single time series when files contain forecasts times - originally put in to deal with GRIB files of forecasts. For example, see this for FNMOC forecast model output:

https://thredds1.pfeg.noaa.gov/thredds/catalog/FNMOC/NAVGEM/onedegree/pressure/6hr/catalog.html

As you can see THREDDS allows you to create a variety of datasets. I believe our ERDDAP links to the " Best FNMOC NAVGEM 1-degree 6-hourly Pressure Time Series" which is a angle series through time.

HTH,

-Roy

> To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/CAG%3Djf3fVsiNVgs3JdXa9v7dxqzxjhgbBE6yomfXPsrhLJB-c1g%40mail.gmail.com.

[Alessandro De Donno]

Dear Bob, dear Roy,

thank you so much for your replies, much appreciated!

I'm relatively new in GIS world, but I feared it would be difficult to handle these kind of datasets...
Maybe I'll try to make new data files, or to setup a THREDDS server (or a THREDDS->ERDDAP chain).

Just a question for @Roy regarding lat/lon:

"you could overcome lat-lon  (it is a mercator projection so while lats or lons are not of a constant spacing,  they do not vary by lat or lon)."

How could I overcome that?
I sometimes find "nav_lat(y, x)" and "nav_lon(y, x)" variables, x and y as dimensions, but I guess it's the same situation.

Best regards,
  Alessandro

[Roy Mendelssohn - NOAA Federal]

If you look at the lat and lon arrays (called XLAT and XLON for example in the file), you will find they just repeat - that is each lon has the same set of lats, but the lats are not evenly spaced, so you can actually define a grid based on 1-D arrays of lats and lons. This is not the case for many projections, so this is not a general thing with projections, but the file says it is Mercator projection for which this is true.. Therefore you could use ncml to define the lats and lons. I would add that though the file claims to be CF-1.6 compliant, my perhaps mistaken understanding is they don't define the projection in the way described in CF-1.6 - for a machine to be able to read and understand a file little things like what some quantities are called and where they appear in the file matter tremendously. I may be able to read through the file and understand what is going on, but for a program to be able to do that on its own, really does require hewing to the details of the standard. Sort of following the standard will break. I know you didn't create the file, but just a warning. Even in THREDDS you may have to do some NcML to have the information it needs.

But as I said, it is most likely not worth doing in ERDDAP, because if I am understanding the file correctly, you have the more fundamental problem of how to deal with forecast times. THREDDS is better setup to deal with projections and forecast times. If you DuckDuckGo "WRF THREDDS" you will find several examples of files like yours being served by THREDDS. The resulting series can then be hooked up into ERDDAP.

HTH and let me know if you have other questions.

-Roy

> On Jun 24, 2022, at 2:50 AM, Alessandro De Donno <alessandr...@cmcc.it> wrote:
>
> Just a question for @Roy regarding lat/lon:
>
> "you could overcome lat-lon (it is a mercator projection so while lats or lons are not of a constant spacing, they do not vary by lat or lon)."
>
> How could I overcome that?
> I sometimes find "nav_lat(y, x)" and "nav_lon(y, x)" variables, x and y as dimensions, but I guess it's the same situation.