1st Draft of NOAA CATS Currents in UGRID NetCDF Format

[Brian Zelenke]

Greetings All,

Background (as I'm new to this group...)

As a physical oceanographer, I've been working with Chris Barker here at the U.S. National Oceanic & Atmospheric Administration (NOAA) Emergency Response Division (ERD) to help develop the next generation of the GNOME (General NOAA Operational Modeling Environment) modeling tool ERD uses to predict the possible trajectory a pollutant might follow in or on a body of water, such as in an oil spill.  Our development team is hoping to make the next GNOME release compatible with Network Common Data Form (NetCDF) files that follow the proposed Unstructured Grid (UGRID) data model.

Example

Among a number of other folks' models, ERD uses with GNOME its own in-house hydrodynamic model, Current Analysis for Trajectory Simulations (CATS).  In its present implementation, CATS outputs flat text files (*.CUR) of current patterns on a triangular mesh.  I've attached a figure of one such mesh here as ElevenPoints.png; which shows a "lake" (very coarsely resolved, for the sake of simplicity) with an inset island.  I've converted CATS' output for this mesh into a NetCDF file and attached it as ElevenPoints.cur.nc.

Questions

In the NetCDF file I've included both variables per the UGRID proposal and a variable with some custom GNOME-specific data.  Would you please give me your feedback as to the following? :

• How best custom data ought be included in UGRID NetCDF files
  
• In the Mesh2_face_links variable, for each (triangular) face, the indices of all other faces that share its edge are given.  I.e., for each triangle, up to three other triangles can be neighbors.  This results in a Mesh2_face_links of size nx3... which makes more sense to me (and to GNOME) than were Mesh2_face_links nx2; as nx3 is a complete specification of all the other faces that are linked to each given face.
• however, triangles on the boundary only link to two other faces... so here I've included a flag value of -1 in Mesh2_face_links to specify this link to the boundary.  ...I'm assuming a flag, if specified, can be included in a variable.
• Following a point Chris Barker raised in an earlier post, it would be nice to add a specification for only those edges on the boundary (especially since model results aren't defined on any edges in CATS).  That way I could make clear (via a flag) that the boundary is land (rather than a mix of land and water, as it would be in other cases).

Best regards,

Brian

[Charles Seaton]

Brian,

We here at CMOP are working on converting our output format to CF compliant netcdf. Since we're doing that, I thought I should re-read the list and look at what other people are doing.

I don't know if you've gotten any feedback on your questions off-list, or if feedback at this point is useful to you, but looking through your example file, I noticed a couple of things that seemed worth mentioning. I am far from an authority on this, these are just my thoughts:

1) I think your variables need "coordinates: face_x face_y" and that you need to define and populate variables "face_x" and "face_y"
        Actually, the standard seems unclear on this: face_x and face_y are optional variables, and it isn't clear if they are "required optional variables" or just optional. By analogy with the CF structure grid convention, these coordinate variables would be "required optional." Since a "coordinates" attribute is required in the structured grid convention, and data on the faces would need these face coordinate variables to meet that requirement.

2) Is the variable Mesh2_DAGtree containing the  "directional_acyclic_graph_tree" the custom data you mention? If so, I don't think it should have a cf_role attribute.

I agree that the mesh face links with dimensions nMesh2_face, nMaxMesh2_face_nodes  seems preferable to the nMesh_face_links, Two structure.

The location index set section of the standard looks like it handles the nodes on the boundary issue, although we haven't tried using it.

Charles

Charles Seaton
Research Associate
Center for Coastal Margin Observation and Prediction
Oregon Health and Science University

--
You received this message because you are subscribed to the Google Groups "UGRID Interoperability" group.
To view this discussion on the web visit https://groups.google.com/d/msg/ugrid-interoperability/-/2wcow1wLG94J.
To post to this group, send email to ugrid-inter...@googlegroups.com.
To unsubscribe from this group, send email to ugrid-interoperab...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ugrid-interoperability?hl=en.

[Christopher Barker]

On Thu, Feb 7, 2013 at 4:05 PM, Charles Seaton <cse...@stccmop.org> wrote:

> I don't know if you've gotten any feedback on your questions off-list,

Actually, not really -- hopefully we can keep the discussion going here.

> or if
> feedback at this point is useful to you,

Sure is -- this is still draft for us.

> 1) I think your variables need "coordinates: face_x face_y" and that you
> need to define and populate variables "face_x" and "face_y"
> Actually, the standard seems unclear on this: face_x and face_y are
> optional variables, and it isn't clear if they are "required optional
> variables" or just optional. By analogy with the CF structure grid
> convention, these coordinate variables would be "required optional." Since a
> "coordinates" attribute is required in the structured grid convention, and
> data on the faces would need these face coordinate variables to meet that
> requirement.

I see this early in the docs:

double Mesh2_face_x(nMesh2_face) ;
Mesh2_face_x:standard_name = "longitude" ;
Mesh2_face_x:long_name = "Characteristics longitude of
2D mesh face." ;
Mesh2_face_x:units = "degrees_east" ;
Mesh2_face_x:bounds = "Mesh2_face_xbnds" ;

I interpret that to mean that face_x and face_y are for defining a
particular point that the value should be applied at. However, in this
case, the velocity is computed in the FE model as essentially the
average velocity over the triangle -- so it doesn't apply at any
particular point within that triangle, but rather over the whole face.
If we did create face_x and face_y variables, they would probably be
arbitrarily assigned to the centroids of the triangles, but as it
would be arbitrary, why not leave that to the end-user, of they need a
single point.

So I think "optional" is correct for these.

> 2) Is the variable Mesh2_DAGtree containing the
> "directional_acyclic_graph_tree" the custom data you mention? If so, I don't
> think it should have a cf_role attribute.

yup -- and you're right -- no cf_role, there is nothing in CF about
our particular data structure.

> The location index set section of the standard looks like it handles the
> nodes on the boundary issue, although we haven't tried using it.

Our use-case is edges on a boundary, rather than nodes, and it's not
totally clear to me that location index set is the way to go for that
-- though it could work. Our main issue is that defining all the
edges, just so we can assign information to a few of them seems
unnecessary. All the edges are implicitly defined by
face_node_connectivity anyway, if the end-user-applicaiton requires
that.

Defining the full set of edges, then defining the subset we need with
a index_set just seems like overkill.

Thanks again for engaging the conversation.

-Chris

> To unsubscribe from this group and stop receiving emails from it, send an
> email to ugrid-interoperab...@googlegroups.com.

>
> To post to this group, send email to
> ugrid-inter...@googlegroups.com.

> Visit this group at
> http://groups.google.com/group/ugrid-interoperability?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Christopher Barker, PhD

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

[Charles Seaton]

Chris,

The main questions are:

1) If face_x and face_y are truly optional variables, and data is defined on faces, what should be used for the value of the coordinates attribute, or should the coordinates attribute be omitted?

2) Is the location index set intended to be sufficient to handle variables defined on boundary edges and is the inconvenience of having to define the full edge_node_connectivity in order to subset it considered an acceptable disadvantage?

My own thoughts:

For the face_x and face_y question, your reasoning makes sense that the values are not associated with the center of the face, but rather with the face, and that an application can calculate the centroid for display purposes from the face_nodes variable. However, at least as I understand it, the CF standard requires that variables that reference a location have a "coordinates" attribute that references the variables that contain the lat and lon location of the data. In the examples given in the UGRID standard, the face located volume data had a coordinates attribute that references face_x and face_y


    Mesh2_volumes:coordinates = "Mesh2_face_x Mesh2_face_y" ;



and nothing notes that as optional. So the UGRID standard seems unclear whether the coordinates attribute is optional for face or edge based variables, or what the coordinates attribute should reference if face_x and face_y are not defined, or if face_x and face_y are "optional required" variables.

For the boundary edges question, the location index set seems to only have the problem that it requires having a full edge_node_connectivity variable in order to be able to subset the relevant face edges, which is mostly a problem because of file size bloat.

The three options that come to my mind are:

1) Under the existing spec, the grid data can be stored in a separate netcdf file and merged with the data netcdf files via ncml.

This means that the space lost to including the extra mesh information is not large (since the mesh information only needs to be stored once and then used with multiple data files).

2) The spec could be modified so that the edge_node_connectivity variable could be incomplete and only contain edges that were used by a variable.

This has the disadvantage that the edge_node_connectivity variable could not be reliably used as a substitute for regenerating the edge_node_connectivity relationship (but that can be regenerated from the face_node_connectivity variable).

3) The concept of the grid boundary could be explicitly included in the UGRID standard (with an optional required attribute: "boundary_node_connectivity") and then variables could be defined with location = "boundary"

The third option seems more intuitive to me than the second option (since it breaks out a special case rather than having one name serve dual purposes), but it would not handle cases where there is a small subset of edges that have variables defined on them, but those edges are not mesh boundaries (such as weirs and gates).

Charles

[Christopher Barker]

On Fri, Feb 8, 2013 at 5:50 PM, Charles Seaton <cse...@stccmop.org> wrote:
> The main questions are:
>
> 1) If face_x and face_y are truly optional variables, and data is defined on
> faces, what should be used for the value of the coordinates attribute, or
> should the coordinates attribute be omitted?

I've always been a little fuzzy on exactly what the requirements are
for the coordinates attribute are. But my instict is to say omit it in
this case -- clients that understand UGRID would use the:

:location = "face"

attribute no know the data is on the faces.

> 2) Is the location index set intended to be sufficient to handle variables
> defined on boundary edges and is the inconvenience of having to define the
> full edge_node_connectivity in order to subset it considered an acceptable
> disadvantage?

It's a matter of how the application thinks about the grid -- in our
case "edges" actually have no meaning, except where we have defined
boundary conditions -- I guess it' s not that big a deal to create
them just for dumping in the output file, but I don't see the point,
either. And as a rule, i don't like redundant info in data files --
the edges themselves are redundant with the cells, unless the
application at hand is using them for something.

> For the face_x and face_y question, your reasoning makes sense that the
> values are not associated with the center of the face, but rather with the
> face, and that an application can calculate the centroid for display
> purposes from the face_nodes variable. However, at least as I understand it,
> the CF standard requires that variables that reference a location have a
> "coordinates" attribute that references the variables that contain the lat
> and lon location of the data.

right, but in this case there is no point location for the data -- I
guess I need to dig in a bit more to what CF does with cell data in
regular grids, but what I've seen assumes that the cells have an
simple relationship to surrounding points.

> In the examples given in the UGRID standard,
> the face located volume data had a coordinates attribute that references
> face_x and face_y
>
> Mesh2_volumes:coordinates = "Mesh2_face_x Mesh2_face_y" ;
>
> and nothing notes that as optional. So the UGRID standard seems unclear
> whether the coordinates attribute is optional for face or edge based
> variables, or what the coordinates attribute should reference if face_x and
> face_y are not defined, or if face_x and face_y are "optional required"
> variables.

yup -- that's what this discussion is for -- to clarify the standard!

> For the boundary edges question, the location index set seems to only have
> the problem that it requires having a full edge_node_connectivity variable
> in order to be able to subset the relevant face edges, which is mostly a
> problem because of file size bloat.

and producing apps need to create the information, perhaps only for
this purpose.

> 1) Under the existing spec, the grid data can be stored in a separate netcdf
> file and merged with the data netcdf files via ncml.
>
> This means that the space lost to including the extra mesh information is
> not large (since the mesh information only needs to be stored once and then
> used with multiple data files).

True -- and the size of the extra data is probably pretty small
compared to the data attached to the mesh anyway.

> 2) The spec could be modified so that the edge_node_connectivity variable
> could be incomplete and only contain edges that were used by a variable.
>
> This has the disadvantage that the edge_node_connectivity variable could not
> be reliably used as a substitute for regenerating the edge_node_connectivity
> relationship

yes -- is this a common need? to use the full edge-node-connectivity
when the original data set doesn't need it?

> 3) The concept of the grid boundary could be explicitly included in the
> UGRID standard (with an optional required attribute:
> "boundary_node_connectivity") and then variables could be defined with
> location = "boundary"

hmm -- actually, I like this one - boundaries are very important to
us, and often model output does not include them. Perhaps having them
an explicit part of the standard would encourage folks to include
them. I had been thinking that boundary info was just a special case
of info-on-the-edges, but maybe it should be explicit.

[note: in any case some standards for variable and attribute names for
boundary info would be great]

> but it would not handle cases where there is a small subset of edges that
> have variables defined on them, but those edges are not mesh boundaries
> (such as weirs and gates).

yeah, that's a problem -- we do need to cover those cases -- an edge
can be a "boundary condition" without being the actual boundary of the
mesh -- but maybe that's OK -- the boundary edges do not need to be
the outer boundary of the mesh.

-Chris

[Brian Zelenke]

Hi Charles,

Thank you for your feedback!  First, ditto for me on Chris Barker's replies.  Also, in regards to item #2 of your reply; yes, the variable "Mesh2_DAGtree" in the example NetCDF file I uploaded containing the "directional_acyclic_graph_tree" represents the kind of custom data I was referring to in my initial post.  I agree that a cf_role attribute shouldn't be contrived for custom (e.g., program-specific) data such as these, and to that end attached is a revised example NetCDF file which omits a cf_role for the custom Mesh2_DAGtree variable.


P.S. -- If anyone knows of a list of "official" Climate & Forecast (CF) metadata roles (cf_role) would you please let me know?

Cheers,
Brian



----------------------------------------------------------------------------------------
   Brian Zelenke
   Oceanographer

   National Oceanic & Atmospheric Administration (NOAA)

   National Ocean Service | Office of Response & Restoration
   7600 Sand Point Way NE
   Seattle, WA 98115-6349

   Phone:  (206)-526-6353
   Fax:  (206)-526-6329
   E-mail:  brian....@noaa.gov
   Web:  http://response.restoration.noaa.gov/

To unsubscribe from this group, send email to ugrid-interoperability+unsub...@googlegroups.com.