Is there a problem with the construction of this NetCDF file?

[Derek Broughton]

The attached html is the html output from pydap for my largest dataset.

Though it would appear to provide everything necessary to provide mappable
data, my wms or kml output is always empty. Since I know nothing about NetCDF
files, I'm guessing wildly, but it seems to me that the coads.nc example always
shows in the bounds selection fields, the names of fields actually in the
dataset, while this file only shows a valid time field, but the y & x fields are
simply called "y" and "x", when my positions are actually "nav_lat" and
"nav_lon". Still there seems to be enough information specified in the file to
make these connections - each variable specifies:
associate
time_counter nav_lat nav_lon
and:
axis
TYX
(or TZYX for 4-dimensional data). Is this a proper NetCDF definition? If so,
is it simply something that hasn't been implemented in pydap? And if that's
so, if someone could point me to the right part of the code, I'd be happy to
make a patch handle it, since it's rather necessary to my project :-)
--
derek

[Roberto De Almeida]

The netcdf file should ideally follow the CF conventions (http://cf-pcmdi.llnl.gov/). The KML/WMS response will display any variables whose axes have the "units" attribute set to "degrees_north", and the other, "degrees_east".

In your case, it looks like you have a non-rectangular grid (not sure if this is the right term), since you have a latitude map (nav_lat) and a longitude map (nav_lon) for each pair (x,y). In this case you'd need to regrid the data to a regular grid before generating the maps. This is a problem that has appeared before, and I can help you with this if you want; I believe it would only be necessary to change the WMS response in order for this to work.

--Rob

--
Dr. Roberto De Almeida
http://dealmeida.net/
http://lattes.cnpq.br/1858859813771449
:wq

[Derek Broughton]

On November 11, 2009 15:12:24 Roberto De Almeida wrote:
> The netcdf file should ideally follow the CF conventions (
> http://cf-pcmdi.llnl.gov/). The KML/WMS response will display any variables
> whose axes have the "units" attribute set to "degrees_north", and the
> other, "degrees_east".

Yes, I worked that out, and nav_lon and nav_lat, do.

> In your case, it looks like you have a non-rectangular grid (not sure if
> this is the right term), since you have a latitude map (nav_lat) and a
> longitude map (nav_lon) for each pair (x,y). In this case you'd need to
> regrid the data to a regular grid before generating the maps. This is a
> problem that has appeared before, and I can help you with this if you want;
> I believe it would only be necessary to change the WMS response in order
> for this to work.

Thanks. I think the first thing I want to do is get some verification from the
creators (who haven't talked to me yet, even though they want me to do this
project) that the data is really correct. I really think it's supposed to be
a strictly rectangular grid, and that nav_lat and nav_lon are supposed to _be_
y and x, not dependent on them.

What determines whether the "map" tab on the html response is enabled? It
isn't for any of these datasets.
--
derek

[Roberto De Almeida]

On Wed, Nov 11, 2009 at 8:06 PM, Derek Broughton <de...@pointerstop.ca> wrote:

What determines whether the "map" tab on the html response is enabled?  It

isn't for any of these datasets.

The page uses javascript to inspect the WMS GetCapabilities response at:

  http://server/dataset.wms?REQUEST=GetCapabilities

This is an XML file specifying the available layers. If there is at least one layer the map tab is activated.

--Rob

[Derek Broughton]

That's what I thought, but a walk(dataset, GridType) in python returned a
list, which should be the layers. I think that just proves that the datasets
aren't really valid, because the filter isn't finding any actual Lat/Lon info.
--
derek

[Roberto De Almeida]

On Thu, Nov 12, 2009 at 8:48 AM, Derek Broughton <de...@pointerstop.ca> wrote:

That's what I thought, but a walk(dataset, GridType) in python returned a

list, which should be the layers.  I think that just proves that the datasets
aren't really valid, because the filter isn't finding any actual Lat/Lon info.

True. The layers are found via:

  from pydap.responses.wms import is_valid

  layers = filter(is_valid, walk(dataset, GridType))

So if your dataset is really non-rectangular we'd need to change the is_valid method (and change its signature too, because it needs to inspect other variables in the dataset; so perhaps this will change to:

  layers = filter(lambda g: is_valid(g, dataset), walk(dataset, GridType))

). And also change the part of the code where the data is plotted with matplotlib, detecting the non-rectangular data and regridding properly.

Best,

--Rob

[Derek Broughton]

I turns out that the data has been stored with integral indexes for x&y which
map to actual lon/lat values as described by:

For all values x, y (and possibly deptht):

> votemper(x,y,deptht,time_counter) where
> lat = nav_lat(x,y)
> long = nav_long(x,y)
> depth = deptht(deptht)
> t = time_counter(time_counter)

So I'm going to look into how I can possibly extract that - generically - from
the data.
--
derek