Creating GridDAP from TableDAP

[Steve Jones]

I have an ERDDAP with a lot of TableDAP datasets, but some users want the data available through GridDAP too. I've been trying to do this with EDDGridFromDap, which mostly works but I end up with two copies of the dataset (one TableDAP and one GridDAP).

I assume there must be a way to set things up so a single dataset contains both the TableDAP and GridDAP versions. If it is, can someone end me a bit of a datasets.xml file so I can see how it should be done please?

Many thanks,

Steve.

[Rich Signell]

Many moons ago we did this at USGS with some ADCP data.   I'm pretty sure we had one netcdf file with the data, but had both TableDAP and GridDAP (virtual) datasets pointing at it.  That's what you *want*, right?
These examples don't seem to exist anymore on the USGS ERDDAP at https://geoport.usgs.esipfed.org/erddap/index.html (in fact there are no GridDAP datasets at all there!), but by web search on "griddap" and "adcp" I found one at https://nwem.apl.washington.edu/erddap/griddap/nemo-ss_adcp.html.    Maybe contact those folks to get their snippet?

[Roy Mendelssohn - NOAA Federal]

HI Steve:

I need more information to clearly understand things. First, can you point me to an example dataset on ERDDAP that has both a Grid and Table of the same data, or at least to the preset table datasets.

Second, I am unclear what it is the people who want to use griddap instead of tabledap are after. Is it a download format that they don't think can be created from tabledap, or just that they prefer the griddap syntax.

Finally, if your question is can you have both a griddap and a tabledap dataset from the same data files without duplicating the files, the answer is yes. If the question is can you have both a griddap and a tabledap dataset under the same datasetid, the answer is no. A lot of tables in ERDDAP are just that, tables, and don't map to grids at all. So writing general code to know when a table can be mapped can become convoluted. I imagine it is possible to do this, but given all the other things we would like to see added to ERDDAP, and given that a solution already exists, albeit imperfect, I doubt this would be high priority.

HTH,

-Roy

> --
> You received this message because you are subscribed to the Google Groups "ERDDAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to erddap+un...@googlegroups.com.
> To view this discussion, visit https://groups.google.com/d/msgid/erddap/e0ddddaa-cfdf-4aba-9d76-2db30201b27fn%40googlegroups.com.

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Men...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[Steve Jones]

That is more or less what I want, and I have achieved it - but I then end up with two datasets for each file - one TableDAP and one GridDAP. I was wondering if it's possible to combine them so that the same dataset contains both TableDAP and GridDAP views.

From what Roy says that isn't possible - so be it.

Roy: For your second question, the user in question has a whole system set up that's based on GridDAP only, and the shortest route to getting them my data was to present my normal TableDAP data as GridDAP. Which is relatively simple if I specify time as the only dimension, and it's good enough for what they need. I don't have an example yet as I'm just noodling around on a local test instance for the minute.

Thanks both for the input!

Steve.

[Roy Mendelssohn - NOAA Federal]

Hi Steve:

Thanks for the followup. Yes that is a good reason to want griddap. When you are done, if you want sends us the link and we can see if we can think of ways to improve the datasets.xml snippet.

As for why it is unlikely you will ever be able to have griddap and tabledap under the same datasetid, each datasetid has a datatype associated with it and a lot of things are generated on the fly based on that datatype. For example, what do I display for https://my_base_url/erddap/myDataset.html if have no way of knowing if it is a table or a grid. There are other things like that. Again, there may be ways of dealing with these sort of issues, but given all the things we need to or would like to add to ERDDAP, I highly doubt that it would be worked on any time soon.

Thanks for the inquiry and I am glad you found a solution.

-Roy

> --
> You received this message because you are subscribed to the Google Groups "ERDDAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to erddap+un...@googlegroups.com.

> To view this discussion, visit https://groups.google.com/d/msgid/erddap/807eea08-3968-4b8d-b163-88489dbc6758n%40googlegroups.com.

[Steve Jones]

Hi Roy,

I've got this running at https://test-erddap.icos-cp.eu/erddap

I built the gridded dataset using EDDGridFromEDDTable. It looks as though it works, but I'm no expert so it's possible I missed something. No doubt my GridDAP user will report things when they get a chance to look, but any feedback from you is also very welcome.

A couple of observations from this:

The GridDAP doesn't have the full F/I/M options in the FGDC, ISO, Metadata column, even though it's built using the same metadata. Is this expected, or should all three be there?

It's a bit of a shame that you have to include the complete <dataset> tag from the TableDAP metadata in the EDDGridFromEDDTable. This results in a lot of duplication and risks the two datasets getting out of sync. It would be nice to just  give EDDGridFromEDDTable the ID of the source dataset, and it pulls in all the info from there. (I'm not asking for it to be implemented any time soon - I also have a big piece of software that has a longer to-do list than can ever possibly be completed...)

Cheers,

Steve.