datasets.xml for ERDDAP local server

[Hugo Gómez Sabucedo]

I am trying to setup a local ERDDAP server with data from different atmospheric parameters, witht the directory structure as follows. For each folder, I want to create a different dataset with all the files. Each of the files has the following fields: CO_column_number_density, latitude, latitude_bounds, longitude, longitude_bounds.

I have defined the datasets.xml file as follows; however, when I enter the ERDDAP server, the datasets list is empty.

[Hugo Gómez Sabucedo]

I see that the images aren't showing, apologies. The structure of the files is the following:

And the datasets.xml is the following:

[bobsimons2.00]

I see problems with the XML file that make the dataset not valid. Notably

• The dataset type must be a specifc type, e.g., "EDDGridFromNcFiles". "griddap" is not a valid option.
• <title>, <summary>, <cdm_data_type> must be within <addAttributes>.
• There are no <dataVariable>'s defined. There must be 1 or more.

In general, I recommend 

1. Use GenerateDatasetsXml to generate the first pass of the chunk of datasets.xml for a dataset.
2. Edit that result by hand if you want to make changes. 
3. Use DasDds to test the dataset. If there is a problem with it, you will see an error message telling you the problem.  
4. Once the dataset works in DasDds, it will work in ERDDAP. 

See https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html#Tools

I hope that helps.

Thanks for using ERDDAP

[bobsimons2.00]

I should have also said:

Here is the list of dataset types: https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html#datasetTypes

[Hugo Gómez Sabucedo]

Hi,

I corrected the XML file to the one attached. However, I'm still not able to see the datasets. The DasDds file gives me the following output. Maybe I am missing something in the XML file?

[Roy Mendelssohn - NOAA Federal]

The log file finds problems in trying to aggregate the datasets. One problem can likely be fixed if you change:

<matchAxisNDigits>20</matchAxisNDigits>

to:

<matchAxisNDigits>12</matchAxisNDigits>

say. The other error is it looks like you are trying to aggregate the datasets over time, but there is no time variable in the files (to be certain can you send a header dump from one of the files). You can create a time variable from the file name.go to:

https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html

and search for timeFormat. You create a new time variable using regex as in the at that link. If you have problems can send further examples of how to use timeFormatl

HTH,

-Roy

PS - And thanks for sending the files. It makes it much easier to see what is happening.

> On Jan 23, 2024, at 9:56 AM, Hugo Gómez Sabucedo <hugo0...@gmail.com> wrote:
>
> Hi,
> I corrected the XML file to the one attached. However, I'm still not able to see the datasets. The DasDds file gives me the following output. Maybe I am missing something in the XML file?
>
> El martes, 23 de enero de 2024 a las 16:47:28 UTC+1, bobsimons2.00 escribió:
> I should have also said:
> Here is the list of dataset types: https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html#datasetTypes
>
>
> On Tuesday, January 23, 2024 at 10:25:16 AM UTC-5 bobsimons2.00 wrote:
> I see problems with the XML file that make the dataset not valid. Notably
> • The dataset type must be a specifc type, e.g., "EDDGridFromNcFiles". "griddap" is not a valid option.
> • <title>, <summary>, <cdm_data_type> must be within <addAttributes>.
> • There are no <dataVariable>'s defined. There must be 1 or more.
> In general, I recommend
> • Use GenerateDatasetsXml to generate the first pass of the chunk of datasets.xml for a dataset.
> • Edit that result by hand if you want to make changes.
> • Use DasDds to test the dataset. If there is a problem with it, you will see an error message telling you the problem.
> • Once the dataset works in DasDds, it will work in ERDDAP.
> See https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html#Tools
>
> I hope that helps.
> Thanks for using ERDDAP
>
> On Tuesday, January 23, 2024 at 10:15:47 AM UTC-5 hugo0...@gmail.com wrote:
> I see that the images aren't showing, apologies. The structure of the files is the following:
>
>

> And the datasets.xml is the following:
>
>
>

> El martes, 23 de enero de 2024 a las 16:08:18 UTC+1, Hugo Gómez Sabucedo escribió:
> I am trying to setup a local ERDDAP server with data from different atmospheric parameters, witht the directory structure as follows. For each folder, I want to create a different dataset with all the files. Each of the files has the following fields: CO_column_number_density, latitude, latitude_bounds, longitude, longitude_bounds.
>

> I have defined the datasets.xml file as follows; however, when I enter the ERDDAP server, the datasets list is empty.
>
>

> --
> You received this message because you are subscribed to the Google Groups "ERDDAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to erddap+un...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/dfcfdce4-b977-43be-854b-27f96e08d1e8n%40googlegroups.com.
> <DasDds.log><datasets.xml>

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Men...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[Hugo Gómez Sabucedo]

The first part seemed to solve the problem, since now I'm seeing on the main screen that there is 1 Dataset. However, when I access it, I can't make any graph with it, since I get "Query error: There are no valid longitude or latitude values".

The header dump of the files is the following:

basically, I have a bunch of files with data, one for each month, and I want to be able to see them from an interface (I'm designing it in JavaScript) so that I can see, for one month at a time, a map with the values plotted. That's why I think it might not be necessary to aggregate the datasets over time, right?

[Roy Mendelssohn - NOAA Federal]

Yes.  you are one the right track.  You just need to use timeFormat to create the time value.  If you want,  if you can send me two files I will create and example datasets.xml snippet for you.  If you want to combine over variables,  send me two of each type and I will set up an example for that.

HTH,

-Roy

On Jan 23, 2024, at 12:30 PM, Hugo Gómez Sabucedo <hugo0...@gmail.com> wrote:

The first part seemed to solve the problem, since now I'm seeing on the main screen that there is 1 Dataset. However, when I access it, I can't make any graph with it, since I get "Query error: There are no valid longitude or latitude values".

The header dump of the files is the following:

To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/97eb9492-a5c4-4821-9412-c0e916c3c221n%40googlegroups.com.

[Hugo Gómez Sabucedo]

I have to send you a drive link (https://drive.google.com/drive/folders/1J3z-zQUDPZ5yCnQT13CM0B4xPELlsj_C?usp=sharing), since the files exceed the maximum size limit. My idea is to have a dataset for the CO, with said files there, so that you can choose which month to query and show the data from the corresponding file, so I don't think it will be necessary to combine over variables,

[bobsimons2.00]

And regarding the missing time variable: Roy is incorrect. Specifying the timeFormat is insufficient. You have to create a time variable.

Since there is no time variable in the data files (including the metadata), you have to grab the time from the file name and make it into a variable. See

https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html#EDDGridFromFiles_AggregationViaFileNames

[Roy Mendelssohn - NOAA Federal]

The attached file worked in my ERDDAP for all four files. You need to:

1. edit the datasetid to be what you want

2. edit <fileDir>/path/to/your/data/</fileDir> to be the actual path

3. Fill in a lot of the metatdata not filled in, and edit the dataset title.

4. The about figuring out a nice graph, and adding a default graph to the xml snippet

Let me know if you get this to work.

-Roy

[Hugo Gómez Sabucedo]

Hi Roy,

I've copied the file into my datasets.xml and now it is working. I can properly see the dataset and generate a map. Thank you so much for your help, I really appreciate it.