Earth2Observe Runoff Reanalysis Data

[Georgy Ayzel]

Dear all,

Dear Ben,

I am struggling to download Water Resources Reanalysis v1 (wrr1) data from https://wci.earth2observe.eu/data/dataset.

Could you help me to identify the error in my workflow?

1. I go to the webpage of the reanalysis I am interested in, e.g., Runoff from HBV-SIMREG: https://wci.earth2observe.eu/data/dataset/jrc_total_runoff_daily-hbv

2. Then, I go to the respective OpENDAT End Point: https://wci.earth2observe.eu/thredds/dodsC/jrc/wrr1-hbv/e2o_jrchbv_wrr1_glob30_day_Runoff_1979-2012.nc.html

3. There I tick "lon" , "lat", "time" and "runoff"

4. For "runoff" I specify slice options.

Output of the request then:

DDS:

Dataset {
    Float64 lon[lon = 720];
    Float64 lat[lat = 360];
    Float64 time[time = 12419];
    Grid {
     ARRAY:
        Float32 Runoff[time = 12419][lat = 360][lon = 720];
     MAPS:
        Float64 time[time = 12419];
        Float64 lat[lat = 360];
        Float64 lon[lon = 720];
    } Runoff;
} jrc/wrr1-hbv/e2o_jrchbv_wrr1_glob30_day_Runoff_1979-2012.nc;

5. Then "Get ASCII" works flawlessly and produces meaningful output.

6. But "Get Binary" leads to .dods file downloading, not .nc.

At the moment, I failed to read .dods file with xarray. ncdump also fails to provide any information (example here: https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2011/msg00009.html).

Thank you!

All the best,

Georgy

[Ben Calton]

Georgy,

 

The binary file that you receive from an ‘Get Binary’ request from Opendap is not, strictly speaking, a netCDF file and so I would not expect you to be able to open it unaltered. You have a couple of options here – I see that you are taking the entire spatial and temporal coverage of the file, so instead you can download the entire original netCDF file. This link provides all variables in the model - http://wci.earth2observe.eu/thredds/catalog/jrc/wrr1-hbv/catalog.html - each variable offers an ‘HTTPServer’ option to download the original file (the full eartH2Observe catalogue in thredds is at http://wci.earth2observe.eu/thredds/catalog-earth2observe.html)

 

This method works well when dealing with relatively small single files (<4GB), but once you get beyond this it becomes less practical. As you are using xarray already you can use the Opendap endpoint to create an xarray dataset or multi file dataset; this allows you to perform spatial and temporal slicing without having to download the entire file. There’s a useful tutorial at https://climate-cms.org/2019/01/18/using-opendap.html and one on multi-file datasets at https://rabernat.github.io/research_computing_2018/xarray-tips-and-tricks.html

 

If there’s anything else that I can help with please shout.

 

Cheers,

Ben

Visit our website www.pml.ac.uk and follow us on Twitter @PlymouthMarine

This message is private and confidential. If you have received this message in error, please notify the sender and remove it from your system. You are reminded that e-mail communications are not secure and may contain viruses; PML accepts no liability for any loss or damage which may be caused by viruses. Our Privacy Notice is available at http://www.pml.ac.uk/System-pages/Privacy_Notice .

Plymouth Marine Laboratory (PML) is a company limited by guarantee registered in England & Wales, company number 4178503. Registered Charity No. 1091222. Registered Office: Prospect Place, The Hoe, Plymouth PL1 3DH, UK. PML Applications Ltd, www.pml-applications.co.uk , is the wholly owned subsidiary of PML, registered as a company in England & Wales, company number 4232750.