HYCOM data download
310 views
Skip to first unread message
Henrike Schmidt
Feb 7, 2017, 12:58:29 PM2/7/17
to fo...@hycom.org
Hello,
I wanted to use the Hycom Reanalysis data of offline trajectory calculation in the Arabian Sea.
So I tried to download the data with the NetcdfSubset to choose depth and area. But it took couple of hours and I need at least ten years of data.
Is there a faster way to get the data e.g. somewhere else, so that I don't have to send a new request for every variable/year/depth. I also found the ftp server, but global data are way too large to store it all for me.
I would be glad for any help!
Best Henrike Schmidt
Michael McDonald
Feb 7, 2017, 5:19:06 PM2/7/17
to Henrike Schmidt, forum
You need to subset "1 day" at a time in order to get a steady response
from our NCSS servers. This is generally true for any NCSS query. NCSS
is designed for small and quick queries, not decades of data. ALL of
the data users request via NCSS is "written" to our servers local
cache disks (which are large but still finite) and then once complete
transfers the output file to you (the client). If too much data is
requested before the scrubbers run, then this scratch space fills up
and NCSS stops working until it gets cleared.
OPeNDAP access methods have no such staging limitations as
subsetted/requested values are immediately streamed back to the user
via the OPeNDAP enabled client/tool. The only limitation is the daily
server restart that we execute daily at 12:30PM EST.
If you are using client side subsetting tools, e.g., ncks, then you
can query as much data as you want within the OPeNDAP constraints.
However, here we also still recommend subsetting "1 day" at a time.
Just create a for loop and then iterate over the date ranges you are
interested in.
Take a look at some of the example NCSS scripts users have used:
https://groups.google.com/a/hycom.org/forum/#!search/ncss$20subset%7Csort:date
and "ncks" subsetting scripts that you can run within a Linux
environment (recommended method)
https://groups.google.com/a/hycom.org/forum/#!search/ncks$20subset%7Csort:date
--
Michael McDonald
HYCOM.org Administrator
from our NCSS servers. This is generally true for any NCSS query. NCSS
is designed for small and quick queries, not decades of data. ALL of
the data users request via NCSS is "written" to our servers local
cache disks (which are large but still finite) and then once complete
transfers the output file to you (the client). If too much data is
requested before the scrubbers run, then this scratch space fills up
and NCSS stops working until it gets cleared.
OPeNDAP access methods have no such staging limitations as
subsetted/requested values are immediately streamed back to the user
via the OPeNDAP enabled client/tool. The only limitation is the daily
server restart that we execute daily at 12:30PM EST.
If you are using client side subsetting tools, e.g., ncks, then you
can query as much data as you want within the OPeNDAP constraints.
However, here we also still recommend subsetting "1 day" at a time.
Just create a for loop and then iterate over the date ranges you are
interested in.
Take a look at some of the example NCSS scripts users have used:
https://groups.google.com/a/hycom.org/forum/#!search/ncss$20subset%7Csort:date
and "ncks" subsetting scripts that you can run within a Linux
environment (recommended method)
https://groups.google.com/a/hycom.org/forum/#!search/ncks$20subset%7Csort:date
Michael McDonald
HYCOM.org Administrator
susan montero salgado
May 15, 2018, 1:17:38 PM5/15/18
to HYCOM.org Forum, hsch...@geomar.de
I have the problem to download hycom data with the following links:
Reply all
Reply to author
Forward
0 new messages