Access to Gulf of Mexico GOM GOMu0.04/expt_90.1m000/FMRC/GOMu0.04_901m000_FMRC

K. J. Schaudt

unread,

Dec 29, 2021, 2:13:29 PM12/29/21

to HYCOM.org Forum

Either old or partial data appear to be available today for GOMu0.04
yesterday and today.

Is there a problem or did other jobs took priority?
https://ncss.hycom.org/thredds/ncss/GOMu0.04/expt_90.1m000/FMRC/GOMu0.04_901m000_FMRC_best.ncd

https://ncss.hycom.org/thredds/ncss/GOMu0.04/expt_90.1m000/FMRC/GOMu0.04_901m000_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=28.34155&longitude=-88.26534&time_start=2021-12-29T00:00:00Z&time_end=2022-01-05T23:00:00Z&accept=csv

Michael McDonald

unread,

Dec 30, 2021, 9:33:51 AM12/30/21

to K. J. Schaudt, HYCOM.org Forum

> Is there a problem or did other jobs took priority?
> https://ncss.hycom.org/thredds/ncss/GOMu0.04/expt_90.1m000/FMRC/GOMu0.04_901m000_FMRC_best.ncd

The 2021-12-29 run looks to be back to the full(er) forecast duration.
There were some outages over the winter break that likely contributed
to the partial runs.

FMRC_RUN_2021-12-29T12:00:00Z
TimeCoverage:
Start: 2021-12-29T12:00:00Z
End: 2022-01-04T00:00:00Z

Kenneth Schaudt

unread,

Mar 24, 2024, 11:44:20 PM3/24/24

to Michael McDonald, HYCOM.org Forum

Am able to apparently obtain single point data in CSV format but the
download of the NCD files fail.   Most recent log is shown below.

Best,

Ken

Last login: Sun Mar 24 22:21:43 on ttys000
(base) cMP12:~ schaudt$ cd
/volumes/sg8gb/users/schaudt/r_projects/roffs_sus/scripts
(base) cMP12:scripts schaudt$ sh
/Volumes/SG8GB/Users/schaudt/R_Projects/ROFFS_SUS/scripts/GOMlores.sh
+ NCKS=ncks
+ DEBUG_LVL=-D0
+ WGET=wget
+ WGET_FLAGS=
+ RETRY_SLEEP=90
+
NCSS_BASE_URL=http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS=var=salinity,water_temp,water_u,water_v
+ LAT='&latitude=26.65696083'
+ LON='&longitude=-91.57923286'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z
+ TIME_START='&time_start=2024-03-25T00:00:00Z'
++ TZ=GMT-167
++ date +%Y-%m-%dT23:00:00Z
+ TIME_END='&time_end=2024-04-01T23:00:00Z'
+ ACCEPT='&accept=csv'
+
NCSS_FULL_URL='http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-03-25T00:00:00Z&time_end=2024-04-01T23:00:00Z&accept=csv'
+ echo 'Site specific forecasts of conditions: acquiring data'
Site specific forecasts of conditions: acquiring data
+ OUT_FILE=../data/WR515.csv
+ true
+ date
Sun Mar 24 22:33:55 CDT 2024
+ wget
'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-03-25T00:00:00Z&time_end=2024-04-01T23:00:00Z&accept=csv'
-O ../data/WR515.csv
--2024-03-24 22:33:55--
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-03-25T00:00:00Z&time_end=2024-04-01T23:00:00Z&accept=csv
Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: ‘../data/WR515.csv’

../data/WR515.csv [ <=>                                          ]
232.16K 1.24KB/s    in 2m 49s

2024-03-24 22:36:51 (1.38 KB/s) - ‘../data/WR515.csv’ saved [237731]

+ break
+
NCSS_BASE_URL=http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS=var=salinity,water_temp,water_u,water_v
+ LAT='&latitude=28.040'
+ LON='&longitude=-89.080'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z
+ TIME_START='&time_start=2024-03-25T00:00:00Z'
++ TZ=GMT-167
++ date +%Y-%m-%dT23:00:00Z
+ TIME_END='&time_end=2024-04-01T23:00:00Z'
+ ACCEPT='&accept=csv'
+
NCSS_FULL_URL='http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=28.040&longitude=-89.080&time_start=2024-03-25T00:00:00Z&time_end=2024-04-01T23:00:00Z&accept=csv'
+ echo 'Site specific forecasts of conditions: acquiring data'
Site specific forecasts of conditions: acquiring data
+ OUT_FILE=../data/Titan.csv
+ true
+ date
Sun Mar 24 22:36:51 CDT 2024
+ wget
'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=28.040&longitude=-89.080&time_start=2024-03-25T00:00:00Z&time_end=2024-04-01T23:00:00Z&accept=csv'
-O ../data/Titan.csv
--2024-03-24 22:36:51--
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=28.040&longitude=-89.080&time_start=2024-03-25T00:00:00Z&time_end=2024-04-01T23:00:00Z&accept=csv
Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: ‘../data/Titan.csv’

../data/Titan.csv [ <=> ] 205.03K 5.59KB/s    in 39s

2024-03-24 22:37:36 (5.32 KB/s) - ‘../data/Titan.csv’ saved [209952]

+ break
+
http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/runs/GLBy0.08_930_FMRC_RUN_2023-01-31T12:00:00Z
/Volumes/SG8GB/Users/schaudt/R_Projects/ROFFS_SUS/scripts/GOMlores.sh:
line 71:
http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/runs/GLBy0.08_930_FMRC_RUN_2023-01-31T12:00:00Z:
No such file or directory
+
DAP_URL=http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS='-v water_temp,water_u,water_v'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z
+ START_DATE=2024-03-25T00:00:00Z
++ TZ=GMT-144
++ date +%Y-%m-%dT23:00:00Z
+ END_DATE=2024-03-31T23:00:00Z
+ TIMERANGE='-d time,2024-03-25T00:00:00Z,2024-03-31T23:00:00Z'
+ LAT_RANGE='-d lat,20.,35.'
+ LON_RANGE='-d lon,260.,280.'
+ echo 'app1: acquiring 0m data'
app1: acquiring 0m data
+ DEPTH_METERS=0
+ DEPTH='-d depth,0.'
+ OUT_FILE=../data/GLBy0.08_930_FMRC_tuv_0m.nc4
+ true
+ date
Sun Mar 24 22:37:36 CDT 2024
+ ncks -D0 -d time,2024-03-25T00:00:00Z,2024-03-31T23:00:00Z -v
water_temp,water_u,water_v -d lat,20.,35. -d lon,260.,280. -d depth,0.
http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
-O ../data/GLBy0.08_930_FMRC_tuv_0m.nc4
oc_open: server error retrieving url: code=? message="Error {
    code = 500;
    message = "java.io.IOException: Stale file handle";
}"ERROR: nco_get_vara() failed to nc_get_vara() variable "water_temp"
nco_err_exit(): ERROR Short NCO-generated message (usually name of
function that triggered error): nco_get_vara()
nco_err_exit(): ERROR Error code is -90. Translation into English with
nc_strerror(-90) is "NetCDF: file not found"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
+ echo 'ERROR: sleeping for 90 seconds and then trying again...'
ERROR: sleeping for 90 seconds and then trying again...
+ sleep 90
+ true
+ date
Sun Mar 24 22:39:08 CDT 2024
+ ncks -D0 -d time,2024-03-25T00:00:00Z,2024-03-31T23:00:00Z -v
water_temp,water_u,water_v -d lat,20.,35. -d lon,260.,280. -d depth,0.
http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
-O ../data/GLBy0.08_930_FMRC_tuv_0m.nc4
ncks: INFO DAP-access to
http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
failed with error code -73. Translation into English with
nc_strerror(-73) is "NetCDF: Malformed or inaccessible DAP2 DATADDS or
DAP4 DAP response"
ncks: WARNING not searching for
http://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
on remote filesystem, using local file
hredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd instead
ERROR: nco__open() unable to open file
"thredds/dodsC/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd"
ERROR NC_ENOTNC Not a netCDF file
HINT: NC_ENOTNC errors can occur for many reasons. If your use-case
matches one of the five listed below, take the corrective action indicated:
1. The netCDF file is truly corrupt, e.g., truncated. To verify whether
the file is corrupt, try, e.g., printing its metadata with 'ncks -m
in.nc'. This test will likely fail on a corrupt file
2. An NCO operator linked only to the classic netCDF3 library attempts
to read netCDF4 (or HDF5) files. However, this executable seems to have
been built with the capability to manipulate netCDF4 files, so it is
unlikely that this command failed only because the input datasets are
netCDF4 format. Something else is going wrong.
3. NCO attempts to read other filetypes (HDF4, HDF-EOS2, PnetCDF/CDF5)
for which support must be (but was not) enabled at netCDF build-time.
NCO can access HDF4 files if NCO is first re-linked to a version of
netCDF configured with the --enable-hdf4 option. This is a non-standard
netCDF build option described here:
http://www.unidata.ucar.edu/software/netcdf/docs/build_hdf4.html. NCO
can access PnetCDF/CDF5 files if NCO is first re-linked to netCDF
version 4.4.0 or later.
4. NCO attempts to utilize diskless (i.e., RAM) files. In this case
remove the diskless switches (e.g., --ram or --diskless) and then
re-issue the command.
5. Access to a DAP URL fails, and the backup method of downloading the
URL using wget obtains a data aggregation file (e.g., a .ncml file)
instead of an actual netCDF file. In this case the problem is with the
DAP server or URL.
nco_err_exit(): ERROR Short NCO-generated message (usually name of
function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -51. Translation into English with
nc_strerror(-51) is "NetCDF: Unknown file format"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
+ echo 'ERROR: sleeping for 90 seconds and then trying again...'
ERROR: sleeping for 90 seconds and then trying again...
+ sleep 90
+ true
+ date

Michael McDonald

unread,

Mar 29, 2024, 8:32:17 AM3/29/24

to Kenneth Schaudt, HYCOM.org Forum

Please specify the time (and time zone) when you are making these queries.

--
Michael McDonald
HYCOM.org Administrator

Kenneth Schaudt

unread,

Mar 29, 2024, 10:17:40 AM3/29/24

to Michael McDonald, HYCOM.org Forum

Problem resolved itself the next day.

Downloads failed in this fashion from about 7PM CDT to after midnight
when I shut the system down here.

Just an observation, it appears that the download availability has
shifted to after 8PM central. Before the time zone change, that was
after 6PM central. Is that now the expected time?

Michael McDonald

unread,

Apr 10, 2024, 1:25:33 PM4/10/24

to Kenneth Schaudt, HYCOM.org Forum

Ken,
Yes. We have been experimenting with transferring the unpacked netcdf
files directly as they are made available on the NRL compute nodes (vs
waiting for the .tar.gz bundles to be created and moved to the archive
server, this takes about 4-6hrs after 00z). This has made the
"unpacked" files 1st visible on the remote side around 00Z (~8pm EST)
and then it takes about 3-4 hours for the files to transfer in serial
with simple rsync (working on improving this to a faster xfer method).
We have this running manually (at present) as a "backup method" to
complement the cron-scheduled transfers we have used for years.

One big issue we have not folded into this workflow is the "restart"
of the THREDDS services once the new "unpacked" files have been
transferred to hycom.org. We are still letting the 2-4am EST tar.gz
bundle transfers kick off the "rolling restart" of all our hycom
services to update the datasets. This window of time when new data has
arrived and the servers have *not all been restarted* results in the
"time1" issues for the FMRC datasets. We are working on cutting down
this time so that we quickly publish the new data and restart all
services ASAP (no rolling restart, instead all services restart at
once in parallel).

Kenneth Schaudt

unread,

Jul 11, 2024, 9:46:51 PM7/11/24

to Michael McDonald, HYCOM.org Forum

Am having problems with accessing the Gulf of Mexico results at 843PM
Central Daylight time.

Downloads one file for a specific location and then fails for the next
point as shown in the log below.

Don't know if this also affects download of fields files in NC4
format. Any suggestions?

+ NCKS=ncks
+ DEBUG_LVL=-D0
+ WGET=wget
+ WGET_FLAGS=
+ RETRY_SLEEP=90
+
NCSS_BASE_URL=http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS=var=salinity,water_temp,water_u,water_v
+ LAT='&latitude=26.65696083'
+ LON='&longitude=-91.57923286'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z

+ TIME_START='&time_start=2024-07-12T00:00:00Z'

++ TZ=GMT-167
++ date +%Y-%m-%dT23:00:00Z

+ TIME_END='&time_end=2024-07-19T23:00:00Z'
+ ACCEPT='&accept=csv'
+
NCSS_FULL_URL='http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv'

+ echo 'Site specific forecasts of conditions: acquiring data'
Site specific forecasts of conditions: acquiring data
+ OUT_FILE=../data/WR515.csv
+ true
+ date

Thu Jul 11 20:41:21 CDT 2024
+ wget
'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv'
-O ../data/WR515.csv
--2024-07-11 20:41:21--
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:22-- (try: 2)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:24-- (try: 3)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:27-- (try: 4)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:29-- (try: 5)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:30-- (try: 6)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.

Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try: 7)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try: 8)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try: 9)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try:10)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try:11)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try:12)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try:13)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80...

failed: Operation timed out.
Retrying.

--2024-07-11 20:41:30-- (try:14)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:40-- (try:15)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:41:50-- (try:16)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:42:00-- (try:17)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:42:10-- (try:18)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:42:20-- (try:19)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:42:30-- (try:20)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Giving up.

+ echo 'ERROR: sleeping for 90 seconds and then trying again...'
ERROR: sleeping for 90 seconds and then trying again...
+ sleep 90
+ true
+ date

Thu Jul 11 20:44:01 CDT 2024
+ wget
'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv'
-O ../data/WR515.csv
--2024-07-11 20:44:01--
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:02-- (try: 2)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:04-- (try: 3)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:07-- (try: 4)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:11-- (try: 5)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:16-- (try: 6)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:22-- (try: 7)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

--2024-07-11 20:44:29-- (try: 8)
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-12T00:00:00Z&time_end=2024-07-19T23:00:00Z&accept=csv

Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... Read error (Connection reset by
peer) in headers.
Retrying.

Kenneth Schaudt

unread,

Jul 19, 2024, 10:26:31 AM7/19/24

to Michael McDonald, HYCOM.org Forum

Are there server problems?   Been trying to download GOM site data and
field files since 7PM CDT yesterday.   Problems vary but the job hangs
or the following message loops.    Let it run for 30 to 90 minutes
before killing the job and trying again.

Best

Ken

+ NCKS=ncks
+ DEBUG_LVL=-D0
+ WGET=wget
+ WGET_FLAGS=
+ RETRY_SLEEP=90
+
NCSS_BASE_URL=http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS=var=salinity,water_temp,water_u,water_v
+ LAT='&latitude=26.65696083'
+ LON='&longitude=-91.57923286'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z

+ TIME_START='&time_start=2024-07-19T00:00:00Z'

++ TZ=GMT-167
++ date +%Y-%m-%dT23:00:00Z

+ TIME_END='&time_end=2024-07-26T23:00:00Z'
+ ACCEPT='&accept=csv'
+
NCSS_FULL_URL='http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-19T00:00:00Z&time_end=2024-07-26T23:00:00Z&accept=csv'

+ echo 'Site specific forecasts of conditions: acquiring data'
Site specific forecasts of conditions: acquiring data
+ OUT_FILE=../data/WR515.csv
+ true
+ date

Fri Jul 19 09:23:37 CDT 2024
+ wget
'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-19T00:00:00Z&time_end=2024-07-26T23:00:00Z&accept=csv'
-O ../data/WR515.csv
--2024-07-19 09:23:37--
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-19T00:00:00Z&time_end=2024-07-26T23:00:00Z&accept=csv

Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

Kenneth Schaudt

unread,

Jul 19, 2024, 10:27:49 AM7/19/24

to Michael McDonald, HYCOM.org Forum

Still no access to the HYCOM model products.

Kenneth Schaudt

unread,

Sep 3, 2024, 11:54:04 PM9/3/24

to Michael McDonald, HYCOM.org Forum

Are there server problems? Been trying to download GOM site data and

field files since Sunday night. Problems vary but the job hangs or

the following message loops. Let it run for 30 to 90 minutes before
killing the job and trying again.

Best

Ken

+ NCKS=ncks
+ DEBUG_LVL=-D0
+ WGET=wget
+ WGET_FLAGS=
+ RETRY_SLEEP=90
+
NCSS_BASE_URL=http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS=var=salinity,water_temp,water_u,water_v
+ LAT='&latitude=26.65696083'
+ LON='&longitude=-91.57923286'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z

+ TIME_START='&time_start=2024-07-19T00:00:00Z'

++ TZ=GMT-167
++ date +%Y-%m-%dT23:00:00Z

+ TIME_END='&time_end=2024-07-26T23:00:00Z'
+ ACCEPT='&accept=csv'
+
NCSS_FULL_URL='http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-19T00:00:00Z&time_end=2024-07-26T23:00:00Z&accept=csv'

+ echo 'Site specific forecasts of conditions: acquiring data'
Site specific forecasts of conditions: acquiring data
+ OUT_FILE=../data/WR515.csv
+ true
+ date

Fri Jul 19 09:23:37 CDT 2024
+ wget

'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-19T00:00:00Z&time_end=2024-07-26T23:00:00Z&accept=csv'
-O ../data/WR515.csv
--2024-07-19 09:23:37--

http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-07-19T00:00:00Z&time_end=2024-07-26T23:00:00Z&accept=csv

Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

Kenneth Schaudt

unread,

Sep 4, 2024, 8:24:22 PM9/4/24

to Michael McDonald, HYCOM.org Forum

Unable to download either point or fields files since Sunday. Today, I
was only able to download 74kb of a 200kb point data file in an hour.
Tried multiple times and gave up today.

(base) CMPdual:~ schaudt$ cd
/Volumes/SG8GB/Users/schaudt/R_Projects/ROFFS_SUS/scripts
(base) CMPdual:scripts schaudt$ sh
/Volumes/SG8GB/Users/schaudt/R_Projects/ROFFS_SUS/scripts/GOMlores.sh

+ NCKS=ncks
+ DEBUG_LVL=-D0
+ WGET=wget
+ WGET_FLAGS=
+ RETRY_SLEEP=90
+
NCSS_BASE_URL=http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd
+ VARS=var=salinity,water_temp,water_u,water_v
+ LAT='&latitude=26.65696083'
+ LON='&longitude=-91.57923286'
++ TZ=GMT+00
++ date +%Y-%m-%dT00:00:00Z

+ TIME_START='&time_start=2024-09-04T00:00:00Z'

++ TZ=GMT-167
++ date +%Y-%m-%dT23:00:00Z

+ TIME_END='&time_end=2024-09-11T23:00:00Z'
+ ACCEPT='&accept=csv'
+
NCSS_FULL_URL='http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-09-04T00:00:00Z&time_end=2024-09-11T23:00:00Z&accept=csv'

+ echo 'Site specific forecasts of conditions: acquiring data'
Site specific forecasts of conditions: acquiring data
+ OUT_FILE=../data/WR515.csv
+ true
+ date

Wed Sep 4 18:27:53 CDT 2024
+ wget
'http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-09-04T00:00:00Z&time_end=2024-09-11T23:00:00Z&accept=csv'
-O ../data/WR515.csv
--2024-09-04 18:27:53--
http://ncss.hycom.org/thredds/ncss/GLBy0.08/expt_93.0/FMRC/GLBy0.08_930_FMRC_best.ncd?var=salinity,water_temp,water_u,water_v&latitude=26.65696083&longitude=-91.57923286&time_start=2024-09-04T00:00:00Z&time_end=2024-09-11T23:00:00Z&accept=csv

Resolving ncss.hycom.org (ncss.hycom.org)... 144.174.97.9
Connecting to ncss.hycom.org (ncss.hycom.org)|144.174.97.9|:80... connected.

HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: '../data/WR515.csv'

../data/WR515.csv [ <=> ] 71.93K 25.9 B/s

Kenneth Schaudt

unread,

Sep 5, 2024, 11:55:39 PM9/5/24

to Michael McDonald, HYCOM.org Forum

Many thanks, everything is now working again.

Best

Ken Schaudt

Kenneth Schaudt

unread,

Sep 9, 2024, 10:36:20 AM9/9/24

to Michael McDonald, HYCOM.org Forum

Will the new output from ESPC-D v2 be available on Thredds as was the previous model?

Best

Ken Schaudt

Kenneth Schaudt

unread,

Sep 10, 2024, 7:33:46 PM9/10/24

to K. J. Schaudt, HYCOM.org Forum

When will the point data from from ESPC be available by depth via NetCDF?

Michael McDonald

unread,

Sep 11, 2024, 4:23:27 PM9/11/24

to Kenneth Schaudt, HYCOM.org Forum

Kenneth,

Are you referring to the NCSS subset service "Grid as Point Dataset" toggle/option, as this is already available? Please see the root THREDDS catalog here for the updated URLs.

https://tds.hycom.org/thredds/

On Tue, Sep 10, 2024 at 7:33 PM Kenneth Schaudt <kjsc...@schaudt.us> wrote:

When will the point data from from ESPC be available by depth via NetCDF?

Michael McDonald

unread,

Sep 11, 2024, 4:39:13 PM9/11/24

to Kenneth Schaudt, HYCOM.org Forum

Ken,

Yes, the ESPC data is now available. Please see the THREDDS catalog root here,

https://tds.hycom.org/thredds/catalog.html

Folder ESPC-D-V02: Global 1/12° Analysis (Aug-10-2024 to Present with 8-day forecast) --
Folder ESPC-D-V02 ice (1-hrly: sic,sih,siu,siv,sss,sst,ssu,ssv,surtx,surty)/ --
Folder ESPC-D-V02 Sssh (1-hrly: steric_ssh = steric SSH)/ --
Folder ESPC-D-V02 ssh (1-hrly: surf_el = Water Surface Elevation)/ --
Folder ESPC-D-V02 s3z (3-hrly: salinity = Salinity)/ --
Folder ESPC-D-V02 t3z (3-hrly: water_temp = Water Temperature)/ --
Folder ESPC-D-V02 u3z (3-hrly: water_u = Eastward Water Velocity)/ --
Folder ESPC-D-V02 v3z (3-hrly: water_v = Northward Water Velocity)/ --

On Mon, Sep 9, 2024 at 10:36 AM Kenneth Schaudt <kjsc...@schaudt.us> wrote:

Will the new output from ESPC-D v2 be available on Thredds as was the previous model?

Yun Liu

unread,

Sep 12, 2024, 12:52:45 AM9/12/24

to Michael McDonald, Kenneth Schaudt, HYCOM.org Forum

Hi Michael,

I tried to grab some subsets with the NetCDF format from (NetcdfSubset: //ncss.hycom.org/thredds/ncss/grid/ESPC-D-V02). I always downloaded the whole field. Would you please check it?

Thank you very much!

Yun Liu

--
--
You received this message because you are a member of HYCOM.org
To ask a question, send an email to fo...@hycom.org

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@hycom.org.

Kenneth Schaudt

unread,

Sep 12, 2024, 1:27:08 AM9/12/24

to Yun Liu, Michael McDonald, HYCOM.org Forum

Dear Yun liu,

Are you able to get more than one level? It appears that I am only getting one level whether I try to grab the full field in NetCDF format or a single site (gridpoint) as a CSV file.

Best,

Ken

Michael McDonald

unread,

Sep 12, 2024, 4:16:33 PM9/12/24

to Yun Liu, HYCOM.org Forum

Yun Liu,

Please elaborate more on the issue you are having and provide the data URLs you are querying. Sharing a screenshot of your screen/output helps out a great deal. Thanks.

Yun Liu

unread,

Sep 12, 2024, 6:29:32 PM9/12/24

to Michael McDonald, HYCOM.org Forum

Hi Michael,

Here are some details. For example, I want to download the SSH hindcast over the Gulf of Mexico region. From https://ncss.hycom.org/thredds/ncss/grid/ESPC-D-V02/ssh/dataset.html, I can configure my subset of (262-280, 18-31N).

I get the URL of ( https://ncss.hycom.org/thredds/ncss/ESPC-D-V02/ssh?var=surf_el&north=31&west=262&east=280&south=18&disableProjSubset=on&horizStride=1&time=2024-09-10T11%3A00%3A00Z&addLatLon=true&accept=netcdf4).

When I wget the URL.The download file includes the global field, which is the same as I do not configure any subset ( https://ncss.hycom.org/thredds/ncss/ESPC-D-V02/ssh?var=surf_el&disableLLSubset=on&disableProjSubset=on&horizStride=1&time=2024-09-10T11%3A00%3A00Z&addLatLon=true&accept=netcdf4)

I hope I have made myself clear. Thank you very much!

-Yun

Parker MacCready

unread,

Sep 12, 2024, 6:49:24 PM9/12/24

to HYCOM.org Forum, Yun Liu, HYCOM.org Forum, Michael McDonald

When I try a simple wget extraction of a region at one time:

wget https://ncss.hycom.org/thredds/ncss/FMRC_ESPC-D-V02_s3z/FMRC_ESPC-D-V02_s3z_best.ncd?var=salinity&north=53&south=39&west=229&east=239&time=2024-09-10-T003A003A00Z&addLatLon=true&accept=netcdf4

I get a 400 error. Also I cannot get the HYCOM page that Yun Liu was working from above to come up. I was able to get there an construct my own url example earlier today.

Thanks,

Parker

Ivica Janeković

unread,

Sep 12, 2024, 9:20:05 PM9/12/24

to HYCOM.org Forum, HYCOM.org Forum, Michael McDonald

Hi Michael,

seems TDS is playing games. The last (as I'm writing this) model run for the "best time series" is having time_origin = "2024-09-08 12:00:00". Is this falling into "download" and unpack problem?

If I try to get only available time variable using bare bone ncdump (version 4.5.0) then it looks wrong as well:

/opt/netcdf-4.5.0_gnu/bin/ncdump -t -v time_run https://tds.hycom.org/thredds/dodsC/FMRC_ESPC-D-V02_ssh/FMRC_ESPC-D-V02_ssh_best.ncd

result:

time_run = "2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20",
"2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20",
"2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20",
"2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20",
"2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20", "2024-08-20",
"2024-08-21", "2024-08-21", "2024-08-21", "2024-08-21", "2024-08-21",
"2024-08-21", "2024-08-21", "2024-08-21", "2024-08-21",...

, "2024-09-11", "2024-09-11", "2024-09-11",
"2024-09-11", "2024-09-11", "2024-09-11", "2024-09-11", "2024-09-11",
"2024-09-11", "2024-09-11", "2024-09-11", "2024-09-11", "2024-09-11",
"2024-09-11", "2024-09-11" ;

Seems it is not getting time attribute right (not CF standard).

Cheers,

I

Kenneth Schaudt

unread,

Jan 25, 2025, 12:56:39 AM1/25/25

to Michael McDonald, K. J. Schaudt, HYCOM.org Forum

Having problems downloading fields files for the past few days. Multiple time values have returned and the server is not delivering the requested time range.

The downloaded files are several times larger than downloaded several weeks ago. The download script is based on an example from Michael McDonald and has not been changed since 16 December and has been running stably until this past week. The download call submitted to thredds from my script is as follows.

+ ncks -D0 -d time,2025-01-25T00:00:00Z,2025-01-31T23:00:00Z -v water_temp,water_u,water_v -d lat,20.,35. -d lon,260.,280. -d depth,0. https://tds.hycom.org/thredds/dodsC/FMRC_ESPC-D-V02_all/FMRC_ESPC-D-V02_all_best.ncd -O ../data/FMRC_ESPC-D-V02_all_best_0m.nc4

Files downloaded before this past few days had one time variable covering a 6 day window (ITEM 1). Now, there are 2 time variables and the data covers 16 days. As a result, the resulting increase in data is causing the server to fail the download.

Anyone have any suggestions?

ITEM 1: 07 January 2025 download
----- Variables -----
time_run:
    run times for coordinate = time in hours since 2024-12-29 12:00:00.000 UTC
    Dimensions: time
water_temp:
    Water Temperature in degC
    Dimensions: lon by lat by depth by time
water_u:
    Eastward Water Velocity in m/s
    Dimensions: lon by lat by depth by time
water_v:
    Northward Water Velocity in m/s
    Dimensions: lon by lat by depth by time

----- Dimensions -----
depth: 1 values from 0 to 0 m
lat: 376 values from 20 to 35 degrees_north
lon: 251 values from 260 to 280 degrees_east
time: 53 values from 2025-01-08 to 2025-01-14 12:00:00

ITEM 2: 24 January 2025 download

----- Variables -----
time2_run:
    run times for coordinate = time2 in hours since 2025-01-15 12:00:00.000 UTC
    Dimensions: time2
time3_run:
    run times for coordinate = time3 in hours since 2025-01-15 12:00:00.000 UTC
    Dimensions: time3
water_temp:
    Water Temperature in degC
    Dimensions: lon by lat by depth by time2
water_u:
    Eastward Water Velocity in m/s
    Dimensions: lon by lat by depth by time3
water_v:
    Northward Water Velocity in m/s
    Dimensions: lon by lat by depth by time3

----- Dimensions -----
depth: 1 values from 0 to 0 m
lat: 376 values from 20 to 35 degrees_north
lon: 251 values from 260 to 280 degrees_east
time2: 129 values from 2025-01-15 12:00:00 to 2025-01-31 12:00:00
time3: 129 values from 2025-01-15 12:00:00 to 2025-01-31 12:00:00
>

Michael McDonald

unread,

Jan 31, 2025, 5:57:01 PM1/31/25

to Kenneth Schaudt, K. J. Schaudt, HYCOM.org Forum

Ken,
reminder: Please try to keep "like email threads together".

I mentioned this before in another post, but I think that the
"usefulness" of this *experimental* "all" aggregation is not working
out "as expected"... In summary, the "all" aggregation is a "use at
your own risk" dataset.

To help avoid these time#... issues that will always be present in
this "all" aggregation (as it mixes temporal frequencies from 1-hrly
ssh/ice/Sssh and 3-hrly from tsuv) please consider breaking up your
queries like the following into "like datasets" (ones the are packaged
together by the Navy):

note: the water_u and water_v data files arrive in the same tar
bundles. similarly, the water_temp and salinity data files also arrive
in their own tar bundles. since ts and uv are arriving in separate tar
files this can create the time1, time2 etc to deal with the
non-uniform times presented to the FMRC when introduced.

for "water_temp", obtain this from the "ts3z" virtual aggregation here,
https://tds.hycom.org/thredds/catalog/FMRC_ESPC-D-V02_ts3z/catalog.html

you can use the "best" like before with this OPeNDAP URL for obtaining
the best "water_temp" data
https://tds.hycom.org/thredds/dodsC/FMRC_ESPC-D-V02_ts3z/FMRC_ESPC-D-V02_ts3z_best.ncd

same goes for your "water_u" and "water_v" variables, get these from
the uv3z aggregation here,
https://tds.hycom.org/thredds/catalog/FMRC_ESPC-D-V02_uv3z/catalog.html

you can use the "best" like before with this OPeNDAP URL for obtaining
the best "water_u" and "water_v" data
https://tds.hycom.org/thredds/dodsC/FMRC_ESPC-D-V02_uv3z/FMRC_ESPC-D-V02_uv3z_best.ncd

rarely there will be time1 issues (specifically, in the ts3z and uv3z
aggregations) between the hours from 4 AM to 12 PM EST when the new
data is unpacking and getting published, but typically these get
cleared quickly with a triggered FMRC update to "read the new data".

e.g., sub in the above for your example.

+ ncks -D5 -d time,2025-01-25T00:00:00Z,2025-01-31T23:00:00Z -v
water_temp -d lat,20.,35. -d lon,260.,280. -d depth,0.
"http://tds.hycom.org/thredds/dodsC/FMRC_ESPC-D-V02_ts3z/FMRC_ESPC-D-V02_ts3z_best.ncd"
-O ../data/FMRC_ESPC-D-V02_ts3z_best_0m.nc4

+ ncks -D5 -d time,2025-01-25T00:00:00Z,2025-01-31T23:00:00Z -v

water_u,water_v -d lat,20.,35. -d lon,260.,280. -d depth,0.

"http://tds.hycom.org/thredds/dodsC/FMRC_ESPC-D-V02_uv3z/FMRC_ESPC-D-V02_uv3z_best.ncd"
-O ../data/FMRC_ESPC-D-V02_uv3z_best_0m.nc4

Ken Schaudt

unread,

Feb 3, 2025, 12:09:51 AM2/3/25

to HYCOM.org Forum, Michael McDonald, K. J. Schaudt, HYCOM.org Forum

Thanks

Reply all

Reply to author

Forward

Access to Gulf of Mexico GOM GOMu0.04/expt_90.1m000/FMRC/GOMu0.04_901m000_FMRC_best.ncd

K. J. Schaudt

Michael McDonald

Kenneth Schaudt

Michael McDonald

Kenneth Schaudt

Michael McDonald

Kenneth Schaudt

Kenneth Schaudt

Kenneth Schaudt

Kenneth Schaudt

Kenneth Schaudt

Kenneth Schaudt

Kenneth Schaudt

Kenneth Schaudt

Michael McDonald

Michael McDonald

Yun Liu

Kenneth Schaudt

Michael McDonald

Yun Liu

Parker MacCready

Ivica Janeković

Kenneth Schaudt

Michael McDonald

Ken Schaudt