Availanility of GLBy0.08_930_FMRC outputs

55 views
Skip to first unread message

Sarah Barbier

unread,
Apr 7, 2022, 5:43:10 PMApr 7
to HYCOM.org Forum

Dear HYCOM Group ,

I have scheduled in my crontab a task to download GLBy0.08_930_FMRC water_u and water_v forecast outputs through the ncss service (wget).

To assign an hour of execution to my task, I have checked on ftp.hycom.org the deposit hour of the hycom_glby_930_YYYYMMDD12_txxx_uv3z.nc files which is approximately between 21:00UTC and 22:00UTC.

So I decided to schedule the execution at 22:30 UTC .
But when my task run the download, my output file is empty.

Therefore, is there a lag between the deposit hour of the forecasts on ftp.hycom.org and the download from the ncss service (wget) ?
Then, at what time do you suggest me to run my job ?

Thank you in advance for your answers.

Best Regards,

BARBIER Sarah
Meteo France Engineer
Martinique, France

Michael McDonald

unread,
Apr 7, 2022, 6:22:14 PMApr 7
to Sarah Barbier, HYCOM.org Forum
Yes. The files getting written to disk do not trigger a THREDDS
catalog/FMRC update until all servers/workers have been restarted.
This can take a few hours to complete after data updates are written
as we do a rolling pool upgrade and try to not interrupt ongoing
OPENDAP operations. I would add a 2hr delay to your cron job.
> --
> --
> You received this message because you are a member of HYCOM.org
> To ask a question, send an email to fo...@hycom.org
>


--
Michael McDonald
HYCOM.org Administrator

Sarah Barbier

unread,
Apr 8, 2022, 11:12:54 AMApr 8
to HYCOM.org Forum, Michael McDonald, HYCOM.org Forum, Sarah Barbier
Micheal,

Thank you for your answer.
After I have posted my question, I read the information in the conversations header that THREDDS server are restarting between 2.00 and 3.00am .

So I re-scheduled my job yesterday at 4.00 am the next day (this morning) (to download the HYCOM run of yesterday 120000UTC then ) but my file is still empty.

I fear that even at 00.30 am I do not have the hycom run.

Than you for your return.

BARBIER Sarah
Meteo France Engineer

Michael McDonald

unread,
Apr 8, 2022, 6:17:57 PMApr 8
to Sarah Barbier, HYCOM.org Forum
Please share the exact/specific URL that you are attempting to
download from and the methods (for reproducibility) used to properly
diagnose and debug the issue you are having.

Sarah Barbier

unread,
Apr 9, 2022, 10:45:02 AMApr 9
to HYCOM.org Forum, Michael McDonald, HYCOM.org Forum, Sarah Barbier
Please find below the cron task my bash script that is running from this cron task. I was unable to attach the bash script itself.

Everayday at 4.00 am, the code downloads, given a spatial domain, a 7-day water_u water_v hycom forecast of the previous run (day-1)
00 04 * * * /home/dirag_app/SARGASSES/bin/get_surface_current_hycom_data.sh > /dev/null 2>&1

#!/bin/bash
#__Author__                         : ECMPF/DEV
# __Date__                          : 08/02/2022
# __Mise_a_jour__                   : 07/04/2022
# __Version__                       : 1.1
                

# Données Modele
DATAHOME='/data/data_SARGASSES/hycom/'
DATASET='GLBy0.08'
EXPT='93.0'
EXP='930'
FORMAT='netcdf4' #ou netcdf4

# Domaine spatiale
lat_north=25
lat_south=-5
lon_east=350
lon_west=280
depth='0.0'
verticale_step=$( echo $depth | wc -l)

# Domaine temporelle
year_start=`date +%Y`
month_start=`date +%m`
day_start=`date --date '1 day ago' +%d`
hour='12:00:00Z'

year_end=`date --date '6 day' +%Y`
month_end=`date --date '6 day' +%m`
day_end=`date --date '6 day' +%d`

start_date=${year_start}${month_start}${day_start}
end_date=${year_end}${month_end}${day_end}

#Le modele fournit des sorties toutes les 3h mais on souhaite toute les 24h
time_step=8

FORECAST_RUN=${year_start}-${month_start}-${day_start}T${hour}
END_RUN=${year_end}-${month_end}-${day_end}T


echo ' '
echo ' '
echo 'Parametre de requete '
echo '----------------------------'
echo 'lat_north      = '$lat_north
echo 'lat_sud        = '$lat_south
echo 'lon_east       = '$lon_east
echo 'lon_west       = '$lon_west
echo 'depth          = '$depth
echo 'verticale_step = '$verticale_step
echo 'year_start     = '$year_start
echo 'month_start    = '$month_start
echo 'day_start      = '$day_start
echo 'hour           = '$hour
echo 'year_end       = '$year_end
echo 'month_end      = '$month_end
echo 'day_end        = '$end_date
echo 'time_step      = '$time_step
echo 'FORECAST_RUN   = '$FORECAST_RUN
echo 'END_RUN        = '$END_RUN
echo ' '
echo ' '



REQUETE="https://ncss.hycom.org/thredds/ncss/${DATASET}/expt_${EXPT}/FMRC/runs/${DATASET}_${EXP}_FMRC_RUN_${FORECAST_RUN}?var=water_u&var=water_v&north=${lat_north}&west=${lon_west}&east=${lon_east}&south=${lat_south}&disableProjSubset=on&horizStride=${verticale_step}&time_start=${year_start}-${month_start}-${day_start}T12%3A00%3A00Z&time_end=${END_RUN}12%3A00%3A00Z&timeStride=${time_step}&vertCoord=${depth}&addLatLon=true&accept=${FORMAT}"

cd ${DATAHOME}
wget -O hycom_${DATASET}_${EXP}_${start_date}120000_${end_date}120000.nc $REQUETE
echo ' '

exit

Michael McDonald

unread,
Apr 13, 2022, 11:52:14 AMApr 13
to Sarah Barbier, HYCOM.org Forum
Thanks for sharing this code. Please also share your full terminal
output when you run this command manually from the command line.
Reply all
Reply to author
Forward
0 new messages