ATTN Michael McDonald hycom pydap issue

[Jason Snyder]

I have some python code in which I am trying to download hycom data and then obtain the date of the hycom data:

#!/usr/bin/python
from pydap.client import open_url
import pickle
import shutil
import sys

url='http://tds.hycom.org/thredds/dodsC/datasets/GLBy0.08/expt_93.0/data/forecasts/hycom_glby_930_2021081112_t012_uv3z.nc?time[0:1:0],tau[0:1:0],depth[0:1:2],lat[0:1:4000],lon[0:1:4499],water_u[0:1:0][0:1:2][0:1:4000][0:1:4499],water_v[0:1:0][0:1:2][0:1:4000][0:1:4499]'

print url
dataset = open_url(url)

date = dataset['time']
dates = date[0].data
print "date is ",dates

However when I run the program I get the following for the date:

date is  A �

How do I tweak this program so as to get an actual numerical value for the variable dates and have the output be:

date is 189456.0

Thank you for your time.

Jason Snyder

[Michael McDonald]

Jason,

Issue #1: "forecasts" is transient and we only retain the "full
forecast" runs for the last ~5 runs/days. So since it is now > 5 days
old, this forecast run has been scrubbed, but the t000~t023 file have
been saved/moved to "hindcasts, e.g.,

url='http://tds.hycom.org/thredds/dodsC/datasets/GLBy0.08/expt_93.0/data/hindcasts/2021/hycom_glby_930_2021081112_t012_uv3z.nc'


Issue #2: Try doing the actual subsetting in python and not via the OPENDAP URL,

i.e., your URL should be
url='http://tds.hycom.org/thredds/dodsC/datasets/GLBy0.08/expt_93.0/data/hindcasts/2021/hycom_glby_930_2021081112_t012_uv3z.nc'

Then do subsetting of this dataset via python methods/functions.

> Snyder

[Jason Snyder]

On Wed, Aug 25, 2021 at 12:40 PM Jason Snyder <jmss...@ucdavis.edu> wrote:

I am running the following opendap python program and it is running extremely slow when it gets to the loop. 

#!/usr/bin/python
from pydap.client import open_url

from datetime import *

import pickle
import shutil
import sys

def my_range(start, end, step):
        while start <= end:
                yield start
                start += step


url='http://tds.hycom.org/thredds/dodsC/datasets/GLBy0.08/expt_93.0/data/forecasts/hycom_glby_930_2021082312_t024_uv3z.nc'
url2='http://tds.hycom.org/thredds/dodsC/datasets/GLBy0.08/expt_93.0/data/forecasts/hycom_glby_930_2021082312_t024_ts3z.nc'
dataset=open_url(url)
dataset2=open_url(url2)

print("list is ",list(dataset.keys()))
date = dataset['time']
print("shape is ",date.shape)
dateT = int(str(date[0].data)[1:-2])
print("date is ",dateT)
dttme=datetime.strptime('2000-01-01 00:00:00', '%Y-%m-%d %H:%M:%S') + timedelta(hours=dateT)
dttme2=datetime.strftime(dttme, "%Y%m%d%H")
print("The date being processed is ",dttme2)

xl=0
xu=4499
yl=0
yu=4000



water_u = dataset['water_u']
water_v = dataset['water_v']
water_temp = dataset2['water_temp']

print("1) ssu file ....")
text_file = open("ssu_"+dttme2+".xyz","w")
for j in my_range(int(yl),int(yu),2):
    for i in my_range(int(xl),int(xu),2):
        latt=int(j)
        lonn=int(i)
        u=float(str(water_u[0,2,latt,lonn][:].array.data)[4:-4])/1000
        ustr=str(u)
        text_file.write(ustr+"\n")

 What can I do to get this to run faster?  The wget works fairly rapidly for whole netcdf files but I only need the first top levels of the hycom data and not the whole netcdf file so that is why I am using opendap.

Thank you for your help.

Jason

--

Jason Snyder PhD

--

Jason Snyder PhD

[Tim Chappell]

Jason - Quick Question. How are you using wget to get the netcdf files. I need to do the same and cant figure it out with python.

-- CONFIDENTIALITY NOTICE: This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to which they are addressed. This communication may contain material protected by HIPAA legislation (45 CFR, Parts 160 & 164) or by 42 CFR Part 2. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. If you have received this email in error, please notify the sender by reply email and destroy all copies of the original message.