wget not subsetting correctly using NCSS

[Sam Kastner]

Hello,

I'm trying to download output from expt_53.X in the western Caribbean sea for the entire run of the experiment (1994-2015). I'm doing this one time step and one variable at a time to avoid overly large requests, and then concatenating locally. I'm primarily a Matlab user, so I've used Matlab to generate the list of urls I need to do this. I've tried to do the download two ways: (1) using Matlab's built-in websave function, and (2) using a shell escape and wget. (1) works correctly, but is slow. (2) is fast, but downloads the entire model domain, not just my region of interest, which would create storage problems for me over the entire time period. Here's the url I'm using for an example time step and surface elevation:

http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/1994?var=surf_el&north=18.5&west=276&east=289&south=8&disableProjSubset=off&horizStride=1&time=1995-01-01T09%3A00%3A00Z&vertCoord=&accept=netcdf4

My syntax with websave is:

websave('examplefilename_websave.nc','http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/1994?var=surf_el&north=18.5&west=276&east=289&south=8&disableProjSubset=off&horizStride=1&time=1995-01-01T09%3A00%3A00Z&vertCoord=&accept=netcdf4');

and my syntax with wget is:

wget -O examplefilename_wget.nc http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/1994?var=surf_el&north=18.5&west=276&east=289&south=8&disableProjSubset=off&horizStride=1&time=1995-01-01T09%3A00%3A00Z&vertCoord=&accept=netcdf4

Thanks for your time!

Sam

[Sam Kastner]

To be clear, I'm trying to understand why wget and websave don't produce the same results. I'd prefer to use wget, but not if I have to download the full grid's worth of output.

[Michael McDonald]

Sam,

A very common tale. Why didn't you enclose your wget 'URL' in single
quotes as you did with websave in MATLAB?

Several of those characters in the URL (e.g., &) are reserved in UNIX,
therefore the actual equivalent URL you are making with "wget" stops
(likely) after the 1st ampersand is encountered, hence the reason why
you get the entire domain for all time.

e.g.,
wget -O examplefilename_wget.nc
http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/1994?var=surf_el

should be,

wget 'http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/1994?var=surf_el&north=18.5&west=276&east=289&south=8&disableProjSubset=off&horizStride=1&time=1995-01-01T09%3A00%3A00Z&vertCoord=&accept=netcdf4'
-O examplefilename_wget.nc