Downloading 4D array of wind data ("ugrdprs") using Matlab

[Julianna Evans]

Hello,

I am using Nctoolbox with MATLAB to download wind velocity data. I have been using this site (http://polar.ncep.noaa.gov/waves/examples/usingmatlab.shtml) as a reference, but I have been running into a problem with the download. 

I am trying to download the "ugrdprs" and "vgrdprs" datasets from the GrADS Data Server (specific URL is http://nomads.ncep.noaa.gov:9090/dods/gfs_0p25_1hr/gfs20170803/gfs_0p25_1hr_00z). I am not able to download the entire dataset--only a 3D section of the array. 

For example, code (1) below will run, but expanding the first index to include the entire time array does not work (2).

(1) UData = double(squeeze(nco{'ugrdprs'}(1,:,:,:)));

(2) UData = double(squeeze(nco{'ugrdprs'}(:,:,:,:)));

Do you know if MATLAB can successfully download 4D arrays from the server using nctoolbox, or is this not supported? 

Here are the first couple lines of the error message:

Error using ncdataset/readdata (line 494)

Java exception occurred:

java.io.IOException: opendap.dap.DataReadException: Inconsistent array length read: 1165128303 != 1914731274

Thank you.

[Rich Signell]

Julianna,

nctoolbox has no problem downloading 4D arrays, but...

In this command:

(1) UData = double(squeeze(nco{'ugrdprs'}(1,:,:,:)));

you are asking for all the vertical levels from a global 1/4 degree
model at every grid point. That downloads 128MB of data via OpenDAP.
Is that what you intended?

(2) (2) UData = double(squeeze(nco{'ugrdprs'}(:,:,:,:)));
now you are asking for all the data from a global model at every level
and ever grid point for all time (121 steps in this case). That is
15GB of data. It's too much for a single request from most OPeNDAP
servers.

The usual workflow is to just download the levels you need in the
region you need them and loop through time, processing them as you go.

-Rich

---
Rich Signell
81 Queen St
Falmouth, MA 02540

> --
> You received this message because you are subscribed to the Google Groups
> "nctoolbox" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nctoolbox+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Julianna Evans]

Rich,

Thank you for your response. In my current code I am looping through and indexing the wind velocity data I want in time, lev, lat, and lon indices like you mentioned. That seemed to be taking a long time, so I was looking into downloading the dataset and directly indexing from that to see if it would be faster. I didn't realize the extensive size of the dataset, and it makes sense now why I am getting the error. 

Thanks,

Julianna