How to read wave data from .Grib2 file?

[Ian Wang]

Hello,

Has anyone successfully used nctoolbox to read wave data from .grb2 files from NOAA NWW3 model (ftp://polar.ncep.noaa.gov/pub/history/waves)?

I've downloaded one of the .grb2 file from the link and saved to local disk and try to extract the wave data from the file. When I run the nctoolbox script I get following errors. Could anyone help? much appreciated for your comments.

Kind regards

Ian

>> setup_nctoolbox
NCTOOLBOX added to Matlab path
>> ds = ncdataset('200502.grb2');
>> ds.variables
ans = 
    'Primary_wave_direction_degree_true_surface'
    'lat'
    'lon'
    'time'
>> ds.data('Primary_wave_direction_degree_true_surface')
Warning: Escape sequence 'c' is not valid. See 'help
sprintf' for valid escape sequences. 
> In ncdataset>ncdataset.readdata at 464
  In ncdataset>ncdataset.data at 159 
Warning: An error occurred while reading
"Primary_wave_direction_degree_true_surface" in
200502.grb2. Cause:
Error using ucar.ma2.ArrayFloat$D3/copyToNDJavaArray
Java exception occurred:
java.lang.OutOfMemoryError: Java heap space
at java.lang.reflect.Array.multiNewArray(Native
    Method)
at java.lang.reflect.Array.newInstance(Unknown Source)
at ucar.ma2.Array.copyToNDJavaArray(Array.java:768)

Error in ncdataset>ncdataset.readdata at 464
  In ncdataset>ncdataset.data at 159 
Error using ncdataset/readdata (line 475)
Failed to open
"Primary_wave_direction_degree_true_surface" in
200502.grb2
Error in ncdataset/data (line 159)
                    d = obj.readdata(variable);
Caused by:
    Error using ucar.ma2.ArrayFloat$D3/toString
    Java exception occurred:
    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at
        java.lang.AbstractStringBuilder.expandCapacity(Unknown
        Source)
    at
        java.lang.AbstractStringBuilder.append(Unknown
        Source)
    at java.lang.StringBuilder.append(Unknown
        Source)
    at java.lang.StringBuilder.append(Unknown
        Source)
    at ucar.ma2.Array.toString(Array.java:1117)
 
>> 

[Rich Signell]

Ian,
I just tried downloading one of the WaveWatch 3 GRIB2 files and it worked fine:

>>  ! wget ftp://polar.ncep.noaa.gov/pub/history/waves/multi_1.at_4m.dp.200607.grb2
>> nc=ncdataset('multi_1.at_4m.dp.200607.grb2');
>> nc.variables

ans =

    'Primary_wave_direction_degree_true_surface'
    'lat'
    'lon'
    'time'

>> dir=nc.data('Primary_wave_direction_degree_true_surface');
>> size(dir)

ans =

   249   481   586

It appears to me that you ran out of Java Memory in Matlab because you are loading the entire array.

A few suggestions:

1) Open these files with "ncgeodataset" rather than "dataset" so that NCTOOLBOX can automatically read the coordinate variables

2) Try accessing just one (or a few) time step at a time instead of reading all the data at once

3) You could increase your Java memory for Matlab by going to "File=>Preferences=>General=>Java Heap Memory".  I have mine set to 1,060.

4) Make sure you have the latest stable NCTOOLBOX from  http://nctoolbox.github.com/nctoolbox/,
    currently:  https://github.com/nctoolbox/nctoolbox/archive/nctoolbox-20130131.zip

5) Try this instead of the above:

nc=ncgeodataset('multi_1.at_4m.dp.200607.grb2');
dirvar=nc.geovariable('Primary_wave_direction_degree_true_surface');
dir=dirvar.data(1,:,:);
g=dirvar.grid_interop(1,:,:);
pcolorjw(g.lon,g.lat,dir);
title(datestr(g.time))

You should get the attached plot.

-Rich

--
You received this message because you are subscribed to the Google Groups "nctoolbox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nctoolbox+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.  
Woods Hole, MA 02543-1598

[Ian Wang]

lol,

Thank you Rich, it is very helpful. I've fixed the problem and now successfully extracted the data I wanted.

Kind regards

Ian

[Shawn Harrison]

Hi Rich, 

I would love to be able to partially access these same grib2 files without downloading the entire file. Basically, hindcast data is saved for the global domain in a unique file for each day. I would like to access that data for a single point location but for many days.. say 1 year. I am able to access local grib2 data file with nctoolbox (ncdataset), however I have been unable to read the file at its remote (ftp) location. How can we partially access these files from an ftp link?

Great tool, BTW.

Many thanks,
Shawn

[Ian Wang]

Hi Shawn, 

Just wondering if you have successfully found a way to partially access the files from the FTP link? It would be much handy and time saving if it works.

Regards

Ian

[Ian]

Hi Rich,

I'm trying to extract the wave data (i.e. the wave height, period and direction ) for a given coordinate from GRiB2 file and has no success so far. I was wondering if you could help? Thank you in advance.

Kind regards

Ian  

On Tuesday, 2 April 2013 22:31:40 UTC+11, Rich Signell wrote:

[Rich Signell]

Ian,

You can use the "nj_tseries" convenience function to extract a time series from a lon,lat location.  The example below is for wave height, but you can use the same approach for wave direction and period, downloading the appropriate grib files.  Unfortunately, it doesn't look like NCEP has the historical data accessible via OPeNDAP.   You can access FORECAST wave data via OPeNDAP at http://thredds.ucar.edu/thredds/idd/modelsNcep.html.  Anyway, here's an example downloading a GRIB2 file for significant wave height and extract a time series at a desired spot:

% download dataset
! wget ftp://polar.ncep.noaa.gov/pub/history/waves/multi_1.at_4m.hs.200607.grb2

% open dataset
nc=ncgeodataset('multi_1.at_4m.hs.200607.grb2');

% list variables
nc.variables

% variable of interest
var='Significant_height_of_combined_wind_waves_and_swell_surface';

% boston buoy location
lon=-70.651
lat=42.346

% time series at specified location
dat=nj_tseries(nc,var,lon,lat,'method','nearest')

% plot it up
plot(dat.time,dat.vals);datetick
title(sprintf('Time series of %s at (%7.3f,%7.3f)',var,lat,lon),'interpreter','none')
grid

attached is the plot it produced.

[Tenoch Juárez]

Hi Rich, 

I need help. I have a problem with nctoolbox. 

[Filehs,PathName] = uigetfile({'*.grb';'*.grb2';'*.nc*'},'Select the file');

>> nc=ncgeodataset('multi_1.at_4m.dp.200607.grb2');

??? Error: File: ncgeodataset.m Line: 60 Column: 38

Undefined function or variable 'private'.

Why this happens?

thanks!

[Richard Signell]

My guess is you need to run "setup_nctoolbox"

--
Rich Signell
81 Queen St
Falmouth, MA

[Tenoch Juárez]

I run the nctoolbox, 

setup_nctoolbox

ds=ncdataset('multi_1.at_4m.dp.200607.grb2')

and return 

??? Error using==> ncdataset>ncdataset.ncdata set at 97

falied to open multi_1.at_4m.dp.200607.grb2

[Brian Schlining]

Tenoch,

Can you please include the following information so I can look into this:

1. In Matlab, run ‘ver’ and include the output.

2.. A link to the dataset you are trying to use so I can test with it.

Thanks

Brian

On July 27, 2015 at 2:46:10 PM, Tenoch Juárez (teno...@gmail.com) wrote:

I run the nctoolbox, 

setup_nctoolbox

ds=ncdataset('multi_1.at_4m.dp.200607.grb2')

and return 

??? Error using==> ncdataset>ncdataset.ncdata set at 97

falied to open multi_1.at_4m.dp.200607.grb2

El lunes, 27 de julio de 2015, 13:33:35 (UTC-7), Rich Signell escribió:

My guess is you need to run "setup_nctoolbox"

-- 
Brian Schlining

[Brian Schlining]

R2008a?? I’ll have to see if I can dig up a copy of R2008a to test with.

On July 27, 2015 at 3:20:57 PM, Tenoch Juárez (teno...@gmail.com) wrote:

!., output from matlab

MATLAB Version 7.6.0.324 (R2008a)

MATLAB License Number: 161051

Operating System: Microsoft Windows Vista Version 6.2 (Build 9200)

Java VM Version: Java 1.6.0 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode

-------------------------------------------------------------------------------------

Warning: No properly formatted Contents.m file was found for '~ -java'.

> In ver>disp_single_toolbox at 424

  In ver at 81

>> ver

-------------------------------------------------------------------------------------

MATLAB Version 7.6.0.324 (R2008a)

MATLAB License Number: 161051

Operating System: Microsoft Windows Vista Version 6.2 (Build 9200)

Java VM Version: Java 1.6.0 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode

-------------------------------------------------------------------------------------

MATLAB                                                Version 7.6        (R2008a)    

Simulink                                              Version 7.1        (R2008a)    

Aerospace Blockset                                    Version 3.1        (R2008a)    

Aerospace Toolbox                                     Version 2.1        (R2008a)    

Bioinformatics Toolbox                                Version 3.1        (R2008a)    

Communications Blockset                               Version 4.0        (R2008a)    

Communications Toolbox                                Version 4.1        (R2008a)    

Control System Toolbox                                Version 8.1        (R2008a)    

Curve Fitting Toolbox                                 Version 1.2.1      (R2008a)    

Database Toolbox                                      Version 3.4.1      (R2008a)    

Datafeed Toolbox                                      Version 3.1        (R2008a)    

Filter Design HDL Coder                               Version 2.2        (R2008a)    

Filter Design Toolbox                                 Version 4.3        (R2008a)    

Financial Derivatives Toolbox                         Version 5.2        (R2008a)    

Financial Toolbox                                     Version 3.4        (R2008a)    

Fixed-Income Toolbox                                  Version 1.5        (R2008a)    

Fixed-Point Toolbox                                   Version 2.2        (R2008a)    

Fuzzy Logic Toolbox                                   Version 2.2.7      (R2008a)    

GARCH Toolbox                                         Version 2.4        (R2008a)    

Genetic Algorithm and Direct Search Toolbox           Version 2.3        (R2008a)    

Image Processing Toolbox                              Version 6.1        (R2008a)    

MATLAB Builder EX                                     Version 1.2.10     (R2008a)    

MATLAB Builder JA                                     Version 2.0.1      (R2008a)    

MATLAB Builder NE                                     Version 2.2.2      (R2008a)    

MATLAB Compiler                                       Version 4.8        (R2008a)    

MATLAB Distributed Computing Server                   Version 3.3        (R2008a)    

MATLAB Report Generator                               Version 3.3        (R2008a)    

Mapping Toolbox                                       Version 2.7        (R2008a)    

Model Predictive Control Toolbox                      Version 2.3.1      (R2008a)    

NCTOOLBOX Tools for read-only access to Common ...    Version 1.1.1-13-g8582810++    

Neural Network Toolbox                                Version 6.0        (R2008a)    

Optimization Toolbox                                  Version 4.0        (R2008a)    

Parallel Computing Toolbox                            Version 3.3        (R2008a)    

Partial Differential Equation Toolbox                 Version 1.0.12     (R2008a)    

RF Blockset                                           Version 2.2        (R2008a)    

RF Toolbox                                            Version 2.3        (R2008a)    

Real-Time Workshop                                    Version 7.1        (R2008a)    

Real-Time Workshop Embedded Coder                     Version 5.1        (R2008a)    

Robust Control Toolbox                                Version 3.3.1      (R2008a)    

Signal Processing Blockset                            Version 6.7        (R2008a)    

Signal Processing Toolbox                             Version 6.9        (R2008a)    

SimBiology                                            Version 2.3        (R2008a)    

SimDriveline                                          Version 1.5        (R2008a)    

SimEvents                                             Version 2.2        (R2008a)    

SimHydraulics                                         Version 1.3        (R2008a)    

SimMechanics                                          Version 2.7.1      (R2008a)    

SimPowerSystems                                       Version 4.6        (R2008a)    

Simscape                                              Version 2.1        (R2008a)    

Simulink Control Design                               Version 2.3        (R2008a)    

Simulink Fixed Point                                  Version 5.6        (R2008a)    

Simulink Parameter Estimation                         Version 1.2.2      (R2008a)    

Simulink Report Generator                             Version 3.3        (R2008a)    

Simulink Response Optimization                        Version 3.1.3      (R2008a)    

Simulink Verification and Validation                  Version 2.3        (R2008a)    

Spline Toolbox                                        Version 3.3.4      (R2008a)    

Spreadsheet Link EX                                   Version 3.0.1      (R2008a)    

Stateflow                                             Version 7.1        (R2008a)    

Stateflow Coder                                       Version 7.1        (R2008a)    

Statistics Toolbox                                    Version 6.2        (R2008a)    

System Identification Toolbox                         Version 7.2        (R2008a)    

SystemTest                                            Version 2.1        (R2008a)    

Video and Image Processing Blockset                   Version 2.5        (R2008a)    

Virtual Reality Toolbox                               Version 4.7        (R2008a)    

Wavelet Toolbox                                       Version 4.2        (R2008a)    

tsplot Toolbox  

and 2., it's just the same file as above

ftp://polar.ncep.noaa.gov/pub/history/waves/multi_1.at_4m.dp.200607.grb2

thaks! 

--

+99 ______________________________
+990_____________________________
++99_____________________________
++99_____________________________
_+990____________________________
_+999_________________________+99
_++990_______________________+99
_++999_____+++99__________+++9
_++9999__+++9990++99____+++99
_+++9999+++9990++9999__++999
__++999++9999+++999999++999
__++99++9999++9999999++999
__++++++999999999999999999
___++++++99999999999999999
___++++++++000999999999999
____+++++++000099999999999
_____+++++++00099999999999
______+++++++099999999999

-- 
Brian Schlining

[Tenoch Juárez]

Hi! 

I have now Matlab R2009a and every run very well. 

Thanks

[Brian Schlining]

Great. Thanks for letting me know.

Cheers

B

On July 30, 2015 at 9:21:09 AM, Tenoch Juárez (teno...@gmail.com) wrote:

Hi! 

I have now Matlab R2009a and every run very well. 

Thanks

El lunes, 27 de julio de 2015, 15:43:31 (UTC-7), Brian Schlining escribió:

R2008a?? I’ll have to see if I can dig up a copy of R2008a to test with.

-- 
Brian Schlining

[soroush]

Hi Rich,

I tried your code with 'nj-tseries' to extract hs for three months. The code works well, but when I try to re-run to code, I receive one of following two sets of errors:

Error using subsref

No appropriate method, property, or field getlonname for class

ncvariable.

Error in ncvariable/subsref (line 295)

                            sref = builtin('subsref',obj,s);

Error in nj_tseries (line 40)

lon=nc.data(vart.getlonname);

or 

Dot name reference on non-scalar structure.

Error in ncdataset (line 72)

                obj.variables = url.variables;

Error in ncgeodataset/geovariable (line 165)

                    ncd = ncdataset(obj);

Error in ncgeodataset/subsref (line 386)

                            B = builtin('subsref',obj,g);

Error in nj_tseries (line 39)

vart=nc.geovariable(var);

 The code itself is as follows:

% download dataset

% ! wget ftp://polar.ncep.noaa.gov/pub/history/waves/multi_1.at_4m.hs.200607.grb2

% open dataset

clear

nc1=ncgeodataset('multi_1.at_10m.hs.201004.grb2');

nc2=ncgeodataset('multi_1.at_10m.hs.201005.grb2');

nc3=ncgeodataset('multi_1.at_10m.hs.201006.grb2');

% list variables

nc1.variables

% variable of interest

var='Significant_height_of_combined_wind_waves_and_swell_surface';

% boston buoy location

lon=-89.658

lat=25.888

% time series at specified location

dat1=nj_tseries(nc1,var,lon,lat,'method','nearest')

dat2=nj_tseries(nc2,var,lon,lat,'method','nearest')

dat3=nj_tseries(nc3,var,lon,lat,'method','nearest')

time = [dat1.time; dat2.time; dat3.time];

vals = [dat1.vals; dat2.vals; dat3.vals];

% plot it up

plot(time,vals);datetick

title(sprintf('Time series of %s at (%7.3f,%7.3f)',var,lat,lon),'interpreter','none')

grid

 To my surprise, when I close MATLAB and re-open it (and re-run nctoolbox) the code works well again. I am using nctoolbox-20130305 and R2013a. Much appreciated for your comments and advice.

Regards,

Soroush

[Rich Signell]

The fact that it works after restarting matlab sounds like it's some kind of memory leak issue or too many files being opened or something.  

Would you mind posting this as an issue on github at:
https://github.com/nctoolbox/nctoolbox/issues

so we can more effectively discuss?

Thanks,
Rich

[Soroush Sorourian]

Sure, I created a post in github in this regard.

Thanks,