Extracting U and V wind components from ECMWF ERA-operational dataset

[DRS]

Hello,

I am new to working with GRiB files and came across the nctoolbox as a simple method to extract the data I need into MATLAB R2012a.

I am trying to extract the 10m U and V components of wind velocity with corresponding longitudes/latitudes from the ECMWF ERA-operational dataset. This is a restricted-access dataset for academic users - so I do hope this doesn't hinder my chances of getting any help here!

My problem is, despite the units of the velocities apparently being [m/s], the velocities being read from the grib files into the U and V arrays (Udata and Vdata - see code below) are of order 1*10^4 [m/s] (or more!). Clearly these cannot be correct.

Code: 

--------------------------------------------------------------------------------------------------------------------

close all

clear

setup_nctoolbox

filename='C:/Users/Documents/WindData/N80/1996/gisf9602/gisf96020100';

nc=ncgeodataset(filename);

 

lat=nc.data('lat'); %Latitude

lon=nc.data('lon'); %Longitude

time=nc.data('time');   %Time

 

Udata=nc.data('10_metre_U_wind_component_surface'); %U Velocity

Vdata=nc.data('10_metre_V_wind_component_surface'); %V Velocity

 

----------------------------------------------------------------------------------------------------------------------

The variables lat, lon and time are all returning the values I expect. However, I am wondering if the velocities are perhaps being interpreted/de-coded from their binary file storage formats incorrectly? I have tried using the nctoolbox to read data sets from other years/months/days/etc., but the outcome is similar each time - suggesting that all of my data are at least of the same format and that an individual grib file is not corrupted.

I hope you can help shed some light on what I might be doing wrong/what is going wrong.

Just to note: ECMWF do offer their own grib extraction tool, GRIB-API, but trying to get this to work on a Windows operating system, requires cygwin to imitate Linux, plus a whole host of other file installations in order to get running. I am still battling with trying to get their method to work as well - however if I could get the nctoolbox working correctly this would be preferable, since I need to eventually read the data into Matlab anyway.  

Many thanks in advance of your help,

David

[Rich Signell]

Do you know if this is a GRIB1 or GRIB2 format?
Can you provide a sample?

> --
> You received this message because you are subscribed to the Google Groups
> "nctoolbox" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nctoolbox+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Rich Signell
81 Queen St
Falmouth, MA 02540

[DRS]

Hi Rich,

Many thanks for your quick reply.

Using the .attributes command of the nctoolbox tells me that its a GRIB-1 format file.

Unfortunately I don't think I can send a sample, as all users are contractually obliged not to... this is not ideal I know :S

However, I have got a little bit further with the GRIB-API method I mentioned in my previous post. I have been able to create a netcdf file of one of the grib files, which I can then read into MATLAB. Unfortunately, its not creating a 'complete' netcdf file and so it only shows parameters of 'soil temperature' and 'soil wetness' - and not of wind velocities. However, looking closer at these netcdf files in MATLAB shows that a scale factor is being applied to each of the parameters (see below):

stl1     

           Size:       320x160x1

           Dimensions: longitude,latitude,time

           Datatype:   int16

           Attributes:

                       scale_factor  = 0.00143

                       add_offset    = 261

                       _FillValue    = -3.28e+04

                       missing_value = -3.28e+04

                       units         = 'K'

                       long_name     = 'Soil temperature level 1'

                       standard_name = 'surface_temperature'

    swl1     

           Size:       320x160x1

           Dimensions: longitude,latitude,time

           Datatype:   int16

           Attributes:

                       scale_factor  = 4.67e-07

                       add_offset    = 0.0153

                       _FillValue    = -3.28e+04

                       missing_value = -3.28e+04

                       units         = 'm of water equivalent'

                       long_name     = 'Soil wetness level 1'

                       standard_name = 'lwe_thickness_of_soil_moisture_content'

If a similar scale factor needs to be applied to the velocity data too, this would probably explain why they appear so high. However, as I can't create a netcdf file which displays the velocity scale factors - I'm stuck. Is there anyway of calling these scale factors using nctoolbox? I've tried using the .attributes command (see below), but it doesn't display it. 'long_name' is just '10_m_U_wind_component_surface' and 'grid_mapping' is 'GaussianLatLon_Projection':

 nc.attributes('10_metre_U_wind_component_surface')

ans = 

    'long_name'                         [1x51 char]

    'units'                   'm.s-1'              

    'missing_value'           [                NaN]

    'grid_mapping'                      [1x25 char]

    'Grib_Variable_Id'        'VAR_98-0-128-165_L1'

    'Grib1_Center'            [                 98]

    'Grib1_Subcenter'         [                  0]

    'Grib1_TableVersion'      [                128]

    'Grib1_Parameter'         [                165]

    'Grib1_Parameter_Name'    '10U'                

    'Grib1_Level_Type'        [                  1]

Many thanks for your support,

David

[Brian Schlining]

Hi David,

Many thanks for your quick reply.

Using the .attributes command of the nctoolbox tells me that its a GRIB-1 format file.

Unfortunately I don't think I can send a sample, as all users are contractually obliged not to... this is not ideal I know :S

Can you do the following code in Matlab:

setup_nctoolbox

filename='C:/Users/Documents/WindData/N80/1996/gisf9602/gisf96020100';

nc=ncgeodataset(filename);

nc.netcdf

Then send us the output from 'nc.netcdf'. That will show us the full structure of the GRIB file and could give us some insight. 

Thanks

Brian 

[DRS]

Hi Brian,

Here's the output:

ans =

 

netcdf C:/Users/Documents/WindData/N80/1996/gisf9602/gisf96020100 {

 dimensions:

   lon = 320;

   lat = 160;

   layer_between_two_depths_below_surface_layer = 1;

   layer_between_two_depths_below_surface_layer1 = 1;

   layer_between_two_depths_below_surface_layer2 = 1;

   layer_between_two_depths_below_surface_layer3 = 1;

   time = 1;

 variables:

   float layer_between_two_depths_below_surface_layer_bounds(layer_between_two_depths_below_surface_layer=1, 2);

     :units = "cm";

     :long_name = "bounds for layer_between_two_depths_below_surface_layer";

   float layer_between_two_depths_below_surface_layer1_bounds(layer_between_two_depths_below_surface_layer1=1, 2);

     :units = "cm";

     :long_name = "bounds for layer_between_two_depths_below_surface_layer1";

   float layer_between_two_depths_below_surface_layer2_bounds(layer_between_two_depths_below_surface_layer2=1, 2);

     :units = "cm";

     :long_name = "bounds for layer_between_two_depths_below_surface_layer2";

   float layer_between_two_depths_below_surface_layer3_bounds(layer_between_two_depths_below_surface_layer3=1, 2);

     :units = "cm";

     :long_name = "bounds for layer_between_two_depths_below_surface_layer3";

   float Soil_temperature_level_1_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer=1, lat=160, lon=320);

     :long_name = "Soil temperature level 1 @ Layer between 2 depths below land surface layer";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-139_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 139; // int

     :Grib1_Parameter_Name = "STL1";

     :Grib1_Level_Type = 112; // int

   float Soil_wetness_level_1_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer=1, lat=160, lon=320);

     :long_name = "Soil wetness level 1 @ Layer between 2 depths below land surface layer";

     :units = "m.of.water";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-140_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 140; // int

     :Grib1_Parameter_Name = "SWL1";

     :Grib1_Level_Type = 112; // int

   float Snow_depth_surface(time=1, lat=160, lon=320);

     :long_name = "Snow depth @ Ground or water surface";

     :units = "m.of.water.equivalent";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-141_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 141; // int

     :Grib1_Parameter_Name = "SD";

     :Grib1_Level_Type = 1; // int

   float Mean_sea_level_pressure_surface(time=1, lat=160, lon=320);

     :long_name = "Mean sea level pressure @ Ground or water surface";

     :units = "Pa";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-151_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 151; // int

     :Grib1_Parameter_Name = "MSL";

     :Grib1_Level_Type = 1; // int

   float 10_metre_U_wind_component_surface(time=1, lat=160, lon=320);

     :long_name = "10 metre U wind component @ Ground or water surface";

     :units = "m.s-1";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-165_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 165; // int

     :Grib1_Parameter_Name = "10U";

     :Grib1_Level_Type = 1; // int

   float 10_metre_V_wind_component_surface(time=1, lat=160, lon=320);

     :long_name = "10 metre V wind component @ Ground or water surface";

     :units = "m.s-1";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-166_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 166; // int

     :Grib1_Parameter_Name = "10V";

     :Grib1_Level_Type = 1; // int

   float 2_metre_temperature_surface(time=1, lat=160, lon=320);

     :long_name = "2 metre temperature @ Ground or water surface";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-167_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 167; // int

     :Grib1_Parameter_Name = "2T";

     :Grib1_Level_Type = 1; // int

   float 2_metre_dewpoint_temperature_surface(time=1, lat=160, lon=320);

     :long_name = "2 metre dewpoint temperature @ Ground or water surface";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-168_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 168; // int

     :Grib1_Parameter_Name = "2D";

     :Grib1_Level_Type = 1; // int

   float Soil_temperature_level_2_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer1=1, lat=160, lon=320);

     :long_name = "Soil temperature level 2 @ Layer between 2 depths below land surface layer";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-170_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 170; // int

     :Grib1_Parameter_Name = "STL2";

     :Grib1_Level_Type = 112; // int

   float Soil_wetness_level_2_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer1=1, lat=160, lon=320);

     :long_name = "Soil wetness level 2 @ Layer between 2 depths below land surface layer";

     :units = "m.of.water";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-171_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 171; // int

     :Grib1_Parameter_Name = "SWL2";

     :Grib1_Level_Type = 112; // int

   float Surface_roughness_surface(time=1, lat=160, lon=320);

     :long_name = "Surface roughness @ Ground or water surface";

     :units = "m";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-173_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 173; // int

     :Grib1_Parameter_Name = "SR";

     :Grib1_Level_Type = 1; // int

   float Albedo_surface(time=1, lat=160, lon=320);

     :long_name = "Albedo @ Ground or water surface";

     :units = "(0.-.1)";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-174_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 174; // int

     :Grib1_Parameter_Name = "AL";

     :Grib1_Level_Type = 1; // int

   float Soil_temperature_level_3_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer2=1, lat=160, lon=320);

     :long_name = "Soil temperature level 3 @ Layer between 2 depths below land surface layer";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-183_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 183; // int

     :Grib1_Parameter_Name = "STL3";

     :Grib1_Level_Type = 112; // int

   float Soil_wetness_level_3_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer2=1, lat=160, lon=320);

     :long_name = "Soil wetness level 3 @ Layer between 2 depths below land surface layer";

     :units = "m.of.water";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-184_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 184; // int

     :Grib1_Parameter_Name = "SWL3";

     :Grib1_Level_Type = 112; // int

   float Skin_reservoir_content_surface(time=1, lat=160, lon=320);

     :long_name = "Skin reservoir content @ Ground or water surface";

     :units = "m.of.water";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-198_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 198; // int

     :Grib1_Parameter_Name = "SRC";

     :Grib1_Level_Type = 1; // int

   float Vegetation_fraction_surface(time=1, lat=160, lon=320);

     :long_name = "Vegetation fraction @ Ground or water surface";

     :units = "(0.-.1)";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-199_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 199; // int

     :Grib1_Parameter_Name = "VEG";

     :Grib1_Level_Type = 1; // int

   float Apparent_surface_humidity_surface(time=1, lat=160, lon=320);

     :long_name = "Apparent surface humidity @ Ground or water surface";

     :units = "kg.kg-1";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-233_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 233; // int

     :Grib1_Parameter_Name = "ASQ";

     :Grib1_Level_Type = 1; // int

   float Logarithm_of_surface_roughness_length_for_heat_surface(time=1, lat=160, lon=320);

     :long_name = "Logarithm of surface roughness length for heat @ Ground or water surface";

     :units = "";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-234_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 234; // int

     :Grib1_Parameter_Name = "LSRH";

     :Grib1_Level_Type = 1; // int

   float Skin_temperature_surface(time=1, lat=160, lon=320);

     :long_name = "Skin temperature @ Ground or water surface";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-235_L1";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 235; // int

     :Grib1_Parameter_Name = "SKT";

     :Grib1_Level_Type = 1; // int

   float Soil_temperature_level_4_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer3=1, lat=160, lon=320);

     :long_name = "Soil temperature level 4 @ Layer between 2 depths below land surface layer";

     :units = "K";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-236_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 236; // int

     :Grib1_Parameter_Name = "STL4";

     :Grib1_Level_Type = 112; // int

   float Soil_wetness_level_4_layer_between_two_depths_below_surface_layer(time=1, layer_between_two_depths_below_surface_layer3=1, lat=160, lon=320);

     :long_name = "Soil wetness level 4 @ Layer between 2 depths below land surface layer";

     :units = "m";

     :missing_value = NaNf; // float

     :grid_mapping = "GaussianLatLon_Projection";

     :Grib_Variable_Id = "VAR_98-0-128-237_L112_layer";

     :Grib1_Center = 98; // int

     :Grib1_Subcenter = 0; // int

     :Grib1_TableVersion = 128; // int

     :Grib1_Parameter = 237; // int

     :Grib1_Parameter_Name = "SWL4";

     :Grib1_Level_Type = 112; // int

   float lat(lat=160);

     :units = "degrees_north";

     :_CoordinateAxisType = "Lat";

   float lon(lon=320);

     :units = "degrees_east";

     :_CoordinateAxisType = "Lon";

   float layer_between_two_depths_below_surface_layer(layer_between_two_depths_below_surface_layer=1);

     :units = "cm";

     :long_name = "Layer between 2 depths below land surface";

     :positive = "down";

     :GRIB1_level_code = 112; // int

     :datum = "ground";

     :bounds = "layer_between_two_depths_below_surface_layer_bounds";

     :_CoordinateAxisType = "Height";

     :_CoordinateZisPositive = "down";

   float layer_between_two_depths_below_surface_layer1(layer_between_two_depths_below_surface_layer1=1);

     :units = "cm";

     :long_name = "Layer between 2 depths below land surface";

     :positive = "down";

     :GRIB1_level_code = 112; // int

     :datum = "ground";

     :bounds = "layer_between_two_depths_below_surface_layer1_bounds";

     :_CoordinateAxisType = "Height";

     :_CoordinateZisPositive = "down";

   float layer_between_two_depths_below_surface_layer2(layer_between_two_depths_below_surface_layer2=1);

     :units = "cm";

     :long_name = "Layer between 2 depths below land surface";

     :positive = "down";

     :GRIB1_level_code = 112; // int

     :datum = "ground";

     :bounds = "layer_between_two_depths_below_surface_layer2_bounds";

     :_CoordinateAxisType = "Height";

     :_CoordinateZisPositive = "down";

   float layer_between_two_depths_below_surface_layer3(layer_between_two_depths_below_surface_layer3=1);

     :units = "cm";

     :long_name = "Layer between 2 depths below land surface";

     :positive = "down";

     :GRIB1_level_code = 112; // int

     :datum = "ground";

     :bounds = "layer_between_two_depths_below_surface_layer3_bounds";

     :_CoordinateAxisType = "Height";

     :_CoordinateZisPositive = "down";

   int time(time=1);

     :units = "Hour since 1996-02-01T00:00:00Z";

     :standard_name = "time";

     :long_name = "Uninitialized analysis / image product / forecast product valid for RT + P1";

     :_CoordinateAxisType = "Time";

 :Originating_or_generating_Center = "European Centre for Medium Range Weather Forecasts (ECMWF) (RSMC)";

 :Originating_or_generating_Subcenter = "0";

 :Conventions = "CF-1.6";

 :history = "Read using CDM IOSP Grib1Collection";

 :featureType = "GRID";

 :file_format = "GRIB-1";

 :_CoordSysBuilder = "ucar.nc2.dataset.conv.CF1Convention";

}

Thanks,

David

[DRS]

Hi Brian and Rich,

I have finally managed to find a work-a-round solution. 

I've managed to generate NetCDF files from the GRIBs, by using the UNIX-based command tools given by ECMWF. This method was originally failing when it stumbled across 'unreadable messages' in the grib files. However, I've since managed to select just the variables I need and extract those individually. Once read into matlab, the NetCDFs are then interpreted correctly - so all sorted!

Many thanks for your help with this problem!

David