Example of saving cube as grib2 ?

[rsignell]

Iris folks,

I'm having trouble writing a cube as grib2.
The only doc I found seems to suggest that this should work:

save_grib2(cube,'my_cube.grib2')

but my simple try here failed:
http://nbviewer.ipython.org/5421188

Does anyone have a real-life example, perhaps loading one of the sample data files and writing a cube as Grib2?

Thanks,
Rich

[bblay]

Hi Rich,

There are some examples of saving grib2 files in iris.tests.test_grib_save.py.
The intended way to save is iris.save(cube, "file.grib2") but your call to iris.fileformats.grib.save_grib2 is probably fine.

Looking at the error message, it seems to come from this line:

grib_message = gribapi.grib_new_from_samples("GRIB2")

That suggests to me that the grib api is not installed properly.
Specifically, I don't think the api can find the sample grib2 file, "GRIB2.tmpl".
This file should live somewhere like this: /usr/local/share/samples
and is installed as part of the grib api installation process.

You could try typing grib_info from the command line, to find out where the api thinks the samples files should live.
Iris' travis testing file, ".travis.yml", installs the gribapi from source. That might be a useful reference?

Out of interest, what version of the grib api are you using?

Kind regards,
Byron

[rsignell]

Byron,

I get:

usgs@gam:/usgs/data2/notebook$ grib_info

grib_api Version 1.9.16

Definition files path from environment variable GRIB_DEFINITION_PATH=/home/rsignell/share/definitions

SAMPLES path from environment variable GRIB_SAMPLES_PATH=/home/rsignell/share/samples

[rsignell]

Even with the GRIB_SAMPLES_PATH and GRIB_DEFINITION_PATH environment variables set, my installaton of Iris still is having problems:

import os
print os.environ


{'LESSOPEN': '| /usr/bin/lesspipe %s', 'LOGNAME': 'usgs', 'USER': 'usgs', 'HOME': '/home/usgs', 'PATH': '/usr/local/python27_epd/bin:/usr/lib/gmt/bin:/usr/local/python27_epd/bin:/usr/lib/gmt/bin:/usr/local/python27_epd/bin:/usr/lib/gmt/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/hdf5/bin:/usr/local/matlab/bin:/usr/local/nco/bin:/usr/local/netcdf/bin:/usr/local/udunits2/bin:/home/rsignell/bin', 'LANG': 'en_US.UTF-8', 'GRIB_SAMPLES_PATH': '/home/rsignell/share/samples', 'TERM': 'xterm-color', 'SHELL': '/bin/bash', 'XDG_SESSION_COOKIE': 'd51b9220f21c4436f8cd43d34b5daafe-1366664435.83924-2094974906', 'SHLVL': '1', 'MANPATH': ':/usr/local/nco/share/man:/usr/local/netcdf/share/man:/usr/local/udunits2/share/man', 'GIT_PAGER': 'cat', '_': '/usr/bin/nohup', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'OLDPWD': '/home/usgs', 'GRIB_DEFINITION_PATH': '/home/rsignell/share/definitions', 'CLICOLOR': '1', 'PWD': '/usgs/data2/notebook', 'MAIL': '/var/mail/usgs' }

#save slice as grib2
iris.save(slice,'hsig.grib2')

---------------------------------------------------------------------------
GribInternalError Traceback (most recent call last)
<ipython-input-10-4411f5d85f01> in <module>()
 1 #save slice as grib2
----> 2 iris.save(slice,'hsig.grib2')

/home/rsignell/epd-7.2-1/lib/python2.7/site-packages/Iris-1.4.0_dev-py2.7.egg/iris/io/__init__.pyc in save(source, target, saver, **kwargs)
 330 # Single cube?
 331 if isinstance(source, iris.cube.Cube):
--> 332 saver(source, target, **kwargs)
 333
 334 # CubeList or sequence of cubes?

/home/rsignell/epd-7.2-1/lib/python2.7/site-packages/Iris-1.4.0_dev-py2.7.egg/iris/fileformats/grib.pyc in save_grib2(cube, target, append, **kwargs)
 605
 606 # Save this slice to the grib file
--> 607 grib_message = gribapi.grib_new_from_samples("GRIB2")
 608 grib_save_rules.run(slice2D, grib_message)
 609 gribapi.grib_write(grib_message, grib_file)

/home/rsignell/epd-7.2-1/lib/python2.7/site-packages/gribapi.pyc in modified(*args, **kw)
 58 assert type(param) in allowed_types, \
 59 "Parameter '%s' should be type %s" % (name, " or ".join([t.__name__ for t in allowed_types]))
---> 60 return _func_(**kw)
 61 return modified
 62 return check_types

/home/rsignell/epd-7.2-1/lib/python2.7/site-packages/gribapi.pyc in grib_new_from_samples(samplename)
 601 """
 602 err,gribid = _internal.grib_c_new_from_samples(0,samplename)
--> 603 GRIB_CHECK(err)
 604 return gribid
 605

/home/rsignell/epd-7.2-1/lib/python2.7/site-packages/gribapi.pyc in modified(*args, **kw)
 58 assert type(param) in allowed_types, \
 59 "Parameter '%s' should be type %s" % (name, " or ".join([t.__name__ for t in allowed_types]))
---> 60 return _func_(**kw)
 61 return modified
 62 return check_types

/home/rsignell/epd-7.2-1/lib/python2.7/site-packages/gribapi.pyc in GRIB_CHECK(errid)
 125 """
 126 if errid:
--> 127 raise GribInternalError(errid)
 128
 129 @require(file=file)

GribInternalError: File not found

[rsignell]

I feel like I'm stuck here. 

When I run "make test" on grib-api, all tests pass.

Is there something else I can test?

-Rich

[rsignell]

I also realize that this google group is probably not the place to
discuss this highly-specific technical issue.

Is there another form of communication we should be using for these
type of issues?

(e.g. help desk,  github issue, one-on-one exchange with expert, etc)

Thanks,
Rich

[bblay]

(Apologies for the slow responses)

ECMWF provide grib api support here:

https://software.ecmwf.int/wiki/display/GRIB/Home

I think your failure can be captured with this snippet:

import gribapi
g = gribapi.grib_new_from_samples("GRIB2")

Please could you confirm, to make sure we're on the right track?

[bblay]

I've pm'd you my personal contact details.

Are you using a 64-bit machine?

Did you install it something like this?

  - cd grib_api-1.9.16/
  - export CFLAGS="-fPIC"
  - ./configure --with-jasper=/usr/local/lib --disable-fortran --enable-python
  - make
  - sudo make install
  - cd python
  - sudo /usr/bin/python setup.py install

[rsignell]

Byron,

You were correct that

import gribapi
g = gribapi.grib_new_from_samples("GRIB2")

was sufficient to create the error.

The first time I built grib_api I did so with

./configure --prefix=$HOME ...

but even with environment variables set, Iris wasn't happy, and couldn't find the samples.

When I built it again, without --prefix set, it worked okay.  Great!

So now I'm onto the next problem, which may be of general interest:

cubes = iris.load('http://cida.usgs.gov/thredds/dodsC/prism');
cube = cubes[2]
slice=cube[1000,:,:]
print(slice)

mean monthly precipitation / mm/month (latitude: 621; longitude: 1405)
     Dimension coordinates:
          latitude                             x               -
          longitude                            -               x
     Scalar coordinates:
          time: 1978-05-01 00:00:00

iris.save(slice,'precip.grib2')

TranslationError: CoordSystem not present

Indeed, if I look at my dimension coordinates, they have no coord_system defined:

print(slice.coord(axis='X').coord_system)

None

So I guess my question now is:

How do I add a GeogCS coord_system to my existing dimensional coords?

Thanks,
Rich

[rsignell]

Iris folks,

With Byron's help I was eventually able to get this to work.   I needed to turn my array with NaN values into a masked array, add a coordinate system, a forecast_time coordinate and a vertical coordinate. 

# add a Geographic Coord system: spherical earth with specified radius
cube.coord(axis='X').coord_system=iris.coord_systems.GeogCS(654321)
cube.coord(axis='Y').coord_system=iris.coord_systems.GeogCS(654321)

# add a forecast_period
cube.add_aux_coord(iris.coords.DimCoord(0, standard_name='forecast_period', units='hours'))
 
# add a vertical coordinate
cube.add_aux_coord(iris.coords.DimCoord(0, "height", units="m"))
 
# convert array with NaNs to masked-array
cube.data=ma.masked_invalid(slice.data)

# Finally... save cube as grib2
iris.save(cube,'hsig.grib2')

 

The full example is here:

http://nbviewer.ipython.org/5492513

It turns out that all GRIB variables are currently being labeled as Air Temperature.

As the GRIB writing progresses, I would hope that the GRIB files written by Iris would be able to be consumed by the Unidata CDM

http://www.unidata.ucar.edu/software/netcdf-java/formats/GribTables.html

I know John Caron at Unidata has struggled mightily with GRIB, and his suggestions for how to disambiguate GRIB hopefully are useful

Thanks,
Rich