Global attribute in MERRA data causes problems

[Sam Hawkins]

Hello,

I've just started trying Iris. After a reasonably traumatic installation, I think I have it working. I'm trying to use it with NASA MERRA data, which (in theory) is CF compliant.

However, the files have a global attribute "missing_value". The last part of ncdump -h shows:

        double XDim_EOS(XDim) ;

                XDim_EOS:long_name = "XDim" ;

                XDim_EOS:coordinates = "XDim" ;

                XDim_EOS:fullpath = "/EOSGRID/Data Fields/XDim" ;

        double YDim(YDim) ;

                YDim:long_name = "latitude" ;

                YDim:units = "degrees_north" ;

                YDim:fullpath = "YDim:EOSGRID" ;

        double YDim_EOS(YDim) ;

                YDim_EOS:long_name = "YDim" ;

                YDim_EOS:coordinates = "YDim" ;

                YDim_EOS:fullpath = "/EOSGRID/Data Fields/YDim" ;

// global attributes:

                :HDFEOSVersion = "HDFEOS_V2.14" ;

                :missing_value = 1.e+15f ;

                :Conventions = "CF-1.0" ;

                :title = "MERRA reanalysis. GEOS-5.2.0" ;

                :institution = "Global Modeling and Assimilation Office, NASA Goddard Space Flight Center, Greenbelt, MD 20771" ;

                :source = "Global Modeling and Assimilation Office. GEOSops_5_2_0" ;

                :references = "http://gmao.gsfc.nasa.gov/research/merra/" ;

                :comment = "GEOS-5.2.0" ;

                :contact = "http://gmao.gsfc.nasa.gov/" ;

                :NCO = "4.3.2" ;

When I try and load the file using Iris, this gives an error, since "missing_value" is a reserved attribute:

 File "<stdin>", line 1, in <module>

  File "iris/__init__.py", line 363, in load_cubes

    collection = _load_collection(uris, constraints, callback).merged()

  File "iris/__init__.py", line 267, in _load_collection

    result = iris.cube._CubeFilterCollection.from_cubes(cubes, constraints)

  File "iris/cube.py", line 138, in from_cubes

    for cube in cubes:

  File "iris/__init__.py", line 254, in _generate_cubes

    for cube in iris.io.load_files(part_names, callback, constraints):

  File "iris/io/__init__.py", line 199, in load_files

    for cube in handling_format_spec.handler(fnames, callback):

  File "iris/fileformats/netcdf.py", line 500, in load_cubes

    cube = _load_cube(engine, cf, cf_var, filename)

  File "iris/fileformats/netcdf.py", line 378, in _load_cube

    engine.activate(_PYKE_RULE_BASE)

  File "/cm/shared/apps/python/2.7.3/Python-2.7.3/lib/python2.7/site-packages/pyke/knowledge_engine.py", line 296, in activate

    for rb_name in rb_names: self.get_rb(rb_name).activate()

  File "/cm/shared/apps/python/2.7.3/Python-2.7.3/lib/python2.7/site-packages/pyke/rule_base.py", line 159, in activate

    self.run_fc_rules(current_rb)

  File "/cm/shared/apps/python/2.7.3/Python-2.7.3/lib/python2.7/site-packages/pyke/rule_base.py", line 147, in run_fc_rules

    for fc_rule in rb.fc_rules: fc_rule.run()

  File "/cm/shared/apps/python/2.7.3/Python-2.7.3/lib/python2.7/site-packages/pyke/fc_rule.py", line 90, in run

    self.rule_fn(self)

  File "iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py", line 13, in fc_default

    build_cube_metadata(engine)

  File "iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py", line 1124, in build_cube_metadata

    cube.attributes[str(attr_name)] = attr_value

  File "iris/_cube_coord_common.py", line 62, in __setitem__

    raise ValueError('%r is not a permitted attribute' % key)

ValueError: 'missing_value' is not a permitted attribute

I don't know if having a global "missing_value" attribute is CF compliant?  If it is, then perhaps Iris needs to not apply the attribute restrictions to global attributes? If it isn't CF compliant, then I will just delete or rename this attribute from the NetCDF files.

I've also ran into  a number of other issues working with MERRA data, I'll post those separately.  Happy to try and contribute to the code via github.

Sam.

[Luke Abraham]

I am seeing a similar problem, although not with missing_value in the global attributes, instead in the variable attributes. This post can be found here:

https://groups.google.com/d/topic/scitools-iris/_9aFHObEJOE/discussion

Luke