How can I set scale_factor and add_offset when saving netCDF files?

[Johan Swanljung]

By default the netCDF4 python library will pack values when saving if you specify add_offset and scale_factor attributes as per:

http://unidata.ucar.edu/software/netcdf/docs/BestPractices.html#bp_Packed-Data-Values

Iris unpacks just fine but I can't figure out any way to set scale_factor and add_offset when saving. Trying to add them as cube attributes gives an error, as they are treated as reserved/forbidden and iris.fileformats.netcdf.save doesn't seem to have any keyword arguments with support for this.

iris.fileformats.netcdf.saver does have a method called update_global_attributes, but that doesn't seem to do what I want either.

Am I missing something obvious? 

[Carwyn Pelley]

Hi Johan,

Gosh, I had not noticed the `scale_factor` and `add_offset`.  I recently added support to iris for `valid_range`, `valid_min` and `valid_max` for the purpose of describing that I had unsigned integers/logical data in my NetCDF-classic file (https://github.com/SciTools/iris/pull/2098).

> Am I missing something obvious?

I don't think it's 'supported' yet so because its a special attribute its excluded.  I think iris just hasn't had a usecase up to now (just like with `valid_range` etc.).
Looks like netcdf4-python should support it fine, so it shouldn't be too much to get things up and running in iris I assume.

Packing support would be great!  Our filesizes are tens of gigabytes larger than they need to be.

Consider me a second usecase :)

Further interesting reading:
http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#_data_types
http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html
http://unidata.github.io/netcdf4-python/#netCDF4.Dataset.set_auto_scale

[Johan Swanljung]

No, it should be very easy, simply a matter of setting the parameters with netCDF4. I'll look into it.

[Johan Swanljung]

If you use gzip compression packing probably doesn't add much, but the option should be there.

[Johan Swanljung]

There's a PR for this now: https://github.com/SciTools/iris/pull/2152

[Johan Swanljung]

There is now netCDF packing support in the master branch (in case someone else reads this).

https://github.com/SciTools/iris/pull/2193

[marqh]

It is excellent to have this on master

many thanks for the contribution Johan