Iris CF Convention functionality

[rsignell]

Iris developers,

At SciPy 2013, Matthew Rocklin gave a compelling lighting talk promoting the concept that "granular software ecosystems enable software evolution".  You can see the basic idea here:
http://matthewrocklin.com/blog/work/2013/07/02/Packages/
or listen to his 5 minute talk here:
http://www.youtube.com/watch?v=ywHqIEv3xXg#t=06m25s

I think it would be great if the Iris CF convention functionality could be split into a separate package that Iris could use, just as the Cartopy package was split into a separate package.   I think this would facilitate co-development as well as usage of the Iris CF functionality in other applications.

This came up because I've built some python tools for ArcGIS to access OPeNDAP and NetCDF files https://github.com/rsignell-usgs/dap2arc, but I'd love to be able to take advantage of the CF conventions (determining the geospatial and time information automatically from the dataset) 

Although importing Iris would provide support for the CF conventions, building Iris is difficult on Windows platforms, as of course building the entire Iris package is overkill for this application.  If we just had the CF functionality in a separate package, I'm thinking it might be much lightweight and easy for applications like this.

Thoughts?
Rich

[Richard Hattersley]

Hi Rich,

Thanks for raising this issue - especially as you have an example of where you'd like to get re-use. For what it's worth, it's an idea which fits nicely with my own view. As you mention, Cartopy is already a separate project, and Biggus aims to take Iris' out-of-core handling the same route.

Do you have a sense of what level of interpretation would be of most use? For example, would it make sense for it to provide Cubes, Coords, Units, etc. (where unit handling/validation would still be something of a blocker on Windows), or would some lower level, perhaps netCDF-specific, model be more appropriate?

On a related note, there have also been discussions concerning the sharing of low-level CF code with a new version of the CF-checker utility, to give it a more light-weight set of dependencies. As far as I know it never went past the theoretical discussion stage.

Richard

[rsignell]

Richard,

Yes, I was thinking that the CF module would provide the Iris common data model cube/coords/units functionality, which, as you mention, would would require a units package. 

The package IRIS is currently using (UDUNITS) has problems compiling on Windows, but Unidata is looking into modifying the build process to use CMAKE and getting this going on windows.

Another possibility would be to use a different python units package.  

Trevor Bekolay gave a talk at SciPy 2013 on python units packages (he looked at 18 of them!):
http://www.youtube.com/watch?v=N-edLdxiM40

He also has a github project where he compares the features of these different units packages, so that those familiar with the features required by Iris could easily see what packages provide them:
https://github.com/tbekolay/quantities-comparison

-Rich

[Richard Hattersley]

Hi Rich,

I've got a branch in my own repo which makes the netCDF (and pyke) and GRIB dependencies optional by tweaking the location of various imports.

It doesn't have the scope of what we were previously talking about in this thread, but I wonder if it's a useful step for you?

[Richard Hattersley]

I've got a branch in my own repo which makes the netCDF (and pyke) and GRIB dependencies optional by tweaking the location of various imports

I forgot to mention it also makes scipy optional.

I'm still working on how to make cartopy optional too.

At what point (if any) does it become feasible to make use of a version of Iris like this within dap2arc (or similar)?