netcdf/numpy typemaps in pupynere

[James Hiebert]

Hi Rob (et al.),

Not sure if this is list-appropriate since it's only peripherally related to Pyap, but I'll give it a go anyways.

I'm in the process of using h5py to write an hdf5 handler for Pydap-3.2, since there isn't yet one. I've found that when I request a netcdf response (generated by pupynere) through my (in development) hdf5 handler, pupynere blows up trying to do the type conversion of any textual attributes, of which there are many.

So pupynere defines the netcdf->numpy typemap and visa-versa as such:

TYPEMAP = { NC_BYTE: dtype(np.byte),
NC_CHAR: dtype('c'),
...

REVERSE = { dtype(np.byte): NC_BYTE,
dtype('c'): NC_CHAR,
...

however, near as I can tell from the numpy docs the code for string types is '|S#'. 'c' is documented as being a complex-floating point. So then what happens is h5py in the handler reads in the attributes to a numpy array of dtype('|S16) (or whatever their length is), and then when pupynere goes to convert them to netcdf type, the lookup into the REVERSE dictionary fails.

So my question is, does this appear to be a correct diagnosis? And if so, should we (I?) fix this, possibly by turning TYPEMAP and REVERSE into functions that are able to handle the "flexible" variable-length numpy types like |S# ?

http://docs.scipy.org/doc/numpy-1.6.0/reference/arrays.dtypes.html

~James

--
James Hiebert
Lead, Computational Support
Pacific Climate Impacts Consortium
http://www.pacificclimate.org
Room 112, University House 1, University of Victoria
PO Box 1700 Sta CSC, Victoria, BC V8V 2Y2
E-mail: hie...@uvic.ca
Tel: (250) 472-4521
Fax: (250) 472-4830