float size 16

[Matthew Turk]

Hi there,

I've got an HDF5 dataset with H5T_NATIVE_LDOUBLE data in it. When
trying to read this, I get back:

TypeError: Unsupported float size (16)

I suspect there's a good reason for this -- my recollection is that
long double tends not to be well-supported, either by compilers or by
HDF5. That being said, is there any way to have this use a numpy
array with dtype "longdouble" or "float128"?

Thanks very much,

-Matt

[Andrew Collette]

Hi Matt,

> TypeError: Unsupported float size (16)

Yes, at the moment the type-translation layer in h5py only handles 4
and 8 byte floats. I've been thinking of relaxing this, as (for
example) people have requested the ability to read & write 2-byte
floats. One wrinkle is that not all installations of numpy support
12/16 byte floats (mine has 12 but not 16), so we would have to think
carefully about whether to complain or risk loss of precision.

> I suspect there's a good reason for this -- my recollection is that
> long double tends not to be well-supported, either by compilers or by
> HDF5.  That being said, is there any way to have this use a numpy
> array with dtype "longdouble" or "float128"?

HDF5 certainly supports long double floats (12 and 16 bytes), as it
has a very generic mechanism for representing floats. I think what
you're bumping into is a limitation of h5py.

I've added your comments under issue 141 (the 2-byte float issue).
Hopefully we'll have some time soon to get this into h5py.

Thanks!
Andrew

[Nathaniel Smith]

On Mar 1, 2012 5:38 PM, "Andrew Collette" <andrew....@gmail.com> wrote:
>
> Hi Matt,
>
> > TypeError: Unsupported float size (16)
>
> Yes, at the moment the type-translation layer in h5py only handles 4
> and 8 byte floats.  I've been thinking of relaxing this, as (for
> example) people have requested the ability to read & write 2-byte
> floats.  One wrinkle is that not all installations of numpy support
> 12/16 byte floats (mine has 12 but not 16), so we would have to think
> carefully about whether to complain or risk loss of precision.

FYI - Your numpy almost certainly supports 10 byte floats, even if it claims to support 12 or 16 byte floats - the float96 and float128 types mean different things on different platforms, but only very rarely mean what they actually say on the tin. If you check the numpy discussion list this has been a somewhat controversial topic in recent weeks.

- Nathaniel