numpy.float128

[nils]

Hi,

I noticed that h5py refuses to automatically create an hdf5 type
corresponding to the long float type numpy.float128. There is even a
unittest assuring this.

Is it impossible to store long floats in hdf5? Is there a workaround?
I need to store many 0-rank numpy arrays and very few rank 4 arrays of
that type.

Incidentally, I may need even more precision and was considering
either Decimal or mpmath for that. Is there a way to make numpy object
arrays of those and then also store them in h5py?

Cheer, Nils

[Paul Anton Letnes]

> --
> You received this message because you are subscribed to the Google Groups "h5py" group.
> To post to this group, send email to h5...@googlegroups.com.
> To unsubscribe from this group, send email to h5py+uns...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/h5py?hl=en.

Hi!

I am not saying this is a good idea, but there is a simple hack. Numpy arrays do not change the bits if you change the datatype. Consider:
>>> from numpy import *
>>> a = arange(4, dtype=int8)
>>> a
array([0, 1, 2, 3], dtype=int8)
>>> a.dtype=int16
>>> a
array([256, 770], dtype=int16)

This means you could take your array, set the dtype to float64 (or int8 for that matter, it should not be important) and save it as a twice as long array. Just remember to read it back in the same manner!

Of course, if you are planning on reading from a different language, this is probably a horrible idea.

Good luck,
Paul.

[Andrew Collette]

Hi Nils,

> Is it impossible to store long floats in hdf5? Is there a workaround?
> I need to store many 0-rank numpy arrays and very few rank 4 arrays of
> that type.

There's no "high-level" support for 128-bit floats at the moment. You
can of course create the appropriate data types and do I/O using the
low-level interface, although that might be more work than you want.
I'm not particularly opposed to supporting long floats; the unit test
is there to ensure the type-translation system doesn't go off into the
woods.

> Incidentally, I may need even more precision and was considering
> either Decimal or mpmath for that. Is there a way to make numpy object
> arrays of those and then also store them in h5py?

Not presently... one of the design choices for h5py was to avoid
storing types that aren't "native" HDF5. For numbers, that means IEEE
floating-point (whatever the precision), and integers. AFAIK the new
IEEE decimal type is not supported by HDF5 at the moment.

Andrew

[nils]

Thanks for the info! I guess I'll try to figure out if the homemade
HDF type is fairly painless, and resort to the hack otherwise...
Nils