Compound datatypes with h5py: datatype inside an attribute

[Seth Johnson]

(I posted this, formatted as HTML instead of Markdown, at
http://stackoverflow.com/questions/5969828/compound-datatypes-with-h5py-datatype-inside-an-attribute
)

I'm using [Silo][1] with HDF5, and I'm having trouble accessing some
of the metadata with h5py. Silo spits out some rather unusual HDF5
structuring, where it puts a `DATATYPE` inside a `DATATYPE`. Here's an
excerpt of the output from `h5dump`:

DATATYPE "sigma_t" H5T_STD_I32LE;
ATTRIBUTE "silo" {
DATATYPE H5T_COMPOUND {
H5T_STRING {
STRSIZE 5;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
} "meshid";
H5T_STRING {
STRSIZE 15;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
} "value0";
H5T_STD_I32LE "ndims";
H5T_STD_I32LE "nvals";
H5T_STD_I32LE "nels";
H5T_IEEE_F32LE "time";
H5T_STD_I32LE "use_specmf";
H5T_STD_I32LE "centering";
H5T_ARRAY { [3] H5T_STD_I32LE } "dims";
H5T_ARRAY { [3] H5T_STD_I32LE } "zones";
H5T_ARRAY { [3] H5T_STD_I32LE } "min_index";
H5T_ARRAY { [3] H5T_STD_I32LE } "max_index";
H5T_ARRAY { [3] H5T_IEEE_F32LE } "align";
}
DATASPACE SCALAR
DATA {
(0): {
"mesh",
"/.silo/#000004",
2,
1,
100,
0,
-1000,
111,
[ 10, 10, 0 ],
[ 9, 9, 0 ],
[ 0, 0, 0 ],
[ 9, 9, 0 ],
[ 0.5, 0.5, 0 ]
}
}
}
ATTRIBUTE "silo_type" {
DATATYPE H5T_STD_I32LE
DATASPACE SCALAR
DATA {
(0): 501
}
}

Basically, `f['sigma_t'].attrs['silo']` returns a `tuple` with all of
the correctly formatted data but without any of the associated labels
for the data types. (I need to know the names `meshid`, `value0`,
etc.) Is there a way to get this? I'm at a loss.

Thanks,
Seth

[1]: https://wci.llnl.gov/codes/silo/index.html

[Andrew Collette]

Hi Seth,

> I'm using [Silo][1] with HDF5, and I'm having trouble accessing some
> of the metadata with h5py. Silo spits out some rather unusual HDF5
> structuring, where it puts a `DATATYPE` inside a `DATATYPE`. Here's an
> excerpt of the output from `h5dump`:

There is a longstanding bug in h5py whereby single elements of
compound types are returned as tuples instead of array scalars, which
I think is what you're bumping in to here. This is one of the things
I'm fixing in h5py 1.4.

Here is an (annoying) workaround for h5py 1.3.1 and earlier. For your
file object 'f' and the names you gave:

group = f['sigma_t']
attr_id = h5py.h5a.open(group.id, 'silo')
data = numpy.array(group.attrs['silo'], dtype=attr_id.dtype)

meshid = data['meshid'] # etc.

Sorry about this. :( In h5py 1.4 you'll be able to do
f['sigma_t'].attrs['silo']['meshid'], as expected.

Andrew

[Seth Johnson]

Andrew,

Thanks for your quick response! This works great.

Seth