transpose in PointSelection

[Matthew Turk]

Hi there,

In profiling our code, which uses point-based selection for some parts
of the IO, I'm seeing a lot of time spent inside the call to transpose
in h5py. This specifically occurs inside __getitem__ in
PointSelection, line 216 in _hl/selections.py. Is there a way to get
around this call? We control the generation of the points we're
selecting, so selecting them in a different order actually isn't such
a big deal.

Any suggestions would be appreciated! Thanks,

Matt

[Andrew Collette]

Hi Matt,

> In profiling our code, which uses point-based selection for some parts
> of the IO, I'm seeing a lot of time spent inside the call to transpose
> in h5py.  This specifically occurs inside __getitem__ in
> PointSelection, line 216 in _hl/selections.py.  Is there a way to get
> around this call?  We control the generation of the points we're
> selecting, so selecting them in a different order actually isn't such
> a big deal.

If you're currently using a boolean array to do point selection, you
could create a new PointSelection object and then use append() to
enter the specific points you want. However, I have to warn you that
as of h5py 2.0 everything in the selections module is non-public and
will probably go away soon, so this may not be a good long-term fix
for you.

You may want to consider using the standard HDF5 "C-style" interface
in h5py.h5s and h5py.h5d to do reads and writes with point-based
selections. It's a little clunkier than using Dataset objects but
provides the absolute best performance possible with h5py.

Andrew

[Matthew Turk]

Hi Andrew,

Ah, brilliant! That worked very well. We do our point selection
inside cython code already so constructing the index array directly
and feeding this to select_points was very easy and reduced our
overall time substantially. Thanks!

-Matt

>
> Andrew
>
> --
> You received this message because you are subscribed to the Google Groups "h5py" group.
> To post to this group, send email to h5...@googlegroups.com.
> To unsubscribe from this group, send email to h5py+uns...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/h5py?hl=en.
>