Is h5py visit and visititems broken?

[Donald Ellis]

When I use either h5py visit or visititems, on the file h5ex_g_visit.h5 (downloaded from http://mirror.fcaglp.unlp.edu.ar/pub/ftp.hdfgroup.org/HDF5/examples/examples-by-api/files/exbyapi/h5ex_g_visit.h5 ) neither work as expected.

A simple program:

import h5py

def print_objs (name):
     print (name)

fd = h5py.File('h5ex_g_visit.h5')
fd.visit(print_objs)

It prints

group1 
group1/dset1 
group1/group3 
group1/group3/group4 
group1/group3/group4/group1 
group1/group3/group4/group2

I think it should print

group1
group1/dset1
group1/group3
group1/group3/dset2
group1/group3/group4
group1/group3/group4/group1
group1/group3/group4/group1/group5
group1/group3/group4/group2 group2
group2/dset2
group2/group4
group2/group4/group1
group2/group4/group1/group5
group2/group4/group1/group5/dset1
group2/group4/group1/group5/group3
group2/group4/group1/group2

You get the same missing objects when using visititems.

I looks like it finds the first group in a level and follows that path, without ever returning to pick up other groups and datasets in that level. It also doesn't seem to go below 4 levels.

This works correctly for the similar function in C.

Are these python methods, the HDF5 file, or me broken?

Running Fedora 31 and python 3.7.8. The h5py is version 2.10.0.

Thanks

[Ken Walker]

I couldn't find your file at the URL you provided. I found a file with the same name at this location:

http://mirror.fcaglp.unlp.edu.ar/pub/ftp.hdfgroup.org/HDF5/examples/examples-by-api/matlab/HDF5_M_Examples/

Check the contents of this file: h5ex_g_visit_output.txt (same folder).

It shows there are both objects and links. The Output of h5dump gives the hardlinks

I suspect the h5py visitor functions only "visit" objects and not links?

Objects in the file:
/  (Group)
/group1 (Group)
/group1/dset1 (Dataset)
/group1/group3 (Group)
/group1/group3/group4 (Group)
/group1/group3/group4/group1 (Group)
/group1/group3/group4/group2 (Group)

Links in the file:
/group1 (Group)
/group1/dset1 (Dataset)
/group1/group3 (Group)
/group1/group3/dset2 (Dataset)
/group1/group3/group4 (Group)
/group1/group3/group4/group1 (Group)
/group1/group3/group4/group1/group5 (Group)
/group1/group3/group4/group2 (Group)
/group2 (Group)

-Ken

[Ray Osborn]

I haven't looked at the file closely enough to see whether soft links are used, but I don't think that's the issue. The h5py visit function will only ever return one instance of each object, I think in order to prevent infinite recursions. This is a weird file that looks like it is designed to generate infinite recursions, since 'group2' is referenced at both the root level and as 'group1/group3/group4/group2'. 

>>> f=h5py.File('/Users/rosborn/Downloads/h5ex_g_visit.h5', 'r')

>>> f.keys()

<KeysViewHDF5 ['group1', 'group2']>

>>> def print_name(name):

    print(name)

>>> f.visit(print_name)

group1

group1/dset1

group1/group3

group1/group3/group4

group1/group3/group4/group1

group1/group3/group4/group2

So 'group2', which is at the root level of the file, has already been returned as 'group1/group3/group4/group2' before the visit function gets back to the root, and the root level reference is ignored. You would have to create your own walk function to get around this, but the version I already use fell into the infinite recursion trap, so you have to be careful.

Ray 

[Thomas Kluyver]

Just to back up what Ray said: HDF5 links (both soft links and hard links) allow cycles, and the 'visit' functions keep track of which objects they've already seen and skip over them, so they can't get stuck in an infinite loop.

h5py's high-level Group.visit() calls H5Ovisit, which goes through objects. The low-level API also exposes H5Lvisit - accessible as group.links.visit() - which goes through links. This should include e.g. '/group2' - a second link to the same object as '/group1/group3'. But it won't list anything under /group2, because that group was already visited.

Thomas

--
You received this message because you are subscribed to the Google Groups "h5py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h5py+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/h5py/90d5a910-cba2-44ba-8c92-042ff30d9d4bo%40googlegroups.com.

[Donald Ellis]

Thanks, That clears things up.