process hung on h5py.File()

[ndjensen]

Hi, I have a weird problem that I'm unable to find a solution to yet.
One of our processes corrupted an hdf5 file thanks to a user killing
the process in the middle of a write. That's fine, not much we can do
about that, but I need to be able to detect that the file is corrupted
and notify them that file is bad.

Here's the weird part. Whenever one of processes goes to open the
corrupted file, it hangs on the h5py.File() constructor, specifically
in the _generate_fid() method on the fid = h5f.open(name,
h5f.ACC_RDONLY, fapl=plist) line. It never returns and doesn't throw
an exception.

Attempting to debug this, I got the corrupted file and attempt to open
it with the cmd line python interpreter in the same way with f =
h5py.File(filename, mode). It kindly throws an exception there, not
hanging.

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/common/njensen/awips2/python/lib/python2.6/site-packages/h5py/
highlevel.py", line 793, in __init__
self.fid = self._generate_fid(name, mode, plist)
File "/common/njensen/awips2/python/lib/python2.6/site-packages/h5py/
highlevel.py", line 827, in _generate_fid
fid = h5f.open(name, h5f.ACC_RDONLY, fapl=plist)
File "h5f.pyx", line 68, in h5py.h5f.open (h5py/h5f.c:1237)
h5py._stub.IOError: Truncated file (File accessability: File has been
truncated)


That would be good enough, but like I said, I'm hung when it's
actually run through our service. The notable difference between cmd
line python and the other process is that the hung one is run through
mod_wsgi. But it is single-thread per process, same python install, I
tried the same user, guaranteed it has a lock on the file and is the
only one attempting to access it, etc. h5py has worked great in this
environment except in this strange case.

I tried to detect corruption before even attempting to open the file,
by using h5py.is_hdf(filename) but that returns True and I couldn't
find any other way to try and detect corruption before the file is
opened. We're using h5py 1.3.0 and hdf5 1.8.4-patch 1.

If you have any ideas about why it is hanging, or how I could detect a
corrupted file earlier, they would be greatly appreciated.