The road to h5py 3.0

[Thomas Kluyver]

Hi all,

While many of us are stuck at home, I'm hoping we can get h5py version 3 out. This is due to include various important changes - some already merged, some still planned. They include:

• Releasing the GIL while reading & writing data, so other Python code can run in threads while HDF5 is working
• N.B. HDF5 has its own coarse-grained lock, so this won't enable thread-level parallel reading/writing
• Changes to how storing and reading strings work, see plans at: https://github.com/h5py/h5py/issues/1338
• Support for HDF5 1.12 (in progress at the moment)
• Reduced overhead for reading small amounts of data
• Changing the default file mode ( File("a.h5") ) to read-only
• Requiring Python 3

These are just some of the most visible changes - there are over 100 merged PRs since 2.10 to fix bugs, clarify documentation and improve maintainability.

So why am I writing about this now? Firstly, if there's anything you'd really like to see in h5py 3.0 that hasn't already been done, now's the time to get on it! I hope that after 3.0 we'll be able to start doing smaller, more frequent releases, so please don't feel that this is the last chance to get your changes in. But it's a good chance if you want to get changes in.

Second, there are pull requests waiting for review (including some of mine - I'm being selfish here ;-). If you're keen to see h5py 3.0 land, maybe you can help out by doing some reviewing?

Thanks,

Thomas