Reading Hdf5 files with many nodes performance

21 views
Skip to first unread message

Jochem Smit

unread,
Nov 15, 2019, 8:26:11 AM11/15/19
to pytables-users
Hi, I've already asked this question on stackoverflow, here, but I thought I'd post it here as well.

Basically what I'm asking is how to speed up the reading of hdf5 files where I have many nodes (~20-40k). I've tried multiprocessing to have 12 processes read simultaneously but that only reduced reading time by a factor of two.

Does anyone have any suggestions to improve the file reading speed?

Thanks,
Jochem
Reply all
Reply to author
Forward
0 new messages