Basically what I'm asking is how to speed up the reading of hdf5 files where I have many nodes (~20-40k). I've tried multiprocessing to have 12 processes read simultaneously but that only reduced reading time by a factor of two.
Does anyone have any suggestions to improve the file reading speed?