Accessing restart files in HDF5 framework outside of WESTPA?

[Hayden Scheiber]

Hello,

My question is fairly straightforward. Let's say I'm running a WESTPA simulation with the per-iteration HDF5 framework enabled. As I understand it, at each iteration, my simulation restart files are combined into a tar.gz file and further archived into the per-iteration HDF5 file (usually located in $WEST_SIM_ROOT/traj_segs/iter_XXXXXX.h5). At the next iteration, WESTPA automatically unpackages these restart files and passes them to the next iteration.

I would like to know the mechanism that is used to access and unpackage these restart files myself, as I would like to access and unpack a subset of these restart files back into their original file types to use as basis states for another westpa simulation.

Any examples of how this could be done would be greatly appreciated.

Thanks,

Hayden

[Leung, Jeremy]

Hi Hayden,

This is a two step process. Restart files are essentially tarred up into the per-iter-h5['restart']['ITER_SEG']['data'] dataset.

1. Load the per-iter H5 file using h5py. These lines in the tutorial files should serve the an example for how to load it.

2. Untar the file. You can reference the restart_writer function to open it in ByteIO, remove the tail protection, then untar it.

-- JL

---
Jeremy M. G. Leung
PhD Candidate, Chemistry
Graduate Student Researcher, Chemistry (Chong Lab)
University of Pittsburgh | 219 Parkman Avenue, Pittsburgh, PA 15260
jml...@pitt.edu | [He, Him, His]

--
You received this message because you are subscribed to the Google Groups "westpa-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to westpa-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/westpa-users/445f5aad-10e0-4652-ae2e-c2a3a2bbf450n%40googlegroups.com.

[Hayden Scheiber]

Hi Jeremy,

Thanks for your reply. I am having difficulty following step 2. In restart_writer, there is a line 

restart = segment.data.pop('iterh5/restart', None)

How do I properly load the segment object? I have tried without loading the segment object:

import h5py
import westpa
from io import BytesIO
from westpa.core.h5io import safe_extract
import tarfile

iseg=0
n_iter=1
iter_h5_filepath = "traj_segs/iter_"+str(int(n_iter)).zfill(6)+".h5"
data = h5py.File(iter_h5_filepath)['restart/'+str(n_iter)+'_'+str(iseg)+'/data']
d = BytesIO(data[()][:-1])
with tarfile.open(fileobj=d, mode='r:gz') as t:
    safe_extract(t, path='traj_segs/'+str(int(n_iter)).zfill(6))

But this gives an error: 

EOFError: Compressed file ended before the end-of-stream marker was reached

I have also tried generating a segment object by importing from westpa

from westpa.core.segment import Segment

segment = Segment
segment.n_iter=n_iter
segment.seg_id=iseg
segment.data['iterh5/restart'] = d.getvalue() + b'\x01'

But this results in an error:

AttributeError: type object 'Segment' has no attribute 'data'

How do I properly load the segment object with data?

Thanks,

Hayden

[Leung, Jeremy]

Hi Hayden,

You have to open the per-iter HDF5File with the WESTIterationFile class.

```

import h5py

import westpa

from io import BytesIO

from westpa.core.h5io import WESTIterationFile, safe_extract

import tarfile

iseg=0

n_iter=1

iter_h5_filepath = f"traj_segs/iter_{n_iter:>06}.h5"

with WESTIterationFile(iter_h5_filepath) as file:

    data = file.read_data(f'/restart/{n_iter}_{iseg}', 'data')

with tarfile.open(fileobj=BytesIO(data[:-1]), mode='r:gz') as t:

    safe_extract(t, path='traj_segs/{n_iter:>06}')

```

Have a good weekend!

Best,

JL

---
Jeremy M. G. Leung
PhD Candidate, Chemistry
Graduate Student Researcher, Chemistry (Chong Lab)
University of Pittsburgh | 219 Parkman Avenue, Pittsburgh, PA 15260
jml...@pitt.edu | [He, Him, His]

To view this discussion on the web visit https://groups.google.com/d/msgid/westpa-users/3ebf3696-2b6e-4835-9fee-23580c9735f6n%40googlegroups.com.

[Hayden Scheiber]

Awesome, that works. Thanks Jeremy!