MAB WESTPA sim fails after few iterations (can't read pcoord, has incorrect shape))

[Rebecca Walters]

Hi, 

Sorry, me again!

I am having some problems with running a steady state MAB WESTPA simulation with AMBER. I am trying to simulation a drug unbinding from a protein. The progress coordinate that I am using is a pre-recorded MD pathway (100 frames) in reduced dimensional space. I am happy to go into more detail about this if need be. 

Basically, my simulation seems to be fine when I initialise it, and it seems to run for a couple of iterations before crashing. The error I get is:

-- ERROR    [westpa.core.propagators.executable] -- could not read pcoord from '/tmp/tmp_oa0_0q7': ValueError('progress coordinate data has incorrect shape (0, 1) [expected (100, 1)] Check pcoord.err or seg_logs for more information.')

I am attaching the west.log for the full output. I have checked the pcoord.err and seg_logs as suggested (also attached) however I don't think they provide more insight into my issue. ('Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL' which appears in my seg log also appears in the seg logs of the successful iterations, so I don't think this is causing the issue).

I thought it might have something to do with the fact that I am running a steady state calculation, but to be honest I am a little lost with where to go from here...

I am attaching the pcoord.err, a relevant seg log, the tstate.file, adaptive.py, west.cfg, and west.log. Any help would be greatly appreciated!!

Kind regards, 
Rebecca

[Anthony Bogetti]

Hi Rebecca,

Thanks for reaching out.  A couple of thoughts on what you've sent and a couple of requests for more files:

From the seg log that you attached, it seems as though the dynamics and pcoord calculation are being executed properly, so at least the general steps that WESTPA goes through are okay.  Let's just check and make sure that those steps are completing and giving you what you expect. 

In this third iteration, are all of the segments failing and giving this error or is it just a few?  This can help us to focus our troubleshooting efforts.  My guess is that it is just a few since your simulation seems fine up until the third iteration.  I wonder if this has something to do with your progress coordinate being very large, it may be worth it to try using the log of the value as your pcoord or dividing through by a common number so the numbers are between 0 and 100 or something like that.  If you are able to identify failing segments, is there anything suspicious about their progress coordinate values?

In your traj_segs, are there seg.log files and seg.rst files for trajectories that completed successfully?  Would you be able to attach one of the seg.log files?  This will confirm that the dynamics are completing successfully.

In addition, would you be able to attach your runseg.sh and the script you use to calculate your progress coordinate?  Also, are you able to save the progress coordinate values to a file after they are calculated?  This can help us confirm that the pcoord calculation script is giving you what you expect.

Best,

Anthony

--
You received this message because you are subscribed to the Google Groups "westpa-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to westpa-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/westpa-users/48cd3f11-2e13-4224-a41b-ca025369baa0n%40googlegroups.com.

[Rebecca Walters]

Hi Anthony, 

Thank you for your quick response! When I was looking through your questions I realised that the progress coordinate was not correct. I was off by a few factors of 10.... whoops! I am attaching the new adaptive.py, west.cfg and west.log. However, I have re-run and the simulation still crashes with the same error, so, to answer your questions:

1. In this third iteration, are all of the segments failing and giving this error or is it just a few? 
   

1. I am not sure... the ERROR message appears 17 times in the west.log and there are 24 segments in iteration 3. However, when I check the traj_segs folder I see that all of the MD trajectories have been completed, they are just all lacking in a 'pcoord.dat' type file.
   
   

1. In your traj_segs, are there seg.log files and seg.rst files for trajectories that completed successfully? 
   

1. I am attaching three seg.log files from iteration 3 (24 segments in total). All seem to have completed dynamics successfully, so I am just adding three to see if you can see if a difference. 
   
   

1. In addition, would you be able to attach your runseg.sh and the script you use to calculate your progress coordinate?
   

1. Yes sure, I will attach these. The script that calculates the progress coordinate is called 'chemdyme_progress_coord.py'. I should note that I am using a python library called ChemDyME to help me calculate how far along the progress coordinate path I am. I think I sent you an email a while ago explaining that I was using an interactive MD in virtual reality pathway in reduced dimensional space as the progress coordinate, so it's a little unconventional. 
   
   

1. Also, are you able to save the progress coordinate values to a file after they are calculated?
   

1. Yes, I am attaching one of the successful trajectory's progress data file (chemdyme_prog.dat). It seems to be going in the wrong direction overall, since I want it to decrease towards 200, rather than increase, the starting pcoord value was around 320.
   
   

Sorry if this all seems a bit confusing, and thank you for your help. Please let me know if you need anymore info from me. 

Kind regards, 

Rebecca

[Rebecca Walters]

I am not sure why my numbering has gone all funny for answering your questions, apologies!!

Kind regards, 

Rebecca

[Anthony Bogetti]

Thanks Rebecca!  Jeremy and I just took a look at your files and have a few thoughts.  It seems like from the seg_logs you sent that the dynamics are completing successfully and the pcoord calculation script is at least being executed.  In iteration 3, I have a feeling that for some of the trajectories the pcoord calculation script is not completing, which causes a blank file to be read into $WEST_PCOORD_RETURN giving you the error you initially encountered (pcoord not correct shape).  Could you try running the pcoord calculation script outside of WESTPA on some of the trajectories that failed and see if you get errors there?  Also, could you attach your west.h5 file so we can take a look at that?

Best,

Anthony

To view this discussion on the web visit https://groups.google.com/d/msgid/westpa-users/ff4f6898-597c-4093-a7d1-4211cdf18843n%40googlegroups.com.

[Rebecca Walters]

Hi Anthony and Jeremy, 

I followed your advice and ran my pcoord calculation script outside of WESTPA on a few trajectories that failed, and it managed to complete in the way I'd expect. I am attaching the output progress coordinate data file that was outputted for one of them ('test_westpa_anthony.dat').

I am also attaching my west.h5 file for you to both look at. 

Thank you both for your help so far. 

Kind regards, 

Rebecca