msm_we: Stratified clustering gives "ModuleNotFoundError: No module named 'msm_we'" error

[Akshay]

WESTPA community,

I am testing the new tutorials (on WESPTA-2022.02) along with the msm_we package for ha-msm analysis. Specifically, I am working on a post-analysis of one of my earlier WESTPA simulation using the msm_we module.

I have succeeded in loading/initalizing the .h5 file and in reducing the dimensionality of the system. However the command: msm_we.modelWE().cluster_coordinates(nclusters=1, streaming=True, first_cluster_iter=100, use_ray=True, stratified=True) throws the following error:

"ModuleNotFoundError: No module named 'msm_we'"

(The full error ouput is attached)

There is no error on using aggregate clustering (stratified=False, use_ray=False). I believe this has something to do with "ray" but I am not sure. Could someone please help with resolving this.

Thanks,

Akshay

PhD student, IIT 

[Leung, Jeremy]

Hi Akshay,

Thanks for writing in. I'll try to help but I might need a little more information.

Since you were able to call the functions correctly, indeed it seems like the ray work manager doesn't have access to the `msm_we` module. 

1) Do you get the same error when you use (stratified=False, use_ray=True)?

2) How did you start your ray server (`ray.init()`)? Is it in the same notebook or was it in another notebook in a different environment?

3) Did you install `msm_we` using this environment file? 

If John Russo can chime in at any time, that'll be great!

Thanks,

Jeremy L.

--
Jeremy M. G. Leung
PhD Candidate, Chemistry
Graduate Student Researcher, Chemistry (Chong Lab)
University of Pittsburgh | 219 Parkman Avenue, Pittsburgh, PA 15260
jml...@pitt.edu | [He, Him, His]

--
You received this message because you are subscribed to the Google Groups "westpa-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to westpa-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/westpa-users/b389ac76-6a02-4b0f-9ebd-f310140905adn%40googlegroups.com.
<err_clustering.txt>

[rus...@ohsu.edu]

Hi,

Looks like there's an issue with the Ray workers' environments, likely an issue with how the server is being launched as Jeremy alluded to.

Jeremy's followup questions will help us pin it down, I'll stay tuned for answers to those and then we should be able to sort this out.

-John

[Akshay]

Hello Jeremy and John,

Thanks a lot for the answers. 

To answer the questions:

1) Yes, I get the same error on using (stratified=False, use_ray=True).

2) The command used for initiating the ray server is: ray.init(num_cpus=4) in the same notebook. I've attached the output logs on starting it.

3) I  installed msm_we from https://github.com/jdrusso/msm_we. Both environment files are the same.

Thanks,

Akshay

[rus...@ohsu.edu]

Can you please provide the full code you're running?

[Akshay]

Sure, I've attached the same. 

Thanks,

Akshay

[rus...@ohsu.edu]

If you've installed `msm_we` correctly, you won't need to manually add it to your `sys.path`. My guess is it's not correctly installed, and the Ray workers don't pick up that modified path. Therefore the main process can find msm_we, but the Ray workers can't.

You can do the following steps:

1. Activate your conda environment

2. cd /home/chemical/phd/chz198152/apps/msm_we

3. pip install .

Now you should not need to modify sys.path, and should be able to directly import msm_we. Additionally, your workers should be able to find it.

[Akshay]

Thank you very much John, that solved my issue.