When running "pysph test" and/or "pysph run <any_example>" ===> ModuleNotFoundError: No module named 'pysph.parallel.parallel_manager'

[alom...@gmail.com]

Hi,

 After struggling a little bit I managed to install PySPH with the parallel dependencies (Zoltan/Pyzoltan). I followed the instructions using Anaconda from the latest version of the docs (https://pysph.readthedocs.io/en/latest/installation.html#using-anaconda) and installed all the core requisites, then installed mpi4py using "conda install -c conda-forge mpi4py", all in a separate venv named sph_env which I activated before starting to install the before mentioned packages.

I also installed trilinos following PyZoltan's doc (https://pyzoltan.readthedocs.io/en/latest/start.html#setting-up-on-ubuntu) "Setting up on Ubuntu", and made sure to export the ZOLTAN variables before running "pip install pyzoltan". Finally, I ran "pip install PySPH" and got a successful installation. However, when I run "pysph test" it gives me the error in the title: ModuleNotFoundError: No module named 'pysph.parallel.parallel_manager'

When I installed PySPH without the the parallel dependencies I was able to run for example, "pysph run elliptical_drop" succesfully, but now, even that doesn't work. When I checked the folder <>/pysph/parallel/ I am missing the .SO files so I assumed it did not compile during PySPH installation. 

Can someone really please help me? I absolutely need the parallel capabilities since I want to run really heavy simulations in a computer cluster. I am not a CS guy so layman language would be appreciated.

Thank you so much for your time and consideration!

Kindest regards,

Alomir

p.s. I am using Ubuntu 18.04

[Prabhu Ramachandran]

On 1/31/19 12:56 AM, alom...@gmail.com wrote:

Hi,

 After struggling a little bit I managed to install PySPH with the parallel dependencies (Zoltan/Pyzoltan). I followed the instructions using Anaconda from the latest version of the docs (https://pysph.readthedocs.io/en/latest/installation.html#using-anaconda) and installed all the core requisites, then installed mpi4py using "conda install -c conda-forge mpi4py", all in a separate venv named sph_env which I activated before starting to install the before mentioned packages.

Sorry to hear of the difficulties.

I also installed trilinos following PyZoltan's doc (https://pyzoltan.readthedocs.io/en/latest/start.html#setting-up-on-ubuntu) "Setting up on Ubuntu", and made sure to export the ZOLTAN variables before running "pip install pyzoltan". Finally, I ran "pip install PySPH" and got a successful installation. However, when I run "pysph test" it gives me the error in the title: ModuleNotFoundError: No module named 'pysph.parallel.parallel_manager'

Did you by any chance do a pip install earlier without zoltan?  If so, it is possible that pip has cached a previous build.  Can you try this:

pip install --no-cache-dir pysph

While retaining the environment variable ZOLTAN.  If this too doesn't work, could you please send us the output of this command:

pip install -vvv --no-cache-dir pysph

That may help us debug this. 

Also, does this work?:

python -c "import pysph.parallel.parallel_manager"

When I installed PySPH without the the parallel dependencies I was able to run for example, "pysph run elliptical_drop" succesfully, but now, even that doesn't work. When I checked the folder <>/pysph/parallel/ I am missing the .SO files so I assumed it did not compile during PySPH installation.

That is interesting, so even though the parallel feature is not installed the elliptical drop itself should run, what is the error you are seeing?

Can someone really please help me? I absolutely need the parallel capabilities since I want to run really heavy simulations in a computer cluster. I am not a CS guy so layman language would be appreciated.

Sorry about these difficulties.  The parallel support is a bit tricky and we will try to see if the next release can clean up this.

cheers,

Prabhu

[alom...@gmail.com]

Hi Prabhu, 

Thank you so much for your reply. You were spot on! The problem was the cached PySPH build. I used "pip install --no-cache-dir pysph" and now the error is gone! I checked the parallel_manager folder and there is a .SO file there now. Also, I can run "pysph test" and it works (I get fails though, but they are related to GPU and OpenCL tests so I think this is fine for now). I can also run individual examples! Just a clarification: barring some hidden bug or problem, I should now be able to use OpenMP and MPI features of PySPH, is that correct?

Thank you once again for your help, and if it is of interest of anyone in the group, attached is a step-by-step how I completed my installation with parallel dependencies on Ubuntu and using Anaconda (no GPU though).

Best regards,

Alomir

[Prabhu Ramachandran]

On 1/31/19 11:53 PM, alom...@gmail.com wrote:

Hi Prabhu, 

Thank you so much for your reply. You were spot on! The problem was the cached PySPH build. I used "pip install --no-cache-dir pysph" and now the error is gone! I checked the parallel_manager folder and there is a .SO file there now. Also, I can run "pysph test" and it works (I get fails though, but they are related to GPU and OpenCL tests so I think this is fine for now). I can also run individual examples! Just a clarification: barring some hidden bug or problem, I should now be able to use OpenMP and MPI features of PySPH, is that correct?

Excellent, I am glad this worked and will update the documentation with this information.

Yes, it should work with openmp and mpi, you will need to setup the number of threads and hosts using the mechanisms provided by your cluster.

Thank you once again for your help, and if it is of interest of anyone in the group, attached is a step-by-step how I completed my installation with parallel dependencies on Ubuntu and using Anaconda (no GPU though).

Thanks, this is handy and I'll see if we can update our instructions on the docs to use any changes.

Also if you have any issues with the documentation please either submit a PR or let us know on the list.

cheers,

Prabhu

[Vasudha Kapre]

Hi Prabhu and Alomir, 

I am trying to install PySPH with MPI on Ubuntu 20.04 and ran into the error "No module named 'pysph.parallel.parallel_manager'" during pysph test, even after using "pip install --no-cache-dir pysph". I uninstalled and tried multiple times using the instructions attached in the following email, but keep getting the same error. Can you please help me figure out what I might be doing wrong? 

Thank you in advance, 

Vasudha

[Prabhu Ramachandran]

Hi Vasudha,

I suspect it has to do with the build isolation in more recent pip versions that is the default.  The fix is documented here:

https://pysph.readthedocs.io/en/latest/installation.html#installation-with-mpi

Basically use pip's --no-build-isolation argument.

HTH.
Prabhu

--
You received this message because you are subscribed to the Google Groups "pysph-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pysph-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pysph-users/87f996ce-66a3-458e-967f-84a6b5de3fddn%40googlegroups.com.