Troubles with installation/execution on a cluster

[Ildar]

Hi, dear devs!

I tried to install gmx_MMPBSA on a cluster via conda (4.8.5 version) using the env.yml file.
When I try to test it by sending the task to slurm, I receive the following error:

/s/ls4/users/username/.conda/envs/gmxMMPBSA/etc/conda/activate.d/gromacs_activate.sh: line 16: /home/users/username/.conda/envs/gmxMMPBSA/bin.SSE2/GMXRC: No such file or directory

The curious thing is that GROMACS have installed and it exists in conda environment list.
When I execute gmx_MMPBSA_test from the command line, being connected to the cluster, it seems to work, but sending the test task(gmx_MMPBSA_test) via sbatch raises the error written above.

Are there options to solve it without root privileges? The point is to make it work ASAP, without introducing system admins.


Kind regards,
Ildar

[Mario Sergio Valdes]

Hi Ildar. 

Sure! We will help you.

Could you show me the config file you are using?

The problem seems to be the file paths. As you can see, the path for the activation script is "/s/ls4/users/username/" while the internal path in this script is "/home/users/username/" Can you check is this solution (https://github.com/Valdes-Tresanco-MS/gmx_MMPBSA/issues/271) works for you?

[Ildar]

There is the task file:

#!/bin/sh
#SBATCH -D /s/ls4/users/coconut/software  #launch directory
#SBATCH -o %j.out
#SBATCH -e %j.err
#SBATCH -p hpc5-el7-gpu-3d
#SBATCH --gres=gpu:4
#SBATCH -n 32
#SBATCH -t 72:00:00



module load anaconda3/default  
source activate /s/ls4/users/coconut/.conda/envs/gmxMMPBSA

gmx_MMPBSA_test  -t 0 -n 10



I didn't notice the file paths, you are right about paths. A path on the cluster is kinda tricky: if use pwd it shows '/home/users', but in fact, it is '/s/ls4/users' and this may really cause an error. But I have no idea how to fix it. Maybe need to change the path in conda environment config file manually?

Also, I tried the suggested solution(https://github.com/Valdes-Tresanco-MS/gmx_MMPBSA/issues/271) right now and it doesn't work, look error below. (And it feels quite irrational to build conda env in every task.)

Resolving valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)... 185.199.108.153, 185.199.109.153, 185
.199.110.153, ...
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|185.199.108.153|:443... failed: Conn
ection timed out.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|185.199.109.153|:443... failed: Conn
ection timed out.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|185.199.110.153|:443... failed: Conn
ection timed out.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|185.199.111.153|:443... failed: Conn
ection timed out.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|2606:50c0:8000::153|:443... failed:
Network is unreachable.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|2606:50c0:8001::153|:443... failed:
Network is unreachable.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|2606:50c0:8002::153|:443... failed:
Network is unreachable.
Connecting to valdes-tresanco-ms.github.io (valdes-tresanco-ms.github.io)|2606:50c0:8003::153|:443... failed:
Network is unreachable.

NotWritableError: The current user does not have write permissions to a required path.
 path: /home/users/coconut/.conda/envs/.conda_envs_dir_test
 uid: 1768
 gid: 1069

среда, 19 апреля 2023 г. в 02:33:18 UTC+8, mariosergi...@gmail.com:

[Mario Sergio Valdes]

Oh, I see... Probably, once you enter the cluster, the internet connection will be lost to avoid security failures. As I checked, this seems to be the best option https://rabernat.medium.com/custom-conda-environments-for-data-science-on-hpc-clusters-32d58c63aa95.
In this case, conda is installed using relative paths. If your home on the HPC was not pointing to your local home, you can create a new variable with the custom path and pass it to the conda installation. Then you define it in your config file, and it should work

[Ildar]

Unfortunately, this doesn't work for me  - I can't install miniconda with python 3.9. I face the following error:

PREFIX=/s/ls4/users/coconut/software/miniconda
Unpacking payload ...
/s/ls4/users/coconut/software/miniconda/conda.exe: /lib64/libc.so.6: version `GLIBC_2.14' not found (required
by /s/ls4/users/coconut/software/miniconda/conda.exe)

Maybe you have another idea on how to overcome my issue?

четверг, 20 апреля 2023 г. в 00:43:37 UTC+8, mariosergi...@gmail.com:

[Ildar]

I'm delighted to notify you that I solved my problem by remaking conda env with the --prefix key with the explicit cluster path.

Thank you so much for your response and help, Mario!
(See you in the next topics xD)

четверг, 20 апреля 2023 г. в 20:15:39 UTC+8, Ildar:

[Mario Sergio Valdes]

Oh, that is good news... You´re welcome. For any help, contact us.

Cheers!

Mario S.