GenPipes on Graham: no module named configparser

[Shraddha Pai]

Hi,

I'm setting up to run GenPipes on Graham. Running "hicseq.py -h" or "methylseq.py -h" returns this error:

Traceback (most recent call last):
  File "/cvmfs/soft.mugqic/root/software/genpipes/genpipes-4.3.1/pipelines/hicseq/hicseq.py", line 33, in <module>
    from bfx import bash_cmd as bash
  File "/cvmfs/soft.mugqic/root/software/genpipes/genpipes-4.3.1/bfx/bash_cmd.py", line 24, in <module>
    from core.job import Job
  File "/cvmfs/soft.mugqic/root/software/genpipes/genpipes-4.3.1/core/job.py", line 28, in <module>
    from .config import config
  File "/cvmfs/soft.mugqic/root/software/genpipes/genpipes-4.3.1/core/config.py", line 21, in <module>
    import configparser
ImportError: No module named configparser

Running "module spider configparser" shows that it's only available for python 2.7.18.:

[spai@gra-login2 GenPipes]$ module spider configparser

-------------------------------------------------------------------------------------------
  configparser: configparser/4.0.2 (E)
-------------------------------------------------------------------------------------------
    This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy.


       python/2.7.18 (StdEnv/2020)

And indeed, if I try to run "module load configparser" I get an incompatibility error.

Can you please help resolve this?

Thanks,Shraddha

[Édouard Henrion]

Hi Shraddha,

You have to use Python3 for Genpipes.

I suggest you load mugqic/python/3.10.4

configparser exists in python3 but was originally called ConfigParser in Python2, so using python2 leads to this error.

Thanks.

[Shraddha Pai]

Thanks Edouard, that fixed it. 

The documentation on this page needs to be updated because it still points to an older version of python. 

https://genpipes.readthedocs.io/en/genpipes-v-3.6.2/deploy/access_gp_pre_installed.html

Am now encountering errors running "bash methylseq.sh". I'm trying to run the bismark methylseq pipeline using the test dataset available here: https://genpipes.readthedocs.io/en/genpipes-v-3.6.2/resources/testdataset.html

Separately, in methylseq.base.ini,  should the "cluster_server" variable be changed to something Graham-specific? And anything else to change for the environment?

This was the code to generate methylseq.sh:

----

export MUGQIC_INSTALL_HOME=/cvmfs/soft.mugqic/CentOS6
module use $MUGQIC_INSTALL_HOME/modulefiles
module load mugqic/python/3.10.4
module load mugqic/genpipes/4.3.1
export JOB_MAIL=<removed>
export RAP_ID=<removed>

configFile=/path/to/locally/downloaded/methylseq.base.ini ## same as default
inDir=/scratch/spai/WGBS_GenPipes_test/input/final
outDir=/scratch/spai/WGBS_GenPipes_test/output/final

methylseq.py -c $configFile \
        --readsets ${inDir}/readset.methylseq.txt \
        --output-dir $outDir \
        --steps 1-14 \
        --type bismark \
        --genpipes_file methylseq.sh

---

Then when I run the .sh file:

---

[spai@gra-login2 GenPipes]$ bash methylseq.sh
sbatch: error: Invalid --distribution specification

---

Thanks,Shraddha

[Édouard Henrion]

It seems you are looking at the doc of GenPipes 3.6.2, which was still using Python2.

But you are running GenPipes 4.3.1, thus the need of python3 ;)

For Methylseq specifically, you should look here : https://genpipes.readthedocs.io/en/genpipes-v4.3.1/user_guide/pipelines/gp_wgs_methylseq.html?highlight=methylseq

You'll see that in your case you have to specify the ini files as follow : 

  -c methylseq.base.ini methylseq.graham.ini

The methylseq.graham.ini file contains graham-specific values and instructions.

Best,

Edouard

[edouard.henrion]

Hi Shraddha,

Sorry for the confusion.

We indeed recently revised the way we store our ini files...

Now server-specific ini files can be found in : `${MUGQIC_PIPELINES_HOME}/pipelines/common_ini/`

There you will find the `graham.ini` which is to be used with all our pipelines on Graham.

Best,

Shraddha

[Shraddha Pai]

Thanks Edouard, that worked. The pipeline ran from end to end and I am able to see the output files. 

Appreciate all the help.

Best,Shraddha