HiC-Pro installation: ImportError: Error - bx-python cannot be imported

[Pim Rullens]

I am currently trying to install HiC-Pro on a cluster using a virtual environment. I run into this error (subject) as soon as I run ```$ make configure```. Previous questions addressing the same error don't really help.

Steps I have taken to install HiC-Pro:

```

# prior to this installation samtools 1.10, bowtie 2.3.4 and R 3.6.3 were installed, all are in my path. 

# create virtual environment with Python dependencies 

$ virtualenv --python=/usr/bin/python2.7 /usr/bin/venv/HiC-Pro

$ source activate /usr/bin/venv/HiC-Pro/bin/activate

(HiC-Pro) $ pip install pysam

(HiC-Pro) $ pip install bx-python

(HiC-Pro) $ pip install numpy

(HiC-Pro) $ pip install scipy

(HiC-Pro) $ pip freeze

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support

bx-python==0.8.9

numpy==1.16.6

pysam==0.16.0.1

scipy==1.2.3

six==1.15.0

# installing HiC-Pro

(HiC-Pro)  $ wget https://github.com/nservant/HiC-Pro/archive/v2.11.4.tar.gz

(HiC-Pro)  $ tar -zxvf v2.11.4.tar.gz

(HiC-Pro)  $ cd HiC-Pro_2.11.4 

(HiC-Pro)  $ nano config-install.txt 

#########################################################################

## Paths and Settings - Start editing here !

#########################################################################

 

PREFIX =

BOWTIE2_PATH =

SAMTOOLS_PATH =

R_PATH = /usr/bin/R-3.6.3/bin/R

PYTHON_PATH =

CLUSTER_SYS =

# configure

(HiC-Pro)  $ make configure

make -f ./scripts/install/Makefile CONFIG_SYS=./config-install.txt

make[1]: Entering directory `/usr/bin/HiC-Pro-2.11.4'

./scripts/install/install_dependencies.sh -c ./config-install.txt -o /usr/local/bin/HiC-Pro_2.11.4 -q

Make sure internet connection works for your shell prompt under current user's privilege ...

Starting HiC-Pro installation ...

export /home/hub_kind/prullens/tmp/R-3.6.3/bin/R in PATH

Checking dependencies ... 

Checking Python libraries ...

Traceback (most recent call last):

  File "../scripts/install/check_pythonlib.py", line 28, in <module>

    raise ImportError('Error - bx-python cannot be imported')  

ImportError: Error - bx-python cannot be imported

Can not proceed without the required Python libraries, please install them and re-run

make[1]: *** [configure] Error 1

make[1]: Leaving directory `/home/hub_kind/prullens/tmp/HiC-Pro-2.11.4'

make: *** [configure] Error 2

```

Best,

[nservant]

Hi 

I think it is now using the right python version.

Could you try to manually set the line 

PYTHON_PATH =

with the path to your python in conda ?

Best

[Pim]

Thank for the quick reply!

I have now manually set: PYTHON_PATH= /usr/bin/venv/HiC-Pro/bin/ 

but: $ make configure

unfortunately still results in the same error.

[nservant]

Are you sure that your `pip` command is from your conda ? and thus install the tools in your conda ?

Could you just try ;

/usr/bin/venv/HiC-Pro/bin/ python

>> import bx.intervals

or 

>> import bx

[Pim]

Yes I did pip inside die venv! Following you suggestions:

$ /usr/bin/venv/HiC-Pro/bin/python

>>> import bx

>>> import bx.intervals

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/usr/bin/venv/HiC-Pro/lib/python2.7/site-packages/bx/intervals/__init__.py", line 7, in <module>

    from bx.intervals.intersection import *  # noqa: F40

ImportError: /usr/bin/venv/HiC-Pro/lib/python2.7/site-packages/bx/intervals/intersection.so: undefined symbol: PyUnicodeUCS2_FromStringAndSize

Strangely only bx.intervals gives an error. Am I doing something wrong?

[nservant]

I do not really know why the bx installation is not correct.

If you want, here is full HiC-Pro conda environment

https://github.com/nservant/HiC-Pro/blob/devel/environment.yml

[Pim]

The issue was solved! Apparently bx-python was built against a python version that was compiled with ucs2. So to fully enable bx utilities I had to recompile the python version I was using from ucs4 to ucs2. I achieved this through: undefined symbol: PyUnicodeUCS2_Decode whilst trying to install psycopg2