[Novice] receiving error when trying to install Majiq

[William Wright]

Hi,

I am very much a novice with Python. I have read through Majiq documentation and watched the authors present how it works online via youtube videos. I am very excited to use this tool to quantify LSVs from my seq data, but I can't seem to get the installation going. I have PyCharm using Python 3.5

pip is v 10.0.01, git is v 2.7.4, and I think I have properly installed gcc. zlib is a little more complicated for me to understand how to install, but I think I have it installed (open the folder as root, and run the command "make install"

I think I have successfully executed the first command (from the Majiq Academic Download website) :

$ pip install cython pysam numpy
Requirement already satisfied: cython in ./venv/lib/python3.5/site-packages (0.28.3)
Requirement already satisfied: pysam in ./venv/lib/python3.5/site-packages (0.14.1)
Requirement already satisfied: numpy in ./venv/lib/python3.5/site-packages (1.14.5)

However, I seem to be getting an error when trying to execute the second command.
This is what I see : (I highlighted the error in yellow at the bottom)

$ pip install git+https://bitbucket.org/biociphers/majiq_stable.git#egg=majiq
Collecting majiq from git+https://bitbucket.org/biociphers/majiq_stable.git#egg=majiq
  Cloning https://bitbucket.org/biociphers/majiq_stable.git to /tmp/pip-install-co0d8gb4/majiq
Requirement already satisfied: Cython>=0.27.3 in ./venv/lib/python3.5/site-packages (from majiq) (0.28.3)
Requirement already satisfied: pysam>=0.14 in ./venv/lib/python3.5/site-packages (from majiq) (0.14.1)
Requirement already satisfied: numpy>=1.14.1 in ./venv/lib/python3.5/site-packages (from majiq) (1.14.5)
Requirement already satisfied: cycler==0.10.0 in ./venv/lib/python3.5/site-packages (from majiq) (0.10.0)
Requirement already satisfied: h5py>=2.7.1 in ./venv/lib/python3.5/site-packages (from majiq) (2.8.0)
Requirement already satisfied: Jinja2==2.10 in ./venv/lib/python3.5/site-packages (from majiq) (2.10)
Requirement already satisfied: MarkupSafe==1.0 in ./venv/lib/python3.5/site-packages (from majiq) (1.0)
Requirement already satisfied: matplotlib==2.1.2 in ./venv/lib/python3.5/site-packages (from majiq) (2.1.2)
Collecting psutil==5.4.3 (from majiq)
  Using cached https://files.pythonhosted.org/packages/e2/e1/600326635f97fee89bf8426fef14c5c29f4849c79f68fd79f433d8c1bd96/psutil-5.4.3.tar.gz
Requirement already satisfied: pyparsing==2.2.0 in ./venv/lib/python3.5/site-packages (from majiq) (2.2.0)
Requirement already satisfied: python-dateutil==2.6.1 in ./venv/lib/python3.5/site-packages (from majiq) (2.6.1)
Requirement already satisfied: pytz==2018.3 in ./venv/lib/python3.5/site-packages (from majiq) (2018.3)
Collecting quicksect==0.1.0 (from majiq)
  Using cached https://files.pythonhosted.org/packages/d4/71/28f9f745dadd37fc51db6ffdb66664bb41549717f89c02016a05fdc5041c/quicksect-0.1.0.tar.gz
Collecting scipy==1.0.0 (from majiq)
  Using cached https://files.pythonhosted.org/packages/1a/83/6aed4f564f3f5d338fd3c642f33d5ded0fc577da5f9a7d85ed6ba23c5d51/scipy-1.0.0-cp35-cp35m-manylinux1_x86_64.whl
Requirement already satisfied: six==1.11.0 in ./venv/lib/python3.5/site-packages (from majiq) (1.11.0)
Collecting SQLAlchemy==1.2.4 (from majiq)
  Using cached https://files.pythonhosted.org/packages/f3/b7/d8725042f105cc6b71c7bae0ffd46e49f762e5a08f421f1eddd855a1f723/SQLAlchemy-1.2.4.tar.gz
Installing collected packages: psutil, quicksect, scipy, SQLAlchemy, majiq
  Running setup.py install for psutil ... error
    Complete output from command /home/bakke/PycharmProjects/MajiqProject/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-co0d8gb4/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-_iob8jso/install-record.txt --single-version-externally-managed --compile --install-headers /home/bakke/PycharmProjects/MajiqProject/venv/include/site/python3.5/psutil:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.5
    creating build/lib.linux-x86_64-3.5/psutil
    copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/__init__.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_common.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_psposix.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_psaix.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_psosx.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_exceptions.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.5/psutil
    copying psutil/_compat.py -> build/lib.linux-x86_64-3.5/psutil
    creating build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.5/psutil/tests
    copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.5/psutil/tests
    running build_ext
    building 'psutil._psutil_linux' extension
    creating build/temp.linux-x86_64-3.5
    creating build/temp.linux-x86_64-3.5/psutil
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/home/bakke/PycharmProjects/MajiqProject/venv/include -I/usr/include/python3.5m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_common.o
    psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
   
    ----------------------------------------
Command "/home/bakke/PycharmProjects/MajiqProject/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-co0d8gb4/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-_iob8jso/install-record.txt --single-version-externally-managed --compile --install-headers /home/bakke/PycharmProjects/MajiqProject/venv/include/site/python3.5/psutil" failed with error code 1 in /tmp/pip-install-co0d8gb4/psutil/

Any help is greatly appreciated. Thank you very much in advance!
Charlie

[cjg...@biociphers.org]

Charlie, 

It looks like you're using a virtual environment.  Let's confirm that you've sourced into the virtual environment. What's the output of "which python"?  

Kindest Regards,

Christopher Green

[William Wright]

Hi Christopher,

The output of "which python" is this:

/home/bakke/PycharmProjects/MajiqProject/venv/bin/python

I would like to say that I looked online for some solutions and was able to successfully find a way to download majiq. I wanted to verify that it is working, so I attempted to follow the tutorial data. I am not able to complete the 1st step (majiq builder).

I navigated to the "workshop_example" folder, which has the following contents :

bam  DB.gff3  README  settings.ini

Then I ran the following command:

$ majiq build db.gff3 -conf settings.ini --output build_groups --nthreads 1 --nogc

what was returned is the following:

usage: majiq [-h] [-v] {build,psi,deltapsi,weights} ...
majiq: error: unrecognized arguments: settings.ini --nthreads 1 --nogc

Could you help me to overcome this error? I still got an "unrecognized arguments : settings.ini" even when I tried it while leaving off the subsequent options (--nthreads and --nogc).
I'm not sure at all if this has anything to do with the error, but I notice that in the settings.ini file, I see this genome_path= /data/WASP_DATA/Genomes/goldenPath/mm10  was that specific to the original tutorial ?

Thanks so much for your help,
Charlie

[cjg...@biociphers.org]

Charlie, 

The genome path is no longer needed in the settings file and will be ignored.  For the error with MAJIQ check "majiq build --help"  for the usage statement. --nogc is now --disable-gc and --nthreads is now --nprocs. 

Kindest Regards,

Christopher Green  

[William Wright]

Christopher,

This worked beautifully, thank you very much. I was able to successfully complete the majiq build stage of the tutorial. Indeed, I need to use -j to specify # processors.

I successfully got majiq build to work by using this command :

majiq build DB.gff3 -c settings.ini -j 4 -o build_out

Now I've followed the documentation to attempt majiq weights and majiq psi, but I can't get either to work.
I notice that majiq weights calls for  .majiq.hdf5 files in the build out directory (in my case the folder is called build_out). However, I only see .majiq files - nothing with the extension .majiq.hdf5

In this folder are .majiq files for workshop_Adr1, 2, 3 and also for workshop_Cer1, 2, and 3. I thought to try magic weights based on these Adr files since I don't see any .majiq.hdf5 files. I tried this :

 majiq weights build_out/workshop_Adr1.majiq workshop_Adr2.majiq workshop_Adr3.majiq -j 4 -o psi_out -n psi_Adr

This is what was returned :

Traceback (most recent call last):
  File "/home/bakke/PycharmProjects/MajiqProject/venv/bin/majiq", line 11, in <module>
    load_entry_point('majiq==1.1.3a0', 'console_scripts', 'majiq')()
  File "/home/bakke/PycharmProjects/MajiqProject/venv/lib/python3.5/site-packages/majiq/run_majiq.py", line 204, in main
    args.func(args)
  File "/home/bakke/PycharmProjects/MajiqProject/venv/lib/python3.5/site-packages/majiq/src/wght_pipeline.py", line 12, in calc_weights
    return pipeline_run(Weights(args))
  File "/home/bakke/PycharmProjects/MajiqProject/venv/lib/python3.5/site-packages/majiq/src/basic_pipeline.py", line 29, in pipeline_run
    return pipeline.run()
  File "/home/bakke/PycharmProjects/MajiqProject/venv/lib/python3.5/site-packages/majiq/src/wght_pipeline.py", line 18, in run
    self.weights_pip()
  File "/home/bakke/PycharmProjects/MajiqProject/venv/lib/python3.5/site-packages/majiq/src/wght_pipeline.py", line 25, in weights_pip
    majiq_logger.create_if_not_exists(self.logger_path)
AttributeError: 'Weights' object has no attribute 'logger_path'

Any advice getting around these obstacles? If I could get weights and psi to work, I will continue with the tutorial and then on to analyzing my own seq data.

Thanks very much for your help this far too!

[William Wright]

UPDATE : I tried using majiq psi on just one .majiq file and it worked for some reason (workshop_Cer3.majiq). Thanks