MAJIQ voila error

Esther Kockelmans

unread,

Jan 30, 2021, 5:43:01 AM1/30/21

to majiq_voila

Hi,

I have downloaded and installed the packages and files for MAJIQ, although, when I try running it, I get an error.

Looks like the --voila-only option is not recognized (see attachment) and I also get an error when I try to install the wheel (also: see attachment),
maybe thats why the --voila-only option does not work?
How do I fix this?

Kind regards,
Esther

ERROR VOILA:

(env) esther@esther-ThinkPad-S3-Yoga-14:~/.local/bin/majiq_academic$ python3.9 -m pip install git+https://bitbucket.org/biociphers/majiq_academic#egg=majiq --install-option="--voila-only"
/home/esther/.local/lib/python3.9/site-packages/pip/_internal/commands/install.py:234: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
cmdoptions.check_install_build_global(options)
Defaulting to user installation because normal site-packages is not writeable
Collecting majiq
Cloning https://bitbucket.org/biociphers/majiq_academic to /tmp/pip-install-5uzqi60e/majiq_4a177fb14c3f4336bbb30c6baa92c888
Collecting Click==7.0
Using cached Click-7.0.tar.gz (286 kB)
Collecting Cython==0.29.14
Using cached Cython-0.29.14.tar.gz (2.1 MB)
Collecting Flask==1.0.2
Using cached Flask-1.0.2.tar.gz (644 kB)
Collecting Flask-Session==0.3.1
Using cached Flask-Session-0.3.1.tar.gz (27 kB)
Collecting Flask-WTF==0.14.2
Using cached Flask-WTF-0.14.2.tar.gz (254 kB)
Collecting gitdb2==2.0.6
Using cached gitdb2-2.0.6.tar.gz (393 kB)
Collecting GitPython==3.0.5
Using cached GitPython-3.0.5.tar.gz (433 kB)
Collecting gunicorn==19.9.0
Using cached gunicorn-19.9.0.tar.gz (415 kB)
Collecting h5py==2.10.0
Using cached h5py-2.10.0.tar.gz (301 kB)
Collecting itsdangerous==1.1.0
Using cached itsdangerous-1.1.0.tar.gz (53 kB)
Collecting Jinja2==2.11.1
Using cached Jinja2-2.11.1.tar.gz (258 kB)
Collecting MarkupSafe==1.1.1
Using cached MarkupSafe-1.1.1.tar.gz (19 kB)
Collecting numpy==1.18.1
Using cached numpy-1.18.1.zip (5.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting psutil==5.6.7
Using cached psutil-5.6.7.tar.gz (448 kB)
Collecting scipy==1.4.1
Using cached scipy-1.4.1.tar.gz (24.6 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting six==1.14.0
Using cached six-1.14.0.tar.gz (33 kB)
Collecting smmap2==2.0.5
Using cached smmap2-2.0.5.tar.gz (22 kB)
Collecting waitress==1.1.0
Using cached waitress-1.1.0.tar.gz (126 kB)
Collecting Werkzeug==0.16.0
Using cached Werkzeug-0.16.0.tar.gz (925 kB)
Collecting WTForms==2.2.1
Using cached WTForms-2.2.1.tar.gz (156 kB)
Skipping wheel build for majiq, due to binaries being disabled for it.
Skipping wheel build for Click, due to binaries being disabled for it.
Skipping wheel build for Cython, due to binaries being disabled for it.
Skipping wheel build for Flask, due to binaries being disabled for it.
Skipping wheel build for Flask-Session, due to binaries being disabled for it.
Skipping wheel build for Flask-WTF, due to binaries being disabled for it.
Skipping wheel build for gitdb2, due to binaries being disabled for it.
Skipping wheel build for GitPython, due to binaries being disabled for it.
Skipping wheel build for gunicorn, due to binaries being disabled for it.
Skipping wheel build for h5py, due to binaries being disabled for it.
Skipping wheel build for itsdangerous, due to binaries being disabled for it.
Skipping wheel build for Jinja2, due to binaries being disabled for it.
Skipping wheel build for MarkupSafe, due to binaries being disabled for it.
Skipping wheel build for psutil, due to binaries being disabled for it.
Skipping wheel build for six, due to binaries being disabled for it.
Skipping wheel build for smmap2, due to binaries being disabled for it.
Skipping wheel build for waitress, due to binaries being disabled for it.
Skipping wheel build for Werkzeug, due to binaries being disabled for it.
Skipping wheel build for WTForms, due to binaries being disabled for it.
Building wheels for collected packages: numpy, scipy
Building wheel for numpy (PEP 517) ... done
Created wheel for numpy: filename=numpy-1.18.1-cp39-cp39-linux_x86_64.whl size=11513446 sha256=b5d74d938697b3275922e9bcc9ed48da56baabf3a3e7ca84c29cb49b2878d727
Stored in directory: /home/esther/.cache/pip/wheels/d7/8f/69/b233132b552877f7f613530ca919c4f566aec2b4c88119979f
Building wheel for scipy (PEP 517) ... done
Created wheel for scipy: filename=scipy-1.4.1-cp39-cp39-linux_x86_64.whl size=46759643 sha256=934809aaa07f6e606061f932172ebd1bf17141923939e5623e65e4df54f38f6c
Stored in directory: /home/esther/.cache/pip/wheels/dd/70/40/456161126b69d62f62b883ad8a5c84b88de0c26575702f5217
Successfully built numpy scipy
Installing collected packages: MarkupSafe, Werkzeug, smmap2, Jinja2, itsdangerous, Click, WTForms, six, numpy, gitdb2, Flask, waitress, scipy, psutil, h5py, gunicorn, GitPython, Flask-WTF, Flask-Session, Cython, majiq
    Running setup.py install for MarkupSafe ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5uzqi60e/markupsafe_69a01687860c434abfda0df76c5d5f0e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5uzqi60e/markupsafe_69a01687860c434abfda0df76c5d5f0e/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-u28ci1gc/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/esther/.local/include/python3.9/MarkupSafe --voila-only
         cwd: /tmp/pip-install-5uzqi60e/markupsafe_69a01687860c434abfda0df76c5d5f0e/
    Complete output (6 lines):
    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: setup.py --help [cmd1 cmd2 ...]
       or: setup.py --help-commands
       or: setup.py cmd --help

    error: option --voila-only not recognized
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5uzqi60e/markupsafe_69a01687860c434abfda0df76c5d5f0e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5uzqi60e/markupsafe_69a01687860c434abfda0df76c5d5f0e/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-u28ci1gc/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/esther/.local/include/python3.9/MarkupSafe --voila-only Check the logs for full command output.

ERROR WHEEL:

(env) esther@esther-ThinkPad-S3-Yoga-14:~/.local/bin/majiq_academic$ pip install wheel
Traceback (most recent call last):
File "/home/esther/.local/bin/majiq_academic/env/bin/pip", line 7, in <module>
    from pip._internal.cli.main import main
File "/home/esther/.local/bin/majiq_academic/env/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

Paul Jewell

unread,

Feb 15, 2021, 1:33:06 PM2/15/21

to majiq_voila

Hi Esther,

There are a few weird things going on here. Based on what I can gather from these log lines, your virtualenv (env) was created with python3.5, but you've tried to install majiq to it using python3.9 .

Neither of these versions are officially supported. You need at least python3.6 and the recommended version is 3.8.

Please start from a fresh terminal, and try to run the steps like this:

$ python3.8 -m venv env_majiq

$ source env_majiq/bin/activate

$ pip install git+https://bitbucket.org/biociphers/majiq_academic#egg=majiq --install-option="--voila-only"

This should allow the voila only install option to work. (Note that the package name is still called 'majiq' , but you will have the "voila" binary available when it finishes installing.)

Let me know if this works for you,

-Paul

Esther Kockelmans

unread,

Feb 16, 2021, 4:19:49 AM2/16/21

to majiq_voila

Thanks for your respond, I have downloaded and installed Python 3.8 and ran the commands that you send me. Although it still seems like something is off with the voila option:

Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done

Collecting psutil==5.6.7 (from majiq)
Using cached https://files.pythonhosted.org/packages/73/93/4f8213fbe66fc20cb904f35e6e04e20b47b85bee39845cc66a0bcf5ccdcb/psutil-5.6.7.tar.gz
Collecting scipy==1.4.1 (from majiq)
Using cached https://files.pythonhosted.org/packages/04/ab/e2eb3e3f90b9363040a3d885ccc5c79fe20c5b8a3caa8fe3bf47ff653260/scipy-1.4.1.tar.gz

Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done

Collecting six==1.14.0 (from majiq)
Using cached https://files.pythonhosted.org/packages/21/9f/b251f7f8a76dec1d6651be194dfba8fb8d7781d10ab3987190de8391d08e/six-1.14.0.tar.gz
Collecting smmap2==2.0.5 (from majiq)
Using cached https://files.pythonhosted.org/packages/3b/ba/e49102b3e8ffff644edded25394b2d22ebe3e645f3f6a8139129c4842ffe/smmap2-2.0.5.tar.gz
Collecting waitress==1.1.0 (from majiq)
Using cached https://files.pythonhosted.org/packages/3c/68/1c10dd5c556872ceebe88483b0436140048d39de83a84a06a8baa8136f4f/waitress-1.1.0.tar.gz
Collecting Werkzeug==0.16.0 (from majiq)
Using cached https://files.pythonhosted.org/packages/5e/fd/eb19e4f6a806cd6ee34900a687f181001c7a0059ff914752091aba84681f/Werkzeug-0.16.0.tar.gz
Collecting WTForms==2.2.1 (from majiq)
Using cached https://files.pythonhosted.org/packages/cd/1d/7221354ebfc32b868740d02e44225c2ce00769b0d3dc370e463e2bc4b446/WTForms-2.2.1.tar.gz
Skipping bdist_wheel for numpy, due to binaries being disabled for it.
Skipping bdist_wheel for scipy, due to binaries being disabled for it.
Installing collected packages: Click, Cython, Werkzeug, MarkupSafe, Jinja2, itsdangerous, Flask, Flask-Session, WTForms, Flask-WTF, smmap2, gitdb2, GitPython, gunicorn, numpy, six, h5py, psutil, scipy, waitress, majiq
Running setup.py install for Click ... error

ERROR: Command errored out with exit status 1:

command: /home/esther/Documents/MAJIQ/env_majiq/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-235psp8s/Click/setup.py'"'"'; __file__='"'"'/tmp/pip-install-235psp8s/Click/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-skr2w91e/install-record.txt --single-version-externally-managed --compile --install-headers /home/esther/Documents/MAJIQ/env_majiq/include/site/python3.8/Click --voila-only
cwd: /tmp/pip-install-235psp8s/Click/

    Complete output (6 lines):
    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: setup.py --help [cmd1 cmd2 ...]
       or: setup.py --help-commands
       or: setup.py cmd --help

    error: option --voila-only not recognized
    ----------------------------------------

ERROR: Command errored out with exit status 1: /home/esther/Documents/MAJIQ/env_majiq/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-235psp8s/Click/setup.py'"'"'; __file__='"'"'/tmp/pip-install-235psp8s/Click/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-skr2w91e/install-record.txt --single-version-externally-managed --compile --install-headers /home/esther/Documents/MAJIQ/env_majiq/include/site/python3.8/Click --voila-only Check the logs for full command output.
WARNING: You are using pip version 19.2.3, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Do you know what is going wrong?

Op maandag 15 februari 2021 om 19:33:06 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 16, 2021, 5:44:11 PM2/16/21

to majiq_voila

Hi Esther,

Something strange is going on, and I'm not sure why I've not seen it surface with any other users yet.

The issue from this log seems to be that the install option "--voila-only" is being passed not just to the majiq package, but in your case, it is being passed to the dependency package "click" , which does not expect this option, and thus it fails.

This seems to be a longstanding and still active, open issue with pip: https://github.com/pypa/pip/issues/1883

I'm guessing, for specific versions of pip, the issue is resolved, or changed such that it works for majiq, but seeing how volatile it looks, I would rather not bother trying to find a specific version of pip that will work for it. Let's try another way:

$ python3.8 -m venv env_majiq

$ source env_majiq/bin/activate

$ git clone git+https://bitbucket.org/biociphers/majiq_academic

$ cd majiq_academic

$ pip install -r requirements.txt

$ python setup.py install --voila-only

Does this method seem to have better success for you?

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 17, 2021, 4:36:14 AM2/17/21

to majiq_voila

Thank you! It didnt give the voila error anymore.

I had to change the " git clone git+https://bitbucket.org/biociphers/majiq_academic"

to "git clone https://bitbucket.org/biociphers/majiq_academic"

because the git+ didn't work.

What is the right way to see if majiq works? I tried to run the following but got a "command not found"

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/majiq_academic$ majiq build DB.gff3 -c settings.ini -j NT -o outputbuild
majiq: command not found

maybe I am not using it right

Op dinsdag 16 februari 2021 om 23:44:11 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 17, 2021, 1:59:00 PM2/17/21

to majiq_voila

Hi Esther,

We provide two binaries. `majiq` and `voila`. Voila is the visualizer, it is pure python and has no library dependencies, and is quick to install. Majiq is the main build tool when you are starting with raw aligned bams.

By specifying --voila-only , only `voila` is installed.

Were you planning to use majiq, or just voila?

You will need to omit --voila-only , and follow the instructions on the sign-up page, including installation of the dependency HTSlib, if you would like to install majiq.

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 17, 2021, 2:19:43 PM2/17/21

to majiq_voila

Hi Paul,

I am planning on visualizing alternative splicing events in genes. Is it possible to use majiq on the bam files and then visualize the outcome with voila?
That would be great! I'm now looking for the right way to do that.

Kind regards,
Esther

Op woensdag 17 februari 2021 om 19:59:00 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 17, 2021, 3:54:21 PM2/17/21

to majiq_voila

Hi Esther,

You will get both `majiq` and `voila` by just following the standard installation instructions on the page you get after signup. You should not use the 'voila only' option, in this case.

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 18, 2021, 5:45:59 AM2/18/21

to majiq_voila

Okay thanks! I found the installation instructions and I get the following message:

esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ$ source $HOME/miniconda3/bin/activate(base) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ$ conda config --add channels bcbio(base) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ$ conda create -n majiq_env -c http://majiq.biociphers.org/download/channel majiq
Collecting package metadata (current_repodata.json): failed

UnavailableInvalidChannel: The channel is not accessible or is invalid.
channel name: download/channel
channel url: http://majiq.biociphers.org/download/channel
error code: 404

You will need to adjust your conda configuration to proceed.
Use `conda config --show channels` to view your configuration's current state,
and use `conda config --show-sources` to view config file locations.

Is this the right way to install MAJIQ and VOILA?

Op woensdag 17 februari 2021 om 21:54:21 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 18, 2021, 10:43:09 AM2/18/21

to majiq_voila

Hi Esther,

The instructions that I'm familiar with involve the virtualenv method:

python3.8 -m venv env_majiq

source env_majiq/bin/activate

pip install git+https://bitbucket.org/biociphers/majiq_academic#egg=majiq

(Basically, this is the same as you started with, but not using the "--voila-only" option)

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 19, 2021, 5:13:42 AM2/19/21

to majiq_voila

Thank you, that worked!

Do you know why I still get the majiq: command not found message?

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/majiq_academic/workshop_example$ majiq build DB.gff3 -c settings.ini -j NT -o outputbuild
majiq: command not found

Op donderdag 18 februari 2021 om 16:43:09 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 19, 2021, 12:01:25 PM2/19/21

to majiq_voila

Hi Esther,

No, I don't know why the pip installation seems to be giving these issues. But seeing as you had problems with it before, let's try the workaround method again:

Deactivate the delete the old virtualenv.

$ python3.8 -m venv env_majiq

$ source env_majiq/bin/activate

$ git clone https://bitbucket.org/biociphers/majiq_academic

$ cd majiq_academic

$ pip install -r requirements.txt

$ python setup.py install

Are you able to see 'majiq' using this series of steps?

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 19, 2021, 1:48:53 PM2/19/21

to majiq_voila

Hi Paul,

Thanks for the quick response, I thought it worked, but then I got this when I ran "python setup.py install"

./majiq/src/internals/io_bam.hpp:8:24: fatal error: htslib/sam.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

Op vrijdag 19 februari 2021 om 18:01:25 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 19, 2021, 5:34:32 PM2/19/21

to majiq_voila

Hi Esther,

To compile Majiq, you will need to have HTSLIB installed. The specific instructions for HTSLIB are covered on the installation page (the one you get after clicking the link in the email sign up). In order to best assist you with it, I would need to know your operating system.

On a debian based system, for example, I install the package "libhts-dev" for the easiest resolution.

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 21, 2021, 1:22:34 PM2/21/21

to majiq_voila

Hi Paul,

Thank you so much, I dont know why I missed these steps.. it is working now!

Kind regards,
Esther

Op vrijdag 19 februari 2021 om 23:34:32 UTC+1 schreef Paul Jewell:

Esther Kockelmans

unread,

Feb 22, 2021, 11:03:52 AM2/22/21

to majiq_voila

Okay so I got MAJIQ and VOILA running on my laptop now, although, when I use the following command line:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example/majiq_academic$ majiq build DB.gff3 -c settings.ini -j 4 -o build_out

I get this message:

Traceback (most recent call last):

File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 66, in __init__
sam_dirlist = general["bamdirs"].split(",")
KeyError: 'bamdirs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq", line 11, in <module>
    load_entry_point('majiq==2.2', 'console_scripts', 'majiq')()
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/run_majiq.py", line 529, in main
    args.func(args)
File "majiq/src/build.pyx", line 659, in majiq.src.build.build
    pipeline_run(Builder(args))
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/basic_pipeline.py", line 13, in pipeline_run
    return pipeline.run()
File "majiq/src/build.pyx", line 668, in majiq.src.build.Builder.run
    majiq_config = Config(self.conf, self)
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 35, in __new__
    Config.instance = Config.__Config(*argv)
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 69, in __init__
    raise UserWarning(
UserWarning: samdir is a deprecated value, please use bamdirs instead

I am using the test data from the workshop.
Do you know how I can fix this?

Op vrijdag 19 februari 2021 om 23:34:32 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 22, 2021, 12:58:31 PM2/22/21

to majiq_voila

Hi Esther,

It looks like, from the error, that in your config file you have a key "samdir" whereas more recent versions of majiq instead expect the key "bamdirs". Could you try changing that and/or posting your config file so I may review it?

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 22, 2021, 1:41:34 PM2/22/21

to majiq_voila

Thanks for your quick response! Its working :)!

Kind regards,
Esther

Op maandag 22 februari 2021 om 18:58:31 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 22, 2021, 3:07:54 PM2/22/21

to majiq_voila

Hi Esther,

Awesome! Please post back if you have any questions about the quantify / view steps , or any other suggestions!

Thanks,

-Paul

Esther Kockelmans

unread,

Feb 23, 2021, 6:40:40 AM2/23/21

to majiq_voila

Hi Paul,

Thank you so much for helping me :)

I do have a question about the "voila psi" options.
I ran the following commands (following the MAJIQ tutorial https://majiq.biociphers.org/docs/TUTORIAL%20-%20V1.1.pdf)

majiq build DB.gff3 -c settings.ini -j 4 -o build_out

majiq deltapsi -grp1 ./build_out/workshop_Cer1.majiq -grp2 ./build_out/workshop_Adr1.majiq --names Cer Adr --output ./psi_out

majiq psi ./build_out/workshop_Cer1.majiq --output ./psi_out --name Psi

and these all worked fine,

although the voila psi option returns the following message:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ voila psi ./psi_out/Psi.psi.pickle --genes-files ./build_out/workshop_Cer1.majiq.splicegraph -o ./voila_out
usage: voila [-h] [-v] {tsv,view} ...
voila: error: invalid choice: 'psi' (choose from 'tsv', 'view')

Is there a another way to create the splice graphs or am I missing a voila package or something?

Thanks,

Esther

Op maandag 22 februari 2021 om 21:07:54 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 23, 2021, 10:31:23 AM2/23/21

to majiq_voila

Hi Esther,

I will assume by "create the splice graphs", you would be interested in viewing the interactive output. This is done with "voila view". You just need to point it to some folders containing the voila files and splicegraph. From your example, I think the command would look something like this:

$ voila view ./psi_out ./build_out

This will create an interactive interface. It will run in the web browser. Basically, a link should appear in the terminal that looks like: http://localhost:38472 (or some port), and you need to click on it, or copy/paste it into a web browser such as chrome, to see the outputs.

Let me know if this makes sense.

-Paul

Esther Kockelmans

unread,

Feb 24, 2021, 3:37:07 AM2/24/21

to majiq_voila

Thanks! Yes I would like to visualize the (alternative) splice events. I ran the command and got this message:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ voila view ./psi_out ./build_out
2021-02-24 09:33:31,371 (PID:3453) - INFO - Command: /home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/voila view ./psi_out ./build_out
2021-02-24 09:33:31,371 (PID:3453) - INFO - Voila v2.2.0-e25c4ac
2021-02-24 09:33:31,371 (PID:3453) - INFO - config file: /tmp/tmp9ka_25vl
2021-02-24 09:33:31,377 (PID:3453) - ERROR - Found Voila files have more then one analysis type.

do I have to change something in the previous steps to avoid this error?

Esther

Op dinsdag 23 februari 2021 om 16:31:23 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 24, 2021, 1:35:35 PM2/24/21

to majiq_voila

Hi Esther,

So it sounds like, you had run before, both a "majiq psi" and a "majiq deltapsi" before. The viewer can only handle viewing one of these two types of analysis at a time. To pick one, just modify my command above to give the exact path to the files to use. Something like:

$ voila view ./psi_out/test1.psi.voila ./build_out/splicegraph.sql

(It will change depending on what the files are called, you will have to look in psi_out and build_out to check for the exact names)

-Paul

Esther Kockelmans

unread,

Feb 25, 2021, 5:55:00 AM2/25/21

to majiq_voila

Thanks! I ran the command and got this:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ voila view ./psi_out/Psi.psi.voila ./build_out/splicegraph.sql
2021-02-25 11:08:30,842 (PID:3407) - INFO - Command: /home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/voila view ./psi_out/Psi.psi.voila ./build_out/splicegraph.sql
2021-02-25 11:08:30,842 (PID:3407) - INFO - Voila v2.2.0-e25c4ac
2021-02-25 11:08:30,842 (PID:3407) - INFO - config file: /tmp/tmpg5dlog14
2021-02-25 11:08:30,847 (PID:3407) - INFO - Using index: /home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/psi_out/Psi.psi.voila
Serving on http://localhost:40667

Its now running for almost an hour, I guess because of the large files, how long does it normally take to make the splicegraph?

And where can I open the splicegraph.sql once its done?

Esther

Op woensdag 24 februari 2021 om 19:35:35 UTC+1 schreef Paul Jewell:

Esther Kockelmans

unread,

Feb 25, 2021, 7:09:08 AM2/25/21

to majiq_voila

Hi Paul,

I read the previous messages and saw the following steps! Thank you so much, it works and I can see the output.
So happy it works now :)

Kind regards,
Esther

Op woensdag 24 februari 2021 om 19:35:35 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Feb 25, 2021, 12:24:21 PM2/25/21

to majiq_voila

Hi Esther,

Sehr Klasse.

Thanks,

-Paul

Esther Kockelmans

unread,

Mar 1, 2021, 5:29:44 AM3/1/21

to majiq_voila

One quick last question: for my internship I want to compare different groups/samples (and different genes) to see the different splice events that occur in these groups/genes and then visualize it with VOILA.

Can I use the same steps for this? And what is the best way to do this?

Thank you very much

Esther

Op donderdag 25 februari 2021 om 18:24:21 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Mar 1, 2021, 12:58:47 PM3/1/21

to majiq_voila

Hi Esther,

I would recommend making multiple psi or dpsi groups / experiments and then running voila with multiple voila-files that you have generated. In this way you can view multiple sets of data together. Have you tried running the multiple dpsi files with the workshop example? Are you able to produce the figures that you expect that way? If not, please describe which kind of comparisons would help most in your case and I will check if we have anything similar.

Thanks,

-Paul

Esther Kockelmans

unread,

Mar 3, 2021, 1:29:37 PM3/3/21

to majiq_voila

Hi Paul,

Thanks, yes I ran the deltapsi command and visialized the output with voila; is it possible to compare more than 2 groups?

I would like to compare 6 groups/samples and look at the splice events that occur to compare these samples.

Kind regards,
Esther

Op maandag 1 maart 2021 om 18:58:47 UTC+1 schreef Paul Jewell:

Esther Kockelmans

unread,

Mar 3, 2021, 1:45:54 PM3/3/21

to majiq_voila

And another quick question; I am working with .bam files and saw that it is possible to pass a set of 6 bam files to majiq builder and then visualize it with voila?

Thank you

Op maandag 1 maart 2021 om 18:58:47 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Mar 4, 2021, 12:13:30 PM3/4/21

to majiq_voila

Hi Esther,

Former question: There is no multi-dpsi currently supported. A similar functionality is planned for a upcoming release.

You may use as many bamfiles as you like, defined in the majiq config file as experiments (and the directorie(s) they are in configured under bamdirs). See the [experiments] section here: https://biociphers.bitbucket.io/majiq/quick.html#conf_file

Thanks,

-Paul

Esther Kockelmans

unread,

Mar 9, 2021, 5:54:22 AM3/9/21

to majiq_voila

Thanks! I have downloaded my own bam and bam.bai files (and pasted them in the "bammetjes" directory) and tried to run majiq builder on these files.

I have changed the settings.ini file to:

[info]
bamdirs=bammetjes
genome= mm10
genome_path= /data/WASP_DATA/Genomes/goldenPath/mm10
[experiments]
Gr1=short_001-001.bam,short_001-002.bam
Gr2=short_001-002.bam,short_001-001.bam

When I run majiq builder I get this:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ majiq build DB.gff3 -c sett.ini -j 4 -o test_build_out
/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py:81: UserWarning: sjdirs parameter not found in config file, using "./" instead
warnings.warn(

Traceback (most recent call last):
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq", line 11, in <module>
load_entry_point('majiq==2.2', 'console_scripts', 'majiq')()
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/run_majiq.py", line 529, in main
args.func(args)
File "majiq/src/build.pyx", line 659, in majiq.src.build.build

File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/basic_pipeline.py", line 13, in pipeline_run
return pipeline.run()
File "majiq/src/build.pyx", line 668, in majiq.src.build.Builder.run

File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 35, in __new__
Config.instance = Config.__Config(*argv)

File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 146, in __init__
raise RuntimeError(
RuntimeError: No matching bam (and bam.bai) or sj file was found matching the experiment short_001-001.bam in any of the provided paths in the build configuration

Although I have these files (the bam files and the bam.bai files):

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example/bammetjes$ ls
short_001-001.bam short_001-001.bam.bai short_001-002.bam short_001-002.bam.bai

Is there something I need to change?

Esther

Op donderdag 4 maart 2021 om 18:13:30 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Mar 15, 2021, 6:50:04 PM3/15/21

to majiq_voila

Hi Esther,

I believe you just need to remove the ".bam" from the config lines, like this:

[info]
bamdirs=bammetjes
genome= mm10
genome_path= /data/WASP_DATA/Genomes/goldenPath/mm10
[experiments]

Gr1=short_001-001,short_001-002
Gr2=short_001-002,short_001-001

Let me know if that helps,

Thanks,

-Paul

Esther Kockelmans

unread,

Mar 16, 2021, 5:12:06 AM3/16/21

to majiq_voila

Hi Paul,

I changed the settings file and ran majiq build again and got this:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ majiq build DB.gff3 -c sett.ini -j 4 -o test_build_out
/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py:81: UserWarning: sjdirs parameter not found in config file, using "./" instead
warnings.warn(

2021-03-16 10:04:33,654 (PID:3957) - INFO - Majiq Build v2.2-e25c4ac
2021-03-16 10:04:33,654 (PID:3957) - INFO - Command: /home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq build DB.gff3 -c sett.ini -j 4 -o test_build_out
2021-03-16 10:04:33,654 (PID:3957) - INFO - Parsing GFF3

Traceback (most recent call last):
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq", line 11, in <module>
    load_entry_point('majiq==2.2', 'console_scripts', 'majiq')()
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/run_majiq.py", line 529, in main
    args.func(args)
File "majiq/src/build.pyx", line 659, in majiq.src.build.build
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/basic_pipeline.py", line 13, in pipeline_run
    return pipeline.run()

File "majiq/src/build.pyx", line 669, in majiq.src.build.Builder.run
File "majiq/src/build.pyx", line 677, in majiq.src.build.Builder.builder
File "majiq/src/build.pyx", line 584, in majiq.src.build._core_build
File "majiq/src/build.pyx", line 428, in majiq.src.build.init_splicegraph
File "majiq/src/build.pyx", line 429, in majiq.src.build.init_splicegraph
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/voila/api/splice_graph.py", line 104, in experiment_names
self.conn.executemany('''
sqlite3.IntegrityError: UNIQUE constraint failed: experiment.name

# to see what files were made:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ cd test_build_out/
(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example/test_build_out$ ls -l
total 52
-rw-rw-r-- 1 esther esther 306 mars 16 10:04 majiq.log
-rw-r--r-- 1 esther esther 43008 mars 16 10:04 splicegraph.sql

So it made the spicegraph.sql but not the .majiq files that I need in order to run the majiq deltapsi command.

Is there something else I need to change?

Thank you so much

Esther

Op maandag 15 maart 2021 om 23:50:04 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Mar 19, 2021, 12:38:10 PM3/19/21

to majiq_voila

Hi Esther,

It looks like I neglected to remember that all of the experiment names must be unique. (even if they are placed into separate groups), as am example:

[info]
bamdirs=bammetjes
genome= mm10
genome_path= /data/WASP_DATA/Genomes/goldenPath/mm10
[experiments]

Gr1=short_001-001-g1,short_001-002-g1
Gr2=short_001-002-g2,short_001-001-g2

Please make sure that all of the experiment names that you list are unique and try once again.

Thanks,

-Paul

Esther Kockelmans

unread,

Mar 21, 2021, 2:19:11 PM3/21/21

to majiq_voila

I changed the sett.ini file to:

[info]
bamdirs=bammetjes
genome= mm10
genome_path= /data/WASP_DATA/Genomes/goldenPath/mm10
[experiments]

Gr1=short_001-001-g1,short_001-002-g1
Gr2=short_001-002-g2,short_001-001-g2

and got this message:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ majiq build DB.gff3 -c sett.ini -j 4 -o test_build_out
/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py:81: UserWarning: sjdirs parameter not found in config file, using "./" instead
warnings.warn(
Traceback (most recent call last):
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq", line 11, in <module>
    load_entry_point('majiq==2.2', 'console_scripts', 'majiq')()
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/run_majiq.py", line 529, in main
    args.func(args)
File "majiq/src/build.pyx", line 659, in majiq.src.build.build
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/basic_pipeline.py", line 13, in pipeline_run
    return pipeline.run()
File "majiq/src/build.pyx", line 668, in majiq.src.build.Builder.run
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 35, in __new__
    Config.instance = Config.__Config(*argv)
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 146, in __init__
    raise RuntimeError(

RuntimeError: No matching bam (and bam.bai) or sj file was found matching the experiment short_001-001-g1 in any of the provided paths in the build configuration

do I also have to rename the .bam files or is something else going wrong?

Esther

Op vrijdag 19 maart 2021 om 17:38:10 UTC+1 schreef Paul Jewell:

Paul Jewell

unread,

Mar 23, 2021, 9:59:46 AM3/23/21

to majiq_voila

Hi Esther,

I'm a little confused about your use case, if these are not already separate files. (I assumed before, that they were simply different inputs that happened to have the same name, in different folders, but I See now there is only one entry in bamdirs so this would not be possible.)

The order you specify the experiments in does not matter, so specifying something like your original:

[experiments]
Gr1=short_001-001.bam,short_001-002.bam
Gr2=short_001-002.bam,short_001-001.bam

Would just make two identical groups.

Could you describe the use case? Then I might be able to help you better.

Thanks,

-Paul

Esther Kockelmans

unread,

Mar 24, 2021, 10:31:31 AM3/24/21

to majiq_voila

Hi Paul,

I've changed the experiments and downloaded 2 extra bam files and their .bam.bai file.

bammetjes now looks like this:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example/bammetjes$ ls
sett.ini               short_001-002.bam      short_001-003.bam.bai
short_001-001.bam      short_001-002.bam.bai short_001-004.bam
short_001-001.bam.bai short_001-003.bam      short_001-004.bam.bai

sett.ini now looks like this:

[info]
bamdirs=bammetjes
genome= mm10
genome_path= /data/WASP_DATA/Genomes/goldenPath/mm10
[experiments]

Gr1=short_001-001-g1,short_001-002-g1
Gr2=short_001-003-g2,short_001-004-g2

so the two groups aren't identical anymore.

I've tried running majiq build (with both Gr1=short_001-001-g1,short_001-002-g1
Gr2=short_001-003-g2,short_001-004-g2 and Gr1=short_001-001.bam,short_001-002.bam
Gr2=short_001-003.bam,short_001-004.bam in the sett.ini file) but got this message for both:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ majiq build DB.gff3 -c sett.ini -j 4 -o test_build_out
/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py:81: UserWarning: sjdirs parameter not found in config file, using "./" instead
warnings.warn(
Traceback (most recent call last):
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq", line 11, in <module>
    load_entry_point('majiq==2.2', 'console_scripts', 'majiq')()
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/run_majiq.py", line 529, in main
    args.func(args)
File "majiq/src/build.pyx", line 659, in majiq.src.build.build
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/basic_pipeline.py", line 13, in pipeline_run
    return pipeline.run()
File "majiq/src/build.pyx", line 668, in majiq.src.build.Builder.run
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 35, in __new__
    Config.instance = Config.__Config(*argv)
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/config.py", line 146, in __init__
    raise RuntimeError(
RuntimeError: No matching bam (and bam.bai) or sj file was found matching the experiment short_001-001.bam in any of the provided paths in the build configuration

What is going wrong?

Op dinsdag 23 maart 2021 om 14:59:46 UTC+1 schreef Paul Jewell:

Caleb Radens

unread,

Mar 24, 2021, 1:07:10 PM3/24/21

to Esther Kockelmans, majiq_voila

Hi Esther,

Did you try the following:

[experiments]

Gr1=short_001-001,short_001-002
Gr2=short_001-003,short_001-004

Best,

Caleb

--
You received this message because you are subscribed to the Google Groups "majiq_voila" group.
To unsubscribe from this group and stop receiving emails from it, send an email to majiq_voila...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/majiq_voila/1aae8022-2530-48cf-8957-8ef5d8f4fcd2n%40googlegroups.com.

Esther Kockelmans

unread,

Mar 25, 2021, 8:30:50 AM3/25/21

to majiq_voila

Thank you very much, that worked,

The .majiq files are made.

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example/test_build_out$ ls
majiq.log short_001-001.sj short_001-002.sj short_001-003.sj short_001-004.sj
short_001-001.majiq short_001-002.majiq short_001-003.majiq short_001-004.majiq splicegraph.sql
(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example/test_build_out$ cd ..

Although when I try to run majiq psi on one of these I get this message:

(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MAJIQ/htslib-1.11/workshop_example$ majiq psi ./test_build_out/short_001-001.majiq --output ./test_psi_out --name Psi
2021-03-25 13:27:07,303 (PID:3962) - INFO - Majiq psi v2.2-e25c4ac
2021-03-25 13:27:07,303 (PID:3962) - INFO - Command: /home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq psi ./test_build_out/short_001-001.majiq --output ./test_psi_out --name Psi
2021-03-25 13:27:07,303 (PID:3962) - INFO - Running Psi ...
2021-03-25 13:27:07,303 (PID:3962) - INFO - GROUP: ['./test_build_out/short_001-001.majiq']
2021-03-25 13:27:07,303 (PID:3962) - INFO - Parsing file: ./test_build_out/short_001-001.majiq

Traceback (most recent call last):
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/bin/majiq", line 11, in <module>
load_entry_point('majiq==2.2', 'console_scripts', 'majiq')()
File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/run_majiq.py", line 529, in main
args.func(args)

File "majiq/src/calc_psi.pyx", line 28, in majiq.src.calc_psi.calcpsi

File "/home/esther/Documents/MAJIQ/htslib-1.11/workshop_example/env_majiq/lib/python3.8/site-packages/majiq/src/basic_pipeline.py", line 13, in pipeline_run
return pipeline.run()

File "majiq/src/calc_psi.pyx", line 127, in majiq.src.calc_psi.CalcPsi.run
File "majiq/src/calc_psi.pyx", line 64, in majiq.src.calc_psi._core_calcpsi
File "majiq/src/io.pyx", line 385, in majiq.src.io.extract_lsv_summary
File "majiq/src/io.pyx", line 389, in majiq.src.io.extract_lsv_summary
File "majiq/src/io.pyx", line 308, in majiq.src.io._extract_lsv_summary
File "majiq/src/io.pyx", line 316, in majiq.src.io._extract_lsv_summary
IndexError: index 0 is out of bounds for axis 0 with size 0
(env_majiq) esther@esther-ThinkPad-S3-Yoga-14:~/Documents/MA

I dont really know what is going wrong here..

Op woensdag 24 maart 2021 om 18:07:10 UTC+1 schreef cra...@biociphers.org:

Paul Jewell

unread,

Mar 26, 2021, 11:38:48 AM3/26/21

to majiq_voila

Hi Esther,

Could you post the run log for the `majiq build` step? I'm guessing the problem is that it found zero LSVs. This is usually caused by the annotation file not naming the genes / chromosomes the same as the the bamfiles.

Thanks,

-Paul

Reply all

Reply to author

Forward

Message has been deleted