Majiq V3 - Voila bugs

[Miriam Martínez]

Hi biociphers team,

first of all, congrats for the release of majiq newest version, incredible job. While trying the new version I have come across two issues when using voila with some data:

i) As you can see on "bug1" file, the sample named PNTR_002_sorted is part of group 1 as  PNTR_001_sorted, but on the splicegraphs above, it displays as if it was part of group 2. 

ii) Regarding the second file named "bug 2", the problem is none the splicegraphs of the samples of group 2 appear for being selected (this also happens with group 1). I could only display the two splicegraphs that appear on the "bug 1" file.

Finally, I was wondering if with this new version there should also be a splicegraph with the data of all the samples of a group combined, or if it is a feature that has been taken down with the new version.

Thanks in advance!

Best regards,

Miriam Martínez

[San Jewell]

Hi Miriam,

Thank you for your report. In order to try to reproduce your results could you briefly describe the kind of run you have done to arrive at this result? It seems like two heterogen groups with three samples each? A list of commands with information redacted would be helpful to make sure I capture the same results that you are seeing.

Thanks,

-San

[Miriam Martínez]

Hi San,

I'll gladly share the info: in this run I had 2 groups: the control one had 4 samples and the affected one 3 samples. Regarding the commands, I'll write them down below:

1) ##Convert gff3 text file to annotation database
majiq-v3 gff3 path_to_annot_file/Homo_sapiens.GRCh38.112.gff3.gz path_to_majiq_folder/majiq_build/annotations/sg.zarr

##Convert bam files into splicejunction databases (this command is in a for loop for all bams, but i'll write just the majiq command)

majiq-v3 sj path_to_bams/${f}.bam  path_to_majiq_folder/majiq_build/annotations/sg.zarr  path_to_majiq_folder/majiq_build/sj/${f}.sj 

2) ##Build command 

majiq-v3 build -j 4 --min-experiments 0.5 --simplify  path_to_majiq_folder/majiq_build/annotations/sg.zarr  path_to_majiq_folder/majiq_build/build/sg.zarr --groups-tsv config.tsv

3) #Quantify commands:
##BD
majiq-v3 psi-coverage  path_to_majiq_folder/majiq_build/build/sg.zarr  path_to_majiq_folder/majiq_build/psi/BD.psicov  path_to_majiq_folder/majiq_build/sj/BD_1.sj  path_to_majiq_folder/majiq_build/sj/BD_2.sj  path_to_majiq_folder/majiq_build/sj/BD_3.sj
##CTL
majiq-v3 psi-coverage  path_to_majiq_folder/majiq_build/build/sg.zarr  path_to_majiq_folder/majiq_build/psi/CTL.psicov  path_to_majiq_folder/majiq_build/sj/CTL_1.sj  path_to_majiq_folder/majiq_build/sj/CTL_2.sj  path_to_majiq_folder/majiq_build/sj/CTL_3.sj  path_to_majiq_folder/majiq_build/sj/CTL_4.sj

4) #Moccasin (The model matrix just has the samples names and as covariate the gender and an intercept column filled with 1's)
majiq-v3 moccasin-pipeline  path_to_majiq_folder/majiq_build/build_after_moccasin  path_to_majiq_folder/majiq_build/psi/BD.psicov  path_to_majiq_folder/majiq_v3_test/majiq_build/psi/CTL.psicov --factors-tsv path_to_majiq_folder/model_matrix_majiq_test_v3_plus_intercept.tsv --confounding gender --overwrite

5) #Heterogen
majiq-v3 heterogen --stats mannwhitneyu --splicegraph  path_to_majiq_folder/majiq_build/build/sg.zarr --output-voila  path_to_majiq_folder/majiq_build/heterogen/BD-CTL.heterogen.voila --output-tsv  path_to_majiq_folder/majiq_build/heterogen/BD-CTL.heterogen.tsv -psi1  path_to_majiq_folder/majiq_build/psi/BD.psicov -psi2  path_to_majiq_folder/majiq_build/psi/CTL.psicov

6) #Voila commands:
##BD
majiq-v3 sg-coverage  path_to_majiq_folder/majiq_build/build/sg.zarr  path_to_majiq_folder/majiq_build/build/BD.sgc  path_to_majiq_folder/majiq_build/sj/BD_1.sj  path_to_majiq_folder/majiq_build/sj/BD_2.sj  path_to_majiq_folder/majiq_build/sj/BD_3.sj
##CTL
majiq-v3 sg-coverage  path_to_majiq_folder/majiq_build/build/sg.zarr  path_to_majiq_folder/majiq_build/build/CTL.sgc  path_to_majiq_folder/majiq_build/sj/CTL_1.sj  path_to_majiq_folder/majiq_build/sj/CTL_2.sj  path_to_majiq_folder/majiq_build/sj/CTL_3.sj  path_to_majiq_folder/majiq_build/sj/CTL_4.sj

7) #View
voila view -p 5000 -j 4   path_to_majiq_folder/majiq_build/build/sg.zarr  path_to_majiq_folder/majiq_build/build/BD.sgc  path_to_majiq_folder/majiq_build/build/CTL.sgc  path_to_majiq_folder/majiq_build/heterogen/BD-CTL.heterogen.voila

This are all the steps I have run. If something is unclear or you need more info, just let me know. 

Thanks in advance!

Miriam

[San Jewell]

Hi Miriam,

Thank you for the detailed instructions. I am not finished with my analysis yet, but I just wanted to give an update and clarify something.

For "bug1" I am using slightly different samples but using the same form of runs as you describe. I have been using the majiq workshop example data to test, for example, this includes three samples Adr1, Adr2, Adr3 belonging to the Adr group, and Cer1, Cer2, Cer3 belonging to the Cer group. Based on the detailed commands above I will assume these are parallels for "BD" (BD_1, BD_2, BD_3) and "CTL" (CTL_1, CTL_2, CTL_3). When I run commands for these, on the heterogen command I specify group names with -n like "-n adr cdr -psi1 workshop_Adr.psicov -psi2  workshop_Cer.psicov" Using -n replaces "grp1 grp2" with "adr cer". After doing this I see splicegraphs labeled as "adr" and "cer" for "group", and in the psi below I see x-axis labels "adr" and "cer" with all samples inside when hovered over showing the correct group name "all adr samples in adr column and all cer samples in cer column"

From this I am unsure if I am reproducing bug1. However, I am seeing something related to bug2, which is that the "experiment" from group1 is not showing up correctly as a second splice graph. There should be six total experiments to view, yet I only see two. I will look into this, I just wanted to check if the bug reports are only related to the splice graph, or if you were also seeing a problem with the het/psi plots, as these seem correct to me.

Thanks,

-San

[San Jewell]

[Miriam Martínez]

Hi San,

Seems that when I wrote the commands for v3, I probably forgot to add the -n option as when I did my runs with v2, I did use the -n option, thanks for pointing it out! 

Regarding the bugs, from what I see on the screenshots, I think you have been able to reproduce at least "bug1" because when I hover over, on my data it shows the samples names on the correct group on the violin plots, whereas on the splicegraph do not show correctly (just as in your screenshot). So, the problem is only with the splicegraphs, the het/psi plots seem to work fine on my side.

Best,

Miriam

[San Jewell]

Hi Miriam,

I have been working on this issue, and in the process I have also found other related issues with the software, so, I appreciate the outreach on it. I have fixed this and other issues and I have pushed a new release. Please reinstall using the same method as before when you have a chance, and let me know if it seems that this issue is resolved.

Thanks!

-San

[Miriam Martínez]

Hi San, 

sorry for the delay in the answer, but I was waiting to get the new version installed in the cluster I work with and they are having issues (not MAJIQ related) to install this new version. Therefore, I have been trying to install MAJIQ myself on my own pc, but im having problems on the final step. I was wondering if I could have some help so I can test again the software with my mock data. Specifically, I'm stuck in the pip installation. The command line "pip install git+https://bitbucket.org/biociphers/moccasin@new_moccasin" has worked smoothly, but the "pip install ." doesn't work (I have also tried with "pip install -v git+https://bitbucket.org/biociphers/majiq_academic.git"). The last lines show this message:

Successfully built rna_majiq_meta rna_moccasin rna_voila
Failed to build rna_majiq
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> rna_majiq

Also, I'm using a conda invironment for the dependencies, in case that is relevant. If you need any more information, please let me know. I would highly appreciate some help with the installation. Thank you very much in advance.

Best,

Miriam

[San Jewell]

Hi Miriam,

I'm sorry to hear that the cluster is having issues right now. I hope it gets resolved soon.

For the error message, usually the 'real' error will be above that part which you listed. I admit the python build errors can be hard to pick apart if it's not something you are used to. Could you please paste all of the lines of the output? A common cause for this error that I'd guess is that htslib isn't installed on your machine, or if a non-admin install, perhaps the right environment variables were not set, but I can't know for sure without seeing the rest of the output.

Conda isn't specifically supported but loads of others have been using it without issue so I don't think using conda is a cause for concern.

Thanks,

-San

[Miriam Martínez]

Hi San,

I have attached a log file with all the info it outputs when trying to do the installation. As you comment, I'm not really used to python installations, so I really appreciate that you invest some time to help me out with this issue. Regarding permissions, I have admin permissions, so that shouldn't be a problem either. Thanks in advance!

Best,

Miriam

[San Jewell]

Hi Miriam,

Thanks!

So as a preface, I'm not as familiar with conda-specific quirks. In this case the part of the log where the real error happens is here:

The error itself confuses me (3.10.18, minimum version required is 3.8)

I found some references to solutions for it online:

https://stackoverflow.com/questions/73496855/how-to-use-the-python-version-active-in-an-anaconda-environment-in-cmake-builds
https://stackoverflow.com/questions/68913248/cmake-error-could-not-find-python-missing-python-numpy-include-dirs-numpy-f

And after talking with some others it seems like it's just in general saying that the development headers are missing and the versions listed are not important. I'm not sure which OS you are on here: if it was linux I would install development headers like $ sudo apt install python3-dev  ;

If it's difficult to resolve using conda I can also recommend trying the default venv appoach:

$ conda deactivate

$ python3.12 -m venv majiq_v3_env

$ source majiq_v3_end/bin/activate

$ pip install (same pip install command you used)

Let me know if either of these approaches work for you.

-San

[Miriam Martínez]

Hi San,

I have good news about the new version and about the installation:

Firstly, regarding the new version, the cluster I work on has been able to install the new version and I have been able to try it with my mock data. When viewing the results with voila, now all the samples appear and I can add them to the splicegraph view. I just wanted to point out two things they were available in past versions but I haven't seen now (i don't know if it was intended or not, so I will comment it here just in case). (A) In version 2 it was possible to add a "combined" splicegraph per group; (B) If I recall correctly, in the last version (2.5 I think), when checking the splicegraphs, the possible transcripts appear on top of the splicegraphs ussing the annotation file. As said, I don't know if these features not appearing now was intended or not, but I just wanted to make sure it isn't a bug.

Secondly, regarding the installation on my PC, I was finally able to resolve the issue. As you pointed out where the error was, I could focus on fixing the problems with cmake. To make the long story short, I hadn't cmake installed via conda, so when it looked for the development headers in python (which is from conda), it couldn't find them because my CMakeLists.txt was inside the conda folder. So basically I updated python to 3.12 and installed cmake via conda with the following command: "conda install -c conda-forge cmake". After that I ran the pip command "pip install -v git+https://bitbucket.org/biociphers/majiq_academic.git" and worked. 

Again, thanks for all your help and time.

Best,

Miriam

[San Jewell]

Hi Miriam,

You are correct on both counts actually! It seems a lot of small features have passed under the radar through the transition period. For point (A), this was a bug, which is now resolved, you may ask for a new installation whenever you have a chance and you should see them. They have been renamed "Median" instead of "Combined".

For point (B), this is a more complex issue in the way that the new version of majiq was laid out. The feature was originally developed primarily for the majiqlopedia app. It has been on our backlog for some time to re-implement this feature in v3 and still is. Please keep a lookout for another minor version bump when the transcripts will be reintroduced.

Thanks,

-San