about running Modulizer

[M K]

Hello dear MAJIQ Team,

I am very confused with the step "voila modulize" to finalise my analysis on a set of patients files. 

In your tutorial-page about modulize ( https://biociphers.bitbucket.io/majiq-docs/modulizer/quick-start.html ), the command is named "categorize". I suppose this is the older version. Could you please update the page with the v3 version of command, as I found from other users here ? >>

``` voila modulize \
-j ${SLURM_CPUS_PER_TASK} \
--show-all --keep-constitutive --overwrite --debug -d ${out_dir}/modulizer/ \
build_output/my.zarr \
        ${work_dir}/sg_coverage/group1.sgc \
        ${work_dir}/sg_coverage/group2.sgc \
        ${work_dir}/deltapsi/voila/.dpsicov ```

Could you therefore confirm that this command is correct ? 

 

Is the help message for voila modulize, the "usage" shows just "files [ files ..]" as input files. 

But are the splicegraph , sg-coverage-files , dpsicov files the 3 obligatory ? In whichever order ? 

Moreover, I don't see a "--license" flag mentioned in the "help", but when I ran, I received an error, reported in the "voila.log" file : 

``` CRITICAL - No Majiq License files were found, exiting```.  

But, when I add the license I 've been using for the other commands (academic), I also received an error:  ```voila: error: unrecognized arguments: --license ```.

Could you clarify me what I should do with the license please ?

[San Jewell]

Hi Maria,

I apologize that the documentation listed the wrong argument for modulizer, I will make it a to-do to get this updated as soon as I am able. 

For your specific question, I am surprised that you are coming across this error only on the modulize step, as the license is required for every command in the majiq pipeline. On the registratioin page there are examples of various ways to specify the license. For example, if can be in the directory where you perform the run, in your home directory, or referenced by an environment variable. In order to use the --license, switch, though, it would go directly after voila (or majiq) ; for example, $ voila --license <path_to_license> modulize <other modulize arguments>

As far as the files specified, you need to specify the splicegraph zarr, at least one quantification file (psicov, deltapsi, heterogen), and for whichever quantification file you should specify .sgc files that cover all of the samples in them. For example, you can specify splicegraph and a psicov file with three samples quantified in it, and an sgc file with those same three samples, or you could specify a combination like a splicegraph, deltapsi, and psicov files, with multiple sgc files as well. All of the different quantifications will be listed side by side as long as they are not exactly the same names/quant types. 

In the case of your example command, I assume you have checked the help text to verify that you want the --show-all and --keep-constitutive switches ; you can remove the --debug switch, as this will not allow multithreading, the files in your command look right as long as group1.sgc and group2.sgc were created with experiments that cover all of the experiments used in the dpsicov file  (technically you can have a dpsicov file just called '.dpsicov' ; but it is unorthadox and I'd recommend actually giving that file a specific name)

Let me know if it helps. 

Thanks!

-San

[M K]

Hello San,

--> " In order to use the --license, switch, though, it would go directly after voila (or majiq)" 

This is what I hadn't figured out correctly ! It worked when put in the right place. The previous commands I had run with license where after `majiq `, so I didn't realize the difference.Thanks.

--> " (technically you can have a dpsicov file just called '.dpsicov' ; but it is unorthadox and I'd recommend actually giving that file a specific name)"

Yes, of course ^_^  I had a real name with my .dpsicov, just omitted when typing !

Here is my command for modulize: 

``` voila --license majiq_license_academic_official.lic modulize -j 6 \

--show-all --overwrite -d majiq_wsp/modulizer/C002I3K/  \

majiq_wsp/build_output/sg_K_vs_all.zarr \

majiq_wsp/sg_coverage/C002I3K.sgc \

majiq_wsp/sg_coverage/all-vs-K.sgc \

majiq_wsp/deltapsi/voila/voila_K_vs_all.dpsicov ```  

For better overview, here are the previous commands to produce sgc and dpsicov : 

My experiment is about 17 patient files, so I use a particular way of running : 1 sample -vs- all others, as you can see in the names of files :

(open to create an other discussion for the experiment design if it is necessary).

> psi

srun pixi run majiq-v3 psi-coverage --license ${lic}

--nthreads 4

./build_output/sg_${CURRENT_S}_vs_all.zarr \

./psi_cov/${CURRENT_S}_vs_all.psicov \

/SJs/${CURRENT_SAMPLE}.sj –overwrite

 

> delta-psi

srun pixi run majiq-v3 deltapsi --license ${lic} --nthreads 4 \

 --splicegraph  ./build_output/sg_${CURRENT_S}_vs_all.zarr \

 --output-voila ./deltapsi/voila/voila_${CURRENT_S}_vs_all.dpsicov \

 --output-tsv ${work_dir}/deltapsi/tsv/dpsi_${CURRENT_S}_vs_all.tsv \

 -psi1 ./psi_cov/${CURRENT_S}_vs_all.psicov \

 -psi2 ./psi_cov/all_vs_${CURRENT_S}.psicov

 

 

> sg-coverage

# For 1 sample/patient :

srun pixi run majiq-v3 sg-coverage --license ${lic} --prefixes ${CURRENT_SAMPLE} --nthreads ${SLURM_CPUS_PER_TASK} ./build_output/sg_${CURRENT_S}_vs_all.zarr ./sg_coverage/${CURRENT_SAMPLE}.sgc ${work_dir}/SJs/${CURRENT_SAMPLE}.sj

 

#for all other samples "vs-patient"

srun pixi run majiq-v3 sg-coverage --license ${lic} --prefixes ${sampleIDs[@]} --nthreads ${SLURM_CPUS_PER_TASK} ./build_output/sg_${CURRENT_S}_vs_all.zarr \

./sg_coverage/all-vs-${CURRENT_S}.sgc ${SJ_FILES[@]}

 

I actually get an error when processing ( same for many genes): 

` WARNING - Some error processing gene gene:ENSG00000251484 , turn on --debug for more info ` 

and there are no results in output of any type of splicing for any sample , and no other message indicating why ! 

I tried only one sample again with --debug option, and this is what I see in my slurm.out : 

 

` Processing Genes and Modules [2/62754]Processing Genes and Modules [3/62754]

Traceback (most recent call last):

  File majiq_wsp/.pixi/envs/default/lib/python3.10/site-packages/rna_voila/classify.py", line 91, in classify_gene

    graph = Graph(gene_id, experiment_names)

  File "majiq_wsp/.pixi/envs/default/lib/python3.10/site-packages/rna_voila/classifier/as_types.py", line 88, in __init__  self._populate()

  File "majiq_wsp/.pixi/envs/default/lib/python3.10/site-packages/rna_voila/classifier/as_types.py", line 671, in _populate

    if self.config.decomplexify_reads_threshold == 0 or self._enough_reads(

  File "majiq_wsp/.pixi/envs/default/lib/python3.10/site-packages/rna_voila/classifier/as_types.py", line 630, in _enough_reads

    for exp in reads:

  File majiq_wsp/.pixi/envs/default/lib/python3.10/site-packages/rna_voila/api/splice_graph_zarr.py", line 284, in junction_reads_exp

    reads = self.exp_reads.junctions_reads[junction['_junc_idx'], self.exp_reads.prefixes.index(experiment_name)].values

ValueError: 'C002I3S_Aligned.sortedByCoord.mrkDups' is not in list` 

 

This value is from another sample, while I run for the C002I3K, not S ! Moreover,

This is not a valid prefix, I have defined the prefixes to be the code of the sample, without the suffix “Aligned.sortedByCoord.mrkDup”. Why and where is it finding this ?

Is this the reason I don't have any events detected? 

I attach you the whole slurm.err file I got after the debugging, if this may help more. 

I feel stuck, I don't really understand what I could change to obtain some splicing events,

any hints and suggestions are welcome, 

Thanks a lot for your availability and response,

Maria

--show-all --overwrite

[San Jewell]

Hi Maria, 

Thank you for the comprehensive post detailing your issue, I think it may take some iteration to nail down exactly what's happening, as it sounds like a setup that I've not tried before (and maybe we should support, depending on how the conversation goes)

-The reason you may see a name like C002I3S_Aligned.sortedByCoord.mrkDups downstream is due to the way that names / prefixes are assigned internally to files. By default, they just use the name of the bam file, but you can override this:

    -When running majiq-v3 sj or majiq-v3 psi-coverage, you can specify the --prefix or --prefixes argument, respectively

    -When running majiq-v3 deltapsi or heterogen, you can specify --names name1 name2 to give names to the groups 

-As for the error itself, it indicates that there is some mismatch between the experiments provided in quantification file (psi-coverage, deltapsi, etc) and the .sgc files In modulizer these become two large groups where all of the experiments in any of the quantification files should appear at least somewhere in the .sgc files. It is difficult to parse due to the environment variables in your code samples, exactly which sj experiments went into each quantification file and sgc file, but that's probably the place to look. 

Let me know if it makes sense!

-San

[M K]

Hello San,

Thanks a lot for your feedback ! It helped ! I wrote more clearly the code with my "CURRENT_SAMPLE" variable for all commands, rerun all the commands again, and didn't put any prefixes (I was only using in 'sg-coverage', so it created a confusion with previous commands or order of samples), so MAJIQ figured out the samples from the input files. I have finally the so-long-desired results for modulize  for every sample! 

I m copying here the terminal's message from the slurm.err file, where we see the order of files is not same as the one I gave in my array in the script, if this can be an indication for other users. (SAMPLES=(C002I3K C002I3L C002I3M C002I3N C002I3P C002I3Q C002I3R C002I3S C002I3U C002I3W C002I3Y C002I3Z C002I44 C002I47 C002I4F C002I4G) )

 (INFO) - Loading input splicegraph from MAJIQ/majiq_wsp/build_output/sg_C002I45.zarr
(INFO) - Saving prefixes and metadata toMAJIQ/majiq_wsp/sg_coverage/all_vs_C002I45.sgc
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3M.sj (1 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3Q.sj (2 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3N.sj (3 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3P.sj (4 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3L.sj (5 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3K.sj (6 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3R.sj (7 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3S.sj (8 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I44.sj (9 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3W.sj (10 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3Z.sj (11 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3U.sj (12 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I3Y.sj (13 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I47.sj (14 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I4F.sj (15 / 16)
(INFO) - Saved coverage from MAJIQ/majiq_wsp/SJs/C002I4G.sj (16 / 16)
(INFO) - Finished successfully!

Thanks again, 

Have a nice day,

Best ,

Maria