Running issue in Jellyfish? (Trinity 2.4.0)

[Kevin McCormick]

Hello,

I had some issues earlier because I did not create a single transcriptome reference, and am currently trying to fix the issue.

However, I have another problem now. I get a memory allocation error now that I don't understand (see screen shot photo).

For some background. This is RNAseq data from 10 individuals, paired end reads. All files are trimmed using trimmomatic, and then I normalized prior to running Trinity due to size issues using insilico_read_normalization.pl (Example: insilico_read_normalization.pl --seqType fq --JM 100G --max_cov 50 --left TNDR1_pe --right TNDR2_pe --pairs_together --PARALLEL_STATS --CPU 10 --output TNDR_Trim_Norm)

After I then feed the results into this script (below) on our cluster with an over estimate of memory and time needed. However, it got the error in the screenshot within a few hours. Any advice at all at this point is welcome.

### walltime - how long you expect the job to run

#PBS -l walltime=120:00:00

### nodes:ppn - how many nodes & cores per node (ppn) that you require

#PBS -l nodes=5:ppn=28

### mem: amount of memory that the job will need

#PBS -l mem=625gb

### you can give your job a name for easier identification

#PBS -N Full_Trinity_New_Trim_Norm

### load necessary modules, e.g.

module load GNU/4.9

module load trinity/2.4.0

Trinity --seqType fq \

         --left /mnt/home/mccor187/Fisi/FastqFiles/Trimmed/ACNE_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/ADS_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/CYBR_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/MCDS_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/NUDI_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/OTIS_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/SLWP_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/SRG_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/TAMI_Trim_Norm/left.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/TNDR_Trim_Norm/left.norm.fq \

         --right /mnt/home/mccor187/Fisi/FastqFiles/Trimmed/ACNE_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/ADS_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/CYBR_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/MCDS_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/NUDI_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/OTIS_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/SLWP_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/SRG_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/TAMI_Trim_Norm/right.norm.fq,/mnt/home/mccor187/Fisi/FastqFiles/Trimmed/TNDR_Trim_Norm/right.norm.fq \

         --CPU 28 --max_memory 125G --min_contig_length 150

[Brian Haas]

This looks like a core dump from jellyfish, so there's something wrong with the jellyfish installation.  You can install a newer version of jellyfish and that'll likely fix it.   It also looks like you're using a pretty old version of Trinity here too, so you might want to upgrade that as well.

best,

~b

--
You received this message because you are subscribed to the Google Groups "trinityrnaseq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-u...@googlegroups.com.
To post to this group, send email to trinityrn...@googlegroups.com.
Visit this group at https://groups.google.com/group/trinityrnaseq-users.
For more options, visit https://groups.google.com/d/optout.

--

--
Brian J. Haas
The Broad Institute
http://broadinstitute.org/~bhaas

 

[Brian Haas]

I took a closer look.  It's actually a jellyfish out of memory error.

There are a couple of ways around this.  The easiest is to include 

    Trinity --min_kmer_cov 2

and that'll greatly reduce memory consumption.

If you're using an old version of Trinity that doesn't do read normalization by default, then you should also include  --normalize_reads

This should hopefully resolve the memory issues. 

Note, when you rerun Trinity, do it in a new workspace so it won't try to reuse the earlier-generated outputs as a starting point.

best,

~brian