Trinity Phase 2 giving memory and thread errors at the very first step of parafly

[Ankit gupta]

Hello All,

I am running Trinity using the following command:

nohup ./Trinity --seqType fq --left KGD_T_R1_filtered.fq --right KGD_T_R2_filtered.fq --CPU 150 --max_memory 450G

The Phase 1 runs fine but the Phase 2 displays follwing errors:

Thursday, June 15, 2017: 06:50:58 CMD: /media/Disk1/PC_Genome/tools/trinityrnaseq-Trinity-v2.4.0/trinity-plugins/parafly/bin/ParaFly -c recursive_trinity.cmds -CPU 150 -v 

Number of Commands: 434391

Error occurred during initialization of VM

java.lang.OutOfMemoryError: unable to create new native thread

We are sorry, commands in file: [failed_butterfly_commands.130130.txt] failed.  :-( 

sh: fork: retry: Resource temporarily unavailable

Trinity run failed. Must investigate error above.

succeeded(0), failed(1)   0.000230207% completed.    Trinity run failed. Must investigate error above.

.

.

Trinity run failed. Must investigate error above.

succeeded(32), failed(24)   0.0128916% completed.    #

# There is insufficient memory for the Java Runtime Environment to continue.

# Cannot create GC thread. Out of system resources.

# An error report file with more information is saved as:

# /media/Disk1/PC_Genome/Transcriptome_Assembly/trinity_out_dir/read_partitions/Fb_0/CBin_0/c84.trinity.reads.fa.out/hs_err_pid143843.log

Error occurred during initialization of VM

java.lang.OutOfMemoryError: unable to create new native thread

succeeded(33), failed(24)   0.0131218% completed.    

These error in the log file is the follwoing:

# There is insufficient memory for the Java Runtime Environment to continue.

# Cannot create GC thread. Out of system resources.

# This output file may be truncated or incomplete.

#

#  Out of Memory Error (gcTaskThread.cpp:48), pid=143843, tid=0x00007f2fc7433700

#

# JRE version:  (8.0_131-b11) (build )

# Java VM: OpenJDK 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)

# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

#

I have 160 CPU and 502 GB of RAM and java 1.8.0_131 installed in my system.

The fastq file size are of 105GB each (100bp paired end reads)

The trinity_out_folder generated around 126 Gb of data before I killed it.

I found multiple solutions in google groups. however, I couldn't figure out the exact problem in my case. 

Thanks in advance for the help.

Best

[Mark Chapman]

Just try with less max memory that's likely to solve the most simple of 'out of memory' problems

Cheers, Mark

--
You received this message because you are subscribed to the Google Groups "trinityrnaseq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-users+unsub...@googlegroups.com.
To post to this group, send email to trinityrnaseq-users@googlegroups.com.
Visit this group at https://groups.google.com/group/trinityrnaseq-users.
For more options, visit https://groups.google.com/d/optout.

[Brian Haas]

Also, try running with --CPU 10 or 20 for comparison.  While you have a lot of CPUs, the

  sh: fork: retry: Resource temporarily unavailable

message suggests that the operating system is having difficulty managing all the processes.

~b

> To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-u...@googlegroups.com.
> To post to this group, send email to trinityrn...@googlegroups.com.

> Visit this group at https://groups.google.com/group/trinityrnaseq-users.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "trinityrnaseq-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-u...@googlegroups.com.
> To post to this group, send email to trinityrn...@googlegroups.com.

> Visit this group at https://groups.google.com/group/trinityrnaseq-users.
> For more options, visit https://groups.google.com/d/optout.

--
--
Brian J. Haas
The Broad Institute
http://broadinstitute.org/~bhaas

 

[Ankit gupta]

Thanks Brian and Mark for your suggestions. I am re-running trinity using the following command:

nohup ./Trinity --seqType fq --left KGD_T_R1_filtered.fq --right KGD_T_R2_filtered.fq --bflyCalculateCPU --CPU 20 --max_memory 350GB

However, I just wanted to make sure that the JavaHeapSize is correctly set. Following are the default settings of java: 

    uintx HeapSizePerGCThread                 = 87241520                            {product}

    uintx InitialHeapSize                           := 2147483648                          {product}

    uintx LargePageHeapSizeThreshold      = 134217728                           {product}

    uintx MaxHeapSize                              := 32210157568                         {product}

This is the command that was running for butterfly previously :

java -Xmx4G -Xms1G -Xss1G -XX:ParallelGCThreads=2 -jar /media/Disk1/PC_Genome/tools/trinityrnaseq-Trinity-v2.4.0/Butterfly/Butterfly.......

Should I change my java heap size ? Is my MaxHeapSize (30 GB) creating an issue while running trinity ??

Thanks again for your help and quick response

> To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-users+unsub...@googlegroups.com.

> To post to this group, send email to trinityrn...@googlegroups.com.
> Visit this group at https://groups.google.com/group/trinityrnaseq-users.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "trinityrnaseq-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-users+unsub...@googlegroups.com.

[Brian Haas]

I think it should be fine.  Butterfly is set to run w/ a max heap of 4G.  I think it was just the OS being overwhelmed by the number of processes.

best,

~b

> To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-users+unsubscribe...@googlegroups.com.

> To post to this group, send email to trinityrn...@googlegroups.com.

> Visit this group at https://groups.google.com/group/trinityrnaseq-users.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "trinityrnaseq-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-users+unsubscribe...@googlegroups.com.

> To post to this group, send email to trinityrn...@googlegroups.com.
> Visit this group at https://groups.google.com/group/trinityrnaseq-users.
> For more options, visit https://groups.google.com/d/optout.




--
--
Brian J. Haas
The Broad Institute
http://broadinstitute.org/~bhaas

 

--

You received this message because you are subscribed to the Google Groups "trinityrnaseq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trinityrnaseq-users+unsub...@googlegroups.com.

To post to this group, send email to trinityrnaseq-users@googlegroups.com.

Visit this group at https://groups.google.com/group/trinityrnaseq-users.
For more options, visit https://groups.google.com/d/optout.