Re: [BGI-SOAP:1240] Digest for bgi-soap@googlegroups.com - 2 updates in 2 topics
24 views
Skip to first unread message
Benzi Galili
Aug 19, 2016, 2:44:21 PM8/19/16
to bgi-...@googlegroups.com
Dear SOAP user,
If you would like the 3 nodes to be turned into a single shared-memory system of 1.5TB RAM, you should use a technology such as ScaleMP's vSMP Foundation. The Free Edition allows for aggregation of up to 1.5TB of RAM, and can be obtained from www.scalemp.com/free after a short registration is filled.
Disclosure: I am with ScaleMP, but have no commercial interest in my reply as this edition of the product is offered free of charge, no strings attached.
Good luck!
Benzi
On Thu, Jun 9, 2016 at 8:55 PM, <bgi-...@googlegroups.com> wrote:
- SOAPdenovo: Problem of sharing memory in cluster - 1 Update
- Problem of sharing memory in cluster - 1 Update
lamz1...@163.com: Jun 09 05:33AM -0700
Hi, dear all!
I had used SOAPdenovo2 to assemble genome. There are four libraries with
120G data respectively, and about 160X (the genome is about 3G). In the
step of pregraph, it didn't report error, but when I ran the step of
contig, it failed immediately. I think the reason is running out of memory
(there are only 500G memory available). Then I turned to cluster machine
(three big nodes, and each node only have 500G memory) and tried to use
memory of these nodes (I have little knowledge about cluster). I installed
openmpi, and used qsub to submit the job with demo data, but it seemed it
just run the SOAPdenovo several times and there are some problems according
to the log. Also, the result is different from when it didn't run in
cluster. Did SOAPdenovo support running in cluster, how can I make the
nodes share memory, or how can I finished my assembly? By the way, although
I run the "hello" script successfully, I didn't know whether I am right in
setting up openmpi.
Any suggestion would be grateful!
Following is what had I done:
1. Run the step of pregraph:
soapPath="/tools/SOAPdenovo2/SOAPdenovo-63mer"
$soapPath pregraph -s config_file -R -p 45 -o graph_prefix > pregraph.log
2> pregraph.err
2. Run the step of contig:
$soapPath contig -g graph_prefix -R -p 45 1>contig.log 2>contig.err
3. The error in contig step:
cat contig.err
Version 2.04: released on July 13th, 2012
Compile Apr 27 2016 15:50:02
********************
Contig
********************
Parameters: contig -g graph_prefix -R -p 45
There are 561510855 kmer(s) in vertex file.
There are 1856621603 edge(s) in edge file.
Ran out of memory while applying -32782782016bytes
There may be errors as follows:
1) Not enough memory.
2) The ARRAY may be overrode.
3) The wild pointers.
4. Install openmpi and submit the job
1) The script of job:
cat mpi.qsub:
#!/bin/bash
#
#$ -cwd
#$ -j y
#$ -S /bin/bash
#
export PATH=/tools/openmpi/bin:$PATH
export LD_LIBRARY_PATH=/tools/openmpi/lib:$LD_LIBRARY_PATH
soapPath="/tools/SOAPdenovo2/SOAPdenovo-63mer"
workPath="/NGS"
outputPath="assembly/soap/demo"
/tools/openmpi/bin/mpirun $soapPath all -s
$workPath/$outputPath/config_file -K 23 -R -F -p 60 -V -o
$workPath/$outputPath/graph_prefix > $workPath/$outputPath/ass.log 2>
$workPath/$outputPath/ass.err
2) Submit the job:
qsub -pe orte 60 mpi.qsub
3) The log in ass.err
cat ass.err | grep "Pregraph" | wc -l
60
less ass.err:
Version 2.04: released on July 13th, 2012
Compile Apr 27 2016 15:50:02
********************
Pregraph
********************
Parameters: pregraph -s /NGS/assembly/soap/demo/config_file -K 23 -p 16 -R
-o /NGS/assembly/soap/demo/graph_prefix
In /NGS/assembly/soap/demo/config_file, 1 lib(s), maximum read length 35,
maximum name length 256.
Version 2.04: released on July 13th, 2012
Compile Apr 27 2016 15:50:02
********************
Pregraph
********************
and so on
cat ass.log:
--------------------------------------------------------------------------
WARNING: A process refused to die despite all the efforts!
This process may still be running and/or consuming resources.
Host: smp03
PID: 75035
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 58 with PID 0 on node c0214.local exited
on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
lamz1...@163.com: Jun 09 04:31AM -0700
Hi, dear all!
I had used SOAPdenovo2 to assemble genome. There are four libraries with
120G data respectively, and about 160X (the genome is about 3G). In the
step of pregraph, it didn't report error, but when I ran the step of
contig, it failed immediately. I think the reason is running out of memory
(there are only 500G memory available). Then I turned to cluster machine
(three big nodes, and each node only have 500G memory) and tried to use
memory of these nodes (I have little knowledge about cluster). I installed
openmpi, and used qsub to submit the job with demo data, but it seemed it
just run the SOAPdenovo several times and there are some problems according
to the log. Also, the result is different from when it didn't run in
cluster. Did SOAPdenovo support running in cluster, how can I make the
nodes share memory, or how can I finished my assembly? By the way, although
I run the "hello" script successfully, I didn't know whether I am right in
setting up openmpi.
Any suggestion would be grateful!
Following is what had I done:
1. Run the step of pregraph:
soapPath="/tools/SOAPdenovo2/SOAPdenovo-63mer"
$soapPath pregraph -s config_file -R -p 45 -o graph_prefix > pregraph.log
2> pregraph.err
2. Run the step of contig:
$soapPath contig -g graph_prefix -R -p 45 1>contig.log 2>contig.err
3. The error in contig step:
cat contig.err
Version 2.04: released on July 13th, 2012
Compile Apr 27 2016 15:50:02
********************
Contig
********************
Parameters: contig -g graph_prefix -R -p 45
There are 561510855 kmer(s) in vertex file.
There are 1856621603 edge(s) in edge file.
Ran out of memory while applying -32782782016bytes
There may be errors as follows:
1) Not enough memory.
2) The ARRAY may be overrode.
3) The wild pointers.
4. Install openmpi and submit the job
1) The script of job:
cat mpi.qsub:
#!/bin/bash
#
#$ -cwd
#$ -j y
#$ -S /bin/bash
#
export PATH=/tools/openmpi/bin:$PATH
export LD_LIBRARY_PATH=/tools/openmpi/lib:$LD_LIBRARY_PATH
soapPath="/tools/SOAPdenovo2/SOAPdenovo-63mer"
workPath="/NGS"
outputPath="assembly/soap/demo"
/tools/openmpi/bin/mpirun $soapPath all -s
$workPath/$outputPath/config_file -K 23 -R -F -p 60 -V -o
$workPath/$outputPath/graph_prefix > $workPath/$outputPath/ass.log 2>
$workPath/$outputPath/ass.err
2) Submit the job:
qsub -pe orte 60 mpi.qsub
3) The log in ass.err
cat ass.err | grep "Pregraph" | wc -l
60
less ass.err:
Version 2.04: released on July 13th, 2012
Compile Apr 27 2016 15:50:02
********************
Pregraph
********************
Parameters: pregraph -s /NGS/assembly/soap/demo/config_file -K 23 -p 16 -R
-o /NGS/assembly/soap/demo/graph_prefix
In /NGS/assembly/soap/demo/config_file, 1 lib(s), maximum read length 35,
maximum name length 256.
Version 2.04: released on July 13th, 2012
Compile Apr 27 2016 15:50:02
********************
Pregraph
********************
and so on
cat ass.log:
--------------------------------------------------------------------------
WARNING: A process refused to die despite all the efforts!
This process may still be running and/or consuming resources.
Host: smp03
PID: 75035
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 58 with PID 0 on node c0214.local exited
on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to bgi-soap+unsubscribe@googlegroups.com.
--B
Reply all
Reply to author
Forward
0 new messages