run abyss-pe on a single machine with multiple cores

[Dong]

Hi Shaun,

I was trying to run abyss-pe on my single machine with multiple
cores, but got the following message:

---------------------------------------------------------------------------------------------------------
/home/usrlocal/bin/abyss-pe np=20 k=55 n=5 j=20 name=test in='/dong/
test11 /dong/test12'
/usr/bin/mpirun -np 20 ABYSS-P -k55 -q3 --coverage-hist=coverage.hist
-s test-bubbles.fa -o test-1.fa /dong/test11 /dong/test12
mpirun: cannot start ABYSS-P on n0 (o): No such file or directory
make: *** [test-1.fa] Error 2
--------------------------------------------------------------------------------------------------------

I installed this assembler without MPI support, because abyss manual
says: "The `np' option of abyss-pe specifies the number of processes
to use for the ABYSS-P parallel MPI job. Without any MPI
configuration,this will allow you to make use of multiple cores on a
single machine.".

I am very confused with it. Could you let me know what's wrong with my
command line or installation?

Thanks!

Dong

[Tony Raymond]

Hi Dong,

It appears that if you installed ABySS with MPI support AND did not configure MPI then you could still use the np option to use ABYSS-P on one machine. ABYSS-P was not generated since you didn't install with mpi support. This means you can only use the threading parameter (j=20) and cannot run abyss-pe with the np parameter, or you must install with MPI support.

Cheers,
Tony
________________________________________
From: abyss...@googlegroups.com [abyss...@googlegroups.com] On Behalf Of Dong [dong...@gmail.com]
Sent: Wednesday, September 07, 2011 10:10 AM
To: ABySS
Subject: run abyss-pe on a single machine with multiple cores

[Tony Raymond]

Hi Dong,

 

I’m actually not too sure about this. Shaun or someone else in the ABySS users group may be able to help you find a suitable value for the s parameter.

 

Cheers,

Tony

 

From: dong he [mailto:dong...@gmail.com]
Sent: Friday, September 09, 2011 5:59 AM
To: Tony Raymond
Subject: Re: run abyss-pe on a single machine with multiple cores

 

Hi Tony,

Yes, it was tmp space problem. Thank you! By the way, I have another question need your suggestion:
I have four libraries with different fragment sizes, these libraries can be assembled together, but how to specify the s parameter?

Thank

Dong

On Thu, Sep 8, 2011 at 12:16 PM, Tony Raymond <tray...@bcgsc.ca> wrote:

Hi Dong

Looks like this is your problem:

sort: write failed: /tmp/sortuatupN: No space left on device

If you don't have access to that directory to clear some space or it is too small you should be able to change the TMPDIR environment variable to change the directory sort uses for temporary files.

Cheers,
Tony
________________________________________
From: dong he [dong...@gmail.com]
Sent: Thursday, September 08, 2011 8:50 AM

To: Tony Raymond
Subject: Re: run abyss-pe on a single machine with multiple cores

Hi Tony,

I re-ran my job, it seems working well, but still got error:

--------------------------------------------------------------------------------
-----
----
sort: write failed: /tmp/sortuatupN: No space left on device
gunzip -c testnohup-3.sam.gz \
|DistanceEst  -j20 -k55 -s100 -n5 -o testnohup-3.dist testnohup-3.hist
warning: the seed-length should be at least twice k: k=55, s=100
DistanceEst: error: no @SQ records in the SAM header
make: *** [testnohup-3.dist] Error 1
make: *** Deleting file `testnohup-3.dist'
-----------------------------------------------------------------------------------------------

could you let me know what's wrong with it?

Thanks!

Dong

On Wed, Sep 7, 2011 at 2:22 PM, Tony Raymond <tray...@bcgsc.ca<mailto:tray...@bcgsc.ca>> wrote:
> Hi Dong,
>
> No problem! With your current installation, the single end assembly (ABYSS) would run on one core. Most of the paired end assembly is threaded and that portion will use 20 cores. You will need to reinstall with MPI support if you wish to have the whole assembly run in parallel. Also, if you do reinstall you will only need to specify the np parameter since the default for j=np.
>
> Cheers,
> Tony
>
> -----Original Message-----

> From: dong he [mailto:dong...@gmail.com<mailto:dong...@gmail.com>]
> Sent: Wednesday, September 07, 2011 12:13 PM
> To: Tony Raymond
> Subject: Re: run abyss-pe on a single machine with multiple cores
>
> Hi Tony,
>
> Thank you for your email. I will try it, and if I use j=20 on my
> current installation, would abyss-pe  behave same as that when np=20
> is used on ABySS with MPI support? it means if abyss-pe would  run on
> 20 cpus when j=20?
>
> Thanks again!
>
> Dong
>
>
>

> On Wed, Sep 7, 2011 at 1:44 PM, Tony Raymond <tray...@bcgsc.ca<mailto:tray...@bcgsc.ca>> wrote:
>> Hi Dong,
>>
>> It appears that if you installed ABySS with MPI support AND did not configure MPI then you could still use the np option to use ABYSS-P on one machine.  ABYSS-P was not generated since you didn't install with mpi support. This means you can only use the threading parameter (j=20) and cannot run abyss-pe with the np parameter, or you must install with MPI support.
>>
>> Cheers,
>> Tony
>> ________________________________________

[Shaun Jackman]

Hi Dong,

As a reasonable default, I'd suggest setting s=200.

Cheers,
Shaun

On Fri, 2011-09-09 at 16:07 -0700, Tony Raymond wrote:
> Hi Dong,
>
>
>
> IοΏ½m actually not too sure about this. Shaun or someone else in the