Difficulties adding mp libraries with 1.3.2
stuartg
insect genome (400Mbp, 300 and 500bp inserts). However, when I add
the ALIGNER_OPTIONS and 2 MP libraries (36bp, 5kb insert) to the same
assembly, it terminates after (I think) the first mapping (all in a
clean dir). Another post regarding a similar error message found it
to be a local memory problem. Is that likely in this case too?
The two command lines and the tail of the terminating log are as
follows:
/share/apps/abyss/1.3.2_96/bin/abyss-pe name=Btk65 k=65 aligner=map
n=4 v=-v np=8 lib='lib1 lib2 lib3 lib4' \ lib1='...works fine
/share/apps/abyss/1.3.2_96/bin/abyss-pe name=Btk65 k=65 aligner=map
n=4 v=-v np=8 lib='lib1 lib2 lib3 lib4' ALIGNER_OPTIONS='-k24'
mp='lib5 lib6' \ lib1='... terminates
log:
Btk65-2.path Btk65-1.fa >Btk65-indel.fa
abyss-map -v -j8 -k65 -k24 Bt_s5_foward_paired.cor.fastq.gz
Bt_s5_reverse_paired.cor.fastq.gz Btk65-3.fa \
|abyss-fixmate -v -h lib1-3.hist \
|sort -snk3 -k4 \
|DistanceEst -v -j8 -k65 -s200 -n4 -o lib1-3.dist lib1-3.hist
Reading from standard input...
Reading `Btk65-3.fa'...
Reading `Btk65-3.fa'...
The alphabet has 5 symbols.
Building the suffix array...
Building the Burrows-Wheeler transform...
Building the character occurence table...
Read 1.22 GB in 6184997 contigs.
Using 11.2 GB of memory and 9.19 B/bp.
Read 7 alignments. Hash load: 3 / 5 = 0.6 using 135 kB.
Read 38 alignments. Hash load: 6 / 11 = 0.545455 using 135 kB.
Read 1000000 alignments. Hash load: 2 / 11 = 0.181818 using 135 kB.
.....(increments of the same process)....
Read 59000000 alignments. Hash load: 4 / 11 = 0.363636 using 135 kB.
Mapped 59597130 of 59877574 reads (99.5%)
Mapped 57783526 of 59877574 reads uniquely (96.5%)
Read 59877574 alignments
Mateless 0
Unaligned 52868 0.177%
Singleton 174708 0.584%
FR 8720300 29.1%
RF 829 0.00277%
FF 22488 0.0751%
Different 20967594 70%
Total 29938787
Stats mean: 254.9 median: 266 sd: 52 n: 8719473 min: 27 max: 904
_▃██▃_
Mate orientation FR: 8720300 (100%) RF: 829 (0.00951%)
Stats mean: 254.9 median: 266 sd: 52 n: 8719473 min: 27 max: 904
_▃██▃_
DistanceEst: MLE.cpp:146: int maximumLikelihoodEstimate(int, int,
const std::vector<int, std::allocator<int> >&, const PMF&, unsigned
int, unsigned int, bool, unsigned int&): Assertion `*it > 2 * (int)l'
failed.
sort: write failed: standard output: Broken pipe
sort: write error
/bin/bash: line 3: 11930 Done abyss-map -v -j8 -k65
-k24 Bt_s5_foward_paired.cor.fastq.gz
Bt_s5_reverse_paired.cor.fastq.gz Btk65-3.fa
11931 | abyss-fixmate -v -h lib1-3.hist
11932 Exit 2 | sort -snk3 -k4
11933 Aborted | DistanceEst -v -j8 -k65 -s200 -n4
-o lib1-3.dist lib1-3.hist
make: *** [lib1-3.dist] Error 134
make: *** Deleting file `lib1-3.dist'
Shaun Jackman
I'm still working out the issues of mapping reads back to an assembly
with a different value of k than the assembly value of k.
DistanceEst must use the same value of k as abyss-map. I've attached a
patched version of abyss-pe that takes care of this. Use it like so:
abyss-pe name=Btk65 k=65 aligner=map
n=4 v=-v np=8 lib='lib1 lib2 lib3 lib4'
mp='lib5 lib6' lib5_k=24 lib6_k=24
lib1='...
DistancEst outputs the value of k that it was run with, and later stages
will complain that it doesn't agree with the assembly value. The
attached patch works around this by patching DistanceEst to not output
the value of k.
I plan to have this all sorted out for the next release of ABySS. Sorry
for the trouble it's causing you.
Cheers,
Shaun
--- DistanceEst/DistanceEst.cpp (revision 75194)
+++ DistanceEst/DistanceEst.cpp (working copy)
@@ -392,10 +392,7 @@ int main(int argc, char** argv)
ostream& out = opt::out.empty() ? cout : outFile;
if (opt::format == DOT)
- out << "digraph dist {\ngraph ["
- "k=" << opt::k << " "
- "s=" << opt::seedLen << " "
- "n=" << opt::npairs << "]\n";
+ out << "digraph dist {\n";
// Read the fragment size distribution.
Histogram distanceHist = loadHist(distanceCountFile);
Steve
With this patch, I can now select a value of K specific to each
library for use with the abyss-map aligner. e.g lib2_k=50, lib3_k=25.
Is this correct?
Cheers,
Stephen
> 8KViewDownload
Shaun Jackman
Yes, that’s correct.
Cheers,
Shaun
Steve
I applied the patch above, verified the changes were made in the
DistanceEst.cpp file, and re-compiled abyss.
I then issued the abyss-pe command like you suggested above:
abyss-pe name=Btk65 k=65 aligner=map
n=4 v=-v np=8 lib='lib1 lib2 lib3 lib4'
mp='lib5 lib6' lib5_k=24 lib6_k=24
lib1='...
mapped back to the assembly (mapped 35518624 of 161489056 - 22%). My
assembly K was 73 and my mp libraries are mixtures of 75bp and 35bp
reads (~55% are 35bp). I suspect it is using the assembly Kmer size of
73 for abyss-map, hence the low mapping.
In the suggested command, the ALIGNER_OPTIONS='-k24' is not used. With
the patch, should I still use this option in my abyss-pe command?
There was also an abyss-pe attachment in the message with the patch.
After re-compiling with the patch, should I be replacing the abyss-pe
file with the attached one?
Thanks for your help,
Stephen
On Feb 20, 7:43 pm, Shaun Jackman <sjack...@bcgsc.ca> wrote:
> Hi Stephen,
>
> Yes, that’s correct.
>
> Cheers,
> Shaun
>
> On 2012-02-20, at 24:21 , Steve wrote:
>
>
>
>
>
>
>
> > Hi Shaun,
>
> > With this patch, I can now select a value of K specific to each
> > Is this correct?
>
> > Cheers,
>
> > Stephen
>
> > On Jan 24, 9:30 pm, Shaun Jackman <sjack...@bcgsc.ca> wrote:
> >> Hi Stuart,
>
> >> I'm still working out the issues of mapping reads back to an assembly
> >> with a different value of k than the assembly value of k.
>
Shaun Jackman
Yes, you should use the attached abyss-pe. You can use the -n,--dry-run option of abyss-pe to ensure that abyss-map will use the intended value of k.
The next release of ABySS (1.3.3) will include this feature.
Cheers,
Shaun