cstacks failure
365 views
Skip to first unread message
Marco Sollitto
Jul 19, 2021, 5:07:16 AM7/19/21
to Stacks
Dear all,
I launched denovo_map.pl.
The command ustacks worked without any issue. However, cstacks failed for an unknowing reason giving me the following output
denovo_map.pl: Aborted because the last command failed (1)
Could you provide some suggestions?
Here, you can find, the last lines of the log file.
/usr/local/bin/cstacks -p 100 -M /mnt/marcos/popmap.txt -n 5 -P /mnt/marcos/stacks
cstacks parameters selected:
Loci matched based on sequence identity.
Number of mismatches allowed between stacks: 5
Gapped alignments: enabled
Constructing catalog from 32 samples.
Initializing new catalog...
Parsing /mnt/marcos/stacks/RA02_R1.trimmed.tags.tsv
Parsing /mnt/marcos/stacks/RA02_R1.trimmed.snps.tsv
Parsing /mnt/marcos/stacks/RA02_R1.trimmed.alleles.tsv
360808 loci were newly added to the catalog.
Processing sample /mnt/marcos/stacks/RA03_R1.trimmed [2 of 32]
Parsing /mnt/marcos/stacks/RA03_R1.trimmed.tags.tsv
Parsing /mnt/marcos/stacks/RA03_R1.trimmed.snps.tsv
Parsing /mnt/marcos/stacks/RA03_R1.trimmed.alleles.tsv
Searching for sequence matches...
Warning: combination of k-mer length (5) and edit distance (5) allows for sequences to be missed by the matching algorithm.
denovo_map.pl: Aborted because the last command failed (1).
Thanks for support,
Marco
Julian Catchen
Jul 19, 2021, 4:11:17 PM7/19/21
to stacks...@googlegroups.com, Marco Sollitto
Hi Marco,
Check your memory usage and lower your number of threads. Using 100
threads is counter productive and will likely use up a lot of extra
memory. I would not use more than 15 or 20 threads. Do you actually have
100 cores on the system you are executing on?
What length are your input reads and how/why did you trim them?
julian
Marco Sollitto wrote on 7/19/21 4:07 AM:
> *
> *
> *
> *
> Thanks for support,
> Marco
>
Check your memory usage and lower your number of threads. Using 100
threads is counter productive and will likely use up a lot of extra
memory. I would not use more than 15 or 20 threads. Do you actually have
100 cores on the system you are executing on?
What length are your input reads and how/why did you trim them?
julian
Marco Sollitto wrote on 7/19/21 4:07 AM:
> Dear all,
>
> I launched denovo_map.pl.
> The command ustacks worked without any issue. However, cstacks failed
> for an unknowing reason giving me the following output
>
> *denovo_map.pl: Aborted because the last command failed (1)
>
> I launched denovo_map.pl.
> The command ustacks worked without any issue. However, cstacks failed
> for an unknowing reason giving me the following output
>
> *
>
> Could you provide some suggestions?
>
> Here, you can find, the last lines of the log file.
>
>
> /usr/local/bin/cstacks -p 100 -M /mnt/marcos/popmap.txt -n 5 -P
> /mnt/marcos/stacks
>
> cstacks parameters selected:
> Loci matched based on sequence identity.
> Number of mismatches allowed between stacks: 5
> Gapped alignments: enabled
> Constructing catalog from 32 samples.
>
> Initializing new catalog...
> Parsing /mnt/marcos/stacks/RA02_R1.trimmed.tags.tsv
> Parsing /mnt/marcos/stacks/RA02_R1.trimmed.snps.tsv
> Parsing /mnt/marcos/stacks/RA02_R1.trimmed.alleles.tsv
> 360808 loci were newly added to the catalog.
>
> Processing sample /mnt/marcos/stacks/RA03_R1.trimmed [2 of 32]
> Parsing /mnt/marcos/stacks/RA03_R1.trimmed.tags.tsv
> Parsing /mnt/marcos/stacks/RA03_R1.trimmed.snps.tsv
> Parsing /mnt/marcos/stacks/RA03_R1.trimmed.alleles.tsv
> Searching for sequence matches...
> Warning: combination of k-mer length (5) and edit distance (5) allows
> for sequences to be missed by the matching algorithm.
>
> denovo_map.pl: Aborted because the last command failed (1).
> *
> Could you provide some suggestions?
>
> Here, you can find, the last lines of the log file.
>
>
> /usr/local/bin/cstacks -p 100 -M /mnt/marcos/popmap.txt -n 5 -P
> /mnt/marcos/stacks
>
> cstacks parameters selected:
> Loci matched based on sequence identity.
> Number of mismatches allowed between stacks: 5
> Gapped alignments: enabled
> Constructing catalog from 32 samples.
>
> Initializing new catalog...
> Parsing /mnt/marcos/stacks/RA02_R1.trimmed.tags.tsv
> Parsing /mnt/marcos/stacks/RA02_R1.trimmed.snps.tsv
> Parsing /mnt/marcos/stacks/RA02_R1.trimmed.alleles.tsv
> 360808 loci were newly added to the catalog.
>
> Processing sample /mnt/marcos/stacks/RA03_R1.trimmed [2 of 32]
> Parsing /mnt/marcos/stacks/RA03_R1.trimmed.tags.tsv
> Parsing /mnt/marcos/stacks/RA03_R1.trimmed.snps.tsv
> Parsing /mnt/marcos/stacks/RA03_R1.trimmed.alleles.tsv
> Searching for sequence matches...
> Warning: combination of k-mer length (5) and edit distance (5) allows
> for sequences to be missed by the matching algorithm.
>
> denovo_map.pl: Aborted because the last command failed (1).
> *
> *
> *
> Thanks for support,
> Marco
>
Marco Sollitto
Jul 20, 2021, 4:16:17 AM7/20/21
to Stacks
Dear dr. Catchen,
I tried even with 20 threads, but it failed again.
My reads are around 37-38 nucleotides. I just removed the adapters, obtaining only the insert size.
I hope I have been clear.
Marco
Julian Catchen
Jul 20, 2021, 2:46:36 PM7/20/21
to stacks...@googlegroups.com, Marco Sollitto
Hi Marco,
With reads of that size, I would not set -n larger than 1 or 2.
At the same time, in Stacks v2.58 I fixed some bugs related to
processing reads of different lengths. You could upgrade, or trim your
reads to a uniform length with process_radtags.
Best,
julian
Marco Sollitto wrote on 7/20/21 3:16 AM:
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>:
With reads of that size, I would not set -n larger than 1 or 2.
At the same time, in Stacks v2.58 I fixed some bugs related to
processing reads of different lengths. You could upgrade, or trim your
reads to a uniform length with process_radtags.
Best,
julian
Marco Sollitto wrote on 7/20/21 3:16 AM:
> Dear dr. Catchen,
>
> I tried even with 20 threads, but it failed again.
>
> My reads are around 37-38 nucleotides. I just removed the adapters,
> obtaining only the insert size.
>
> I hope I have been clear.
>
> Marco
>
>
>
> Il giorno lunedì 19 luglio 2021 alle 22:11:17 UTC+2 jcatchen ha scritto:
>
> Hi Marco,
>
> Check your memory usage and lower your number of threads. Using 100
> threads is counter productive and will likely use up a lot of extra
> memory. I would not use more than 15 or 20 threads. Do you actually
> have
> 100 cores on the system you are executing on?
>
> What length are your input reads and how/why did you trim them?
>
> julian
>
> Marco Sollitto wrote on 7/19/21 4:07 AM:
> > Dear all,
> >
> > I launched denovo_map.pl
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>.
>
> I tried even with 20 threads, but it failed again.
>
> My reads are around 37-38 nucleotides. I just removed the adapters,
> obtaining only the insert size.
>
> I hope I have been clear.
>
> Marco
>
>
>
> Il giorno lunedì 19 luglio 2021 alle 22:11:17 UTC+2 jcatchen ha scritto:
>
> Hi Marco,
>
> Check your memory usage and lower your number of threads. Using 100
> threads is counter productive and will likely use up a lot of extra
> memory. I would not use more than 15 or 20 threads. Do you actually
> have
> 100 cores on the system you are executing on?
>
> What length are your input reads and how/why did you trim them?
>
> julian
>
> Marco Sollitto wrote on 7/19/21 4:07 AM:
> > Dear all,
> >
> > I launched denovo_map.pl
>
> > The command ustacks worked without any issue. However, cstacks
> failed
> > for an unknowing reason giving me the following output
> >
> > *denovo_map.pl
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>:
> > The command ustacks worked without any issue. However, cstacks
> failed
> > for an unknowing reason giving me the following output
> >
> > *denovo_map.pl
Giulia Trauzzi
Jul 20, 2021, 6:05:29 PM7/20/21
to stacks...@googlegroups.com, Marco Sollitto
Hi Julian,
so the new version allows reads with different length to be processed?
Thanks
Giulia
--
Stacks website: http://catchenlab.life.illinois.edu/stacks/
---
You received this message because you are subscribed to the Google Groups "Stacks" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stacks-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/stacks-users/6088c5dd-2e6b-ece4-8d4e-306828cdfa42%40illinois.edu.
Julian Catchen
Jul 21, 2021, 10:47:06 AM7/21/21
to stacks...@googlegroups.com, Giulia Trauzzi, Marco Sollitto
Hi Guilia,
Short answer: yes.
Longer answer: Stacks has always allowed reads of different lengths,
such as caused by incorporating indels but we haven't prioritized
trimmed input reads -- when all raw input reads are variable lengths.
There were a few bugs affecting this in ustacks, but it should now work
properly.
Opinionated answer: it is not typically useful to trim your input reads
-- you will get a worse result than not trimming. Particularly since
people who trim often are starting with lower quality sequencing data
(which is why they are considering trimming in the first place). In RAD
data, trimming creates artificially low coverage in your loci at the 3'
end of the read, affecting SNP calling. Better to keep the reads intact
and let the SNP calling model handle small amounts of error. An
exception may be if you have lots of adaptor, which has to be removed.
Here the question is: is it better to have a region of the loci with low
coverage, or to just uniformly truncate all reads to a reasonable length
that gets rid of most of the adaptor (I would favor this latter
approach, which is how process_radtags is programmed).
julian
Giulia Trauzzi wrote on 7/20/21 5:05 PM:
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21tqEoVXiG_vuva1maxB6osKkXsYTEPCz5-luU7hJF8au2vpD00af-1_SofRFOHOfrpo8$>
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21tqEoVXiG_vuva1maxB6osKkXsYTEPCz5-luU7hJF8au2vpD00af-1_SofRFOHOfrpo8$>
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/stacks-users/CALo9B-t-_k0y0rMu-nr_AMZT2SoiKjJFLwyU2weD3bNc65DHvQ*40mail.gmail.com?utm_medium=email&utm_source=footer__;JQ%21%21DZ3fjg%21tqEoVXiG_vuva1maxB6osKkXsYTEPCz5-luU7hJF8au2vpD00af-1_SofRFOTGKGImE$>.
--
Julian M Catchen, Ph.D.
Assistant Professor
Department of Evolution, Ecology, and Behavior
Carl R. Woese Institute for Genomic Biology
University of Illinois, Urbana-Champaign
--
jcat...@illinois.edu; @jcatchen
Short answer: yes.
Longer answer: Stacks has always allowed reads of different lengths,
such as caused by incorporating indels but we haven't prioritized
trimmed input reads -- when all raw input reads are variable lengths.
There were a few bugs affecting this in ustacks, but it should now work
properly.
Opinionated answer: it is not typically useful to trim your input reads
-- you will get a worse result than not trimming. Particularly since
people who trim often are starting with lower quality sequencing data
(which is why they are considering trimming in the first place). In RAD
data, trimming creates artificially low coverage in your loci at the 3'
end of the read, affecting SNP calling. Better to keep the reads intact
and let the SNP calling model handle small amounts of error. An
exception may be if you have lots of adaptor, which has to be removed.
Here the question is: is it better to have a region of the loci with low
coverage, or to just uniformly truncate all reads to a reasonable length
that gets rid of most of the adaptor (I would favor this latter
approach, which is how process_radtags is programmed).
julian
Giulia Trauzzi wrote on 7/20/21 5:05 PM:
> >
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>.
> >
> > > The command ustacks worked without any issue. However, cstacks
> > failed
> > > for an unknowing reason giving me the following output
> > >
> > > *denovo_map.pl
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21tqEoVXiG_vuva1maxB6osKkXsYTEPCz5-luU7hJF8au2vpD00af-1_SofRFOHOfrpo8$>
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>.
> >
> > > The command ustacks worked without any issue. However, cstacks
> > failed
> > > for an unknowing reason giving me the following output
> > >
> > > *denovo_map.pl
> >
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>:
> > Aborted because the last command failed (1).
> > > *
> > > *
> > > *
> > > *
> > > Thanks for support,
> > > Marco
> > >
> >
>
> --
> Stacks website: http://catchenlab.life.illinois.edu/stacks/
> ---
> You received this message because you are subscribed to the Google
> Groups "Stacks" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to stacks-users...@googlegroups.com
> <mailto:stacks-users%2Bunsu...@googlegroups.com>.
> <https://urldefense.com/v3/__http://denovo_map.pl__;%21%21DZ3fjg%21u-_gcFPNAYSp97SUCh8VaN7dUX7qh2ZRKvtukHjkosbN1f0U_XdhfSqB79dgDBO9TqI$>:
> > Aborted because the last command failed (1).
> > > *
> > > *
> > > *
> > > *
> > > Thanks for support,
> > > Marco
> > >
> >
>
> --
> Stacks website: http://catchenlab.life.illinois.edu/stacks/
> ---
> You received this message because you are subscribed to the Google
> Groups "Stacks" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to stacks-users...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/stacks-users/6088c5dd-2e6b-ece4-8d4e-306828cdfa42%40illinois.edu
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/stacks-users/6088c5dd-2e6b-ece4-8d4e-306828cdfa42*40illinois.edu__;JQ%21%21DZ3fjg%21tqEoVXiG_vuva1maxB6osKkXsYTEPCz5-luU7hJF8au2vpD00af-1_SofRFO512i8As$>.
> https://groups.google.com/d/msgid/stacks-users/6088c5dd-2e6b-ece4-8d4e-306828cdfa42%40illinois.edu
>
> --
> Stacks website: http://catchenlab.life.illinois.edu/stacks/
> ---
> You received this message because you are subscribed to the Google
> Groups "Stacks" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to stacks-users...@googlegroups.com
> <mailto:stacks-users...@googlegroups.com>.
> --
> Stacks website: http://catchenlab.life.illinois.edu/stacks/
> ---
> You received this message because you are subscribed to the Google
> Groups "Stacks" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to stacks-users...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/stacks-users/CALo9B-t-_k0y0rMu-nr_AMZT2SoiKjJFLwyU2weD3bNc65DHvQ%40mail.gmail.com
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/stacks-users/CALo9B-t-_k0y0rMu-nr_AMZT2SoiKjJFLwyU2weD3bNc65DHvQ*40mail.gmail.com?utm_medium=email&utm_source=footer__;JQ%21%21DZ3fjg%21tqEoVXiG_vuva1maxB6osKkXsYTEPCz5-luU7hJF8au2vpD00af-1_SofRFOTGKGImE$>.
--
Julian M Catchen, Ph.D.
Assistant Professor
Department of Evolution, Ecology, and Behavior
Carl R. Woese Institute for Genomic Biology
University of Illinois, Urbana-Champaign
--
jcat...@illinois.edu; @jcatchen
Reply all
Reply to author
Forward
0 new messages