VARiD fails with error message: assertion: (ret.data != NULL) failed, Last errno=12
carlo.lucchesi
I am currently using VARid on an alignement issued from SOLiD color
space data. Every line of the SAM file contains CIGAR code strictly
equal to 50M and the Hg18 reference fasta file contains tags for
chromosomes 1 to 22,X,Y,M.
We are on a 64bits Linux server.
After launch, the software reports:
- alignments: /bioinfo/projects/data/Human/mRNA_expression/
Sequencing/ABI/SolidV3Plus/NGS_Curie/EWING/ANALYSES/
P5_T3_F3.rmdupS.genomic.mapped.sam
- ref-file: /bioinfo/projects/data/Human/mRNA_expression/
Sequencing/ABI/SolidV3Plus/NGS_Curie/EWING/ANALYSES/IN/
hg18.sorted.reduced.fa
Detected:
chr1:247249719 nt
chr2:242951149 nt
chr3:199501827 nt
chr4:191273063 nt
chr5:180857866 nt
chr6:170899992 nt
chr7:158821424 nt
chr8:146274826 nt
chr9:140273252 nt
chr10:135374737 nt
chr11:134452384 nt
chr12:132349534 nt
chr13:114142980 nt
chr14:106368585 nt
chr15:100338915 nt
chr16:88827254 nt
chr17:78774742 nt
chr18:76117153 nt
chr19:63811651 nt
chr20:62435964 nt
chr21:46944323 nt
chr22:49691432 nt
chrX:154913754 nt
chrY:57772954 nt
chrM:16571 nt
Splitting reads
Total reads: 57208847
Used reads: 57208847
Done: in: 2423.943 secs
A quick look at the temporary folder allow to confirm that the aligned
reads have been split in 25 groups, each group corresponding to a file
holding the reads for the corresponding chromosome. The reports
continues as follows:
Processing group number 1
Loading reads ...
Done in 53.582 secs
Aligning reads ...
Done in: 1.033 secs
Starting VARiD Algorithm
Building Transition Matrix...
Done in: 0.026 secs
0 135
0 207
1 186
0 378
0 557
3 588
2 591
2 602
.
.
.
.
.
3 247115875
1 247115880
3 247115881
3 24711589 covmaps built in: 177.266 secs, Used reads in the
covmaps: 4123467/4123467, Skipped for 0 index:0
Computing emissions...
2
0 247115901
1 247115915
.
.
.
2 247199593
0 247199599
3 247199626
2 247199652
0 247199653
2 247199668
<va_utils.c:402> assertion: (ret.data != NULL) failed, Last errno=12
<run_wrap.c:242> varid exiting...
<run_wrap.c:245> varid exited
and the software stops there. The last line reports the starting
position (+1) of the last read aligned on chromosome 1. The algorithm
apparently found something abnormal while moving to chromosome 2.
Has anybody an idea of why this happens?
Thanks for your help
Carlo Lucchesi
UNIT 830 INSERM "Biology and Genetics of Cancers"
Curie Institute - Research Center
26 rue d'Ulm
75248 PARIS CEDEX 05
FRANCE
Tél : 01 56 24 65 17
Fax : 01 56 24 66 30
adijr
Thank you for the message.
First off, apologies for the extra printed numbers - they are a debug
print leftover. :(
We're fairly confident that this error has to do with the fact that
you are using whole chromosomes of the genome and varid will run out
of memory in most cases.
In both the current version and the new version of VARiD we are about
to release in the next couple of days, we provide some scripts
(varidhelper.sh) to automatically chop up the reference into smaller
regions,
good for memory usage and parallelization.
Can you give a look at these scripts and try to run it with smaller
regions?
Thank you,
Adrian
Carlo Lucchesi
I've run the varidhelper.sh script on the same sam file and reference fasta.
This time the program has come to completion but deceivably it produces the
results only for one chromosome, the chromosome M. Pretty weird! After a
quick look at the lines in varidhelper.sh, I have spotted something unusual
in the sorting line:
sort -t$'\t' -k 4n ${ALIGNMENTS_FILE} --temporary-directory=/mnt/disk2/tmp >
${TEMP_DIR}/`basename ${ALIGNMENTS_FILE}`.tmp
This means that you sort the original sam file by the genomic position
whichever the chromosome. Chromosome M has the first genomic positions
amongst all other chromosomes, that why only chrM is outpout. Is this what
it is expected to do?
So, in conclusion, we can't use Annovar at the moment. Are we the only one
laboratory suffering this problem?
Following an extract of the error log (please remove the trace in the next
version!)
Cheers
Carlo
===================================================
running sort -t$'\t' -k 4n
/mnt/disk4/WORK_SPACE/DEV/SAMPLES/Hs/EWING_A673/P5_T3_F3/OUTPUT_HG18_2010062
1_classic/P5_T3_F3.rmdupS.sam > /mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.t
mp
running python /mnt/disk4/TOOLS/NGS/varid-1.0.1c/helper/splitreads_helper.py
--sam --alignments /mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp
--output-prefix /mnt/disk2/tmp/VARID
/P5_T3_F3.rmdupS.sam.tmp --split-size 10000 --overlap-size 100
running python /mnt/disk4/TOOLS/NGS/varid-1.0.1c/helper/splitgenome.py
--input
/mnt/disk4/WORK_SPACE/DEV/ANNOTATIONS/Hs/BUILT_18/hg18.sorted.reduced.fa
--output-prefix /mnt/disk
2/tmp/VARID/hg18.sorted.reduced.fa.tmp --split-size 10000 --overlap-size 100
running /mnt/disk4/TOOLS/NGS/varid-1.0.1c/varid --sam
--alignments=/mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.0
--ref-file=/mnt/disk2/tmp/VARID/hg18.sorted.reduced.fa.tmp.0 --
donor-name=EWING_A673 --temp-dir=/mnt/disk2/tmp/VARID
--output=/mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.0
- alignments: /mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.0
- ref-file: /mnt/disk2/tmp/VARID/hg18.sorted.reduced.fa.tmp.0
Detected:
chrM_0:10050 nt
Splitting reads
Total reads: 19817
Used reads: 11545
Done: in: 0.201 secs
------------------------------
Processing group number 1
Loading reads ...
Done in 0.105 secs
Aligning reads ...
Done in: 0.000 secs
Starting VARiD Algorithm
Building Transition Matrix...
Done in: 0.018 secs
3 51
2 1
3 52
0 3
.
.
.
0 10044
1 10045
0 10046
3 10047
3 10048
1 10049
<run_wrap.c:242> varid exiting...
<run_wrap.c:245> varid exited
running /mnt/disk4/TOOLS/NGS/varid-1.0.1c/varid --sam
--alignments=/mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.1
--ref-file=/mnt/disk2/tmp/VARID/hg18.sorted.reduced.fa.tmp.1 --
donor-name=EWING_A673 --temp-dir=/mnt/disk2/tmp/VARID
--output=/mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.1
- alignments: /mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.1
- ref-file: /mnt/disk2/tmp/VARID/hg18.sorted.reduced.fa.tmp.1
Detected:
chrM_1:6621 nt
Splitting reads
Total reads: 21284
Used reads: 10286
Done: in: 0.210 secs
------------------------------
Processing group number 1
Loading reads ...
Done in 0.095 secs
Aligning reads ...
Done in: 0.000 secs
Starting VARiD Algorithm
Building Transition Matrix...
Done in: 0.016 secs
2 52
3 53
0 54
0 56
.
.
.
2 6570
0 6620
3 6621
2 6622
<run_wrap.c:242> varid exiting...
<run_wrap.c:245> varid exited
running python /mnt/disk4/TOOLS/NGS/varid-1.0.1c/helper/mergeresults.py
--input-prefix /mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp --output
/mnt/disk4/WORK_SPACE/DEV/PROJECTS/E
WING_A673_PROTOCOL_COMPARISON_classic/REPORT/VARID_GENETIC_VARIANTS/P5_T3_F3
.rmdupS.sam --contig-count 2 --split-size 10000 --overlap-size 100
Infile, <open file '/mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.0', mode
'r' at 0x2aaaaab42dc8>
Infile, <open file '/mnt/disk2/tmp/VARID/P5_T3_F3.rmdupS.sam.tmp.1', mode
'r' at 0x2aaaaab42e40>
adijr
I apologize, we hadn't seen this problem before. Is it possible for
you to just split up the multifasta file into multiple fasta files and
run varidhelper with each?
You can keep the alignment file containing all contigs!
We have released a new version of VARiD (1.0.7), much improved, which
you can find on the website, however if you want to use the (new)
varidhelper it will still require one fasta file at a time. We will
make it a priority to allow mfa files with varidhelper.
I hope this helps,
Adrian
Carlo Lucchesi
I prefer to convey energies to the version 1.0.7. I've tried to install it
but I receive a error message at compile time.
./VARID/varid_backward.c: In function varid_backward:
./VARID/varid_backward.c:123: internal compiler error: in gimplify_expr, at
gimplify.c:5836
Following you can find the make summary report.
What is the likely cause of this error at compile time?
Thanks, Carlo Lucchesi
=========================================================================
[root@u830-tro-6-63-6517 varid-1.0.7c]# ./configure --prefix=`pwd`
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for exp in -lm... yes
checking for clock_gettime in -lrt... yes
checking how to run the C preprocessor... gcc -E
checking for egrep... grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/timeb.h usability... yes
checking sys/timeb.h presence... yes
checking for sys/timeb.h... yes
checking syslog.h usability... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking for unistd.h... (cached) yes
checking for an ANSI C-conforming const... yes
checking for size_t... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for working memcmp... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... yes
checking whether lstat dereferences a symlink specified with a trailing
slash... yes
checking whether stat accepts an empty string... no
checking for atexit... yes
checking for clock_gettime... yes
checking for memset... yes
checking for sqrt... yes
checking for strchr... yes
checking for strdup... yes
checking for strrchr... yes
checking for OpenMP flag of C compiler... -fopenmp
configure: creating ./config.status
config.status: creating Makefile
config.status: creating helper/Makefile
config.status: creating m4/Makefile
config.status: creating config.h
config.status: executing depfiles commands
[root@u830-tro-6-63-6517 varid-1.0.7c]# make
make all-recursive
make[1]: Entering directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c'
Making all in helper
make[2]: Entering directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c/helper'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c/helper'
Making all in m4
make[2]: Entering directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c/m4'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c/m4'
make[2]: Entering directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c'
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
run_wrap.o -MD -MP -MF ".deps/run_wrap.Tpo" -c -o run_wrap.o run_wrap.c; \
then mv -f ".deps/run_wrap.Tpo" ".deps/run_wrap.Po"; else rm -f
".deps/run_wrap.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
varid_single.o -MD -MP -MF ".deps/varid_single.Tpo" -c -o varid_single.o
varid_single.c; \
then mv -f ".deps/varid_single.Tpo" ".deps/varid_single.Po"; else rm
-f ".deps/varid_single.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
get_pred.o -MD -MP -MF ".deps/get_pred.Tpo" -c -o get_pred.o `test -f
'./VARID/get_pred.c' || echo './
then mv -f ".deps/get_pred.Tpo" ".deps/get_pred.Po"; else rm -f
".deps/get_pred.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
predict.o -MD -MP -MF ".deps/predict.Tpo" -c -o predict.o `test -f
'./VARID/predict.c' || echo './'`./
then mv -f ".deps/predict.Tpo" ".deps/predict.Po"; else rm -f
".deps/predict.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
varid.o -MD -MP -MF ".deps/varid.Tpo" -c -o varid.o `test -f
'./VARID/varid.c' || echo './'`./VARID/va
then mv -f ".deps/varid.Tpo" ".deps/varid.Po"; else rm -f
".deps/varid.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
varid_emissions.o -MD -MP -MF ".deps/varid_emissions.Tpo" -c -o
varid_emissions.o `test -f './VARID/va
then mv -f ".deps/varid_emissions.Tpo" ".deps/varid_emissions.Po";
else rm -f ".deps/varid_emissions.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
get_transitions.o -MD -MP -MF ".deps/get_transitions.Tpo" -c -o
get_transitions.o `test -f './VARID/ge
then mv -f ".deps/get_transitions.Tpo" ".deps/get_transitions.Po";
else rm -f ".deps/get_transitions.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -Wall -g -O2 -fopenmp -MT
varid_backward.o -MD -MP -MF ".deps/varid_backward.Tpo" -c -o
varid_backward.o `test -f './VARID/varid
then mv -f ".deps/varid_backward.Tpo" ".deps/varid_backward.Po";
else rm -f ".deps/varid_backward.Tpo"; exit 1; fi
./VARID/varid_backward.c: In function âvarid_backwardâ:
./VARID/varid_backward.c:123: internal compiler error: in gimplify_expr, at
gimplify.c:5836
Please submit a full bug report,
with preprocessed source if appropriate.
See <URL:http://bugzilla.redhat.com/bugzilla> for instructions.
Preprocessed source stored into /tmp/ccM0ZCqn.out file, please attach this
to your bugreport.
make[2]: *** [varid_backward.o] Error 1
make[2]: Leaving directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/mnt/disk4/TOOLS/NGS/varid-1.0.7c'
make: *** [all] Error 2
Misko Dzamba
I am not sure what to make of this. The code in this area does not
seem to be strange in anyway, other then that it is in a OpenMP
parallelized region. GCC should support OpenMP since version 4.2 (May
2007, http://gcc.gnu.org/gcc-4.2/). It doesn't seem like it directly
complains about the OpenMP or varid code, and I do not know what could
be causing this. Can you please let us know what kind of system this
is ? (OS, RAM, compiler name and version, 64bit vs 32bit), this might
help us find the source of the error. Have you tried compiling it on a
different machine or different partition? Maybe its the filesystem
acting up? In the meantime I have compiled two linux binaries with
statically linked libraries and made the available at,
http://compbio.cs.utoronto.ca/varid/releases/varid_1_07_c_32.bit.gz
and
http://compbio.cs.utoronto.ca/varid/releases/varid_1_07_c_64.bit.gz
These binaries should hopefully work on your system, I would recommend
using the 64bit version if possible.
Thank you for the feedback! Please let us know if you have any more troubles,
Misko
> -----Original Message-----
> From: varid-c...@googlegroups.com
> [mailto:varid-c...@googlegroups.com] On Behalf Of adijr
> Sent: mercredi 6 octobre 2010 05:34