Variant calling error

117 views

Skip to first unread message

Vanessa Guerra

unread,

Dec 7, 2017, 5:40:59 PM12/7/17

to trinityrnaseq-users

Dear all,

I'm having problems with the variant calling pipeline. I'm copying below the error that I'm getting. I used the same script that I used to run the test files that worked well (instead of copying the output of the test file to the post, I am attaching it as a document to this post to reduce any confusion).

To minimize problems due to the variability of sequences between my two biological groups, I'm using the data of just one individual (A7773) at a time (assembly also built from just one individual (~90,00 transcripts)).

I used paired and unpaired sequences to make my Trinity transcript. But for the variant calling step, I'm only using the files with paired sequences (no unpaired sequence). Could this be creating the problem?

Best,

Vanessa

===

command

===

WORKING_DIR=/home/vguerra/scratch/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5

if [[ ! -d $WORKING_DIR ]]; then mkdir -p $WORKING_DIR; fi

cd $WORKING_DIR

OUT_DIR="/home/vguerra/scratch/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5"

if [[ ! -d $OUT_DIR ]]; then mkdir -p $OUT_DIR; fi

/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/intel2016.4/trinity/2.5.0/trinityrnaseq-Trinity-v2.5.0/Analysis/SuperTranscripts/AllelicVariants/run_variant_calling.py \

--threads 32 \

--st_fa trinity_genes.fasta\

--st_gtf trinity_genes.gtf \

-p A37773_1_combined_first-line-edited_paired-clean.fq.gz A37773_2_combined_first-line-edited_paired-clean.fq.gz \

--maxram 69873294549 \

-o test_varcalling

===

slurm output

===

/var/spool/slurmd/job3170009/slurm_script: line 13: /home/vguerra/ENV_MiH5/bin/activate: No such file or directory

To execute picard run: java -jar $EBROOTPICARD/picard.jar

To execute GATK run: java -jar $EBROOTGATK/GenomeAnalysisTK.jar

Generating SuperTranscript index.

CMD: samtools faidx /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/trinity_genes.fasta already processed. Skipping.

Generating Picard dictionary.

Generating genome folder for STAR

CMD: mkdir star_genome_idx already processed. Skipping.

Running: STAR --runThreadN 32 --runMode genomeGenerate --genomeDir star_genome_idx --genomeFastaFiles /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/trinity_genes.fasta --sjdbGTFfile /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/trinity_genes.gtf --sjdbOverhang 150 --limitGenomeGenerateRAM 69873294549

Running: touch /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/test_varcalling/trinity_genes.fasta.gatk_chkpts/star_genome_generate.ok

Running STAR alignment.

Running: STAR --runThreadN 32 --genomeDir star_genome_idx --runMode alignReads --twopassMode Basic --alignSJDBoverhangMin 10 --outSAMtype BAM SortedByCoordinate --limitBAMsortRAM 69873294549 --readFilesIn /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/A37773_1_combined_first-line-edited_paired-clean.fq.gz /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/A37773_2_combined_first-line-edited_paired-clean.fq.gz --readFilesCommand gunzip -c

Running: touch /scratch/vguerra/analyses_Cryptasterina/SuperTranscripts/SuperTranscripts_MiH5/Individual_MiH5/test_varcalling/trinity_genes.fasta.gatk_chkpts/star_aln.ok

Cleaning and Converting sam file with Picard-Tools.

Running: java -jar /cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/picard/2.10.7/picard.jar AddOrReplaceReadGroups I=Aligned.sortedByCoord.out.bam O=rg_added_sorted.bam SO=coordinate RGID=id RGLB=library RGPL=platform RGPU=machine RGSM=sample

[Thu Dec 07 13:58:12 PST 2017] picard.sam.AddOrReplaceReadGroups INPUT=Aligned.sortedByCoord.out.bam OUTPUT=rg_added_sorted.bam SORT_ORDER=coordinate RGID=id RGLB=library RGPL=platform RGPU=machine RGSM=sample VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false

[Thu Dec 07 13:58:12 PST 2017] Executing as vgu...@cdr465.int.cedar.computecanada.ca on Linux 3.10.0-693.5.2.el7.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_121-b13; Deflater: Intel; Inflater: Intel; Picard version: 2.10.7-SNAPSHOT

INFO 2017-12-07 13:58:13 AddOrReplaceReadGroups Created read group ID=id PL=platform LB=library SM=sample

INFO 2017-12-07 13:58:22 AddOrReplaceReadGroups Processed 1,000,000 records. Elapsed time: 00:00:09s. Time for last 1,000,000: 9s. Last read position: TRINITY_DN24378_c4_g2:413

INFO 2017-12-07 13:58:29 AddOrReplaceReadGroups Processed 2,000,000 records. Elapsed time: 00:00:15s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN27638_c1_g1:15

INFO 2017-12-07 13:58:35 AddOrReplaceReadGroups Processed 3,000,000 records. Elapsed time: 00:00:22s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN25455_c0_g1:1,580

INFO 2017-12-07 13:58:42 AddOrReplaceReadGroups Processed 4,000,000 records. Elapsed time: 00:00:28s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN28030_c1_g3:159

INFO 2017-12-07 13:58:48 AddOrReplaceReadGroups Processed 5,000,000 records. Elapsed time: 00:00:34s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN23475_c6_g1:687

INFO 2017-12-07 13:58:54 AddOrReplaceReadGroups Processed 6,000,000 records. Elapsed time: 00:00:41s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN23453_c1_g2:1,705

INFO 2017-12-07 13:59:01 AddOrReplaceReadGroups Processed 7,000,000 records. Elapsed time: 00:00:47s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN24233_c5_g3:2,593

INFO 2017-12-07 13:59:07 AddOrReplaceReadGroups Processed 8,000,000 records. Elapsed time: 00:00:54s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN27537_c2_g1:8,157

INFO 2017-12-07 13:59:13 AddOrReplaceReadGroups Processed 9,000,000 records. Elapsed time: 00:01:00s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN25587_c4_g2:2,516

INFO 2017-12-07 13:59:20 AddOrReplaceReadGroups Processed 10,000,000 records. Elapsed time: 00:01:06s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN23815_c3_g1:1,250

INFO 2017-12-07 13:59:26 AddOrReplaceReadGroups Processed 11,000,000 records. Elapsed time: 00:01:12s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN24444_c0_g1:874

INFO 2017-12-07 13:59:32 AddOrReplaceReadGroups Processed 12,000,000 records. Elapsed time: 00:01:18s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN25596_c2_g1:2,397

INFO 2017-12-07 13:59:38 AddOrReplaceReadGroups Processed 13,000,000 records. Elapsed time: 00:01:25s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN28518_c0_g1:7,981

INFO 2017-12-07 13:59:45 AddOrReplaceReadGroups Processed 14,000,000 records. Elapsed time: 00:01:31s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN25242_c7_g5:214

INFO 2017-12-07 13:59:51 AddOrReplaceReadGroups Processed 15,000,000 records. Elapsed time: 00:01:38s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN29035_c6_g1:530

INFO 2017-12-07 13:59:58 AddOrReplaceReadGroups Processed 16,000,000 records. Elapsed time: 00:01:44s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN28834_c4_g1:352

INFO 2017-12-07 14:00:04 AddOrReplaceReadGroups Processed 17,000,000 records. Elapsed time: 00:01:51s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN25894_c2_g2:673

INFO 2017-12-07 14:00:11 AddOrReplaceReadGroups Processed 18,000,000 records. Elapsed time: 00:01:57s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN28789_c4_g2:324

INFO 2017-12-07 14:00:17 AddOrReplaceReadGroups Processed 19,000,000 records. Elapsed time: 00:02:03s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN28509_c3_g1:6,552

INFO 2017-12-07 14:00:23 AddOrReplaceReadGroups Processed 20,000,000 records. Elapsed time: 00:02:10s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN24477_c11_g1:1,624

INFO 2017-12-07 14:00:30 AddOrReplaceReadGroups Processed 21,000,000 records. Elapsed time: 00:02:16s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN27264_c4_g1:475

INFO 2017-12-07 14:00:36 AddOrReplaceReadGroups Processed 22,000,000 records. Elapsed time: 00:02:22s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN27220_c2_g2:1,000

INFO 2017-12-07 14:00:42 AddOrReplaceReadGroups Processed 23,000,000 records. Elapsed time: 00:02:29s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN22929_c0_g1:1,602

INFO 2017-12-07 14:00:49 AddOrReplaceReadGroups Processed 24,000,000 records. Elapsed time: 00:02:35s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN26734_c3_g2:2,810

INFO 2017-12-07 14:00:55 AddOrReplaceReadGroups Processed 25,000,000 records. Elapsed time: 00:02:41s. Time for last 1,000,000: 6s. Last read position: TRINITY_DN22909_c0_g1:825

[Thu Dec 07 14:00:58 PST 2017] picard.sam.AddOrReplaceReadGroups done. Elapsed time: 2.77 minutes.

Runtime.totalMemory()=9236381696

To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp

Exception in thread "main" htsjdk.samtools.SAMFormatException: SAM validation error: ERROR: Read name HS28_47:1:1114:2015:7790, Read CIGAR M operator maps off end of reference

at htsjdk.samtools.SAMUtils.processValidationErrors(SAMUtils.java:451)

at htsjdk.samtools.BAMRecord.getCigar(BAMRecord.java:253)

at htsjdk.samtools.SAMRecord.getAlignmentEnd(SAMRecord.java:603)

at htsjdk.samtools.SAMRecord.computeIndexingBin(SAMRecord.java:1547)

at htsjdk.samtools.SAMRecord.isValid(SAMRecord.java:2054)

at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:811)

at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:797)

at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:765)

at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:576)

at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:548)

at picard.sam.AddOrReplaceReadGroups.doWork(AddOrReplaceReadGroups.java:140)

at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:228)

at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:94)

at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:104)

Traceback (most recent call last):

File "/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/intel2016.4/trinity/2.5.0/trinityrnaseq-Trinity-v2.5.0/Analysis/SuperTranscripts/AllelicVariants/run_variant_calling.py", line 212, in <module>

main()

pipeliner.run()

run_cmd(cmd.get_cmd())

raise RuntimeError("Error while running command \"" + str(cmd) + "\":\n" + error)

RuntimeError: Error while running command "['java', '-jar', '/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/picard/2.10.7/picard.jar', 'AddOrReplaceReadGroups', 'I=Aligned.sortedByCoord.out.bam', 'O=rg_added_sorted.bam', 'SO=coordinate', 'RGID=id', 'RGLB=library', 'RGPL=platform', 'RGPU=machine', 'RGSM=sample']":