elprep sfm and filter runs die
17 views
Skip to first unread message
Stephane Plaisance
Aug 29, 2022, 4:59:00 AM8/29/22
to elprep
Dear,
I am trying elprep in both sfm and filter modes and both die with little info on what went wrong but apparently related to GO.
Thanks for your help
Stephane
* running on ubuntu 20.4 server with 88 threads (using 40) and 512 GB RAM
* elprep version 5.1.3 compiled with go1.17.6
* running on ubuntu 20.4 server with 88 threads (using 40) and 512 GB RAM
* elprep version 5.1.3 compiled with go1.17.6
The command used was:
elprep filter bwa_mappings/524_rawmappings.bam elprep_results/elprep/524_elprep_filter_mappings.bam
elprep filter bwa_mappings/524_rawmappings.bam elprep_results/elprep/524_elprep_filter_mappings.bam
--filter-unmapped-reads
--optical-duplicates-pixel-distance 2500
--mark-duplicates
--mark-optical-duplicates
elprep_results/elprep/524.filter.output.metrics
--sorting-order coordinate
--bqsr
elprep_results/elprep/524.filter_bqsr.output.recal
elprep_results/elprep/524.filter_bqsr.output.recal
--quantize-levels 16
--known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites
--reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta
--haplotypecaller elprep_results/elprep/524.filter.g.vcf.gz
--reference-confidence GVCF
--nr-of-threads 40
--timed
--log-path elprep_results/elprep
Below is the log of the filter aborted run.
Below is the log of the filter aborted run.
```
cat elprep-2022-08-29-10-13-30-780711439-CEST.log
cat elprep-2022-08-29-10-13-30-780711439-CEST.log
elprep version 5.1.3 compiled with go1.17.6 - see http://github.com/exascience/elprep for more information.
2022/08/29 10:13:30 Created log file at elprep_results/elprep/logs/elprep/elprep-2022-08-29-10-13-30-780711439-CEST.log
2022/08/29 10:13:30 Command line: [elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --optical-duplicates-pixel-distance 2500 --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --sorting-order coordinate --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --quantize-levels 16 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --reference-confidence GVCF --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles]
2022/08/29 10:13:30 Executing command:
elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --optical-duplicates-pixel-distance 2500 --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --quantize-levels 16 --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles --intermediate-files-output-prefix 524_rawmappings --intermediate-files-output-type bam
2022/08/29 10:13:30 Splitting...
elprep version 5.1.3 compiled with go1.17.6 - see http://github.com/exascience/elprep for more information.
2022/08/29 10:13:30 Created log file at elprep_results/elprep/logs/elprep/elprep-2022-08-29-10-13-30-786746552-CEST.log
2022/08/29 10:13:30 Command line: [elprep split elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/ --nr-of-threads 40 --timed --log-path elprep_results/elprep --output-prefix 524_rawmappings --output-type bam]
2022/08/29 10:13:30 Executing command:
elprep split elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/ --output-prefix 524_rawmappings --output-type bam --nr-of-threads 40 --timed --log-path elprep_results/elprep
2022/08/29 10:13:30 Splitting paired-end files.
2022/08/29 10:16:00 Elapsed time: 2m30.122458572s
2022/08/29 10:16:00 Filtering (phase 1)...
elprep version 5.1.3 compiled with go1.17.6 - see http://github.com/exascience/elprep for more information.
2022/08/29 10:16:00 Created log file at elprep_results/elprep/logs/elprep/elprep-2022-08-29-10-16-00-943482612-CEST.log
2022/08/29 10:16:00 Command line: [elprep filter elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/splits/524_rawmappings-group00005.bam elprep_results/tmpfiles/elprep-splits-processed-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --filter-unmapped-reads --mark-duplicates --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --pg-cmd-line elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --optical-duplicates-pixel-distance 2500 --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --quantize-levels 16 --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles --intermediate-files-output-prefix 524_rawmappings --intermediate-files-output-type bam --bqsr-tables-only elprep_results/tmpfiles/elprep-tabs-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam.elrecal --mark-optical-duplicates-intermediate elprep_results/tmpfiles/elprep-metrics-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --optical-duplicates-pixel-distance 2500]
2022/08/29 10:16:00 Executing command:
elprep filter elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/splits/524_rawmappings-group00005.bam elprep_results/tmpfiles/elprep-splits-processed-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates-intermediate elprep_results/tmpfiles/elprep-metrics-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --optical-duplicates-pixel-distance 2500 --bqsr-tables-only elprep_results/tmpfiles/elprep-tabs-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam.elrecal --max-cycle 500 --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --pg-cmd-line elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --optical-duplicates-pixel-distance 2500 --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --quantize-levels 16 --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles --intermediate-files-output-prefix 524_rawmappings --intermediate-files-output-type bam
2022/08/29 10:16:00 Reading SAM into memory and applying filters.
2022/08/29 10:16:46 Elapsed time: 45.684517013s
2022/08/29 10:16:58 Marking optical duplicates.
2022/08/29 10:17:00 Elapsed time: 2.264542147s
2022/08/29 10:17:00 Base recalibration
2022/08/29 10:17:43 Elapsed time: 43.099216583s
2022/08/29 10:17:48 exit status 2
panic: exit status 2
goroutine 1 [running]:
log.Panic({0xc000215060, 0x1c, 0x1b})
/opt/conda/conda-bld/elprep_1651164620400/_build_env/go/src/log/log.go:354 +0x65
github.com/exascience/elprep/v5/internal.RunCmd(0x7ffed986cf6d)
/opt/conda/conda-bld/elprep_1651164620400/work/internal/misc.go:41 +0x46
github.com/exascience/elprep/v5/cmd.Sfm()
/opt/conda/conda-bld/elprep_1651164620400/work/cmd/sfm.go:682 +0xb985
main.main()
/opt/conda/conda-bld/elprep_1651164620400/work/main.go:81 +0x18b
```
2022/08/29 10:13:30 Created log file at elprep_results/elprep/logs/elprep/elprep-2022-08-29-10-13-30-780711439-CEST.log
2022/08/29 10:13:30 Command line: [elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --optical-duplicates-pixel-distance 2500 --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --sorting-order coordinate --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --quantize-levels 16 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --reference-confidence GVCF --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles]
2022/08/29 10:13:30 Executing command:
elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --optical-duplicates-pixel-distance 2500 --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --quantize-levels 16 --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles --intermediate-files-output-prefix 524_rawmappings --intermediate-files-output-type bam
2022/08/29 10:13:30 Splitting...
elprep version 5.1.3 compiled with go1.17.6 - see http://github.com/exascience/elprep for more information.
2022/08/29 10:13:30 Created log file at elprep_results/elprep/logs/elprep/elprep-2022-08-29-10-13-30-786746552-CEST.log
2022/08/29 10:13:30 Command line: [elprep split elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/ --nr-of-threads 40 --timed --log-path elprep_results/elprep --output-prefix 524_rawmappings --output-type bam]
2022/08/29 10:13:30 Executing command:
elprep split elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/ --output-prefix 524_rawmappings --output-type bam --nr-of-threads 40 --timed --log-path elprep_results/elprep
2022/08/29 10:13:30 Splitting paired-end files.
2022/08/29 10:16:00 Elapsed time: 2m30.122458572s
2022/08/29 10:16:00 Filtering (phase 1)...
elprep version 5.1.3 compiled with go1.17.6 - see http://github.com/exascience/elprep for more information.
2022/08/29 10:16:00 Created log file at elprep_results/elprep/logs/elprep/elprep-2022-08-29-10-16-00-943482612-CEST.log
2022/08/29 10:16:00 Command line: [elprep filter elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/splits/524_rawmappings-group00005.bam elprep_results/tmpfiles/elprep-splits-processed-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --filter-unmapped-reads --mark-duplicates --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --pg-cmd-line elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --optical-duplicates-pixel-distance 2500 --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --quantize-levels 16 --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles --intermediate-files-output-prefix 524_rawmappings --intermediate-files-output-type bam --bqsr-tables-only elprep_results/tmpfiles/elprep-tabs-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam.elrecal --mark-optical-duplicates-intermediate elprep_results/tmpfiles/elprep-metrics-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --optical-duplicates-pixel-distance 2500]
2022/08/29 10:16:00 Executing command:
elprep filter elprep_results/tmpfiles/elprep-splits-ad5922de-0c26-4e77-98e1-60d9727ad888/splits/524_rawmappings-group00005.bam elprep_results/tmpfiles/elprep-splits-processed-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates-intermediate elprep_results/tmpfiles/elprep-metrics-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam --optical-duplicates-pixel-distance 2500 --bqsr-tables-only elprep_results/tmpfiles/elprep-tabs-ad5922de-0c26-4e77-98e1-60d9727ad888/524_rawmappings-group00005.bam.elrecal --max-cycle 500 --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --pg-cmd-line elprep sfm elprep_results/../bwa_mappings/524_rawmappings.bam elprep_results/elprep/_elprep_sfm_mappings.bam --filter-unmapped-reads --mark-duplicates --mark-optical-duplicates elprep_results/elprep/524.sfm.output.metrics --optical-duplicates-pixel-distance 2500 --bqsr elprep_results/elprep/524.sfm_bqsr.output.recal --reference /data/biodata/references/galGal7b/Gallus_gallus.bGalGal1.mat.broiler.GRCg7b.dna.toplevel.elfasta --quantize-levels 16 --max-cycle 500 --known-sites /data/biodata/references/galGal7b/ens_galGal7b.elsites --haplotypecaller elprep_results/elprep/524.sfm.g.vcf.gz --sorting-order coordinate --nr-of-threads 40 --timed --log-path elprep_results/elprep --tmp-path elprep_results/tmpfiles --intermediate-files-output-prefix 524_rawmappings --intermediate-files-output-type bam
2022/08/29 10:16:00 Reading SAM into memory and applying filters.
2022/08/29 10:16:46 Elapsed time: 45.684517013s
2022/08/29 10:16:58 Marking optical duplicates.
2022/08/29 10:17:00 Elapsed time: 2.264542147s
2022/08/29 10:17:00 Base recalibration
2022/08/29 10:17:43 Elapsed time: 43.099216583s
2022/08/29 10:17:48 exit status 2
panic: exit status 2
goroutine 1 [running]:
log.Panic({0xc000215060, 0x1c, 0x1b})
/opt/conda/conda-bld/elprep_1651164620400/_build_env/go/src/log/log.go:354 +0x65
github.com/exascience/elprep/v5/internal.RunCmd(0x7ffed986cf6d)
/opt/conda/conda-bld/elprep_1651164620400/work/internal/misc.go:41 +0x46
github.com/exascience/elprep/v5/cmd.Sfm()
/opt/conda/conda-bld/elprep_1651164620400/work/cmd/sfm.go:682 +0xb985
main.main()
/opt/conda/conda-bld/elprep_1651164620400/work/main.go:81 +0x18b
```
Stephane Plaisance
Aug 29, 2022, 5:26:48 AM8/29/22
to elprep
the github states elprep does not need any dependencies
do I need to install go on my server?
Stephane Plaisance
Aug 29, 2022, 6:52:24 AM8/29/22
to elprep
Stupid me, I used mappings against a previous reference.
Never mind!
@Admin You may delete this irrelevant track!
Reply all
Reply to author
Forward
0 new messages