Linkage Disequilibrium never completing

[Mary LaPorte]

Hello! I am having an issue where Linkage Disequilibrium is never running past 10% and I'm not sure how to move forward.

The hapmap file is 53.84 GB and contains 92,790,713 sites after filtering. It is running on 94 cores with 5GB/core (470GB). I let it run for about 24 hours before terminating the run (I can schedule it for 96 hours max). Is there a more efficient way to run LD in TASSEL? For context, GWAS using this hapmap worked like a charm. 

Some other options I have considered: 

• Selecting every 100th marker and running LD on this (not sure if it makes sense/is rigorous, not sure how to do it yet)
• running it on a 32GB core (would wait a while in the queue, and not sure it would help)

Could you advise on what to do in this situation, and what may be causing the "hanging" at 10%? Thank you very much for your help! 

TASSEL output:

$ run_pipeline.pl -Xmx460g -debug -h /xdisk/group/user/temp.hmp.txt -ld -export /xdisk/group/user/ld_out.txt

/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/trove-3.0.3.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jackson-annotations-2.13.2.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/phg.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/error_prone_annotations-2.19.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/biojava-genome-6.0.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/kotlin-stdlib-1.6.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/ahocorasick-0.2.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jhdf5-14.12.5.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/protobuf-java-util-3.23.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/gs-core-1.3.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/slf4j-simple-1.7.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/json-simple-1.1.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/sshj-0.32.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/biojava-alignment-6.0.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jackson-core-2.13.2.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/commons-codec-1.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jfreesvg-3.2.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/forester-1.039.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/kotlin-stdlib-jdk8-1.6.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jackson-module-kotlin-2.13.2.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/postgresql-42.6.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/protobuf-kotlin-3.23.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jfreechart-1.0.19.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/javax.json-1.0.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/commons-math3-3.4.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/colt-1.2.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/log4j-core-2.21.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/guava-22.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/protobuf-java-3.23.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jackson-databind-2.13.2.2.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/slf4j-api-1.7.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/gs-ui-1.3.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/jcommon-1.0.23.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/ini4j-0.5.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/kotlinx-coroutines-core-jvm-1.6.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/junit-4.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/snappy-java-1.1.8.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/sqlite-jdbc-3.39.2.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/log4j-api-2.21.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/ejml-core-0.41.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/ejml-ddense-0.41.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/scala-library-2.10.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/kotlin-reflect-1.6.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/kotlin-stdlib-jdk7-1.6.10.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/fastutil-8.2.2.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/htsjdk-2.24.1.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/itextpdf-5.1.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/mail-1.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/biojava-core-6.0.4.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/commons-io-2.11.0.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/lib/biojava-phylo-4.2.12.jar:/opt/ohpc/pub/apps/tassel/5.2.96/bin/sTASSEL.jar
Memory Settings: -Xms512m -Xmx460g
Tassel Pipeline Arguments: -debug -h /xdisk/group/user/temp.hmp.txt -ld -export /xdisk/group/user/ld_out.txt
[main] INFO net.maizegenetics.tassel.TasselLogging - Tassel Version: 5.2.96  Date: March 31, 2025
[main] INFO net.maizegenetics.tassel.TasselLogging - Max Available Memory Reported by JVM: 418702 MB
[main] INFO net.maizegenetics.tassel.TasselLogging - Java Version: 1.8.0_452
[main] INFO net.maizegenetics.tassel.TasselLogging - OS: Linux
[main] INFO net.maizegenetics.tassel.TasselLogging - Number of Processors: 94
[main] INFO net.maizegenetics.tassel.TasselLogging - Tassel Citation: Bradbury PJ, Zhang Z, Kroon DE, Casstevens TM, Ramdoss Y, Buckler ES. (2007) TASSEL: Software for association mapping of complex traits in diverse samples. Bioinformatics 23:2633-2635.
[main] INFO net.maizegenetics.tassel.TasselLogging -
[main] INFO net.maizegenetics.tassel.TasselLogging - Tassel Using Library: Practical Haplotype Graph (PHG): Version: 1.10 Date: August 20, 2024
[main] INFO net.maizegenetics.tassel.TasselLogging - PHG Citation: Bradbury PJ, Casstevens T, Jensen SE, Johnson LC, Miller ZR, Monier B, Romay MC, Song B, Buckler ES. The Practical Haplotype Graph, a platform for storing and using pangenomes for imputation. Bioinformatics. 2022 Aug 2;38(15):3698-3702. doi: 10.1093/bioinformatics/btac410. PMID: 35748708; PMCID: PMC9344836.
[main] INFO net.maizegenetics.pipeline.TasselPipeline - Tassel Pipeline Arguments: [-fork1, -h, /xdisk/group/user/temp.hmp.txt, -ld, -export, /xdisk/group/user/ld_out.txt, -runfork1]
net.maizegenetics.analysis.data.FileLoadPlugin
   net.maizegenetics.analysis.popgen.LinkageDisequilibriumPlugin
      net.maizegenetics.analysis.data.ExportMultiplePlugin
[pool-2-thread-1] INFO net.maizegenetics.plugindef.AbstractPlugin - Starting net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:51:31
[pool-2-thread-1] INFO net.maizegenetics.plugindef.AbstractPlugin -
FileLoadPlugin Parameters
format: Hapmap
sortPositions: false
keepDepth: false

[pool-2-thread-1] INFO net.maizegenetics.analysis.data.FileLoadPlugin - Start Loading File: /xdisk/group/user/temp.hmp.txt time: Jul 21, 2025 10:51:31
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:26: progress: 0%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:28: progress: 10%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:30: progress: 20%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:32: progress: 30%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:34: progress: 40%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:36: progress: 50%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:38: progress: 60%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:40: progress: 70%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:42: progress: 80%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:54:44: progress: 90%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 10:55:1: progress: 100%
[pool-2-thread-1] INFO net.maizegenetics.plugindef.AbstractPlugin - net.maizegenetics.analysis.data.FileLoadPlugin  Citation: Bradbury PJ, Zhang Z, Kroon DE, Casstevens TM, Ramdoss Y, Buckler ES. (2007) TASSEL: Software for association mapping of complex traits in diverse samples. Bioinformatics 23:2633-2635.
[pool-2-thread-1] INFO net.maizegenetics.analysis.data.FileLoadPlugin - Finished Loading File: /xdisk/group/user/temp.hmp.txt time: Jul 21, 2025 11:00:28
Genotype Table Name: temp
Number of Taxa: 256
Number of Sites: 92790713
Sites x Taxa: 23754422528
Chromosomes...
NC_007578.1: start site: 0 (1704) last site: 151 (148393) total: 152
NC_042756.1: start site: 152 (35830) last site: 1649 (362549) total: 1498
NC_056623.2: start site: 1650 (2858) last site: 10307871 (252569385) total: 10306222
NC_056624.2: start site: 10307872 (8416) last site: 20067427 (236377555) total: 9759556
NC_056625.2: start site: 20067428 (11353) last site: 31351525 (324655322) total: 11284098
NC_056626.2: start site: 31351526 (21723) last site: 46667277 (410279543) total: 15315752
NC_056627.2: start site: 46667278 (5151) last site: 59783373 (371825205) total: 13116096
NC_056628.2: start site: 59783374 (10423) last site: 66553094 (206649905) total: 6769721
NC_056629.2: start site: 66553095 (10117) last site: 73196181 (209884597) total: 6643087
NC_056630.2: start site: 73196182 (7970) last site: 85215915 (343479930) total: 12019734
NC_056631.2: start site: 85215916 (13513) last site: 92786607 (229402556) total: 7570692
NW_026440193.1: start site: 92786608 (15921) last site: 92786764 (635921) total: 157
NW_026440194.1: start site: 92786765 (3489) last site: 92786959 (389962) total: 195
NW_026440195.1: start site: 92786960 (19840) last site: 92787014 (359666) total: 55
NW_026440196.1: start site: 92787015 (19839) last site: 92787094 (108454) total: 80
NW_026440197.1: start site: 92787095 (320) last site: 92787443 (146153) total: 349
NW_026440198.1: start site: 92787444 (60655) last site: 92787458 (96137) total: 15
NW_026440199.1: start site: 92787459 (643) last site: 92788589 (140840) total: 1131
NW_026440200.1: start site: 92788590 (46656) last site: 92788603 (133629) total: 14
NW_026440201.1: start site: 92788604 (51735) last site: 92788605 (84164) total: 2
NW_026440202.1: start site: 92788606 (7923) last site: 92788608 (8093) total: 3
NW_026440207.1: start site: 92788609 (1220) last site: 92788706 (67581) total: 98
NW_026440208.1: start site: 92788707 (24477) last site: 92788710 (45007) total: 4
NW_026440209.1: start site: 92788711 (639) last site: 92788717 (40963) total: 7
NW_026440211.1: start site: 92788718 (9580) last site: 92788718 (9580) total: 1
NW_026440212.1: start site: 92788719 (20072) last site: 92788736 (26160) total: 18
NW_026440213.1: start site: 92788737 (37885) last site: 92788737 (37885) total: 1
NW_026440215.1: start site: 92788738 (35112) last site: 92788739 (35113) total: 2
NW_026440222.1: start site: 92788740 (1543) last site: 92788740 (1543) total: 1
NW_026440228.1: start site: 92788741 (27550) last site: 92788756 (27710) total: 16
NW_026440229.1: start site: 92788757 (3113) last site: 92788995 (29282) total: 239
NW_026440230.1: start site: 92788996 (489) last site: 92789374 (31447) total: 379
NW_026440233.1: start site: 92789375 (7621) last site: 92789383 (7942) total: 9
NW_026440237.1: start site: 92789384 (4029) last site: 92789393 (10637) total: 10
NW_026440238.1: start site: 92789394 (2242) last site: 92789400 (2420) total: 7
NW_026440239.1: start site: 92789401 (25868) last site: 92789401 (25868) total: 1
NW_026440242.1: start site: 92789402 (255) last site: 92790617 (25210) total: 1216
NW_026440244.1: start site: 92790618 (12658) last site: 92790638 (23813) total: 21
NW_026440250.1: start site: 92790639 (9138) last site: 92790658 (9416) total: 20
NW_026440253.1: start site: 92790659 (7312) last site: 92790659 (7312) total: 1
NW_026440255.1: start site: 92790660 (10256) last site: 92790670 (14197) total: 11
NW_026440260.1: start site: 92790671 (4729) last site: 92790690 (5212) total: 20
NW_026440263.1: start site: 92790691 (4114) last site: 92790712 (4230) total: 22

[pool-2-thread-1] INFO net.maizegenetics.plugindef.AbstractPlugin - Finished net.maizegenetics.analysis.data.FileLoadPlugin: time: Jul 21, 2025 11:00:28
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.popgen.LinkageDisequilibriumPlugin: time: Jul 21, 2025 11:00:29: progress: 0%
[pool-2-thread-1] INFO net.maizegenetics.pipeline.TasselPipeline - net.maizegenetics.analysis.popgen.LinkageDisequilibriumPlugin: time: Jul 21, 2025 12:40:11: progress: 10%

[cut off by user the next day around 12:00:00]

Other variations I have tried: 

run_pipeline.pl -Xmx460g -debug -h /xdisk/group/user/temp.hmp.txt -ld -ldRapidAnalysis true -ldWinSize 100000 -export /xdisk/group/user/ld_out.txt

run_pipeline.pl -Xmx460g -debug -h /xdisk/group/user/temp.hmp.txt -ld -ldRapidAnalysis true -ldWinSize 50 -export /xdisk/group/user/ld_out.txt

run_pipeline.pl -Xmx400g -h ./xdisk/group/user/temp.hmp.txt -ld -ldType SiteByAll -export /xdisk/group/user/ld_out.txt

[Muhammad Atif Wahid]

It depends on type of genetic markers...if it's SSRs I can tell you some methods of using tassel 2.1

--
You received this message because you are subscribed to the Google Groups "TASSEL - Trait Analysis by Association, Evolution and Linkage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tassel+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/tassel/b97386a9-b92c-43d5-9483-4ce0f387636fn%40googlegroups.com.

[Mary LaPorte]

Yes, SSRs! Thank you very much

[Muhammad Atif Wahid]

We consider several informations before running the LD Plot.

No 1. Type of Genetic Marker

No 2. Individuals in the Population Sample

According to my experience We use Tassel 2.1 or Tassel 3.o for SSR Marker.

Other softwares which are latest and updated now a days are Farm CPU and P Link 

--