MELT merge error

[Hyokyung Lee]

Hi, Eugene, I have a question about merge.

I already finished pre-processing.

And then I run each sample.

(because my data didn't work at once.)

First sample worked normally. But second sample didn't work in part of .tsv file merge.

Can I have some advice this problem?

- error :

Failed at merging genotype files, please see below for stack trace...

Exiting!

-------------------JAVA STACK TRACE-------------------

java.lang.Exception: Incorrect genotyping file sizes

        at MELT.MELTIllumina.mergeTSV.MergeTSV.CheckFileLengths(MergeTSV.java:427)

        at MELT.MELTIllumina.mergeTSV.MergeTSV.doWork(MergeTSV.java:203)

        at MELT.MELTIllumina.runtimes.StepFour.RunStepFour(StepFour.java:31)

        at MELT.MELTImplement.main(MELTImplement.java:85)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)

-------------------END STACK TRACE!-------------------

[egar...@umaryland.edu]

Hello,

Can you share the lengths of all the genotyped tsv files (with something like `wc -l`)? Additionally, please ensure MELT Genotype finished properly for all files.

Best,

Eugene

[Hyo Kyung Lee]

Hi, 

I didn't find .tsv file. So, I think .tsv file is not made. 

When running each sample, I change the working directory('-w option') that contains each directory of all samples to the directory of one sample. 

And tried to run melt again.

ex) '-w ~/melt/' -> '-w ~/melt/sample1/'

The running successes.

And the result is in the sample directory. ex) ~/sample1/ALU.final_comp.vcf

However, I expected that merge all samples .vcf results.

FYI, the length of ALU.final_comp.vcf is 1344, second is 2449.

Can I have some advice this problem?

Thank you.

2021년 4월 14일 수요일 오후 5시 51분 0초 UTC+9에 egar...@umaryland.edu님이 작성:

[egar...@umaryland.edu]

Hello,

The .tsv file has to be made and should be in the folder that you provided to -w/ when running Genotype. If you are trying to merge VCF files from separate runs, this is not what MELT or MakeVCF does. If you could share full command lines for at least a subset of your entries I might be able to help more.

Best,

Eugene

[Hyo Kyung Lee]

Hi, I'll show you my command line and error lines.

- command line

MELT.jar SGE -bamlist /MELT/PM-AA-0051-T-A1/PM-AA-0051-T-A1.depth.mean.txt.repath -exec /Tools/MELTv2.1.4/MELT.jar -h /gatk-bundle/Homo_sapiens_assembly38.fasta -i /opt/Gridengine -n /Tools/MELTv2.1.4/add_bed_files/Hg38/Hg38.genes.bed -u -v JAVA_TOOL_OPTIONS=-XX:ParallelGCThreads=1 -V -t /Tools/MELTv2.1.4/me_refs/Hg38/mei_list.txt -w /MELT/ -verbose -x 43200000 -m 8

- error line

Performing MELT analysis...

--------------------------------------------------------------------

Performing initial MEI discovery on the 4 mobile elements in provided list.

Current time: Apr 23, 2021 2:09:36 PM

IndivAnalysis cycle 1 finished; 0 fails...

--------------------------------------------------------------------

Determining statistics about each mobile element type.

Current time: Apr 23, 2021 7:18:17 PM

GroupAnalysis cycle 1 finished; 0 fails...

--------------------------------------------------------------------

Calling final genotypes in each genome.

Current time: Apr 23, 2021 7:56:09 PM

Genotyping cycle 1 finished; 0 fails...

--------------------------------------------------------------------

Merging all genotype calls and statistics into final VCF output.

Current time: Apr 23, 2021 8:36:04 PM

MakeVCF cycle 1 finished; 4 fails...

MakeVCF cycle 2 finished; 4 fails...

MakeVCF cycle 3 finished; 4 fails...

MakeVCF cycle 4 finished; 4 fails...

MakeVCF cycle 5 finished; 4 fails...

MakeVCF cycle 6 finished; 4 fails...

MakeVCF cycle 7 finished; 4 fails...

MakeVCF cycle 8 finished; 4 fails...

MakeVCF cycle 9 finished; 4 fails...

MakeVCF cycle 10 finished; 4 fails...

MakeVCF cycle 11 finished; 4 fails...

MakeVCF cycle 12 finished; 4 fails...

MakeVCF cycle 13 finished; 4 fails...

MakeVCF cycle 14 finished; 4 fails...

MakeVCF cycle 15 finished; 4 fails...

MakeVCF cycle 16 finished; 4 fails...

Failed at VCF creation after 15 cycles...

additionally, show you my success command line.

- command line (bold mark is different.)

MELT.jar SGE -bamlist /MELT/PM-AA-0051-T-A1/PM-AA-0051-T-A1.depth.mean.txt.repath -exec /Tools/MELTv2.1.4/MELT.jar -h /gatk-bundle/Homo_sapiens_assembly38.fasta -i /opt/Gridengine -n /Tools/MELTv2.1.4/add_bed_files/Hg38/Hg38.genes.bed -u -v JAVA_TOOL_OPTIONS=-XX:ParallelGCThreads=1 -V -t /Tools/MELTv2.1.4/me_refs/Hg38/mei_list.txt -w /MELT/PM-AA-0051-T-A1/ -verbose -x 43200000 -m 8

- result file
/MELT/PM-AA-0051-T-A1/ALU.final_comp.vcf

/MELT/PM-AA-0051-T-A1/HERVK.final_comp.vcf

/MELT/PM-AA-0051-T-A1/LINE1.final_comp.vcf

/MELT/PM-AA-0051-T-A1/SVA.final_comp.vcf

So, I wonder that final result should merge one .vcf file contained all samples.

not one .vcf file contained each sample like this result file.

Please an advice for these problems...:(

Thank you,

Hyo Kyung Lee

2021년 4월 23일 금요일 오전 12시 13분 30초 UTC+9에 egar...@umaryland.edu님이 작성:

[egar...@umaryland.edu]

Hello Hyo Kyung Lee,

The SGE mode of MELT is based on some old code, and I would not recommend using it. I should mark it as deprecated and remove it from the command line. As such, it is going to be difficult for me to debug your issues if you continue to use SGE.

My recommendation is to use the five step MELT pipeline (Preprocess, IndivAnalysis, GroupAnalysis, Genotype, MakeVCF) and schedule the jobs on your own and see if it fixes your problems. Apologies for this not being obvious from the documentation and please come back to me with any questions you may have.

Best,

Eugene 

[Hyo Kyung Lee]

Hi, 

Okay, I see.
I'll try to use the five step MELT pipeline.

And then, I come back to you if have any problem.

Thank you for your recommendation.

Hyo Kyung Lee.

2021년 4월 27일 화요일 오후 4시 44분 12초 UTC+9에 egar...@umaryland.edu님이 작성: