Error when running Juicer Pipeline

638 views
Skip to first unread message

xian...@gmail.com

unread,
Oct 17, 2018, 1:57:45 PM10/17/18
to 3D Genomics
Hi,

I encountered an error when i was running the Juicer Pipeline and i was able to pinpoint to the following command:

~/tools/juicer/CPU/common/juicer_tools pre -f ~/tools/juicer/misc/hg19_MboI.txt -s ~/juicer/test/aligned/inter.txt -g ~/juicer/test/aligned/inter_hists.m -q 1 ~/juicer/test/aligned/merged_nodups.txt ~/juicer/test/aligned/inter.hic ~/reference/Juicer/hg19.chrom.sizes

Start preprocess
Writing header
Writing body
..................................................................................................................................................................................................................................................................................
Writing footer

Finished preprocess
Error reading datasetnull
java.io.EOFException
        at htsjdk.tribble.util.LittleEndianInputStream.readString(LittleEndianInputStream.java:119)
        at juicebox.data.DatasetReaderV2.read(DatasetReaderV2.java:182)
        at juicebox.tools.utils.original.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:78)
        at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:112)
        at juicebox.tools.HiCTools.main(HiCTools.java:96)

Any comments / suggestions are greatly appreciated!

I am using the latest version of Juicer and juicer tools. The content of "inter.txt" file is:
Experiment description: Juicer version 1.5.6;8 threads; java version "1.7.0_141"; ./juicer.sh -d ~/juicer/test/ -s MboI -p ~/reference/Juicer/hg19.chrom.sizes -y ~/tools/juicer/misc/hg19_MboI.txt -z ~/reference/hg19.fa -D ~/tools/juicer/ -t 8 -S final                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
Sequenced Read Pairs:  10,000
 Normal Paired: 4,253 (42.53%)
 Chimeric Paired: 4,583 (45.83%)
 Chimeric Ambiguous: 1,070 (10.70%)
 Unmapped: 94 (0.94%)
 Ligation Motif Present: 3,853 (38.53%)
Alignable (Normal+Chimeric Paired): 8,836 (88.36%)
Unique Reads: 8,836 (88.36%)
PCR Duplicates: 0 (0.00%)
Optical Duplicates: 0 (0.00%)
Library Complexity Estimate: 0
Intra-fragment Reads: 223 (2.23% / 2.52%)
Below MAPQ Threshold: 726 (7.26% / 8.22%)
Hi-C Contacts: 7,887 (78.87% / 89.26%)
 Ligation Motif Present: 1,998  (19.98% / 22.61%)
 3' Bias (Long Range): 60% - 40%
 Pair Type %(L-I-O-R): 24% - 25% - 24% - 26%
Inter-chromosomal: 1,345  (13.45% / 15.22%)
Intra-chromosomal: 6,542  (65.42% / 74.04%)
Short Range (<20Kb): 2,597  (25.97% / 29.39%)
Long Range (>20Kb): 3,945  (39.45% / 44.65%)



Best,
--Xiang

Neva Durand

unread,
Oct 17, 2018, 2:57:03 PM10/17/18
to xian...@gmail.com, 3D Genomics
What size is the .hic file right now? Based on your error message, it looks like adding norm failed. You could try loading the hic file in Juicebox or running the Python "read_hic_header.py" code from Straw to see if it's readable.

--
You received this message because you are subscribed to the Google Groups "3D Genomics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 3d-genomics...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/7a7b9515-8417-4bb6-be73-f8947e1353fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Neva Cherniavsky Durand, Ph.D.
Staff Scientist, Aiden Lab

xian...@gmail.com

unread,
Oct 17, 2018, 3:21:51 PM10/17/18
to 3D Genomics
Thank you for your quick reply!
The .hic file is 34Mb. It contains some binary characters, which i don't know if this is normal.

"aligned/inter.hic" may be a binary file.  See it anyway?
HIC^@^H^@^@^@^?@<D1>^A^@^@^@^@~/reference/Juicer/hg19.chrom.sizes^@^D^@^@^@software^@Juicer Tools Version 1.9.8^@statistics^@Experiment description: Juicer version 1.5.6;8 threads; java version "1.7.0_141"; ./juicer.sh -d ~/juicer/test/ -s MboI -p ~/reference/Juicer/hg19.chrom.sizes -y ~/tools/juicer/misc/hg19_MboI.txt -z ~/reference/hg19.fa -D ~/tools/juicer/ -t 8 -S final
Sequenced Read Pairs:  10,000
 Normal Paired: 4,253 (42.53%)
 Chimeric Paired: 4,583 (45.83%)
 Chimeric Ambiguous: 1,070 (10.70%)
 Unmapped: 94 (0.94%)
 Ligation Motif Present: 3,853 (38.53%)
Alignable (Normal+Chimeric Paired): 8,836 (88.36%)
Unique Reads: 8,836 (88.36%)
PCR Duplicates: 0 (0.00%)
Optical Duplicates: 0 (0.00%)
Library Complexity Estimate: 0
Intra-fragment Reads: 223 (2.23% / 2.52%)
Below MAPQ Threshold: 726 (7.26% / 8.22%)
Hi-C Contacts: 7,887 (78.87% / 89.26%)
 Ligation Motif Present: 1,998  (19.98% / 22.61%)
 3' Bias (Long Range): 60% - 40%
 Pair Type %(L-I-O-R): 24% - 25% - 24% - 26%
Inter-chromosomal: 1,345  (13.45% / 15.22%)
Intra-chromosomal: 6,542  (65.42% / 74.04%)
Short Range (<20Kb): 2,597  (25.97% / 29.39%)
Long Range (>20Kb): 3,945  (39.45% / 44.65%)
^@graphs^@A = [
38 33 1078 56 45 41 51 44 48 31 46 35 42 53 43 59 47 53 60 69 60 59 70 50 64 59 71 75 87 77 70 63 61 62 57 61 70 60 69 48 62 55 61 64 66 48 63 44 62 62 65 60 51 55 52 49 64 62 66 49 61 48 56 55 59 54 52 61 65 44 57 73 56 53 66 60 52 47 62 58 39 47 51 45 50 36 48 46 58 50 49 56 48 55 49 45 35 48 49 40 60 50 49 45 43 34 45 47 32 46 47 36 33 42 41 44 46 37 46 54 43 43 51 51 37 51 41 39 50 42 40 47 47 43 35 47 46 54 46 50 28 41 46 47 47 49 50 38 41 40 40 44 45 35 37 46 45 37 43 27 47 25 33 35 39 35 40 46 50 34 37 43 45 30 44 43 28 27 33 35 31 32 22 22 38 44 30 33 30 31 29 34 36 30 29 37 31 29 30 29 34 31 27 31 22 26 44 31 28 28 34 34 23 39 39 39 33 29 31 23 35 31 32 23 30 24 27 22 21 21 15 33 23 29 27 24 31 34 31 22 11 27 22 22 30 21 23 24 22 29 30 23 20 31 17 16 16 17 19 11 31 22 27 32 21 14 14 25 21 11 21 14 18 11 12 21 20 17 12 21 12 11 16 17 27 21 18 9 17 13 18 19 15 11 20 14 16 15 9 16 15 13 9 14 10 19 11 13 14 13 9 8 12 16 16 17 11 13 9 8 10 7 17 12 11 9 10 10 11 7 14 11 15 8 10 11 7 9 14 10 11 10 9 12 10 14 6 16 9 12 11 9 10 4 8 15 2 8 8 10 7 6 7 11 8 12 9 8 7 4 4 9 7 10 5 10 10 6 9 9 9 4 7 6 5 5 7 12 2 7 8 5 5 9 4 3 5 7 2 8 7 7 3 3 5 9 7 7 9 8 5 11 3 8 4 2 3 4 9 6 3 1 6 3 5 3 7 5 3 10 3 7 3 6 6 5 2 2 6 1 4 7 4 5 4 3 6 7 4 5 1 4 3 1 5 5 5 5 3 8 4 5 8 1 6 1 4 5 1 0 2 5 3 6 4 0 5 0 2 3 2 1 1 1 1 4 1 2 2 0 1 5 6 4 1 2 4 4 1 4 1 5 3 1 1 6 1 2 3 4 3 5 3 3 3 4 2 3 4 1 1 2 2 3 3 3 1 0 2 1 2 4 4 1 1 1 2 1 1 2 3 2 2 4 2 4 2 1 2 0 2 2 2 3 2 2 0 4 1 1 1 3 2 1 2 2 2 1 3 0 3 1 1 1 2 3 1 2 1 0 2 1 0 2 1 0 2 2 4 7 0 2 1 2 2 2 2 1 0 5 2 0 1 1 1 2 2 0 3 1 1 2 0 0 2 1 0 2 1 3 4 2 2 2 2 0 1 2 0 1 2 0 2 0 2 0 0 0 4 0 1 1 1 0 1 1 3 1 0 0 0 0 0 2 2 0 1 1 1 2 0 1 3 0 1 1 1 0 1 1 0 2 0 0 0 0 2 1 0 0 0 0 6 2 0 2 0 2 2 0 1 1 1 1 1 2 2 1 0 2 2 0 4 1 0 1 1 2 1 0 1 1 0 1 0 1 0 1 0 0 0 1 0 0 1 1 1 1 2 1 2 1 0 2 2 2 2 0 3 1 1 1 0 0 2 2 0 1 2 3 1 1 1 1 4 0 1 1 3 0 0 2 1 0 2 0 1 1 2 3 1 2 1 0 1 0 2 2 1 0 3 0 0 3 0 1 0 0 0 1 0 1 3 3 0 0 2 0 0 1 4 2 1 0 0 0 0 1 0 0 0 4 0 1 1 0 1 0 2 2 0 0 0 0 0 0 0 1 0 0 1 1 1 0 1 1 1 1 0 1 0 0 1 1 0 1 1 2 1 0 0 0 0 0 0 1 1 0 1 2 1 2 1 1 0 0 0 1 1 1 1 0 0 1 0 0 0 0 0 0 1 0 1 0 2 0 0 1 0 1 0 0 1 1 0 0 0 2 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 1 1 2 0 0 0 0 0 0 0 0 1 0 1 0 2 0 0 1 0 2 0 0 1 2 0 1 0 0 0 0 3 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 0 0 1 2 0 0 0 1 0 1 0 1 0 0 1 0 0 0 1 1 0 0 0 1 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1 0 0 1 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
];
B = [
0 0 0
 16 11 5
 18 8 10
......

xian...@gmail.com

unread,
Oct 17, 2018, 3:40:08 PM10/17/18
to 3D Genomics
Emm, when i tried to load .hic file into Juicebox, it said "error loading .hic file"...

Neva Durand

unread,
Oct 17, 2018, 3:45:43 PM10/17/18
to xian...@gmail.com, 3D Genomics
I'm not sure of the problem; it looks like your header was written but not data. Do your chromosome names in merged_nodups.txt match those in ~/reference/Juicer/hg19.chrom.sizes ? Could you paste ~/reference/Juicer/hg19.chrom.sizes?



To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/f79f6d9a-26d8-4c53-8553-7ee0c1ae0523%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

xian...@gmail.com

unread,
Oct 17, 2018, 3:50:18 PM10/17/18
to 3D Genomics
Thank you for your time, Neva!

Here is my hg19.chrom.sizes file:
chr1    249250621
chr2    243199373
chr3    198022430
chr4    191154276
chr5    180915260
chr6    171115067
chr7    159138663
chrX    155270560
chr8    146364022
chr9    141213431
chr10   135534747
chr11   135006516
chr12   133851895
chr13   115169878
chr14   107349540
chr15   102531392
chr16   90354753
chr17   81195210
chr18   78077248
chr20   63025520
chrY    59373566
chr19   59128983
chr22   51304566
chr21   48129895
chrM    16571

Neva Durand

unread,
Oct 17, 2018, 3:51:08 PM10/17/18
to xian...@gmail.com, 3D Genomics
Could you send "head" of merged_nodups?

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/d50cfb3c-076c-44e1-9ec3-0e5f652e2fe6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

xian...@gmail.com

unread,
Oct 17, 2018, 4:08:57 PM10/17/18
to 3D Genomics
Here it is:
  head aligned/merged_nodups.txt

0 chr1 1315265 2679 0 chr1 1355017 2789 60 10M1D98M43S ACTCCGTCTCAAAAAAAAAAAAGAAAAAAATTTTAAAAACAAATTAGCCAGGCATGGAGGCGTGAGCTTGTGGTCTCAGCTACTTGGGAGGCTGAGGTGAGAGGATCGATCCTTGGATTTGCTCTAAGGACTCTTAGGGGGAAAAGTGGAC 60 125M26S CTGTCCCCGCAGCGGCCTGTCTCTCCTGACACCACTTCCTCATGTTGTGAGGCCGAACCCAAGGCTGGCCTGCTTGGAAACGTCCACTTTTCCCCCTAAGAGTCCTTAGAGCAAATCCAAGGATCGATCCTCTCACCTCAGCCTCCCAAGT E00224:346:H7Y5TALXX:3:1101:26111:23407/2 E00224:346:H7Y5TALXX:3:1101:26111:23407/1
16 chr1 1538653 3312 16 chr1 1539521 3314 60 151M AGGGCTCAGTCCTCACTGTAGCAGGGGAGCTGCCGGCCCTCTAGGCAGGCCGCCCTCTGGGAACCCCTCTTCTCTCCAACTGCCCCAGCTCCAGGACAGGTTGGAAATGCTTTTGCCTGGCAGGGGAAGGCTGGACCAGTAACGATGGAAG 60 151M CCTGGGTCCCATCCCAAGAGCATCCACAGTGGCGTCGCCAGTGGAGGGGTGAAGCTGAGGGGCCCCACAGACCCCAGGCCCAGTGCTTGGTGGCTGCAAGAGCTGGGAGGAGCCCCATCCTCCAGGGCCTTCAGGGAAGGTGGAGCGGACT E00224:346:H7Y5TALXX:3:1102:28699:24691/1 E00224:346:H7Y5TALXX:3:1102:28699:24691/2
0 chr1 1920313 4422 0 chr1 1922283 4428 60 83M68S GATCCTCCGAGTTACCTGCTGTGTCCCGGGTCCACGTCATCCTCAGCTTCCTGTAGAAGACATTTGATGTCAAACTCCGGATCGATCATCTTTTGCAGCTAATTTCTATCTCCCAGCATGGAGGATGACGGCAGCCTGCTCCCTGAGGACG 60 148M3S TGTCTTTTTGCTGAAACTTCTGTTACTGTGAAAGAAGCTTTATTACCATCTTCCAAAAGAAGGGCATCGGCCAAAAGCTCGTCCTCAGGGAGCAGGCTGCCGTCATCCTCCATGCTGGGAGATAGAAATTAGCTGCAAAAGATGATCGATC E00224:346:H7Y5TALXX:3:1101:22426:37454/1 E00224:346:H7Y5TALXX:3:1101:22426:37454/2
0 chr1 2421999 5555 0 chr1 2805019 6146 60 151M ATGCATGGGGAGATAATTGTGCAGCGTGCTCCTGGCCACGCGGCCAACCATTTTAGAAACTCCTGAGGCAGCAGCACTCTTCCTGCCTGGGATGTGCGGCACACGCCTCTGTGTGGCCCACGTGATGGCAAAGGCAGCTCCACGGTGCTCC 60 151M CTGCCTAGGCATCATGCACGGGGGCCTTCACCCTCCAGGGCTGCTTGGCCAGTGGTGCTGGGCTGGGCCCGTGAAAAGAGCAGGTCGGGTGAGCAGAGGTGGCTGCTCTGGCCCCCTGGGAGGTGAGAGGGCGCGTGCCTGTGTCAGGGCA E00224:346:H7Y5TALXX:3:1103:22254:9484/1 E00224:346:H7Y5TALXX:3:1103:22254:9484/2
0 chr1 4627232 9932 0 chr1 4628432 9939 60 151M TGGGCAAATTACTCAACCTCTCTGTGCCTTATTTTTTCCAGTTGTAAAATGGAAGATAATACTAGCACCTCTCACCGAAGATTGTTGAGGTGGGTCCAGAACTGTGACACTTACATCCGGGCAGTTTACTTATCAGGACTGAAATGAAGGC 60 151M GATCAGACCTAAAAGCATCAAATAGCTTTTAATGCTCAAAAATGGCTGCAAGTGAACAAGCAAAGACAAAGAAGATCTGTCACTGGAAGATAAAATAAAAGCACTTGTCAGAATTGCAGCTGCACTGTTGGGGCCTCCGTGGGGCATTTTT E00224:346:H7Y5TALXX:3:1101:23776:10890/1 E00224:346:H7Y5TALXX:3:1101:23776:10890/2
16 chr1 4712026 10129 16 chr1 4712728 10130 60 54S10M3I84M TGATATAGCCGTATATATTATTTATTAATACACTTAGTAAGCAATTGACAGATCACTCCATCTCAATAATAATAATAATAATAATAATAATAATAATAATAATAGGTGGATTGCTCTTTTTGGAGAGAGGCAGACCTGAGTTACCCTAGAG 60 151M CATAGAATTGGTTTCCTAATAATCCTGGGCTTTTTCATCTATGCACTTAGAAGCCATTCTGAGAGAGGTCAGCAGGTGTCCACAAGGCTGCCCAAGGGGTTCCTGGAGCCGACACGGTCCGGTCCGGAGCCCCTCCCTGCCCTGCTGGCTC E00224:346:H7Y5TALXX:3:1102:6115:48195/1 E00224:346:H7Y5TALXX:3:1102:6115:48195/2
0 chr1 5091670 11019 16 chr1 5090847 11017 60 151M TTATAATGCAAAAGCAGAAATATACACTACATAAGTGAATGGGCTTGTCTGTGTTCCAATAAAACTTTATTTATAAAACAGCTGGTGAGCCAGATTTGGCCTATGAGCTATAATTTGTCAATTCTTGTTCTAGGCCAGTAACTTGGGTCTT 60 98H53M GATCTATCTCATCAGGGAATGGATGATGTACTCAAATTAGTGATTGGAGAGAT E00224:346:H7Y5TALXX:3:1101:14794:9923/2 E00224:346:H7Y5TALXX:3:1101:14794:9923/1
0 chr1 5257721 11386 0 chr1 5262893 11395 60 151M CACCACCTGATGCGCCAACTCCCCTTCCCCTTCCATCATAATTGGAAGCTTCCTGAGGCTCTCACCAGATGCAGATGCTGGTGCCATATTTCATGTACAACCTGCAGAACTGTGAGCCAAATAAACATTTTTTCTTTATAAGTTTCCCAGC 60 151M AGGCTTAACATGTCACTTAGTCCTTAAGGAGCTTGCAACATAAAGGCGGAAGCCACCTATTCATCTCCAGCCATGCTGTTTACCAGTGTGGTCAGCTAGAAATGCAAGTAACAGTCAGGAGTTTCATCTGTTTTCCTCAAACTGGCCTCCT E00224:346:H7Y5TALXX:3:1101:1467:15109/2 E00224:346:H7Y5TALXX:3:1101:1467:15109/1
16 chr1 6551528 14760 16 chr1 20173862 53999 60 77H54M20H GATCCCCACGTCTTCACCTGCTCCCCCAAAACTACTTAGCCAAAACCCAGGAGT 60 151M AGCGGGAGTACAGGGATTCACACCTTCCTTCTCTGAGTCCAAGTCTCTGCTCTGTGCCTAACTCCACTGGGTTGCCTTACACAGTTTCCAGGTTCTCCATTCATTCCAAGTAGAAAATAAGGTCAATGTTAAAAGAAAAGCTATGCCTCAT E00224:346:H7Y5TALXX:3:1101:32515:55140/2 E00224:346:H7Y5TALXX:3:1101:32515:55140/1
0 chr1 7300021 16823 16 chr1 35297515 98229 60 151M TAATGACGTTAATCCATTCATGACCTAATCACCTCTTATTAGGCCTGACCTCCCAACAGTGTTGTAGTGGGGATTAAGTTTCCAACACATGACCTTTAGGGGGACACATCCAAACCATAGCAGAGGACATGTACTATCTGGTTGTCTCTCT 60 151M AGCAAAGATGGATGTAGGGACCTGGGCCAAGGAAACTGCTCAGGCAGTTTCCAATTCCTGGTCTGGTGCTGCCTTCCCTGTTCTGTCTGCTGACATGAGCTCACCGCCAGGTGCAGCTGAACAATGCACACTCTCTATGCCTTCAGGTCTC E00224:346:H7Y5TALXX:3:1103:23685:11646/2 E00224:346:H7Y5TALXX:3:1103:23685:11646/1

Neva Durand

unread,
Oct 17, 2018, 4:15:56 PM10/17/18
to xian...@gmail.com, 3D Genomics

I’m sorry to give you this very unsatisfying answer, but I would try running “pre” again. I.e. run this command:

~/tools/juicer/CPU/common/juicer_tools pre -f ~/tools/juicer/misc/hg19_MboI.txt -s ~/juicer/test/aligned/inter.txt -g ~/juicer/test/aligned/inter_hists.m -q 1 ~/juicer/test/aligned/merged_nodups.txt ~/juicer/test/aligned/inter.hic ~/reference/Juicer/hg19.chrom.sizes
To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/61bb4764-babe-4532-8776-d3c2dab4ff9c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

xian...@gmail.com

unread,
Oct 17, 2018, 4:23:25 PM10/17/18
to 3D Genomics
Many thanks! But actually i ran it for at least 5 times, with the same error each time...

Start preprocess
Writing header
Writing body
..................................................................................................................................................................................................................................................................................
Writing footer

Finished preprocess
Error reading datasetnull
java.io.EOFException
        at htsjdk.tribble.util.LittleEndianInputStream.readString(LittleEndianInputStream.java:119)
        at juicebox.data.DatasetReaderV2.read(DatasetReaderV2.java:182)
        at juicebox.tools.utils.original.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:78)
        at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:112)
        at juicebox.tools.HiCTools.main(HiCTools.java:96)

Neva Durand

unread,
Oct 17, 2018, 4:24:33 PM10/17/18
to xian...@gmail.com, 3D Genomics
It looks like your merged_nodups file is small, could you send it to me and I'll try to debug?

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/e277bec8-4e77-4781-b32c-f1ade876db60%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Neva Durand

unread,
Oct 17, 2018, 4:24:59 PM10/17/18
to xian...@gmail.com, 3D Genomics
Also can you try running without the fragment map (without the "-f" flag) and see what happens?

xian...@gmail.com

unread,
Oct 17, 2018, 4:58:40 PM10/17/18
to 3D Genomics
I also tested without -f. Still the same error.
I have attached my input files for you to test, thanks a lot!
And you probably already have hg19_MboI.txt file.

If you need my original fastq files, i can also send you over. Since they are very small (2Mb).

Best,
--Xiang
hg19.chrom.sizes
inter_hists.m
inter.txt
merged_nodups.txt

Neva Durand

unread,
Oct 17, 2018, 5:12:26 PM10/17/18
to xian...@gmail.com, 3D Genomics
I am able to run it without any problems. I am running Java 8, which is the only difference I see between your setup and mine.

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/1de99cbe-ca62-4884-b2ef-03a7568509ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

xian...@gmail.com

unread,
Oct 17, 2018, 5:38:58 PM10/17/18
to 3D Genomics
Emm...
I also checked the "juicer_tools" file and i manually ran this command using two different Java version. However, none of them worked...

First test:
/usr/bin/java -Djava.awt.headless=true -Xmx16000m -jar ~/tools/juicer/scripts/common/juicer_tools.jar pre -f ~/tools/juicer/misc/hg19_MboI.txt -s ~/juicer/test/aligned/inter.txt -g ~/juicer/test/aligned/inter_hists.m -q 1 ~/juicer/test/aligned/merged_nodups.txt ~/juicer/test/aligned/inter.hic ~/reference/Juicer/hg19.chrom.sizes

/usr/bin/java -version
java version "1.7.0_141"
OpenJDK Runtime Environment (rhel-2.6.10.1.el6_9-x86_64 u141-b02)
OpenJDK 64-Bit Server VM (build 24.141-b02, mixed mode)

-------------------------------------------------------------------------------------
Second test:
/usr/lib/jvm/jre-1.8.0/bin/java -Djava.awt.headless=true -Xmx16000m -jar ~/tools/juicer/scripts/common/juicer_tools.jar pre -f ~/tools/juicer/misc/hg19_MboI.txt -s ~/juicer/test/aligned/inter.txt -g ~/juicer/test/aligned/inter_hists.m -q 1 ~/juicer/test/aligned/merged_nodups.txt ~/juicer/test/aligned/inter.hic ~/reference/Juicer/hg19.chrom.sizes

/usr/lib/jvm/jre-1.8.0/bin/java -version
openjdk version "1.8.0_31"
OpenJDK Runtime Environment (build 1.8.0_31-b13)
OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)

Muhammad Shamim

unread,
Oct 18, 2018, 6:16:51 PM10/18/18
to 3D Genomics
Thank you for helping us catch this bug!
I've pushed a fix and rebuilt the jar.
Can you retry running pre?

Best,

xian...@gmail.com

unread,
Oct 18, 2018, 6:56:38 PM10/18/18
to 3D Genomics
Thank you, Muhammad.
I confirmed that the issue is solved:-)
Message has been deleted

Neva Durand

unread,
Dec 11, 2018, 9:55:49 AM12/11/18
to xian...@gmail.com, 3D Genomics
Hello,

Sorry for the late reply. The error message usually gives a hint to the problem. Make sure you have a correctly formatted restriction site file and that the chromosome names match. (This is not similar to your previous error)

On Fri, Nov 2, 2018 at 6:42 PM <xian...@gmail.com> wrote:
Hi,

I think this issue is related with the previous one. So i reply here:

Last time, after the pipeline update, i was able to successfully run the program. However, using another dataset today, i encountered another similar error:

~/software/jdk1.8.0_101/bin/java -jar ~/tools/juicer/CPU/common/juicer_tools.1.9.8_jcuda.0.8.jar pre -f ${TEMP}/$$_resfrag.juicebox ${TEMP}/$$_allValidPairs.pre_juicebox_sorted sample1_allValidPairs.hic ~/tools/HiC-Pro_2.10.0/annotation/hg19.chrom.sizes
Problem with creating fragment-delimited maps, NullPointerException.
This could be due to a null fragment map or to a mismatch in the chromosome name in the fragment map vis-a-vis the input file or chrom.sizes file.
Exiting.

Could anyone take a look at this? Thanks a lot!

Best,
--Xiang
To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/b40abef4-4803-4e15-8032-a3200cf4febe%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages