Re: Either inter.hic or inter_30.hic were not created

[Moshe Olshansky]

Hi Nolan,

If you have enough RAM you should increase the java heap space by including the Xmx option.  For example, 

java -Xmx60G

will allow java to use 60GB of RAM (please notice that there are no spaces in -Xmx60G).

Regards,

Moshe.

On Friday, March 25, 2022 at 6:47:31 PM UTC+11 nolan.r...@outlook.fr wrote:

Hi,

I sent some mails before this one for some issues at the end of the pipeline.

► I solve some of them but i have the same problem than previously : Either inter.hic or inter_30.hic were not created.

When i check files in the folder "aligned", i saw these files.

Moreover, these files are used by the pipeline so, i think there are already created ^^

► I have another issue but i don't know if it was essential or not : "inter_30_loops could not be parsed". I don't know if it could be a problem for the next steps of the pipeline ...

► I send here the final output :

------------------------------------------------------------------------------------------------------

Picked up _JAVA_OPTIONS: -Xmx1024m -Xms1024m

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Using 1 CPU thread(s) for primary task 

Picked up _JAVA_OPTIONS: -Xmx1024m -Xms1024m 

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Using 1 CPU thread(s) for primary task 

Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m 

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Using 1 CPU thread(s) for primary task 

Using 10 CPU thread(s) for secondary task 

Not including fragment map 

Start preprocess Writing header

Writing body .Exception in thread "main" java.lang.OutOfMemoryError: Java heap space          at java.base/java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:86) 

          at java.base/java.lang.StringBuilder.<init>(StringBuilder.java:103) 

          at juicebox.tools.utils.original.MatrixZoomDataPP.incrementCount(MatrixZoomDataPP.java:212) 

  at juicebox.tools.utils.original.MatrixPP.incrementCount(MatrixPP.java:148) 

  at juicebox.tools.utils.original.Preprocessor.writeBody(Preprocessor.java:743)

   at juicebox.tools.utils.original.Preprocessor.preprocess(Preprocessor.java:436)

   at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:165) 

          at juicebox.tools.HiCTools.main(HiCTools.java:94) real 13m26.963s user 18m9.102s sys 0m14.405s 

Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m 

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Using 10 CPU thread(s) for primary task 

  Calculating norms for zoom BP_2500000java.lang.NullPointerException at juicebox.data.iterator.ListOfListIterator.hasNext(ListOfListIterator.java:44) at juicebox.data.iterator.IteratorContainer.getNumberOfContactRecords(IteratorContainer.java:54) 

  at juicebox.data.iterator.ListOfListIteratorContainer.getIsThereEnoughMemoryForNormCalculation(ListOfListIteratorContainer.java:56) 

  at juicebox.tools.utils.norm.NormalizationCalculations.<init>(NormalizationCalculations.java:59) 

    at        juicebox.tools.utils.norm.GenomeWideNormalizationVectorUpdater.getWGVectors(GenomeWideNormalizationVectorUpdater.java:167) 

  at juicebox.tools.utils.norm.GenomeWideNormalizationVectorUpdater.updateHicFileForGWfromPreAddNormOnly(GenomeWideNormalizationVectorUpdater.java:132) 

  at juicebox.tools.utils.norm.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:159)

    at juicebox.tools.clt.old.AddNorm.launch(AddNorm.java:83) 

          at juicebox.tools.clt.old.AddNorm.run(AddNorm.java:137) 

         at juicebox.tools.HiCTools.main(HiCTools.java:94) 

  real 0m0.484s user 0m0.983s sys 0m0.061s Thu Mar 24 18:24:58 CET 2022 Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m 

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Using 1 CPU thread(s) for primary task 

Using 10 CPU thread(s) for secondary task 

Not including fragment map 

Start preprocess Writing header Writing body .

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space 

  at juicebox.tools.utils.original.MatrixZoomDataPP.incrementCount(MatrixZoomDataPP.java:212) 

  at juicebox.tools.utils.original.MatrixPP.incrementCount(MatrixPP.java:148) 

  at juicebox.tools.utils.original.Preprocessor.writeBody(Preprocessor.java:743) 

  at juicebox.tools.utils.original.Preprocessor.preprocess(Preprocessor.java:436) 

  at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:165) 

  at juicebox.tools.HiCTools.main(HiCTools.java:94) 

  real 11m56.773s user 15m57.144s sys 0m13.904s 

Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m 

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Using 10 CPU thread(s) for primary task 

  Calculating norms for zoom BP_2500000java.lang.NullPointerException

    at juicebox.data.iterator.ListOfListIterator.hasNext(ListOfListIterator.java:44) 

          at juicebox.data.iterator.IteratorContainer.getNumberOfContactRecords(IteratorContainer.java:54) 

          at juicebox.data.iterator.ListOfListIteratorContainer.getIsThereEnoughMemoryForNormCalculation(ListOfListIteratorContainer.java:56) 

          at juicebox.tools.utils.norm.NormalizationCalculations.<init>(NormalizationCalculations.java:59) 

  at juicebox.tools.utils.norm.GenomeWideNormalizationVectorUpdater.getWGVectors(GenomeWideNormalizationVectorUpdater.java:167) 

  at juicebox.tools.utils.norm.GenomeWideNormalizationVectorUpdater.updateHicFileForGWfromPreAddNormOnly(GenomeWideNormalizationVectorUpdater.java:132) 

  at juicebox.tools.utils.norm.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:159) 

  at juicebox.tools.clt.old.AddNorm.launch(AddNorm.java:83) 

  at juicebox.tools.clt.old.AddNorm.run(AddNorm.java:137) 

  at juicebox.tools.HiCTools.main(HiCTools.java:94) 

  real 0m0.468s user 0m1.001s sys 0m0.105s /media/nolan.ruffenach/DATA/juicer/scripts/common/juicer_tools is post-processing Hi-C for mm10 

Data read from /media/nolan.ruffenach/DATA/Documents/MLL3_project_serveur/HiC/aligned/inter_30.hic. Motifs read from /media/nolan.ruffenach/DATA/juicer/references/motif 

  ARROWHEAD: 

  Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Reading file: /media/nolan.ruffenach/DATA/Documents/MLL3_project_serveur/HiC/aligned/inter_30.hic Using 1 CPU thread(s) for primary task Unable to assess map sparsity; continuing with Arrowhead Default settings for 10kb being used No valid chromosome matrices at given resolution HiCCUPS: Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Reading file: /media/nolan.ruffenach/DATA/Documents/MLL3_project_serveur/HiC/aligned/inter_30.hic No valid configurations specified, using default settings Using 1 CPU thread(s) for primary task Unable to assess map sparsity; continuing with HiCCUPS Default settings for 5kb, 10kb, and 25kb being used Running HiCCUPS for resolution 5000 Running HiCCUPS for resolution 10000 Running HiCCUPS for resolution 25000 HiCCUPS complete APA: 

  Picked up _JAVA_OPTIONS: -Xmx1024m -Xms512m 

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 

Reading file: /media/nolan.ruffenach/DATA/Documents/MLL3_project_serveur/HiC/aligned/inter_30.hic Using 1 CPU thread(s) for primary task Processing APA for resolution 25000 File /media/nolan.ruffenach/DATA/Documents/MLL3_project_serveur/HiC/aligned/inter_30_loops could not be parsed Loop list is empty or incorrect path provided. 

***! Can't find folder /media/nolan.ruffenach/DATA/juicer/references/motif 

***! Not running motif finder 

  (-: Feature annotation successfully completed (-:

***! Error! Either inter.hic or inter_30.hic were not created 

Either inter.hic or inter_30.hic were not created. Check for results

----------------------------------------------------------------------------------------

Can you help me to fix these problems please ? 

Have a nice day,

Nolan.