Re: can't creat temporary file HIC_tmp

[Neva Durand]

Sorting uses the tmpdir, so if you don't have writing permissions that could cause problems.

On Fri, Apr 9, 2021 at 6:56 AM YN HAO <23885...@qq.com> wrote:

Hi,

Thank you for your help. There is a warning when I run the juicer.sh.

sort: cannot create temporary file in ‘/data/home/juicer_result/2221/HIC_tmp’: No such file or directory.

It seems have no impact to my result, but I'm still worried about it. And I noticed that in the juicer.sh code line 425,remove tmpdir, and in the line 460 required tmpdir. It this warning caused by it? And I delete line425, there is no warning.  If there any influence to my result when I delete line425?

Thanks and Best Regards,

yanan

--
You received this message because you are subscribed to the Google Groups "3D Genomics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 3d-genomics...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/ee2dcc2b-54d3-4b36-aa37-9dedb780536dn%40googlegroups.com.

--

Neva Cherniavsky Durand, Ph.D. | she, her, hers

Assistant Professor |  Molecular and Human Genetics

Aiden Lab | Baylor College of Medicine

www.aidenlab.org

[YN HAO]

Hi Neva,

Thanks for your reply!   Can I delete the  line425 code (rm -r ${tmpdir}) of juicer.sh? When I retain this line, there is still the warning that 

sort: cannot create temporary file in ‘/data/home/juicer_result/2221/HIC_tmp’: No such file or directory. although the tmpdir has all the  permissions.

Thanks again,

yanan

[Neva Durand]

Again if the sort command is giving you that error, just deleting the directory won’t help. Make sure you have permissions and that there’s enough space. 

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/4dcafb1f-de2d-45b3-8168-6569642d637bn%40googlegroups.com.

[linyong shen]

I have encountered the same problem recently, and I want to know how you solved it? Delete the line425 code (rm -r ${tmpdir} ) or ignore the warning "sort: cannot create temporary file in ‘*/HIC_tmp’: No such file or directory"?

[Neva Durand]

You would need to delete the portion of the sort call that uses the directory (i.e. remove the -T $tmpdir ). Note that if you're having trouble creating directories, you might be running out of space, which will cause problems with the pipeline finishing; and the reason we set a tmpdir is that otherwise one often runs out of space on /var/tmp

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/71780dcd-d500-4760-96a1-86f39c7e0207n%40googlegroups.com.

[linyong shen]

I am glad you responded to my question so quickly. The result of my juicer pipline looks very normal. The warning "sort: cannot create temporary file in'*/HIC_tmp': No such file or directory" is correct for my juicer result. Does it matter? 

 Here is the code near this warning:

   3439 [M::mem_process_seqs] Processed 8313612 reads in 3093.883 CPU sec, 28.707 real sec

   3440 [main] Version: 0.7.17-r1188

   3441 [main] CMD: bwa mem -SP5M -t 144 /storage/SLY68/hic/juicer/fat1/references/Gallus_gallus.GRCg6a.dna.toplevel.fa /storage/SLY68/hic/juicer/fat1/splits/AF224727_R1.fas

   3442 [main] Real time: 8044.238 sec; CPU: 535953.625 sec

   3443 (-:  Align of /storage/SLY68/hic/juicer/fat1/splits/AF224727.fastq.gz.sam done successfully

   3444 (-: Finished sorting all sorted files into a single merge.

   3445 Picked up _JAVA_OPTIONS: -Xmx16384m

   3446 sort: cannot create temporary file in ‘/storage/SLY68/hic/juicer/fat1/HIC_tmp’: No such file or directory

   3447 Picked up _JAVA_OPTIONS: -Xmx16384m

   3448 Not including fragment map

   3449 Start preprocess

   3450 Writing header

   3451 Writing body

   3452 .....................................................................................................................................................................

   3453 Writing footer

   3454 

   3455 Finished preprocess

   3456 

   3457 Calculating norms for zoom BP_2500000

   3458 Calculating norms for zoom BP_1000000

[Neva Durand]

Thank you for posting the whole output. Is this the CPU version? I have a feeling this is collisions dedupping and not affecting anything else.

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/c4d5d10e-154f-4787-a94f-dcf9856e809dn%40googlegroups.com.

[linyong shen]

This is the juicer pipline of the CPU version. I checked all the log files and found no Errors or Warnings in other places. And the result does not seem to be abnormal.

[Neva Durand]

Yes, I think it's fine, just an issue with the collisions dedup.

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/af9cfd66-b242-4f3d-9597-ee7054cfccbbn%40googlegroups.com.

[etiennedanis]

I

1. 3IAM n

Sent from my T-Mobile 4G LTE Device

3w

-------- Orig jminal message --------

From: 'linyong shen' via 3D Genomics <3d-ge...@googlegroups.com>

Date: 7/7/21 18:40 (GMT-07:00)

To: 3D Genomics <3d-ge...@googlegroups.com>

Subject: Re: can't creat temporary file HIC_tmp

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/71780dcd-d500-4760-96a1-86f39c7e0207n%40googlegroups.com.

[Noor-ul-Ain]

Hello, Dear Neva, i am running juicer and i am facing the following error ,

[M::mem_process_seqs] Processed 1341058 reads in 2773.295 CPU sec, 111.545 real sec

[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -SP5M -t 25 /public/home/agis_noor/m_sativa/xinjiangdaye/ont+hifi/allhic/juicer/references/m.sativa.fa /public/home/agis_noor/m_sativa/xinjiangdaye/ont+hifi/allhic/juicer/splits/hic_R1.fastq /public/home/agis_noor/m_sativa/xinjiangdaye/ont+hifi/allhic/juicer/splits/hic_R2.fastq
[main] Real time: 62260.083 sec; CPU: 1547267.582 sec
(-:  Align of /public/home/agis_noor/m_sativa/xinjiangdaye/ont+hifi/allhic/juicer/splits/hic.fastq.sam done successfully
sort: cannot create temporary file in '/public/home/agis_noor/m_sativa/xinjiangdaye/ont+hifi/allhic/juicer/HIC_tmp': Too many links
***! Failure during sort of /public/home/agis_noor/m_sativa/xinjiangdaye/ont+hifi/allhic/juicer/splits/hic.fastq
 

After so many attempts finally i got up to this step, and it took a long time to create a sam file of 444 GB, can you guide me to solve this error, and secondly how can I resume my process from here because it will consume alot of time and resources if I start over. 

output of \splits ; I have deleted sorted sam file which was 0 in size and tmp dir as well