HDF error when saving the dump files for post-processing

[LIMING CHAO]

Dear all,

When the code write the dump files,  the following error occurs, what should I do?

Best,

Li-MIng

Writing visualization files...

HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 0:
  #000: H5F.c line 444 in H5Fcreate(): unable to create file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1509 in H5F_open(): unable to open file: time = Tue Aug 16 11:11:27 2022
, name = 'viz_3d/visit_dump.00200/processor_cluster.00000.samrai', tent_flags = 13
    major: File accessibilty
    minor: Unable to open file
  #002: H5FD.c line 734 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = 'viz_3d/visit_dump.00200/processor_cluster.00000.samrai', errno = 24, error message = 'Too many open files', flags = 13, o_flags = 242
    major: File accessibilty
    minor: Unable to open file
P=00000:Program abort called in file ``../../../../SAMRAI-2.4.4/source/toolbox/restartdb/HDFDatabase.C'' at line 2397
P=00000:ERROR MESSAGE:
P=00000:Unable to open HDF5 file viz_3d/visit_dump.00200/processor_cluster.00000.samrai
P=00000:
[kn-cb-c022:17586] *** Process received signal ***
[kn-cb-c022:17586] Signal: Aborted (6)
[kn-cb-c022:17586] Signal code:  (-6)
[kn-cb-c022:17586] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f8adb31e980]
[kn-cb-c022:17586] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f8adaf59e87]
[kn-cb-c022:17586] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f8adaf5b7f1]
[kn-cb-c022:17586] [ 3] ./main3d(+0x75a8b2)[0x55fa037d98b2]
[kn-cb-c022:17586] [ 4] ./main3d(+0x733d45)[0x55fa037b2d45]
[kn-cb-c022:17586] [ 5] ./main3d(+0x15f574)[0x55fa031de574]
[kn-cb-c022:17586] [ 6] ./main3d(+0xacf23)[0x55fa0312bf23]
[kn-cb-c022:17586] [ 7] ./main3d(+0x6e410)[0x55fa030ed410]
[kn-cb-c022:17586] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f8adaf3cc87]
[kn-cb-c022:17586] [ 9] ./main3d(+0x888aa)[0x55fa031078aa]
[kn-cb-c022:17586] *** End of error message ***
Aborted (core dumped)

[Boyce Griffith]

On Aug 16, 2022, at 10:03 AM, LIMING CHAO <chaoli...@gmail.com> wrote:

Dear all,

When the code write the dump files,  the following error occurs, what should I do?

I think that the key reported error is “Too many open files”. This is a system error, not a problem in IBAMR per se. You might be able to use ulimit to increase the number of files that you can have open, but I suggest to check with your system administrator.

Although this kind of error can occur if a program “leaks” file handles, I am pretty sure that IBAMR is not doing this because we run very long simulations without encountering this kind of problem.

--
You received this message because you are subscribed to the Google Groups "IBAMR Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ibamr-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ibamr-users/42dfc724-89a8-48f1-9d1b-6274be65c676n%40googlegroups.com.

[LIMING CHAO]

Dear Prof. Griffth,

Yes, you are right. Actually, I run more than 20 cases and store the result in a online storage system at the same time. However, only this case would encounter the problem. 

Best,

Li-Ming

[Boyce Griffith]

On Aug 16, 2022, at 10:16 AM, LIMING CHAO <chaoli...@gmail.com> wrote:

Dear Prof. Griffth,

Yes, you are right. Actually, I run more than 20 cases and store the result in a online storage system at the same time. However, only this case would encounter the problem.

Are there any obvious differences between the cases? E.g. is this running with a different number of cores than the other cases that succeed?

To view this discussion on the web visit https://groups.google.com/d/msgid/ibamr-users/97217c8f-90bb-4cf9-907a-19e36dc8ffc5n%40googlegroups.com.

[LIMING CHAO]

Dear Prof. Griffith,

All cases are simulated with single core. The difference between the failed one and succeed one is that the failed case is 3D simulation and the succeed cases are 2D simulation. On the other hand, the failed 3D simulation can save the dump files at the first saving interval but always be failed at the second saving interval.

Best,

Li-Ming