Computation terminated Time step size change encountered.

[Yicheng Chen]

Hi all,

Thank you for reading. Some strange errors occurs in my IBAMR, and I hope to know the cause of these errors.

1. Time step size change encountered

When I try to simulate the move of a flexible plate, the calculation will stop between step 3 and 7, thus the terminal reports message A. This problem occured both in soft plate simulations and 3d human airway simulations.  

2. Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has aborted

This error occured when I running human airway simulation by mpi, and I cannot find what induced it. Detail information is included in message B.

Thank you again for your kind help!

message A

At beginning of timestep # 3
Simulation time is 9.76563e-07
P=00000:Program abort called in file ``../../IBAMR/lib/../src/IB/IBHierarchyIntegrator.cpp'' at line 184
P=00000:ERROR MESSAGE:
P=00000:IBHierarchyIntegrator::preprocessIntegrateHierarchy():  Time step size change encountered.
P=00000:Aborting.
P=00000:
[Eason:09382] *** Process received signal ***
[Eason:09382] Signal: Aborted (6)
[Eason:09382] Signal code:  (-6)
[Eason:09382] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f51e517d980]
[Eason:09382] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f51e33a4fb7]
[Eason:09382] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f51e33a6921]
[Eason:09382] [ 3] ./main2d(+0x64ed12)[0x558364360d12]
[Eason:09382] [ 4] ./main2d(+0x1571c2)[0x558363e691c2]
[Eason:09382] [ 5] ./main2d(+0x13b63c)[0x558363e4d63c]
[Eason:09382] [ 6] ./main2d(+0x452184)[0x558364164184]
[Eason:09382] [ 7] ./main2d(+0x79af2)[0x558363d8baf2]
[Eason:09382] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f51e3387bf7]
[Eason:09382] [ 9] ./main2d(+0x8e37a)[0x558363da037a]
[Eason:09382] *** End of error message ***
Aborted (core dumped)

message B

Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has aborted.

-----------------------------------------------------------------------------------------------------------------------------

MPI_ABORT was invoked on rank 7 in communicator MPI_COMM_WORLD with errorcode 50162059.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them.

[Yicheng Chen]

Add:

The error will change to "at least one node in the current mesh is inside the fluid domain and not associated with any patch." if I increase the MU_S and NU_S (*100 -> e8) of solid. The error is in message C.

Decreasing of MAX_LEVELS (4 -> 3) and FALSE of ERROR_ON_DT_CHANGE will extend computation time but it finally stop in step 2251, detail information in message D.

 message C

At beginning of timestep # 3

Simulation time is 4.88281e-07
P=00000:Program abort called in file ``../../../IBAMR/ibtk/lib/../src/lagrangian/FEDataManager.cpp'' at line 717
P=00000:ERROR MESSAGE:
P=00000:At least one node in the current mesh is inside the fluid domain and not associated with any patch. This class currently assumes that all elements are on the finest level and will not work correctly if this assumption does not hold. This usually happens when you use multiple patch levels and set the regrid CFL interval to a value larger than one. To change this check set node_outside_patch_check to a different value in the input database: see the documentation of FEDataManager for more information.
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode -1.

message D

At beginning of timestep # 2251
Simulation time is 1.04938e-06
P=00000:Program abort called in file ``../../../IBAMR/ibtk/lib/../src/utilities/HierarchyIntegrator.cpp'' at line 258
P=00000:ERROR MESSAGE:
P=00000:IBHierarchyIntegrator::advanceHierarchy():
P=00000:  at time = 1.04938e-06: time step size dt = 3.43707e-15 is zero to machine precision.
P=00001:Program abort called in file ``../../../IBAMR/ibtk/lib/../src/utilities/HierarchyIntegrator.cpp'' at line 258
P=00001:ERROR MESSAGE:
P=00001:IBHierarchyIntegrator::advanceHierarchy():
P=00001:  at time = 1.04938e-06: time step size dt = 3.43707e-15 is zero to machine precision.

The result of D attached in picture followed. U in face 0 was set as u = 2.0 and U in face 1 is set as ∂u/∂n =0. But there seems some errors.

[Boyce Griffith]

These errors and results suggest that you are exceeding the largest stable time step size for your problem set up.

In IBAMR, both IBMethod and IBFEMethod use conditionally stable explicit time stepping, which means that there is a largest stable time step size. If you exceed the largest stable time step size, then there is an exponential blow up in the energy, and that can look like the structure “exploding”. The flow field visualization suggests that may be happening in your model.

Setting ERROR_ON_DT_CHANGE to TRUE gives an error message when the time step size changes — which, for stiff elastic structures, can be an indication that the model is unstable at the given time step size. Setting ERROR_ON_DT_CHANGE to FALSE lets the code dynamically determine the time step size based on an advective CFL condition and some other parameters — but if the model has become unstable, using a CFL-type condition to determine the time step size is not reliable. This is why most of the IB and IBFE examples have ERROR_ON_DT_CHANGE set too TRUE.

I do not know an a priori way to determine the largest stable time step size, but for elastic bodies, the scaling is roughly dt ~ 1/sqrt(stiffness) if you hold all other physical and numerical parameters fixed. I usually use the method of bisection to determine an approximately optimal time step size.

You usually can use larger time step sizes with “smoother” IB kernel functions, although depending on the model, you may realize lower accuracy with those smoother kernels — see https://arxiv.org/abs/2105.14536.

— Boyce

-- 
You received this message because you are subscribed to the Google Groups "IBAMR Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ibamr-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ibamr-users/60b504b0-50a2-433d-b4b7-23d1aa686ad2n%40googlegroups.com.
<下载.png>

[Yicheng Chen]

Thank you for you help, professor! These word helps me a lot.

According your guidance, I checked the program by decreasing the time step. It does appear to be the same as what you described, and an abnormal source items appear in the flow field due to my wrong setting (thus it induced the "exploding"). 

It is really strange that the pressure becomes strange from time step 2. The flow goes right in step one, but reverse its dirction each time step. I am not sure what induced this error. What do you think this error is more likely to come from? The .CPP file?  Or, more likely, induced by the input file. 

For convenience, I have attached the corresponding CPP file and input file.

    time step 1

 time step 2

 time step 3

[Boyce Griffith]

On Sep 3, 2021, at 10:37 AM, Yicheng Chen <cychi...@gmail.com> wrote:

Thank you for you help, professor! These word helps me a lot.

According your guidance, I checked the program by decreasing the time step. It does appear to be the same as what you described, and an abnormal source items appear in the flow field due to my wrong setting (thus it induced the "exploding"). 

It is really strange that the pressure becomes strange from time step 2. The flow goes right in step one, but reverse its dirction each time step. I am not sure what induced this error. What do you think this error is more likely to come from? The .CPP file?  Or, more likely, induced by the input file. 

These could be reflections (if the structure is very stiff) or the signs of a growing instability. Notice that these pressures are gigantic. If the magnitude of the pressure makes sense given the units that you are using, and if this simulation remains stable, you might try gradually turning on the flow instead of starting it all at once.

— Boyce

For convenience, I have attached the corresponding CPP file and input file.

 <1.png>   time step 1

<2.png> time step 2

To view this discussion on the web visit https://groups.google.com/d/msgid/ibamr-users/c0610d1e-173a-430c-bf61-56a564a900c8n%40googlegroups.com.
<input2d><3.png><2.png><example.cpp><1.png>

[Yicheng Chen]

Hi professor,

I test most parameters in "input2d"  this weekend. With a decrease of flow rate, the simulation could run slightly better than before, but the error still occurs. However, the fluid mass density shows a strong dominance over computational stability. When I input air parameters like "ρ = 1.18 kg/m3 && ν = 1.68e-5 m2/s && dt = 0.0125*CFL_MAX/U_MAX ", time step size change encountered. But this could be elimated in "ρ = 11.8 kg/m3 && ν = 1.68e-5 m2/s && dt = 0.5*CFL_MAX/U_MAX ".

Do we applied density parameter in N-S solver or IB solver? As I know, the N-S solvers usually dimensionalize fluid density in incompressible simulation, and the key cannot be found in .cpp files. Does the IB solver brings this limitation? If possible, could you please tell me where I can learn more about the calculation process of our program.  

-Yicheng Chen

[Boyce Griffith]

On Sep 5, 2021, at 11:08 PM, Yicheng Chen <cychi...@gmail.com> wrote:

Hi professor,

I test most parameters in "input2d"  this weekend. With a decrease of flow rate, the simulation could run slightly better than before, but the error still occurs. However, the fluid mass density shows a strong dominance over computational stability. When I input air parameters like "ρ = 1.18 kg/m3 && ν = 1.68e-5 m2/s && dt = 0.0125*CFL_MAX/U_MAX ", time step size change encountered. But this could be elimated in "ρ = 11.8 kg/m3 && ν = 1.68e-5 m2/s && dt = 0.5*CFL_MAX/U_MAX “.

It sounds like at your time step size and your original mass density, your material stiffness is about a factor of 10 too large for the time step size that you are trying to use.

Do we applied density parameter in N-S solver or IB solver? As I know, the N-S solvers usually dimensionalize fluid density in incompressible simulation, and the key cannot be found in .cpp files. Does the IB solver brings this limitation?

You can use any consistent system of units that you wish. The code does not nondimensionalize, nor does it assume or require the use of nondimensional variables.

— Boyce

To view this discussion on the web visit https://groups.google.com/d/msgid/ibamr-users/f520f9cb-9bc4-40df-a4ef-fb21b62ba1c5n%40googlegroups.com.

[Yicheng Chen]

Thank you so much, professor! 

With a 0.01 factor applied in kappa_s, it works much better than before. 

I will continue to push forward my work.

-Yicheng Chen