Skip to first unread message
Assigned to lik...@wp.pl by inz.karol...@gmail.com
Karol Lewandowski
Oct 6, 2021, 10:20:05 AM10/6/21
to MoFEM Q&A
I am building a multi-level fieldsplit solver, starting with splitting the fields.
Here is my param file:
-ksp_monitor
# plastic strains
-pc_type fieldsplit
-pc_fieldsplit_type schur
-fieldsplit_0_ksp_type preonly
-fieldsplit_0_ksp_monitor
-fieldsplit_0_pc_type lu
-fieldsplit_0_pc_factor_mat_solver_type mumps
-fieldsplit_1_pc_type fieldsplit
-fieldsplit_1_pc_fieldsplit_type schur
-fieldsplit_1_ksp_type preonly
-fieldsplit_1_ksp_monitor
# plastic multiplier
-fieldsplit_1_fieldsplit_0_pc_type lu
-fieldsplit_1_fieldsplit_0_ksp_type preonly
-fieldsplit_1_fieldsplit_0_pc_factor_mat_solver_type mumps
-fieldsplit_1_fieldsplit_0_ksp_monitor
# boundary
-fieldsplit_1_fieldsplit_1_pc_type lu
-fieldsplit_1_fieldsplit_1_ksp_type preonly
-fieldsplit_1_fieldsplit_1_ksp_monitor
and the output is
] <inform> [petsc] 0 SNES Function norm 8.276359569249e+02 [ 8.276359569249e+02 , 6.166468273909e-07 , 0.000000000000e+00 ]
[0] <inform> [petsc] 0 KSP Residual norm 9.641039121356e+00
[0] <inform> [petsc] 1 KSP Residual norm 4.844583274202e-14
[0] <inform> [petsc] 1 SNES Function norm 3.515207777594e+02 [ 3.515207777594e+02 , 6.366138602407e-07 , 1.530979979816e-13 ]
[0] <inform> [petsc] 0 KSP Residual norm 4.252398977796e+00
[0] <inform> [petsc] 1 KSP Residual norm 1.021194767660e-14
[0] <inform> [petsc] 2 SNES Function norm 5.196932266325e+01 [ 5.196932266325e+01 , 2.300586250842e-07 , 1.949889382638e-14 ]
Does anyone have an idea, why do I see only two KSP iterations? There should be tree of iterations. Am I missing some parameters?
Karol Lewandowski
Oct 6, 2021, 10:40:01 AM10/6/21
to MoFEM Q&A
I think ksp_type has to be set to fgmres or gmres, to be able to see the residuals properly
Anyway, setting the boundary problem as the last split, is still not straightforward to implement.
Lukasz Kaczmraczyk
Oct 6, 2021, 10:50:25 AM10/6/21
to MoFEM Q&A
Karol,
You set
-fieldsplit_1_pc_type fieldsplit
-fieldsplit_1_pc_fieldsplit_type schur
-fieldsplit_1_ksp_type preonly
-fieldsplit_1_ksp_monitor
so that means that on the second level you kick the strat field preconditioner on the second level. You can see KSP iteration on second level, which in KSP iterations if you do
-fieldsplit_1_pc_type fieldsplit
-fieldsplit_1_pc_fieldsplit_type schur
-fieldsplit_1_ksp_type fgmres
-fieldsplit_1_ksp_monitor
L.
Lukasz Kaczmraczyk
Oct 6, 2021, 10:54:21 AM10/6/21
to MoFEM Q&A
To add, your problem is here,
-fieldsplit_1_ksp_type preonly you do not run KSP iterations on this level, just apply a preconditioner.
Karol Lewandowski
Oct 6, 2021, 11:04:11 AM10/6/21
to MoFEM Q&A
Thank you, Lukasz. Now it is clear.
So I try to apply the last boundary fieldsplit preconditioner using DM_tau and this is what I get
[0] <inform> [petsc] 0 SNES Function norm 4.102663298115e+01 [ 4.102663298115e+01 , 0.000000000000e+00 , 0.000000000000e+00 ]
[0] <inform> [petsc] Residual norms for fieldsplit_1_fieldsplit_1_fieldsplit_0_ solve.
[0] <inform> [petsc] 0 KSP Residual norm 4.088320778375e+01
[0] <inform> [petsc] 0 SNES Function norm 1.026654709890e+01 [ 1.026654709890e+01 , 0.000000000000e+00 , 0.000000000000e+00 ]
[0] <inform> [petsc] Residual norms for fieldsplit_1_fieldsplit_1_fieldsplit_0_ solve.
[0] <inform> [petsc] 0 KSP Residual norm 1.023065622906e+01
and the rest of my param_file:
# boundary
-fieldsplit_1_fieldsplit_1_pc_type lu
-fieldsplit_1_fieldsplit_1_ksp_type preonly
-fieldsplit_1_fieldsplit_1_pc_factor_mat_solver_type mumps
# -fieldsplit_1_fieldsplit_1_ksp_monitor
-fieldsplit_1_fieldsplit_1_pc_type fieldsplit
-fieldsplit_1_fieldsplit_1_pc_fieldsplit_type multiplicative
-fieldsplit_1_fieldsplit_1_fieldsplit_0_pc_type lu
-fieldsplit_1_fieldsplit_1_fieldsplit_0_ksp_type fgmres
-fieldsplit_1_fieldsplit_1_fieldsplit_0_ksp_monitor
-fieldsplit_1_fieldsplit_1_fieldsplit_0_pc_factor_mat_solver_type mumps
-fieldsplit_1_fieldsplit_1_fieldsplit_1_pc_type lu
-fieldsplit_1_fieldsplit_1_fieldsplit_1_ksp_type fgmres
-fieldsplit_1_fieldsplit_1_fieldsplit_1_pc_factor_mat_solver_type mumps
-fieldsplit_1_fieldsplit_1_fieldsplit_1_ksp_monitor
Lukasz Kaczmraczyk
Oct 6, 2021, 3:46:39 PM10/6/21
to MoFEM Q&A
Hi Karol,
1) run code with option -h -log_trace, it will show you all the options, It will show you at exactly what point code is breaking. Can you upload log file after you run -log_trace?
2) I will check what will happen, when you use -pc_type jacobi. Mumps can fail because of memory, or the matrix is singular.
L.
Karol Lewandowski
Oct 6, 2021, 3:48:16 PM10/6/21
to MoFEM Q&A
I will do that, but I started to think now how and when should I assemble the boundary contributions.
Karol Lewandowski
Oct 7, 2021, 6:30:03 AM10/7/21
to MoFEM Q&A
Little update, the field split is working. The only issue I get now is when I try to setup the last split in parallel
Karol Lewandowski
Oct 7, 2021, 8:35:07 AM10/7/21
to MoFEM Q&A
CHKERR PCSetUp(tau_pc);
[0]PETSC ERROR: Argument out of range
[0]PETSC ERROR: key 1416 is greater than largest key allowed 795
I get this error when I try to SetUp second preconditioner (for TAU) in parallel computations.
Lukasz Kaczmraczyk
Oct 7, 2021, 8:43:13 AM10/7/21
to MoFEM Q&A
Karol,
I hard to help here. But is clear that you have a problem with indices. Possibly the size of Schur complement and your Matrix in sub-problem does not match.
L.
Karol Lewandowski
Oct 7, 2021, 8:45:45 AM10/7/21
to MoFEM Q&A
but why only in parallel? I checked if in Eshelbian plasticity you do anything special for parallel computations and I have not found anything specific.
Lukasz Kaczmraczyk
Oct 7, 2021, 8:51:37 AM10/7/21
to MoFEM Q&A
hard to say,
Karol Lewandowski
Oct 7, 2021, 10:20:51 AM10/7/21
to MoFEM Q&A
What to do if boundary elements for fieldsplit are only on one core, but not others?
Lukasz Kaczmraczyk
Oct 7, 2021, 10:21:46 AM10/7/21
to MoFEM Q&A
That should not be a problem.
Reply all
Reply to author
Forward
0 new messages