Using periodic boundary conditions with parallell processing

[Raghunandan Pratoori]

Hello team,

I am running a 3D simulation with periodicity in all 3 directions.

I have defined the periodic boundary conditions using (similarly in the other 2 directions) -

DoFTools::make_periodicity_constraints(dof_handler,
                                            /*b_id*/ 0,
                                            /*b_id*/ 1,
                                            /*direction*/ 0,
                                            constraints);

and the deformation of the box using -

IndexSet selected_dofs_x;
       std::set< types::boundary_id > boundary_ids_x= std::set<types::boundary_id>();
               boundary_ids_x.insert(0);

       DoFTools::extract_boundary_dofs(dof_handler,
                                      fe.component_mask(y_displacement),
                                           selected_dofs_x,
                                           boundary_ids_x);
       unsigned int nb_dofs_face_x = selected_dofs_x.n_elements();
       IndexSet::ElementIterator dofs_x = selected_dofs_x.begin();

       double relative_displacement_x = 0.0;

       if(timestep<=020)
        relative_displacement_x = 5e-4;
       else if(timestep%02==0)
        relative_displacement_x = 2e-4;
       else
        relative_displacement_x = 0.0;

       for(unsigned int i = 0; i < nb_dofs_face_x; i++)
       {
        constraints.add_line (*dofs_x);
         constraints.set_inhomogeneity(*dofs_x, (apply_dirichlet_bc ? relative_displacement_x : 0.0));
           dofs_x++;
       }

This works fine when run using a single process. But when increased to 2 processes, it pauses after few time steps without any error message. When used more than 2 processes, it does not converge in the first time step itself. I am not sure what is causing the problem.

I also tested it with step-45. It works fine with 1 and 2 processes. But when 4 processes are used, it gives out the following error:

[100%] Built target step-45
Refinement cycle 0
   Assembling...
   Computing preconditioner...
   Solving...

----------------------------------------------------
Exception on processing:


----------------------------------------------------
Exception on processing:

--------------------------------------------------------
An error occurred in line <457> of file </usr/include/deal.II/lac/solver_cg.h> in function
    void dealii::SolverCG<VectorType>::solve(const MatrixType&, VectorType&, const VectorType&, const PreconditionerType&) [with MatrixType = Step45::SchurComplement<dealii::TrilinosWrappers::PreconditionJacobi>; PreconditionerType = dealii::TrilinosWrappers::PreconditionAMG; VectorType = dealii::TrilinosWrappers::MPI::Vector]
The violated condition was:
    false
Additional information:
Iterative method reported convergence failure in step 65. The residual in the last step was 4.56146e-06.

This error message can indicate that you have simply not allowed a sufficiently large number of iterations for your iterative solver to converge. This often happens when you increase the size of your problem. In such cases, the last residual will likely still be very small, and you can make the error go away by increasing the allowed number of iterations when setting up the SolverControl object that determines the maximal number of iterations you allow.

The other situation where this error may occur is when your matrix is not invertible (e.g., your matrix has a null-space), or if you try to apply the wrong solver to a matrix (e.g., using CG for a matrix that is not symmetric or not positive definite). In these cases, the residual in the last iteration is likely going to be large.

I would appreciate any pointers on the possible mistakes I might be making.

Best regards,

Raghunandan.

[peterrum]

Dear Raghunandan,

I don't quite understand what you want to accomplish. You are setting PBC constraint between the left and right face and after that you apply DBC on the left face? Is that about right?

My first guess would be that you constraints are somewhere not globally consistent. Maybe you could try to check them with AffineConstraints::is_consistent_in_parallel(). There is also the function AffineConstraints::make_consistent_in_parallel(), which might also help. Latter is quite a new function so we might have missed some corner cases.

Hope this helps,

PM

[Raghunandan Pratoori]

Hi Peter,

Thanks for your reply. I have PBC in all the 3 directions ( I have included the code snippet above for only 1 direction). In addition to this, I want to apply compression in 1 direction.

I checked the constraints using AffineConstraints::is_consistent_in_parallel() and it returned 1. So, that does not seem to be the issue. I found this problem with multiple processes even with the step-45 example. Everything works fine when I have DBC instead of PBC.

Thanks in advance,

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/442c4f61-d509-4875-aa4f-5b8eb10d5ae9n%40googlegroups.com.

--

Raghunandan Pratoori

Graduate Student

Aerospace Engineering | Iowa State University

2048 Howe Hall | 515-708-6767

[Jean-Paul Pelteret]

Hi Raghunandan,

Quick question: This sort of setup sounds like something that you would do for homogenisation problems… Have you fixed your periodicity frame? (This amounts to completely fixing one point to remove rigid body translations, and partially constraining n-dim other points to remove rigid body rotations. Without doing so, the solution is “floating” and the convergence of the linear solver may stagnate. See the article below if you need some explanation as to what I mean about the periodicity frame.) 

@Article{Miehe2003a,

  author    = {Miehe, C.},

  title     = {Computational micro-to-macro transitions for discretized micro-structures of heterogeneous materials at finite strains based on the minimization of averaged incremental energy},

  journal   = {Computer Methods in Applied Mechanics and Engineering},

  year      = {2003},

  volume    = {192},

  number    = {5--6},

  pages     = {559--591},

  month     = jan,

  doi       = {10.1016/s0045-7825(02)00564-9},

}

Best,

Jean-Paul

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/CALFC2WBLhY%2BMHHS4Xv7FeidW3Vxs8Puga7rwfYBTe7mKi9dr-A%40mail.gmail.com.