Question on resolving chains of constraints containing both periodic and hanging node constraints

[Sambit Das]

Hi All,

I have a question about resolving chains of constraints in a ConstraintMatrix class object after calling close() in a parallel distributed case:

I created a ConstraintMatrix with both periodic and hanging node constraints, and called close().

Then I picked a constrained degree of freedom, lets say with global dof id = “i” and printed the constraint equation pairs (j,a_ij) corresponding to “i” on the processor for which “i” is locally owned as well as the processors for which “i” is a ghost. I expected the constraint equation to be the same for owning processor and the ghost processor, however we have encountered a case where printing the constraint equation entries shows different “j” for owning and ghost processorsalthough a_ij are same. When we printed the coordinates of “j” which were different, we found that those two nodes to be periodic images of each other.

Should I except the constraint equation to be the same for the owning processor and the ghost processors?

Thank you,

Sambit

[Wolfgang Bangerth]

> I created a ConstraintMatrix with both periodic and hanging node constraints,
> and called close().
>

> Then I pickeda constrained degree of freedom, lets say with global dof id =

> “i” and printed the constraint equation pairs (j,a_ij) corresponding to “i” on
> the processor for which “i” is locally owned as well as the processors for
> which “i” is a ghost. I expected the constraint equation to be the same for

> owning processor and the ghost processor, howeverwe have encountered a case

> where printing the constraint equation entries shows different “j” for owning
> and ghost processorsalthough a_ij are same. When we printed the coordinates of
> “j” which were different, we found that those two nodes to be periodic images
> of each other.
>
>
> Should I except the constraint equation to be the same for the owning
> processor and the ghost processors?

Yes, they should really be the same. Or, more correctly, if two processors
both store the constraints for a node, they better be the same. On the other
hand, of course not every processor will know every constraint.

Can you try to construct a minimal testcase for what you observe?

Best
Wolfgang


--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Sambit Das]

Dear Prof. Bangerth,

Yes, they should really be the same. Or, more correctly, if two processors 
both store the constraints for a node, they better be the same. On the other 
hand, of course not every processor will know every constraint. 

Thanks you for clarifying this.

 Can you try to construct a minimal testcase for what you observe? 

Yes, I am going to construct a minimal test case .

Best,

Sambit 

[Sambit Das]

Dear Prof. Bangerth,

I have now reproduced the above issue in the attached minimal example.

Below is the algorithm of the minimal example 

1) Create a hypercube (-20,20) with origin at the center

2) Set periodic boundary conditions on all faces of the hypercube

3) Refine mesh by first doing global refinement once to get 8 cells and then refine 

    the cell containing the corner (-20,-20,-20) two times iteratively. Finally I get 71 cells (see attached image)

    with hanging nodes on three faces.

4) Create constraint matrix with both hanging node and periodic constraints, and call close().

5) Print the constraint equation (j,a_ij) for global dof id-52 on processors for which global dof id-52 is relevant, when run on two mpi tasks:

   $ mpirun -n 2 ./minimalExample

   number of elements: 71

   taskId: 1, globalDofId-i: 52, coordinates-i: -20 20 -10, globalDofId-j: 16, coordinates-j: -20 -20 -10, scalarCoeff-aij: 1

   taskId: 0, globalDofId-i: 52, coordinates-i: -20 20 -10, globalDofId-j: 32, coordinates-j: 20 -20 -10, scalarCoeff-aij: 1

  

   Clearly "j" in the constraint equation is different across processors for the same constrained global dof id.

Thank you,

Sambit

[Denis Davydov]

Thanks for the MWE, Sambit.

I created a Github issue to track this further https://github.com/dealii/dealii/issues/7053 

Denis.

[Sambit Das]

Thank you, Denis.

Best,

Sambit