Issue of extrude_triangulation for distributed mesh

[zhaowen...@gmail.com]

Dear All,

I am a new for deal.ii. I am tring to test the step-28, an example of neutron diffusion problem. In step-28 there are only four assemblies(each assembly consists of 17 by 17 fuel pins) in core and the serial mesh is enough for this case.

In my case the core consists of about 200 assemblies, and the number of cells are 200(assemblies) * 17*17(fuel pins) * 100(axial z-direction). It needs huge computing resouce and memory storage.

I had changed the step-28 to a MPI parallel version.

However, I found that the mesh is not distributed to all process, which is still created using "subdivided_hyper_cube".

Because fuel pins are different between each other in radial x-y plane, the mesh could not be coaser in radial x-y plane. Fuel pins are almost similar in z-directions.

I found that deal.ii GridIn could not read so huge mesh and distributing. Maybe GrindIN only read radial x-y plane mesh and then use "distributed" extrude_triangulation to solve this case?

Or is there any other solutions?

I am looking forward to your suggestions!

Best

Zhao Wen-bo

[zhaowen...@gmail.com]

Dear All,

I found the similar case in link below

https://groups.google.com/forum/#!searchin/dealii/Question$20regarding$20P$3AD$3ATria/dealii/m_IEKPeJhaM/jua7VHonBAAJ

The radial mesh can't be coarsen because of cell-wise material. 
Could this question be solved using link below

https://github.com/dealii/dealii/pull/3956

Thanks

BEST,

ZHAO Wen-bo


在 2017年3月12日星期日 UTC+8下午2:27:53，zhaowen...@gmail.com写道：

[Bruno Turcksin]

Zhao,

here is what you can do. Create a mesh with only one cell and then do global/adaptive refinements until you get the mesh that you want. Now that you have the right mesh, you can assign the material ID and the boundary ID to every cells. The mesh will be totally distributed. There is only thing that you need to be careful about, if you refine the mesh or move the cells on a different processors, you will need to reassign the IDs to all the cells (see http://dealii.org/8.4.1/doxygen/deal.II/classparallel_1_1distributed_1_1Triangulation.html). If every processors knows the "real" coarse mesh, you wouldn't have this problem.

> The radial mesh can't be coarsen because of cell-wise material. 
> Could this question be solved using link below

> https://github.com/dealii/dealii/pull/3956
No, this is not ready to be used. Once it works, then yes it would solve your problem.

Best,

Bruno

[zhaowen...@gmail.com]

Bruno,

Thanks for your reply.
I think it exisists a problem that cell numbers does not match.
For example, the radial mesh i want is with 289*289 cells, but the refinement mesh might be with 512*512 cells. Am i correct? If so, what should i do? Could i delete those nonuse mesh after refinement?

BEST,

Wenbo

[Bruno Turcksin]

Wenbo,

2017-03-12 21:52 GMT-04:00 <zhaowen...@gmail.com>:
> I think it exisists a problem that cell numbers does not match.
> For example, the radial mesh i want is with 289*289 cells, but the refinement mesh might be with 512*512 cells. Am i correct? If so, what should i do? Could i delete those nonuse mesh after refinement?

Yes, 289 is really ugly because it has very few factors but since now,
your mesh is distributed having too many cells is not as bad. But note
that you don't have to start with only one cell, if you want a mesh
with 34*34 cells, you don't need to use 64*64 cells. You can start
with a 17*17 mesh and then refine it. Also, you can use
subdivided_hyper_rectangle so that the initial mesh is not uniform and
then, do adaptive mesh refinement where you flag the cells to refine
yourself. You cannot remove the cells after refinement, you can only
do that on the coarse mesh.

Best,

Bruno

[zhaowen...@gmail.com]

Bruno,
Thanks.

I am sorry. I should give my case more details.
The reactor core consists of 17*17 assembly, which consists of 17*17 fuel pins.
If the material propeties, named cross sections, varies in assembly wise, your solution is OK.
If varies in fuel pin wise, it doesn't work in my opinion. Is it correct?
In my opinion the refinement can not be performed on only one direction.

In factor, the geometry in my case is not complex except the materiales varied on pin wise.

BEST,

Wenbo

[Bruno Turcksin]

Wenbo,

2017-03-12 23:01 GMT-04:00 <zhaowen...@gmail.com>:
> I am sorry. I should give my case more details.
> The reactor core consists of 17*17 assembly, which consists of 17*17 fuel pins.
> If the material propeties, named cross sections, varies in assembly wise, your solution is OK.
> If varies in fuel pin wise, it doesn't work in my opinion. Is it correct?

No it works. Let's say for simplicity that you just want to mesh a
single assembly. You can first create a mesh with 4 cells, i.e.
(A+B)*(A+B) with A = 16*B Now you can refine A so to get the right
number of cells. At the end, you won't get a 17*17 mesh but you have a
mesh that is distributed and you can assign a single material ID to
each cell.

Best,

Bruno

[zhaowen...@gmail.com]

Bruno,

Thanks.

I am not clear about your method.

The mesh with 4 cells is like below:

A C

D B
Then A is refined 4 times, and will be with 16*16 cells.

 And now, B, C and D with 1 cells. 

How could I make C with 16*1 cells and D with 1*16 cells?

BEST,

Wenbo

在 2017年3月13日星期一 UTC+8下午8:40:56，Bruno Turcksin写道：

Wenbo,  

[Bruno Turcksin]

Wenbo,

2017-03-13 9:19 GMT-04:00 <zhaowen...@gmail.com>:
> The mesh with 4 cells is like below:
> A C
> D B
> Then A is refined 4 times, and will be with 16*16 cells.
> And now, B, C and D with 1 cells.
> How could I make C with 16*1 cells and D with 1*16 cells?

Well you can't. You could use anisotropic refinement but it doesn't
work in parallel. So you will need to refine C and D to be 16*16 which
means that B will also be refined in the process. Like I said you
won't get a 17*17 mesh but since the mesh is distributed that's not a
problem. At the end, the mesh that you will be using will be too
refined but again this is not a problem.

Best,

Bruno

[zhaowen...@gmail.com]

Bruno,

Thanks.

Yes, you are right. Though it is not perfect, it does work.

There is not an easy and perfect solution for my case through discussion with you.

I think that pre-splitting mesh and distributing to procs looks like a more general solution to my cases. 

Though it is a hard work, it is very important for application on neutron transport simulation.

And I know Timo is developing it.

After finishing, it will be perfect!!

Thanks again!

BEST,

Wenbo

[zhaowen...@gmail.com]

Bruno,

I have one more question about your methods.

For example, the case is with 289 * 289 cells. Firstly we define coase mesh with 19*19. And then coarse mesh is refined 4 times. The fine mesh would be with 19*16 * 19*16  = 304 * 304 which coverd 289*289 cells and is with more layers near boundary. Then I can define some "virtual" material on these layers to decrease the influence of vitrual material on real 289*289 cells. It will be ok.

Now the mesh has been distributed, We assign material ID for distributed mesh.

It is true that assembly is with 17*17 squares( cell). But the step width of cells are not of same values, with tiny difference.

 step width on x-axis is like this : 1.265, 15*1.26 (repeat 15 times), 1.265

 step width on y-axis is like this : 1.265, 15*1.26 (repeat 15 times), 1.265

How to move these lines in 2-D(x-y) or planes in 3-D?

BEST,

Wenbo

在 2017年3月13日星期一 UTC+8上午8:45:12，Bruno Turcksin写道：

[Bruno Turcksin]

Wenbo,

2017-03-14 9:49 GMT-04:00 <zhaowen...@gmail.com>:
> For example, the case is with 289 * 289 cells. Firstly we define coase mesh
> with 19*19. And then coarse mesh is refined 4 times. The fine mesh would be
> with 19*16 * 19*16 = 304 * 304 which coverd 289*289 cells and is with more
> layers near boundary. Then I can define some "virtual" material on these
> layers to decrease the influence of vitrual material on real 289*289 cells.
> It will be ok.

Virtual material? This seems strange.

> It is true that assembly is with 17*17 squares( cell). But the step width of
> cells are not of same values, with tiny difference.
> step width on x-axis is like this : 1.265, 15*1.26 (repeat 15 times), 1.265
> step width on y-axis is like this : 1.265, 15*1.26 (repeat 15 times), 1.265
>
> How to move these lines in 2-D(x-y) or planes in 3-D?

Alright here is how I would do it for the 17*17 mesh, you can do
something similar for the 289*289: First you create a square using a
subdivided_hyper_rectangle
http://dealii.org/developer/doxygen/deal.II/namespaceGridGenerator.html#a0e245f7c5788b6dcfcf93279a67fdbbb
for the step_sizes, you create a vector of vector of size (2,3) with
the values 1.265, 15*1.36, 1.265. So you get a mesh with 9 cells.
Then, you refine the largest cell (you will need to do something
similar for four other cells). Now the 15*1.26 becomes 7.5*1.26 +
7.5*1.26 but you can loop over the vertices of the cells that you just
created and moved the vertices manually
(http://dealii.org/developer/doxygen/deal.II/classTriaAccessor_3_010_00_01dim_00_01spacedim_01_4.html#a2574a51c028bf1c31dabdbc77dbca7bc).
So you move the vertices to have 11*1.26 + 4*1.26. For the second cell
you can do global refinements. For the first cell, you do 11 ->
(5.5+5.5) -> (3+8). The 8*1.26 can be globally refined. The 3*1.26
becomes (1.5+1.5) -> (2+1) . This creates the mesh that you want BUT
you have to be very careful. If you move vertex 0 on processor 0 and
vertex 0 is also on processor 2, you need to move it yourself on
processor 2. If you don't the mesh will make no sense and you will get
the wrong result.

Best,

Bruno

[zhaowen...@gmail.com]

Bruno,

Thanks for your reply!

"Virtual material? This seems strange. "

We can fill with "strong absorption" material which is similar with zero boundary condition.

The method you give is great!

You mean vertex move on the same level of refinement is OK. 

But after refinement, vertex which belong to more than one procs need much more care!

" If you move vertex 0 on processor 0 and vertex 0 is also on processor 2, you need to move it yourself on 

processor 2."

How about moving the vertex of cells (both local and ghost) according to its real location on each procs? Because final refinement mesh is certain before refinment. Is it correct? It might be a crude way.

BEST,

Wenbo

[Bruno Turcksin]

Wenbo,

2017-03-14 11:21 GMT-04:00 <zhaowen...@gmail.com>:
> "Virtual material? This seems strange. "
> We can fill with "strong absorption" material which is similar with zero
> boundary condition.

So you would use strong absorption in a diffusion equation? But the
diffusion equation is derived assuming that there is a lot of
diffusion and almost not absorption :-) I guess it would work though.

> The method you give is great!
> You mean vertex move on the same level of refinement is OK.
> But after refinement, vertex which belong to more than one procs need much
> more care!

So the problem is if you move a vertex on a processor, there is no way
for the other processors to know that the vertex has been moved. So
you need to be very careful once you move the vertex (that's why we
don't advertise that is possible to do). It doesn't matter on which
level of refinement they are.

> " If you move vertex 0 on processor 0 and vertex 0 is also on processor 2,
> you need to move it yourself on
> processor 2."
> How about moving the vertex of cells (both local and ghost) according to its
> real location on each procs? Because final refinement mesh is certain before
> refinment. Is it correct? It might be a crude way.

You mean create a refined mesh and then move all the vertices at the
same time? Yes, that works too but then you can't do exactly a 17*17
mesh.

Best,

Bruno