Evaluating FE-solution on distributed mesh, semi-Lagrangian method

[Konrad Simon]

Dear all,

I am currently working on a semi-Lagrangian method for an advection-diffusion equation. During runtime I must evaluate the (known) FE-solution at a previous time step but the mesh is distributed.

Problem: It can well be that I must evaluate the solution at a point that is in a cell that is not locally owed by the processor requesting that value or not even a ghost cell. I am aware of the FEFieldFunction class and that it says that an exception will be thrown if the point is found to be in an artificial cell. 

Now, does anyone here have experience with that? Am I doomed and will I have to manually MPI communicate such points to any processor and test if they own the relevant cell?

Any help would be much appreciated.

Best,
Konrad

[Bruno Turcksin]

Konrad,

Unfortunately, you will need to do the communication yourself. You can only evaluate the solution on cells that locally owned or on ghost cells.

Best,

Bruno

[Konrad Simon]

Thank you, Bruno. What is sometimes done for semi-Lagrangian methods is that one defines a halo region around the locally owned cells (like ghost cells) that contains information from previous time steps and then limits the time step so that one never leaves the halo region when asking for information from previous time steps. Is there a way in Deal.ii to control the size of the halo region. In my case I have pretty uniform grids if it helps.

Best,
Konrad

[Bruno Turcksin]

Konrad,

There is nothing out of the box. However, deal.II uses p4est which can
use more that one layer of ghost cells. So you should take a look
there to see how hard it is to change the code. If that's the way you
want to go, I am sure that someone will be able to provide you with
some guidance.

Best,

Bruno

> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/9li9twVxrGE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/bfc6dd81-abb3-421c-8f6c-539c234b624f%40googlegroups.com.

[Konrad Simon]

Thank you, Bruno. :-)

On Tuesday, May 12, 2020 at 2:50:35 PM UTC+2, Bruno Turcksin wrote:

Konrad,

There is nothing out of the box. However, deal.II uses p4est which can
use more that one layer of ghost cells. So you should take a look
there to see how hard it is to change the code. If that's the way you
want to go, I am sure that someone will be able to provide you with
some guidance.

Best,

Bruno

Le mar. 12 mai 2020 à 06:04, Konrad Simon <ksimo...@gmail.com> a écrit :
>
> Thank you, Bruno. What is sometimes done for semi-Lagrangian methods is that one defines a halo region around the locally owned cells (like ghost cells) that contains information from previous time steps and then limits the time step so that one never leaves the halo region when asking for information from previous time steps. Is there a way in Deal.ii to control the size of the halo region. In my case I have pretty uniform grids if it helps.
>
> Best,
> Konrad
>
> On Monday, May 11, 2020 at 2:49:21 PM UTC+2, Bruno Turcksin wrote:
>>
>> Konrad,
>>
>> Unfortunately, you will need to do the communication yourself. You can only evaluate the solution on cells that locally owned or on ghost cells.
>>
>> Best,
>>
>> Bruno
>
> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/9li9twVxrGE/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to dea...@googlegroups.com.

[Konrad Simon]

Dear all,

I finally had time to go back to my code. Quick problem summary: I am currently working on a semi-Lagrangian method for an advection-diffusion equation with a multiscale method that I designed for advection dominated processes. During runtime I must evaluate the (known) multiscale FE-solution object at a previous time step. 

Problem: The mesh is distributed and the object that will give me the evaluated data at a point p is possibly owned on another processor. The processor that owes the object is the same as the one owing the cell that surrounds p. To find the correct cell on the same processor that needs the information I am doing things similarly as the FEFieldFunction class to find the cell. But what if p is in a cell that is not locally owned? It could be in a ghost cell or an artificial one (also ghosts no not owe the data I need).

Now, I have have to manually MPI communicate such points to find out the MPI rank that owes the cell surrounding p as I found out with your help. Any way of doing this efficiently? I would like to avoid the brute force version: communicating all points to all ranks and search the owned cells. (Hint: Mostly (but not always) p will be in a ghost layer. )

Any way to get the actual MPI rank of the owner?

Any help would be much appreciated.

Best,
Konrad

[heena patel]

Hi Konrad,

                        I have the following suggestion. You can use

(1) https://www.dealii.org/current/doxygen/deal.II/p4est__wrappers_8cc.html check vertices_with_ghost_neighbors.

(2)p4est_search_local check link https://github.com/cburstedde/p4est/blob/master/src/p4est_search.h and video by Carsten https://www.youtube.com/watch?v=VHcebHg8gK8 start from 49 min.

(3) extract_rtree_level() https://www.dealii.org/current/doxygen/deal.II/rtree_8h.html#af9bdb0c5cd6bf687d9d04c1834c7c121

(4) compute_index_owner()

https://www.dealii.org/current/doxygen/deal.II/namespaceUtilities_1_1MPI.html#a19d5bdd546cc6440259a4df6f945f3d9

https://www.dealii.org/current/doxygen/deal.II/mpi__compute__index__owner__internal_8h.html

Regards,

Heena

You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/1b60b7a8-361a-4f2f-9216-50190ab7e950n%40googlegroups.com.