new examples

[Stephane Popinet]

Dear all,

As promised, I have just published two new examples:

http://basilisk.fr/src/examples/atomisation.c
http://basilisk.fr/src/examples/isotropic.c

They show how to run large parallel (MPI) simulations for multiphase
flows with adaptivity, load-balancing etc... (first example), or large
parallel, regular Cartesian grid simulations (second example).

enjoy,

Stephane

[Antoon van Hooft]

Dear Stephane, 

Cool! great work. I am eager to run basilisk with MPI. I am new to MPI as the following may confirm:

I have not been able to run MPI on my machine, i think because i simply do not know howto.

In the new examples the qcc command is not used anymore?  Does this mean i have to recompile basilisk, as currently the config file links to config.gcc. Does th still makes sense?

I did install openmpi following this page:
http://lsi.ugr.es/~jmantas/pdp/ayuda/datos/instalaciones/Install_OpenMPI_en.pdf

then i tried this (note: this code works fine with qcc compilation and -fopenmp flag)

CC99='mpicc -std=c99' qcc -Wall -O2 -D_MPI=1 mycode.c -o mpitest -lm

then i only get a warning as discussed here (absent for qcc compilation):
 https://groups.google.com/forum/#!/basilisk-fr/yGedVYOCz8E

then i do

mpirun -np x ./mpitest

However, depending on the number ' x' i get different results/errors. for x=1, the code runs but i get " Convergence not reached" warnings
for x=2 the convergence errors do not appear but code crashes after a few timesteps. for x>2 the code crashes after the first timestep. (all with somewhat different MPI related errors). 

i have a quadcore proc. with HT

When i try the compilation method from the examples, it cannot compile because it cannot find the "#include ...." files (navierstokes for example)

suggestions are welcome

[Stephane Popinet]

Hi Antoon,

Did you read:

http://www.basilisk.fr/src/Tips#parallel-runs-with-mpi

Stephane

[Antoon van Hooft]

Hello Stephane,

This time, i actually did read this page already, but thus gave me also an unwanted result. i end up running the same simulation multiple (np) times (also called parallel).

antoon@antoon-XPS-15-9550:~/basilisk/src/Chielcase$ CC='mpicc -D_MPI=1' qcc chielLES.c -o example -lm
antoon@antoon-XPS-15-9550:~/basilisk/src/Chielcase$ mpirun -np 8 ./example
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738
3D    t=0    i=0    DT=42.2738

However, i reinstalled my open MPI packages and now it works via the method below. sorry for my confusion. 

antoon@antoon-XPS-15-9550:~/basilisk/src/Chielcase$ CC99='mpicc -std=c99' qcc -Wall -O2 -D_MPI=1 chielLES.c -o mpitest -lm
/home/antoon/basilisk/src/grid/fpe.h: In function ‘gdb’:
/home/antoon/basilisk/src/grid/fpe.h:16:3: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
   system (command);
   ^
antoon@antoon-XPS-15-9550:~/basilisk/src/Chielcase$ mpirun -np 4 ./mpitest
3D    t=0    i=0    DT=42.2738
3D    t=1    i=1    DT=10.5665

Thank you, now  I hope to be able update my sandbox soon.
gr

Antoon

[Stephane Popinet]

Hi Antoon,

You are right, the qcc compilation should be something like:

C99='mpicc -std=c99' qcc -Wall -O2 -D_MPI=1 example.c -o example -lm

I will fix the 'TIPS' page.

cheers

Stephane

[Antoon van Hooft]

Dear Stephane,

I have a question regarding the "isotropic.c" example. I think i do not understand the stencil definitions correctly. In the example turbulence is forced with the following acceleration term (av):

 foreach_face() 
    av.x[] += 0.1*((u.x[] + u.x[-1])/2. - ubar.x);

with ubar of type "coord". I do not understand how this is consistent with the stencil definitions: both av.x and u.x are values that live on faces, correct? then why take the average of u.x[ ] and u.x[-1]. I understood that u.x[ ] and av.x[ ] live at the same face location. this average value (with u.x[-1]) would be an first order approximation of its 'centered' located value. If i assume that the code above is the correct way of implementation than i do not understand why i implement the following buoyancy (in y direction) term with centered scalar field T the following way:

 

  coord Del = { 0 , 1, 0};
  foreach_face()
    {
      av.x[] = Del.x* (((T[]+T[-1])/2)

Here T[] is a centered scalar field. I understood that the average of T[] and T[-1] would be the first order approximation of the value at the face location where av.x lives?

thanks in advance

[Stephane Popinet]

> I do not understand how this is consistent
> with the stencil definitions: both av.x and u.x are values that live on
> faces, correct?

no, av lives on faces but u is centered, see:

http://www.basilisk.fr/src/navier-stokes/centered.h

(it's why it's called 'centered.h').

> Here T[] is a centered scalar field. I understood that the average of
> T[] and T[-1] would be the first order approximation of the value at the
> face location where av.x lives?

Yes, that is correct (although I would say "second order").

What may be confusing is that indeed 'u' (the velocity) and 'a' (the
acceleration) are not defined at the same location. This is necessary to
ensure balance between the acceleration and the pressure gradient (which
is also defined on faces).

See also the comments here:

http://www.basilisk.fr/src/navier-stokes/centered.h#acceleration-term

cheers

Stephane

[Antoon van Hooft]

Dear Stephane,

Thank you, this is important information. Apparently i have been implementing a damping layer (opposite of the forcing term) incorrectly. I now realize this may have caused strange behavior of the adaption that I earlier related to gravity waves.

I think i was confused because I assumed general validity of the following figure:

from JFM's tutorial in the sand box. It is stated there that u is a centered field)

Sorry for all the off topic discussions