TurbChannel example, error on running-AMG files not found, dumping data

716 views
Skip to first unread message

sohaib malik

unread,
May 28, 2021, 10:53:08 PM5/28/21
to Nek5000
Dear All,
I am trying to run TurbChannel example. The case compiled successfully. However on running nekbmpi it gives an error and stops. I have included the "logfile" below.
I saw on some previous posts about making tools "amg_setup" , "amg_hypre", it tried both, however that gives me the error
on typing ./maketools amg_setup and ./maketools amg_hypre inside ~/Nek5000/tools$ gives errors
building amg_setup ... failed
building amg_hypre ... failed.

Can anyone please guide me the way to proceed from here. I will be highly grateful.
Best Regards,
Sohaib


*******************************************************************************
/----------------------------------------------------------\\
|      _   __ ______ __ __  ______  ____   ____   ____     |
|     / | / // ____// //_/ / ____/ / __ \\/ __ \\/ __ \\   |
|    /  |/ // __/  / ,<   /___ \\ / / / // / / // / / /    |
|   / /|  // /___ / /| | ____/ / / /_/ // /_/ // /_/ /     |
|  /_/ |_//_____//_/ |_|/_____/  \\___/ \\___/ \\___/      |
|                                                          |
|----------------------------------------------------------|
|                                                          |
| COPYRIGHT (c) 2008-2019 UCHICAGO ARGONNE, LLC            |
| Version:  19.0                                           |
| Web:      https://nek5000.mcs.anl.gov                    |
|                                                          |
\\----------------------------------------------------------/

 Number of MPI ranks :          10
  
 Reading /home/mtdl/Nek5000/run/turbchannel3/turbChannel.par                                                                                 
general = [(null)]
general:polynomialorder = [7]
general:pnpnformulation = [yes]
general:minnumprocesses = [4]
general:stopat = [endTime]
general:endtime = [400]
general:dt = [0]
general:timestepper = [bdf2]
general:extrapolation = [OIFS]
general:variabledt = [yes]
general:targetcfl = [3.5]
general:writecontrol = [runTime]
general:writeinterval = [50.0]
general:userparam01 = [200]
general:userparam02 = [20]
general:filtering = [hpfrt]
general:filterweight = [10]
general:filtercutoffratio = [0.9]
problemtype = [(null)]
problemtype:equation = [incompNS]
pressure = [(null)]
pressure:preconditioner = [semg_amg]
pressure:residualtol = [1e-4]
pressure:residualproj = [yes]
velocity = [(null)]
velocity:residualtol = [1e-6]
velocity:density = [1]
velocity:viscosity = [-10000]
 Reading /home/mtdl/Nek5000/run/turbchannel3/turbChannel.re2                                                                                 
 nelgt/nelgv/lelt:        1536        1536         387
 lx1  /lx2  /lx3 :           8           8           8

 partioning elements to MPI ranks
 Reading /home/mtdl/Nek5000/run/turbchannel3/turbChannel.ma2                                                                                 
 Max neighbors: 8 | Min neighbors: 4 | Avg neighbors: 6.600000
 Max nvolume: 115 | Min nvolume: 6 | Avg nvolume: 58.600000
 Max volume: 446 | Min volume: 287 | Avg volume: 379.600000
 Max elements: 154 | Min elements: 153 | Balance: 1.006536
  
 done :: partioning   0.15970E-02 sec

  preading mesh 
  preading bc for ifld           1
 done :: read .re2 file    0.48E-02 sec

 RK4 substeps:           2
 setup mesh topology
   Right-handed check complete for        1536 elements. OK.
   setvert3d:   8      201344      533120      201344      201344
 call usrsetvert
 done :: usrsetvert

gs_setup: 55527 unique labels shared
   pairwise times (avg, min, max): 7.33485e-05 7.02196e-05 7.68486e-05
   crystal router                : 0.000217823 0.000211655 0.00022334
   all reduce                    : 0.000466757 0.000463672 0.000469364
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 691517 639068 724404
   buffer bytes (avg, min, max): 197056 148592 234176
   setupds time 5.6245E-02 seconds   0  8      201344        1536
           8  max multiplicity
 done :: setup mesh topology
  
 call usrdat
 done :: usrdat

 generate geometry data
 NOTE: All elements deformed , param(59) ^=0
 done :: generate geometry data
  
 call usrdat2
 done :: usrdat2

  8.8818E-16  1.1102E-16  4.4409E-16  8.8818E-16  1.1102E-16  4.4409E-16 xyz repair 1
  8.8818E-16  1.1102E-16  4.4409E-16  1.6624E-15  2.9172E-16  8.3122E-16 xyz repair 2
  8.8818E-16  2.2204E-16  4.4409E-16  1.1324E-15  2.2204E-16  4.4409E-16 xyz repair 3
  4.4409E-16  1.1102E-16  0.0000E+00  4.4409E-16  1.1102E-16  0.0000E+00 xyz repair 4
 regenerate geometry data           1
 NOTE: All elements deformed , param(59) ^=0
 done :: regenerate geometry data           1
  
 regenerate geometry data           1
 NOTE: All elements deformed , param(59) ^=0
 done :: regenerate geometry data           1
  
 verify mesh topology
   0.0000000000000000        6.2831853071795862       Xrange
  -1.0000000000000000        1.0000000000000000       Yrange
   0.0000000000000000        3.1415926535897931       Zrange
 done :: verify mesh topology
  
 mesh metrics:
 GLL grid spacing min/max    : 1.28E-03 8.22E-02
 scaled Jacobian  min/max/avg: 1.00E+00 1.00E+00 1.00E+00
 aspect ratio     min/max/avg: 1.02E+00 1.97E+01 6.30E+00

 IFTRAN    = T
 IFFLOW    = T
 IFHEAT    = F
 IFSPLIT   = T
 IFLOMACH  = F
 IFUSERVP  = F
 IFUSERMV  = F
 IFPERT    = F
 IFADJ     = F
 IFSTRS    = F
 IFCHAR    = T
 IFCYCLIC  = F
 IFAXIS    = F
 IFMVBD    = F
 IFMELT    = F
 IFNEKNEK  = F
 IFNEKNEKC = F
 IFSYNC    = F
   
 IFVCOR    = T
 IFINTQ    = F
 IFGEOM    = F
 IFSURT    = F
 IFWCNO    = F
   
 IFTMSH for field           1    =  F
 IFADVC for field           1    =  T
 IFNONL for field           1    =  F
   
 Dealiasing enabled, lxd=          12
  
 Estimated eigenvalues
 EIGAA =    1.2391337000907798     
 EIGGA =    1782331.7387356486     
 EIGAE =   0.25000000000000000     
 EIGAS =    1.6812821192585647E-002
 EIGGE =    1782331.7387356486     
 EIGGS =    2.0000000000000000     
  
 initialize pressure solver
 mg_nx:           1           3           7
 mg_ny:           1           3           7
 mg_nz:           1           3           7
 call usrsetvert
 done :: usrsetvert

gs_setup: 1005 unique labels shared
   pairwise times (avg, min, max): 7.53002e-06 7.19214e-06 7.73631e-06
   crystal router                : 8.37512e-06 8.18362e-06 8.71997e-06
   all reduce                    : 2.70083e-05 2.66408e-05 2.7305e-05
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 19243.2 17660 20644
   buffer bytes (avg, min, max): 6073.6 4592 7136
   setupds time 1.9648E-03 seconds   1  2        1664        1536
   setvert3d:   4       30336       42624       30336       30336
 call usrsetvert
 done :: usrsetvert

gs_setup: 9939 unique labels shared
   pairwise times (avg, min, max): 1.39416e-05 1.34318e-05 1.43573e-05
   crystal router                : 3.42916e-05 3.36364e-05 3.55694e-05
   all reduce                    : 0.000121313 0.000120565 0.000122
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 135142 124380 142868
   buffer bytes (avg, min, max): 40166.4 30320 47488
   setupds time 1.1552E-02 seconds   2  4       30336        1536
   setvert3d:   4       30336       42624       30336       30336
 call usrsetvert
 done :: usrsetvert

gs_setup: 9939 unique labels shared
   pairwise times (avg, min, max): 1.11813e-05 1.08846e-05 1.14233e-05
   crystal router                : 3.22127e-05 3.1723e-05 3.28341e-05
   all reduce                    : 0.000104974 0.000104466 0.000105343
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 135142 124380 142868
   buffer bytes (avg, min, max): 40166.4 30320 47488
   setupds time 8.5347E-03 seconds   3  4       30336        1536
   setvert3d:   6       96896      195200       96896       96896
 call usrsetvert
 done :: usrsetvert

gs_setup: 28113 unique labels shared
   pairwise times (avg, min, max): 1.57162e-05 1.52217e-05 1.60855e-05
   crystal router                : 6.09751e-05 5.9979e-05 6.24605e-05
   all reduce                    : 0.000185336 0.000184672 0.000185925
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 359234 331516 377476
   buffer bytes (avg, min, max): 103827 78320 123168
   setupds time 1.6406E-02 seconds   4  6       96896        1536
 regenerate geometry data           1
 NOTE: All elements deformed , param(59) ^=0
 done :: regenerate geometry data           1
  
 h1_mg_nx:           1           3           7
 h1_mg_ny:           1           3           7
 h1_mg_nz:           1           3           7
 call usrsetvert
 done :: usrsetvert

gs_setup: 1005 unique labels shared
   pairwise times (avg, min, max): 4.88949e-06 4.75291e-06 5.0609e-06
   crystal router                : 6.28828e-06 6.15995e-06 6.42715e-06
   all reduce                    : 2.51787e-05 2.49648e-05 2.54618e-05
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 19243.2 17660 20644
   buffer bytes (avg, min, max): 6073.6 4592 7136
   setupds time 1.4462E-03 seconds   5  2        1664        1536
   setvert3d:   4       30336       42624       30336       30336
 call usrsetvert
 done :: usrsetvert

gs_setup: 9939 unique labels shared
   pairwise times (avg, min, max): 1.11081e-05 1.08495e-05 1.13766e-05
   crystal router                : 2.9694e-05 2.9154e-05 3.0311e-05
   all reduce                    : 0.000103786 0.000103096 0.000104337
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 135142 124380 142868
   buffer bytes (avg, min, max): 40166.4 30320 47488
   setupds time 7.2923E-03 seconds   6  4       30336        1536
   setvert3d:   4       30336       42624       30336       30336
 call usrsetvert
 done :: usrsetvert

gs_setup: 9939 unique labels shared
   pairwise times (avg, min, max): 1.14842e-05 1.13531e-05 1.16308e-05
   crystal router                : 3.01681e-05 2.96975e-05 3.08862e-05
   all reduce                    : 0.000106174 0.000105841 0.000106576
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 135142 124380 142868
   buffer bytes (avg, min, max): 40166.4 30320 47488
   setupds time 6.6853E-03 seconds   7  4       30336        1536
   setvert3d:   6       96896      195200       96896       96896
 call usrsetvert
 done :: usrsetvert

gs_setup: 28113 unique labels shared
   pairwise times (avg, min, max): 1.55165e-05 1.50272e-05 1.59015e-05
   crystal router                : 6.17337e-05 6.07557e-05 6.32794e-05
   all reduce                    : 0.000196045 0.000195095 0.000197069
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 359234 331516 377476
   buffer bytes (avg, min, max): 103827 78320 123168
   setupds time 1.6103E-02 seconds   8  6       96896        1536
   setvert3d:   8      201344      533120      201344      201344
 call usrsetvert
 done :: usrsetvert

gs_setup: 55527 unique labels shared
   pairwise times (avg, min, max): 2.07367e-05 2.02475e-05 2.1439e-05
   crystal router                : 0.000116731 0.000114785 0.000119497
   all reduce                    : 0.000337297 0.000335657 0.000338303
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 691517 639068 724404
   buffer bytes (avg, min, max): 197056 148592 234176
   setupds time 3.4411E-02 seconds   9  8      201344        1536
   setvert3d:  10      343680     1130112      343680      343680
 call usrsetvert
 done :: usrsetvert

gs_setup: 92181 unique labels shared
   pairwise times (avg, min, max): 2.6399e-05 2.55616e-05 2.70362e-05
   crystal router                : 0.000192006 0.000188827 0.000196574
   all reduce                    : 0.000651193 0.000644554 0.000654292
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 1.13199e+06 1047036 1183652
   buffer bytes (avg, min, max): 319853 241136 380512
   setupds time 6.9617E-02 seconds  10 10      343680        1536
 setup h1 coarse grid, nx_crs=           2
 call usrsetvert
 done :: usrsetvert

gs_setup: 1005 unique labels shared
   pairwise times (avg, min, max): 3.98878e-06 3.85847e-06 4.0764e-06
   crystal router                : 5.45556e-06 5.32372e-06 5.58672e-06
   all reduce                    : 2.56074e-05 2.54075e-05 2.5844e-05
   used all_to_all method: pairwise
   handle bytes (avg, min, max): 19243.2 17660 20644
   buffer bytes (avg, min, max): 6073.6 4592 7136
AMG files not found, dumping data ...
AMG writing data from proc 0
AMG writing data from proc 1
AMG writing data from proc 2
AMG writing data from proc 3
AMG writing data from proc 4
AMG writing data from proc 5
AMG writing data from proc 6
AMG writing data from proc 7
AMG writing data from proc 8
AMG writing data from proc 9
AMG dump successful
  
an error occured: dying ...

stek...@gmail.com

unread,
Jun 9, 2021, 5:59:30 AM6/9/21
to Nek5000
Hi, this is my workflow for running a simulation with AMG enabled:

1. run nek
2. run nekamg_setup
3. run nek again

you only did step 1. The "an error occured: dying ..." message at the end is normal behaviour I think. Don't use amg_hypre, it was used in older versions, now it's called nekamg_setup.

see also these Threads:

hope that helps!

sohaib malik

unread,
Jun 14, 2021, 4:14:36 PM6/14/21
to Nek5000
Hi, Thank you so much for your help. I have tried to run nekamg_setup, by going inside tools and typing "./maketools nekamg_setup" but it shows "building nekamg_setup ... failed". I can't proceed from here. I am not what might be the reason. I have checked the other threads you shared and couldn't figure out this issue. Would appreciate any help. 
Thanks, Sohaib

YuHsiang Lan

unread,
Jun 14, 2021, 4:43:36 PM6/14/21
to Nek5000
Hi Sohaib,

What's the error message from the build.log?
(check    ~/Nek5000/tools/nekamg_setup/build.log)


BTW, You need cmake to compile that tool.

Hope this helps,
Yu-Hsiang
--

sohaib malik

unread,
Jun 14, 2021, 5:44:51 PM6/14/21
to Nek5000
Hi Yu-Hsiang, Thank you for your help. Following is the error message from build.log inside the nekamg_setup directory, the path you mentioned,
<
make: Entering directory '/home/mtdl/Nek5000/tools/nekamg_setup'

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
makefile:13: recipe for target 'lib' failed
make: *** [lib] Error 2
make: Leaving directory '/home/mtdl/Nek5000/tools/nekamg_setup'
>

Also, if you may please help me how to use 'cmake', do i need to first install 'cmake'.

Best Regards,
Sohaib

YuHsiang Lan

unread,
Jun 14, 2021, 5:51:45 PM6/14/21
to Nek5000
Hi Sohab,

That seems to indicate the tar file is corrupted.
Maybe something happen when downloading the hypre.

Can you try
  cd /tools/nekamg_setup/hypre; rm *tar.gz
  cd /tools/nekamg_setup/; make clean

Then re-install again with
  ./maketools nekamg_setup

Hope this helps,
Yu-Hsiang
--

sohaib malik

unread,
Jun 14, 2021, 6:08:12 PM6/14/21
to Nek5000
I have tried the same. But once again it shows failure to make tools nekamg_setup. I have attached both terminal window output and build.log. Build.log suggests tar file is not missing at one of the websites.


Terminal window output
<
mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools/nekamg_setup$ cd hypre

mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools/nekamg_setup/hypre$ ls
hypre  install  v2.15.1.tar.gz

mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools/nekamg_setup/hypre$ rm *tar.gz

mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools/nekamg_setup/hypre$ ls
hypre  install

mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools/nekamg_setup$ make clean
mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools/nekamg_setup$ ls
amg_setup.c  amg_setup.h  build.log  hypre  makefile

mtdl@mtdl-Precision-5820-Tower-X-Series:~/Nek5000/tools$ ./maketools nekamg_setup
building nekamg_setup ... failed
>

build.log output
<
make: Entering directory '/home/mtdl/Nek5000/tools/nekamg_setup'
Resolving github.com (github.com)... 140.82.113.4
Connecting to github.com (github.com)|140.82.113.4|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-06-14 16:55:42 ERROR 404: Not Found.

makefile:13: recipe for target 'lib' failed
make: *** [lib] Error 8
make: Leaving directory '/home/mtdl/Nek5000/tools/nekamg_setup'

YuHsiang Lan

unread,
Jun 14, 2021, 6:24:51 PM6/14/21
to Nek5000
That is a know issue, which is resolved in the latest repo version.
At some point, HYPRE changes their github project name. Therefore, the path is changed.

I will recommend you to use the nek_amgsetup tool under the latest repo version.
It will be fine if you want to keep the main source code.
That tools should backward support the version of Nek5000 you used.

Hope this helps,
Yu-Hsiang
--




sohaib malik

unread,
Jun 14, 2021, 7:00:20 PM6/14/21
to Nek5000
Thank you so much for your help.
"I will recommend you to use the nek_amgsetup tool under the latest repo version."
Can you please help me with the steps required to do this. I mean do I have to modify the source directory for hypre to the correct repo, if yes, where can I do that.

YuHsiang Lan

unread,
Jun 14, 2021, 7:08:45 PM6/14/21
to Nek5000
I usually keep a Nek5000 latest repo by doing
   git clone https://github.com/Nek5000/Nek5000 Nek5000_repo
Then you go to Nek5000_repo/tools/ to use the tools there.

Or, if you are confident enough, you can simply replace the old install file with the newer one from here:

Hope this helps,
Yu-Hsiang
--

sohaib malik

unread,
Jun 16, 2021, 11:15:52 PM6/16/21
to Nek5000
Hi YuHsiang, Thank you so much for your help. I did git clone for getting Nek5000_repo and then tried to maketools nekamg_setup inside the /Nek5000-repo. that worked. As below.

~/Nek5000-repo/tools$ ./maketools nekamg_setup
building nekamg_setup ... done

However, when I went inside the case directory first run Nekbmpi and it showed the same error of dumping files and dying as I expected. But when I went to the second step and typed "nekamg_setup" it showed command not found.
nekamg_setup
nekamg_setup: command not found

I am not getting what is going on, although the tool is made successfully. I was wondering if I made tool by going into the ~/Nek5000-repo/tools and not inside actual source code holding directory which also has case files inside the run folder i.e.  ~/Nek5000/tools. So, that might be the reason it is not getting the command "nekamg_setup" or is there any other reason.

Thank you so much for your help which is highly appreciated.
Sohaib

YuHsiang Lan

unread,
Jun 16, 2021, 11:25:17 PM6/16/21
to Nek5000
Hi Sohaib,

That's because your Nek5000_repo/bin is not included as the default PATH.
You should find the tool at the path
    ~/Nek5000_repo/bin/nekamg_setup

Jundi He

unread,
Jun 19, 2021, 11:41:43 AM6/19/21
to Nek5000
Hi,

Thanks for the explanation, I have used the latest release of hypre tool on Github, and I have successfully compiled the nekamg_setup tool. But when I tried to run it, an error shown in the terminal: 

ERROR: Invalid size of last level, try different coarsening!

Does anyone know what's going wrong? Thanks!

Regards!
Jundi

YuHsiang Lan

unread,
Jun 19, 2021, 12:38:00 PM6/19/21
to Nek5000
Hi Jundi,

Do you have the entire output log from the tool?

The amg setup tool is just an interface to call HYPRE.
It probably not uses the best option as the default, and you can probably try other coarsening methods.
In this sense, using the on-the-fly HYPRE as the coarse grid solver might be advisible.

Thanks,
Yu-Hsiang
--

Jundi He

unread,
Jun 19, 2021, 12:45:09 PM6/19/21
to Nek5000
Hi Yu-Hsiang:

This is the output of the hypre from my end:

jhe@heft-vis2:~/Nek5000/run/turbChannel$ nekamg_setup 
Enter name prefix of input file(s):
turbChannel
Choose a coarsening method. Available options are:
 - 3: Ruege-Stuben,
 - 6: Falgout (default),
 - 8: PMIS,
 - 10: HMIS,
Choice: 8
Choose an interpolation method. Available options are:
 - 0: classical modified interpolation,
 - 6: extended + i interpolation (default),
Choice: 
Enter smoother tolerance [0.5]:

Reading AMG dump files... Done.
BoomerAMGSetup... 

 Num MPI tasks = 1

 Num OpenMP threads = 1


BoomerAMG SETUP PARAMETERS:

 Max levels = 25
 Num levels = 6

 Strength Threshold = 0.250000
 Interpolation Truncation Factor = 0.000000
 Maximum Row Sum Threshold for Dependency Weakening = 0.900000

 Coarsening Type = PMIS 
 measures are determined locally

 Interpolation = extended+i interpolation

Operator Matrix Information:

            nonzero         entries per row        row sums
lev   rows  entries  sparse  min  max   avg       min         max
===================================================================
 0    2160    49708  0.011     8   27  23.0  -4.108e-15   4.040e-01
 1     698    37074  0.076    20  104  53.1  -3.509e-02   6.534e-01
 2     143     7937  0.388    21  103  55.5  -6.801e-04   6.280e-01
 3      25      549  0.878    14   25  22.0  -6.849e-09   1.321e+00
 4       3        9  1.000     3    3   3.0   9.480e-01   1.520e+00
 5       1        1  1.000     1    1   1.0   1.881e+00   1.881e+00


Interpolation Matrix Information:
                 entries/row    min     max         row sums
lev  rows cols    min max     weight   weight     min       max 
=================================================================
 0  2160 x 698     1   4  -5.344e+00 6.188e+00 5.557e-01 1.000e+00
 1   698 x 143     1   4  -3.048e-01 1.000e+00 3.428e-01 1.008e+00
 2   143 x 25      1   4  -6.487e-01 8.401e-01 2.863e-01 1.001e+00
 3    25 x 3       1   3   2.989e-02 6.206e-01 3.484e-01 1.000e+00
 4     3 x 1       1   1   1.663e-01 2.447e-01 1.663e-01 1.000e+00


     Complexity:    grid = 1.402778
                operator = 1.916754
                memory = 2.102961




BoomerAMG SOLVER PARAMETERS:

  Maximum number of cycles:         20 
  Stopping Tolerance:               1.000000e-07 
  Cycle type (1 = V, 2 = W, etc.):  1

  Relaxation Parameters:
   Visiting Grid:                     down   up  coarse
            Number of sweeps:            1    1     1 
   Type 0=Jac, 3=hGS, 6=hSGS, 9=GE:     13   14     9 
   Point types, partial sweeps (1=C, -1=F):
                  Pre-CG relaxation (down):   0
                   Post-CG relaxation (up):   0
                             Coarsest grid:   0

Done.
--------
Level 1
--------
Computing diagonal smoother... Done.
Running Lanczos... k = 4, [lambda[0], lambda[3]] = [0.067282, 1.960160], rho = 0.933629
Chebyshev smoother iterations: 6 Contraction: 0.208644
ERROR: Smoother iterations too large, try different coarsening!




Which option of coarsening method I should use? There is:

 - 3: Ruege-Stuben,
 - 6: Falgout (default),
 - 8: PMIS,
 - 10: HMIS,

I keep getting this error when I used option 6, 8 and 10, when I use option 3, there is no error, and it shows: 

Setup finished... Exporting data.
8 matrices written to turbChannel.amg_W.dat
8 matrices written to turbChannel.amg_AfP.dat
8 matrices written to turbChannel.amg_Aff.dat
12980 doubles written to turbChannel.amg.dat


Regards!
Jundi




YuHsiang Lan

unread,
Jun 19, 2021, 1:10:23 PM6/19/21
to Nek5000
Hi Jundi,

I'm not really familiar with all of the options here.
However, I do know the amg_setup is different between the on-the-fly HYPRE and the nekamg_setup tool even if you use the same options.
There are some extra options used in the on-the-fly one, which is pasted at the end.

You can probably try the successful one with option 3.
Please observe the pressure iteration and the time spend inside the coarse grid solvers to see if it is reasonable.



BoomerAMG SETUP PARAMETERS:

Max levels = 25
Num levels = 6

Strength Threshold = 0.250000
Interpolation Truncation Factor = 0.000000
Maximum Row Sum Threshold for Dependency Weakening = 0.900000

Coarsening Type = PMIS
measures are determined locally

Interpolation = extended+i interpolation



BoomerAMG SOLVER PARAMETERS:

 Maximum number of cycles:         1
 Stopping Tolerance:               0.000000e+00
 Cycle type (1 = V, 2 = W, etc.):  1

 Relaxation Parameters:
  Visiting Grid:                     down   up  coarse
           Number of sweeps:            1    1     3
  Type 0=Jac, 3=hGS, 6=hSGS, 9=GE:     16   16    16
  Point types, partial sweeps (1=C, -1=F):
                 Pre-CG relaxation (down):   0
                  Post-CG relaxation (up):   0
                            Coarsest grid:   0   0   0



Hope this helps,
Yu-Hsiang
--

Jundi He

unread,
Jun 19, 2021, 1:40:26 PM6/19/21
to Nek5000
Hi Yu-Hsiang,

Thanks for the suggestion, I have downloaded the latest hypre package and replaced folder ./Nek5000/tools/nekamg_setup/hypre, but still, it only works with coarsening method 3: Ruege-Stuben, and for other methods, this error will be shown: 

ERROR: Invalid size of last level, try different coarsening!

As long as it works with one of the coarsening method, I'll use it for now. Thanks for your help and suggestions. 

Regards!
Jundi

YuHsiang Lan

unread,
Jun 19, 2021, 1:56:42 PM6/19/21
to Nek5000
Hi Jundi,

Newer HYPRE won't help since it will still run the parameters we set inside the tool.
I meant the on-the-fly HYPRE as an option in Nek5000's coarse grid solver.

which can be activated by
  preconditioner = semg_amg_hypre   or
  param(40)=2
(Need to put HYPRE into the PPLIST inside makenrs, and compile again, which requires cmake as well).

It will save you time from manually dump/read the dat files and it runs the amg_setup in parallel.
The iteration numbers will differ a little if you runs with different MPI ranks.

Hope this clarifies,
Yu-Hsiang
--

Jundi He

unread,
Jun 19, 2021, 5:17:13 PM6/19/21
to Nek5000
Hi Yu-Hsiang,

OK, thanks for the explanation, it's clear now. 

Regards!
Jundi

Reply all
Reply to author
Forward
0 new messages