Openmp and KROME

Mats Esseldeurs

unread,

Feb 26, 2024, 11:35:53 PM2/26/24

to KROMEusers

Hi All

I am trying to run the test suite of KROME to understand how to use the code. I am trying to run the test "atmosphere", compiled with gfortran. If I run the code in series (with no openmp) the code runs correctly, but when I compile with the additional flag -fopenmp and run the test with more than 1 thread, it results in segmenation faults. Am I doing something wrong when I compile or run the code? Or did something change and the test break?

Thanks in advance!

Mats

Troels Haugbølle

unread,

Feb 27, 2024, 1:55:14 AM2/27/24

to Mats Esseldeurs, KROMEusers

Dear Mats,

This is a guess, so I may wrong. But if it segfaults without any error message, it could be many things.

One classic reason is that there is not enough stack space. When you compile with openmp, does the code run on one thread? E.g. try set

export OMP_NUM_THREADS=1

And rerun. If that works, try increase to two threads. If it segfaults, try increasing stack size

export OMP_NUM_THREADS=2

export OMP_STACKSIZE=32m

and rerun. Does it work?

Best

Troels

--
You received this message because you are subscribed to the Google Groups "KROMEusers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kromeusers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kromeusers/bd2038a2-f749-4933-aad8-7c1d12c21524n%40googlegroups.com.

Mats Esseldeurs

unread,

Feb 27, 2024, 2:59:35 AM2/27/24

to KROMEusers

Hi Troels

Thanks for the reply! Running with OMP_NUM_THREADS=1 works, but export OMP_NUM_THREADS=2 breaks (also when increasing OMP_STACKSIZE=32m).

Sometimes I get Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

But other times it gives me a wrong solver exit status (istate = -2 or -7)

Kind regards

Mats

Op dinsdag 27 februari 2024 om 07:55:14 UTC+1 schreef haug...@gmail.com:

Troels Haugbølle

unread,

Feb 27, 2024, 9:45:33 AM2/27/24

to Mats Esseldeurs, KROMEusers

Ok, that sounds like an “out-of-bounds problem”. Try to run with full debug

Sendt fra min iPhone

Den 27. feb. 2024 kl. 08.59 skrev Mats Esseldeurs <matse...@gmail.com>:

Hi Troels

To view this discussion on the web visit https://groups.google.com/d/msgid/kromeusers/5d018804-650e-4e77-a242-d35caf20decdn%40googlegroups.com.

Mats Esseldeurs

unread,

Feb 27, 2024, 11:27:00 PM2/27/24

to KROMEusers

Hi,

This results in the following output:

At line 3678 of file opkda1.f
Fortran runtime error: Index '-1874508807' of dimension 1 of array 'row' below lower bound of 1

Error termination. Backtrace:
ERROR: wrong solver exit status!
istate:          -7
iter count:           2
max iter count:         100
SEE KROME_ERROR_REPORT file

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
At line 1823 of file opkda1.f
Fortran runtime error: Index '-2440' of dimension 1 of array 'wk' below lower bound of 1

Error termination. Backtrace:
ERROR: wrong solver exit status!
istate:          -7
iter count:           7
max iter count:         100
SEE KROME_ERROR_REPORT file
At line 330 of file krome.f90 (unit = 99, file = 'KROME_ERROR_REPORT')
Fortran runtime error: Cannot change STATUS parameter in OPEN statement

Error termination. Backtrace:
#0 0x15044b64251f in ???
       at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#0 0x41cc59 in nnfc_
       at /home/matse/codes/krome/build/opkda1.f:3678
#1 0x408093 in cdrv_
       at /home/matse/codes/krome/build/opkda1.f:2768
#2 0x428ce7 in dprjs_
       at /home/matse/codes/krome/build/opkda1.f:1862
#3 0x4254b4 in dstode_
       at /home/matse/codes/krome/build/opkda1.f:905
#0 0x42862e in dprjs_
#1 0x41bea7 in nntc_
       at /home/matse/codes/krome/build/opkda1.f:3857
       at /home/matse/codes/krome/build/opkda1.f:1823
#2 0x4085dc in cdrv_
#1 0x4254b4 in dstode_
       at /home/matse/codes/krome/build/opkda1.f:905
       at /home/matse/codes/krome/build/opkda1.f:2768
#3 0x4293f5 in dsolss_
       at /home/matse/codes/krome/build/opkda1.f:1954
#4 0x4259fa in dstode_
       at /home/matse/codes/krome/build/opkda1.f:933
#4 0x4456e7 in dlsodes_
       at /home/matse/codes/krome/build/opkdmain.f:1760
#2 0x4456e7 in dlsodes_
       at /home/matse/codes/krome/build/opkdmain.f:1760
#5 0x4c6cc1 in __krome_main_MOD_krome
       at /home/matse/codes/krome/build/krome.f90:102
#3 0x4c6cc1 in __krome_main_MOD_krome
       at /home/matse/codes/krome/build/krome.f90:102
#5 0x4456e7 in dlsodes_
#6 0x4c8c83 in MAIN__._omp_fn.0
       at /home/matse/codes/krome/build/test.f90:110
#4 0x4c8c83 in MAIN__._omp_fn.0
       at /home/matse/codes/krome/build/opkdmain.f:1760
       at /home/matse/codes/krome/build/test.f90:110
#6 0x4c6cc1 in __krome_main_MOD_krome
       at /home/matse/codes/krome/build/krome.f90:102
#7 0x4c8c83 in MAIN__._omp_fn.0
       at /home/matse/codes/krome/build/test.f90:110
#7 0x15044c21beb5 in gomp_thread_start
       at /home/user/sdk2-tmp/build/gcc/libgomp/team.c:129
#5 0x15044c21beb5 in gomp_thread_start
       at /home/user/sdk2-tmp/build/gcc/libgomp/team.c:129
#8 0x15044c21beb5 in gomp_thread_start
       at /home/user/sdk2-tmp/build/gcc/libgomp/team.c:129
#8 0x15044b694ac2 in start_thread
       at ./nptl/pthread_create.c:442
#9 0x15044b72684f in ???
#6 0x15044b694ac2 in start_thread
       at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
       at ./nptl/pthread_create.c:442
#10 0xffffffffffffffff in ???
#7 0x15044b72684f in ???
#9 0x15044b694ac2 in start_thread
       at ./nptl/pthread_create.c:442
       at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
#10 0x15044b72684f in ???

Could it be that the omp flags in opkdmain.f or opkda1.f are not recognized?

The compilation happens with the following commands:
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -w -fallow-argument-mismatch -c opkda2.f -o opkda2.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -w -fallow-argument-mismatch -c opkda1.f -o opkda1.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -w -fallow-argument-mismatch -c opkdmain.f -o opkdmain.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_commons.f90 -o krome_commons.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_constants.f90 -o krome_constants.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_user_commons.f90 -o krome_user_commons.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_fit.f90 -o krome_fit.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_getphys.f90 -o krome_getphys.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_gadiab.f90 -o krome_gadiab.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_grfuncs.f90 -o krome_grfuncs.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_phfuncs.f90 -o krome_phfuncs.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_subs.f90 -o krome_subs.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_stars.f90 -o krome_stars.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_dust.f90 -o krome_dust.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_photo.f90 -o krome_photo.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_tabs.f90 -o krome_tabs.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_coolingGH.f90 -o krome_coolingGH.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_cooling.f90 -o krome_cooling.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_heating.f90 -o krome_heating.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_ode.f90 -o krome_ode.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_user.f90 -o krome_user.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome_reduction.f90 -o krome_reduction.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c krome.f90 -o krome.o
gfortran -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp -c test.f90 -o test.o
gfortran opkda2.o opkda1.o opkdmain.o krome_commons.o krome_constants.o krome_user_commons.o krome_fit.o krome_getphys.o krome_gadiab.o krome_grfuncs.o krome_phfuncs.o krome_subs.o krome_stars.o krome_dust.o krome_photo.o krome_tabs.o krome_coolingGH.o krome_cooling.o krome_heating.o krome_ode.o krome_user.o krome_reduction.o krome.o test.o -o test -fbacktrace -g -ffpe-trap=zero,overflow,invalid -fbounds-check -ffree-line-length-none -fopenmp

Kind regards

Mats

Op dinsdag 27 februari 2024 om 15:45:33 UTC+1 schreef haug...@gmail.com:

Troels Haugbølle

unread,

Feb 28, 2024, 2:36:11 AM2/28/24

to Mats Esseldeurs, KROMEusers

Hi Mats,

This looks suspicious. Either some memory (at both wk and work) is not properly recognised or it is overwritten from somewhere else.

I would have to look at the actual experiment to help more, and this will not be possible before late next week. Tommaso, any suggestions to what could be happening?

Best,

Troels

To view this discussion on the web visit https://groups.google.com/d/msgid/kromeusers/f4033320-9645-45f4-bdfa-0349026d5f88n%40googlegroups.com.

tommaso grassi

unread,

Feb 29, 2024, 10:02:56 AM2/29/24

to KROMEusers

Hi Mats,

thanks for using KROME.

I did some tests also using Intel Inspector. With Intel ifort compiler, there are no problems.

The same error as you appears with gfortran compiler instead, but at the moment is unclear to me why.

Let me do some more tests, but the question here is if you really need the "atmosphere" test.

Keep in mind that this is rather obsolete, and there are probably better codes to study chemistry in planetary atmospheres.

cheers,

-tg

Mats Esseldeurs

unread,

Feb 29, 2024, 10:39:44 PM2/29/24

to KROMEusers

Hi,

Thanks! I will have a try later with ifort.

The atmosphere test itself is not important for me. I was trying to call krome from inside the hydro code phantom, and ran into parallelization problems there. This led me to the test cases that included parallelisation, and noticed the same problem there.

Thanks!

Mats

Op donderdag 29 februari 2024 om 18:02:56 UTC+3 schreef tommas...@gmail.com:

Mats Esseldeurs

unread,

Mar 5, 2024, 10:51:54 AM3/5/24

to KROMEusers

Hi all,

The ifort compilation seems to work in phantom as well. Will keep on using ifort for now!

Thanks!

Mats

Op vrijdag 1 maart 2024 om 04:39:44 UTC+1 schreef Mats Esseldeurs:

Reply all

Reply to author

Forward