FATAL ERROR: CCE OpenMP does not yet support GNU entry point "GOMP_taskgroup_start"

[bcsj]

Hi, 

I've been trying to get slate running on the LUMI supercomputer. I installed slate using spack:

$> spack install slate+rocm amdgpu_target==gfx90a

I have compiled the slate_lu.cc and been trying to run it as a test.

this code: https://github.com/icl-utk-edu/slate/blob/master/examples/slate_lu.cc

But I get the error you see in the title:

FATAL ERROR: CCE OpenMP does not yet support GNU entry point "GOMP_taskgroup_start"

Clearly the problem is related to openMP, possibly a version thing? I'm not quite sure how I am to approach this issue. If anyone has any suggestion I'm happy to hear it.

Best regards,

Bjørn

[Mark Gates]

Which compiler are you using to install SLATE, and which to compile the example? Hopefully the same compiler in both cases, e.g., both g++ or both Cray's compiler (craycxx). It's a little weird on Cray since the compilers are typically hidden behind the CC compiler wrapper. What version of the compiler? E.g., output from these:

spack find slate

CC --version

I haven't seen this particular error, and a Google search doesn't show occurrences.

Mark

[bcsj]

Thanks for replying :)

I've thought some on the issue since I posted the first message and I'm thinking you are right on the mark that there is a compiler disparity. 

I have since tried to rebuild with different compilers, but run into some other trouble with that. Anyway, first things first:

> spack find slate
-- linux-sles15-zen2 / g...@11.2.0 -------------------------------
sl...@2022.07.00
==> 1 installed package

> CC --version
Cray clang version 15.0.0  (324a8e7de6a18594c06a0ee5d8c0eda2109c6ac6)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/cray/pe/cce/15.0.0/cce-clang/x86_64/share/../bin

So, yea, the compilers were not the same.

spack has a bunch of compilers available though

> spack compilers
==> Available compilers
-- cce sles15-any -----------------------------------------------
c...@15.0.1  c...@14.0.2

-- clang sles15-any ---------------------------------------------
cl...@14.0.0-rocm5.2.3

-- gcc sles15-any -----------------------------------------------
g...@12.2.0  g...@11.2.0  g...@7.5.0

and I tried to rebuild slate using c...@15.0.1 instead, 

spack install slate+rocm amdgpu_target==gfx90a %c...@15.0.1

but in a attempting hat I got an error on the blaspp building

==> blaspp: Executing phase: 'cmake' ==> Error: ProcessError: Command exited with status 1: '/appl/lumi/spack/23.03/0.19.2/opt/spack/cmake-3.24.3-jbx7zbe/bin/cmake' '-G' 'Unix Makefiles' '-DCMAKE_INSTALL_PREFIX:STRING=/project/<...>/spack/23.03/0.19.2/blaspp-2022.07.00-7j6r65k' '-DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo' '-DBUILD_TESTING:BOOL=OFF' '-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF' '-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON' '-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=ON' '-DCMAKE_INSTALL_RPATH:STRING=/project/<...>/spack/23.03/0.19.2/blaspp-2022.07.00-7j6r65k/lib;/project/<...>/spack/23.03/0.19.2/blaspp-2022.07.00-7j6r65k/lib64;/opt/cray/pe/libsci/23.02.1.1/CRAY/9.0/x86_64/lib;/opt/rocm-5.2.3/hip/lib;/opt/rocm-5.2.3/lib;/opt/rocm-5.2.3/llvm/lib;/opt/rocm-5.2.3/lib64' '-DCMAKE_PREFIX_PATH:STRING=/appl/lumi/spack/23.03/0.19.2/opt/spack/cmake-3.24.3-jbx7zbe;/opt/rocm-5.2.3;/opt/rocm-5.2.3/llvm;/opt/rocm-5.2.3;/opt/rocm-5.2.3/hip;/opt/cray/pe/libsci/23.02.1.1/CRAY/9.0/x86_64' '-Dbuild_tests=False' '-Duse_openmp=True' '-DBUILD_SHARED_LIBS=True' '-Dgpu_backend=hip' '-DBLAS_LIBRARIES=/opt/cray/pe/libsci/23.02.1.1/CRAY/9.0/x86_64/lib/libsci_cray.so' '-DBLA_VENDOR:STRING=CRAY' '/tmp/<...>/spack-stage/spack-stage-blaspp-2022.07.00-7j6r65ksmxiklkspx6t5g5rtzixc3zio/spack-src' 1 error found in build log: 20 -- Building HIP/ROCm support in BLAS++ 21 -- Found OpenMP_CXX: -fopenmp (found version "5.0") 22 -- Found OpenMP: TRUE (found version "5.0") 23 -- Found BLAS: /opt/cray/pe/libsci/23.02.1.1/CRAY/9.0/x86_64/lib/libsci_cray.so 24 -- Checking BLAS library version 25 -- Checking BLAS complex return type >> 26 CMake Error at cmake/BLASConfig.cmake:182 (message): 27 Error - Cannot detect zdotc return value. Please check the BLAS 28 installation. 29 Call Stack (most recent call first): 30 CMakeLists.txt:379 (include)

I've also tried to switch to compiling the slate_lu.cc code with g++ by switching to a different provided environment

> CC --version
g++ (GCC) 11.2.0 20210728 (Cray Inc.)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

in which case the error I got when running the code was instead

terminate called recursively
terminate called after throwing an instance of 'std::out_of_range'
terminate called after throwing an instance of 'std::out_of_range'
terminate called recursively
  what():  map::at
terminate called recursively
terminate called after throwing an instance of 'std::out_of_range'
terminate called recursively
  what():  map::at
terminate called recursively
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
srun: error: nid005141: tasks 0-3: Aborted
srun: launch/slurm: _step_signal: Terminating StepId=3624832.0

...

Any good suggestions for how to proceed from here?

Best,

bcsj