Bug#1031799: cmake-data: cmake does not search multiarch paths for HIP

[Cordell Bloor]

Package: cmake-data
Version: 3.25.1-1
Severity: normal
X-Debbugs-Cc: cg...@slerp.xyz, debi...@lists.debian.org

Dear Maintainer,

It is not possible to use CMake's HIP language support together with
the Debian package for HIP. Consider this sample project:

CMakeLists.txt:

cmake_minimum_required(VERSION 3.22)
project(example LANGUAGES HIP)
add_executable(ex main.hip)

main.hip:

#include <stdio.h>
#include <stdlib.h>
#include <hip/hip_runtime.h>

#define CHECK_HIP(expr) do { \
hipError_t result = (expr); \
if (result != hipSuccess) { \
fprintf(stderr, "%s:%d: %s (%d)\n", \
__FILE__, __LINE__, \
hipGetErrorString(result), result); \
exit(EXIT_FAILURE); \
} \
} while(0)

__global__ void sq_arr(float *arr, int n) {
int tid = blockDim.x*blockIdx.x + threadIdx.x;
if (tid < n) {
arr[tid] = arr[tid] * arr[tid];
}
}

int main() {
enum { N = 5 };
float hArr[N] = { 1, 2, 3, 4, 5 };
float *dArr;
CHECK_HIP(hipMalloc(&dArr, sizeof(float) * N));
CHECK_HIP(hipMemcpy(dArr, hArr, sizeof(float) * N, hipMemcpyHostToDevice));
sq_arr<<<dim3(1), dim3(32,1,1), 0, 0>>>(dArr, N);
CHECK_HIP(hipMemcpy(hArr, dArr, sizeof(float) * N, hipMemcpyDeviceToHost));
for (int i = 0; i < N; ++i) {
printf("%f\n", hArr[i]);
}
CHECK_HIP(hipFree(dArr));
return 0;
}

Build log:

# apt install hipcc cmake
# HIPFLAGS="--rocm-path=/usr --rocm-device-lib-path=/usr/lib/x86_64-linux-gnu/amdgcn/bitcode" \
HIPCXX=clang++-15 cmake -S. -Bbuild --debug-output \
-DCMAKE_LIBRARY_ARCHITECTURE=x86_64-linux-gnu
Running with debug output on.
-- The HIP compiler identification is Clang 15.0.7
Called from: [3] /usr/share/cmake-3.25/Modules/CMakeDetermineCompilerId.cmake
[2] /usr/share/cmake-3.25/Modules/CMakeDetermineHIPCompiler.cmake
[1] /root/CMakeLists.txt
CMake Error at /usr/share/cmake-3.25/Modules/CMakeDetermineHIPCompiler.cmake:106 (message):
The ROCm root directory:

/usr

does not contain the HIP runtime CMake package, expected at:

/usr/lib/cmake/hip-lang/hip-lang-config.cmake

Call Stack (most recent call first):
CMakeLists.txt:2 (project)


Called from: [2] /usr/share/cmake-3.25/Modules/CMakeDetermineHIPCompiler.cmake
[1] /root/CMakeLists.txt
-- Configuring incomplete, errors occurred!

This error is because when provided as part of the libamdhip64-dev
package, the HIP runtime CMake package is installed to
/usr/lib/$(DEB_HOST_MULTIARCH)/cmake/hip-lang/hip-lang-config.cmake
CMake should probably check both locations to ensure compatibility with
the layout of both the upstream ROCm project and the Debian HIP package.

In total, this path appears in three places:

/usr/share/cmake-3.25/Modules/CMakeDetermineHIPCompiler.cmake:105
/usr/share/cmake-3.25/Modules/CMakeDetermineHIPCompiler.cmake:110
/usr/share/cmake-3.25/Modules/CMakeHIPInformation.cmake:145

Sincerely,
Cory Bloor

-- System Information:
Debian Release: bookworm/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-3-amd64 (SMP w/32 CPU threads; PREEMPT)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: unable to detect

-- no debconf information

[Cordell Bloor]

There is a related issue once finding hip-lang-config.cmake is handled. The
next problem is this:

    CMake Error at
/usr/share/cmake-3.25/Modules/CMakeFindDependencyMacro.cmake:47
(find_package):
      By not providing "Findamd_comgr.cmake" in CMAKE_MODULE_PATH this
project
      has asked CMake to find a package configuration file provided by
      "amd_comgr", but CMake did not find one.

      Could not find a package configuration file provided by
"amd_comgr" with
      any of the following names:

        amd_comgrConfig.cmake
        amd_comgr-config.cmake

      Add the installation prefix of "amd_comgr" to CMAKE_PREFIX_PATH
or set
      "amd_comgr_DIR" to a directory containing one of the above files.  If
      "amd_comgr" provides a separate development package or SDK, be
sure it has
      been installed.

    Call Stack (most recent call first):

/usr/lib/x86_64-linux-gnu/cmake/hip-lang/hip-lang-config.cmake:98
(find_dependency)
      /usr/share/cmake-3.25/Modules/CMakeHIPInformation.cmake:146
(find_package)
/root/build/CMakeFiles/CMakeScratch/TryCompile-9ltYBc/CMakeLists.txt:2
(project)


    CMake Error at
/usr/share/cmake-3.25/Modules/CMakeDetermineCompilerABI.cmake:57
(try_compile):
      Failed to configure test project build system.

    Call Stack (most recent call first):

      /usr/share/cmake-3.25/Modules/CMakeTestHIPCompiler.cmake:29
(CMAKE_DETERMINE_COMPILER_ABI)
      CMakeLists.txt:6 (enable_language)

    -- Configuring incomplete, errors occurred!

I haven't pinned down the exact cause, but I think this is because
project(example LANGUAGE HIP) will call hip-lang-config.cmake before
CMAKE_HIP_LIBRARY_ARCHITECTURE is detected and therefore
/usr/lib/x86_64-linux-gnu/amd_comgr is not in the search path. It only
checks
/usr/lib/amd_comgr. You can workaround the issue by passing the architecture
explicitly or by adding /usr/lib/x86_64-linux-gnu/amd_comgr to your PATH.

It's not immediately clear to me whether this aspect of the failure
should be
considered a HIP bug or a CMake bug. I'm tempted to ask for some advice from
upstream.

[Cordell Bloor]

I have been discussing this issue with the upstream developers and have
opened a merge request with a fix [1].

On 2/22/23 15:34, Cordell Bloor wrote:
> It is not possible to use CMake's HIP language support together with
> the Debian package for HIP. Consider this sample project:
>
> CMakeLists.txt:
>
> cmake_minimum_required(VERSION 3.22)
> project(example LANGUAGES HIP)
> add_executable(ex main.hip)

The fix for this issue will require the user to instead write their
CMakeLists.txt as:

cmake_minimum_required(VERSION 3.22)
project(example LANGUAGES CXX HIP)
add_executable(ex main.hip)

CMake needs to load the hip-lang information that is provided by the HIP
runtime before the HIP ABI can be determined, but CMake needs to know
the ABI to determine which multiarch directories so search for the HIP
runtime. Enabling C or CXX before HIP will ensure that the appropriate
architecture is known before starting the search and a warning will be
emitted for projects that do not enable C or CXX before HIP.

Sincerely,
Cory Bloor

[1]: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/8356

[Cordell Bloor]

tag 1031799 patch
thanks

Brad King has mostly fixed this problem upstream [1]. I've opened a pull
request [2] that backports his patch series and adds another small fix
required to close this bug.

Sincerely,
Cory Bloor

[1]: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/8525
[2]: https://salsa.debian.org/cmake-team/cmake/-/merge_requests/12