Is there anyone using CUDA to run ISCE2 on a GPU?

[gjustin]

When using the Docker provided by ISCE2, I encounter the following error.
https://github.com/isce-framework/isce2/issues/784


Therefore, I am attempting to use scons, but I'm experiencing various errors, and the installation is not progressing.
Since I'm new to using scons, I can't figure out where the problem is occurring.

Is there someone who is actually using ISCE2 with GPU, and if so, could you please share your development environment? Installation is proving to be quite challenging.
(Is it possible that it doesn't work on Ubuntu?)


My environment is as follows:"
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 40
GPUI: NVIDIA RTX A4000
NVIDIA-SMI 510.54
Driver Version: 510.54
CUDA Version: 11.6
Docker version: 24.0.2

[Yujie Zheng]

I follow Yunjun's installation instructions here:

https://github.com/yunjunz/conda_envs/blob/main/isce2/README.md

Hopefully, this is helpful.

Yujie

[gjustin]

Thanks for your information.

I have completed the build and installation based on the provided website.

Is there a way to check if CUDA is currently enabled for GPU usage?"

Thanks.

2023년 11월 3일 금요일 오전 2시 23분 56초 UTC+9에 Yujie Zheng님이 작성:

[Yujie Zheng]

The instruction was for one of the machines called kamb at Caltech. 

Cuda, you will need to load the version installed in your system.

I think for testing, you can try:

cuDenseOffsets.py -h

Yujie

--
You received this message because you are subscribed to a topic in the Google Groups "MintPy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mintpy/tufuDALXv1o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mintpy+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mintpy/46e1bd58-5cf6-4351-911c-fbf3623bfd83n%40googlegroups.com.

[gjustin]

Thank you for your assistance.
I went through all the processes with the provided materials, but I encountered various errors and need help.

1. During the cmake process, the following error occurred:
The part I need to focus on in the error seems to be
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
What could be the reason for not being able to find it even though CUDA 11.2 is properly loaded, and the server was originally used for deep learning and should have access to the GPU?
The full error msg is like:
(isce2) user@ws:~/tools/isce2/build$ cmake ~/tools/isce2/src/isce2 -DCMAKE_INSTALL_PREFIX=~/tools/isce2/install${ISCE_VERSION} -DCMAKE_CUDA_FLAGS="-arch=sm_86" -DCMAKE_PREFIX_PATH=${CONDA_PREFIX} -DCMAKE_BUILD_TYPE=Release
-- The C compiler identification is GNU 12.3.0
-- The CXX compiler identification is GNU 12.3.0
-- The Fortran compiler identification is GNU 12.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/tools/mambaforge/envs/isce2/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/tools/mambaforge/envs/isce2/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /home/tools/mambaforge/envs/isce2/bin/x86_64-conda-linux-gnu-gfortran - skipped
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
-- Found Python: /home/tools/mambaforge/envs/isce2/bin/python3.11 (found suitable version "3.11.6", minimum required is "3.5") found components: Interpreter Development NumPy Development.Module Development.Embed
-- Found PkgConfig: /home/tools/mambaforge/envs/isce2/bin/pkg-config (found version "0.29.2")
-- Found FFTW: /home/tools/mambaforge/envs/isce2/lib/libfftw3f.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3f_threads.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3f_omp.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3_threads.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3_omp.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3l.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3l_threads.so;/home/tools/mambaforge/envs/isce2/lib/libfftw3l_omp.so
-- Found Motif: /home/tools/mambaforge/envs/isce2/lib/libXm.so
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP_Fortran: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5") found components: C CXX Fortran
-- Found OpenCV: /home/tools/mambaforge/envs/isce2 (found version "4.8.1") found components: core highgui imgproc
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- Found GDAL: /home/tools/mambaforge/envs/isce2/lib/libgdal.so (found version "3.7.2")
-- Found X11: /home/tools/mambaforge/envs/isce2/include  found components: Xau Xt
-- Looking for XOpenDisplay in /home/tools/mambaforge/envs/isce2/lib/libX11.so;/home/tools/mambaforge/envs/isce2/lib/libXext.so
-- Looking for XOpenDisplay in /home/tools/mambaforge/envs/isce2/lib/libX11.so;/home/tools/mambaforge/envs/isce2/lib/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found Cython: /home/tools/mambaforge/envs/isce2/bin/python3.11;-m;cython
-- Found Python: /home/tools/mambaforge/envs/isce2/include/python3.11 (found version "3.11.6") found components: Development Development.Module Development.Embed
-- Found Cython:  3.0.5
-- Performing Test C_FNO_COMMON
-- Performing Test C_FNO_COMMON - Success
-- Performing Test CXX_FNO_COMMON
-- Performing Test CXX_FNO_COMMON - Success
-- ISCE2's Stanford-licensed components will NOT be built.
-- Configuring done (18.4s)
-- Generating done (0.8s)
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CUDA_FLAGS

-- Build files have been written to: /home/tools/isce2/build


2. There is a part in the process where (optional) git branches are changed, right?
cd ~/tools/isce2/src/isce2
git checkout pycuampcor
It doesn't seem like there is a separate version for pycuampcor.


3. After installing everything, when I run cuDenseOffsets.py -h
from contrib.PyCuAmpcor import PyCuAmpcor
ModuleNotFoundError: No module named 'contrib.PyCuAmpcor'
Such an error occurs.
Regarding this error, it seems to be discussed in the following issue:
https://github.com/isce-framework/isce2/issues/334
I tried reinstalling pybind11 with "conda install pybind11" as mentioned above, but I still encounter the same error.

I would appreciate any advice on the above errors.
Thank you.

2023년 11월 4일 토요일 오전 1시 38분 55초 UTC+9에 Yujie Zheng님이 작성: