[slurm-users] Building Slurm with UCX support

[Matthias Leopold]

Hi,

I'm compiling Slurm with ansible playbooks from NVIDIA deepops framework
(https://github.com/NVIDIA/deepops). I'm trying to add UCX support. How
can I tell if UCX is actually included in the resulting binaries
(without actually using Slurm)? I was looking at executables and *so
files with ldd, but found no reference to the UCX installation in
/usr/lib/ucx.

Background:
- I'm struggling with the build system using a non-existent path
(PMIXP_UCX_LIBPATH=\"/usr/lib64\"). The last ugly hack was to create a
symlink from /usr/lib/ucx to /usr/lib64/ucx
- I can't easily test actual operation of MPI with UCX because I'm on a
limited test/dev system and (frankly) because I'm not yet a MPI expert

The configure string used is:
./configure --prefix=/usr/local --disable-dependency-tracking
--disable-debug --disable-x11 --enable-really-no-cray
--enable-salloc-kill-cmd --with-hdf5=no --sysconfdir=/etc/slurm
--enable-pam --with-pam_dir=/lib/x86_64-linux-gnu/security
--with-shared-libslurm --without-rpath --with-pmix=/opt/deepops/pmix
--with-hwloc=/opt/deepops/hwloc --with-ucx=/usr

thx
Matthias

[Ozeryan, Vladimir]

I am not sure about the rest of the Slurm world, but since I will most likely update OpenMPI more often than Slurm, I've configured and built OpenMPI with UCX and Slurm support and I think they are both default unless you specify "--without" option. Works great so far!

-----Original Message-----
From: slurm-users <slurm-use...@lists.schedmd.com> On Behalf Of Matthias Leopold
Sent: Wednesday, January 12, 2022 11:54 AM
To: Slurm User Community List <slurm...@lists.schedmd.com>
Subject: [EXT] [slurm-users] Building Slurm with UCX support

APL external email warning: Verify sender slurm-use...@lists.schedmd.com before clicking links or attachments 

[Matthias Leopold]

Am 12.01.22 um 17:54 schrieb Matthias Leopold:

it looks like a similar problem with the "/usr/lib64" path was reported
in https://bugs.schedmd.com/show_bug.cgi?id=10288
I'm seeing exactly what is reported in comment 5 + 6
unfortunately it's not clear if the problem was resolved

Matthias