Experiences building WarpTools v2.0.0dev36, dev37, or dev38 from source?

[Chris]

Hi All,

Does anyone have experience building WarpTools v2.0.0dev36, dev37, or dev38 on a RHEL 8.x (e.g. Rocky Linux 8.4 or 8.5) machine with glibc 2.28?

I am hoping to get at least dev36 running on a Rocky Linux 8.x machine with glibc 2.28 quickly then upgrade the OS later.  I also expect to build from source with Rocky Linux 9.x since glibc is 2.34, and the WarpTools 2.0.0dev37 pre-built conda package asked for glibc 2.38.

Since the most likely outcome is building from source no matter what, I was interested to know if there any positive experiences (or any watch-outs) for building from source. I would like to know if building dev36, dev37 or even dev38 would enable compatibility with older glibc 2.28 on RHEL 8.

Thanks!

Chris

[Warp Bot]

Hi Chris,

Building from source is documented in the README at the root of the repository. The steps are:

conda env create -f warp_build.yml
conda activate warp_build
./scripts/build-native-unix.sh
./scripts/publish-unix.sh

Binaries end up in Release/linux-x64/publish. The publish step produces self-contained executables, so you do not need .NET installed on the target machine.

A few things to be aware of regarding glibc and versions:

1) dev36 and dev37 used .NET 8 and CUDA 11.8. dev38 moved to .NET 10 and CUDA 12.9. This is a major change, so building dev38 requires the newer CUDA toolkit and driver (>= 525.60.13).

2) The glibc 2.38 requirement you saw in the dev37 conda package comes from how the conda package was built (on a newer base image), not necessarily from the code itself. Building from source on your system compiles the C++ native libraries (NativeAcceleration and LibTorchSharp) against your local glibc 2.28, which should lower that barrier.

3) The main watch-out is SkiaSharp (used for PNG output). It ships pre-built native libraries via NuGet, and those have their own glibc floor. A previous user with glibc 2.17 (CentOS 7) hit exactly this issue. glibc 2.28 should be fine for SkiaSharp, but if you do hit a failure there, it will show up as a libSkiaSharp loading error at runtime.

4) The other dependency to watch is PyTorch. The build environment pulls PyTorch from conda, and the version available for your platform needs to be compatible with glibc 2.28. conda-forge PyTorch packages generally support glibc 2.17+, so this should be fine.

5) For dev36/dev37, your Rocky Linux 8 with glibc 2.28 has a good chance of working when built from source. For dev38, it depends on whether .NET 10 and CUDA 12.9 conda packages are available for glibc 2.28. Rocky Linux 9 with glibc 2.34 would be the safer option for dev38.

In summary: building dev36 or dev37 from source on Rocky Linux 8.x is your best bet for a quick solution. For dev38, I would recommend going straight to Rocky Linux 9.x.

-- Warp Bot
THIS IS AN AUTOMATED MESSAGE GENERATED BY AN LLM. IT MAY OR MAY NOT SOLVE YOUR PROBLEM. IF YOU'D LIKE TO SPEAK TO A HUMAN, SAY SO IN YOUR MESSAGE.

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/warp-em/98c91b4d-9b45-4bbd-baa4-74c36b08d608n%40googlegroups.com.

[Ricardo Righetto]

Dear Chris,

Not the same distro/glibc version, but FWIW just yesterday I built dev38 (latest source on Github main branch) on Ubuntu 22.04.5 with glibc 2.35 successfully.

I basically followed the README instructions. The only thing I got a little confused is that then WarpTools should be run under the 'warp_build' conda env, not the regular 'warp' env that you would use if installing from the conda package.

Please let me know if you need more specific guidance.

Best wishes,

--
Ricardo Diogo Righetto

To view this discussion visit https://groups.google.com/d/msgid/warp-em/0100019dac633023-16b70ffe-2bdd-467c-8b0b-84d2bec0cff3-000000%40email.amazonses.com.