Limiting number of threads for specific programs

[Steve Trotter]

Hi all,

I have a problem currently with using ninja to compile large executables/libraries on a fast computer with less memory essentially. What I want to know is, can I pass options to ninja somehow such that it limits the number of ld (GNU linker, though this isn't important) processes that can be spawned but only for ld processes?

In particular I've been using ninja for building debug builds of LLVM and related projects where there are many libraries and executables built from many .o files. It's using 6 threads by default, which is great when I'm compiling .cpp files into .o files, however when compiling the .o files into .a files there can be many ld processes spawned to do this which each take a lot of memory and the system starts swapping and badness ensues. So what I'd like is 6 threads (or maybe it's processes, I haven't checked) that spawn as many g++ or clang++ processes as they like, but so that only one ld process will ever run to do the linking step, is that possible?

Thanks in advance,

Steve

[Nico Weber]

Hi Steve,

This can be done using pools, see https://ninja-build.org/manual.html#ref_pool I believe newer versions of cmake expose this somehow — and looking through their docs for a bit finds https://cmake.org/cmake/help/v3.0/variable/CMAKE_JOB_POOL_LINK.html#variable:CMAKE_JOB_POOL_LINK , so maybe they already put links into their own pool already. Try passing -DCMAKE_JOB_POOL_LINK=1 to cmake, maybe that already does what you want.

Nico