Avoiding overallocation of CPU

[Øyvind Harboe]

Q: How can I indicate that a build step should not run in parallel with other build steps?

Background:

I'm using Bazel for EDA tools (OpenROAD, open source ASIC tools) and I'm wondering how I can avoid the CPU being overloaded.

The OpenROAD Bazel build flow consists of many steps, most of these steps run an executable that uses a single CPU and doesn't use too much memory, but the final detailed routing uses all the available CPUs.

The problem: if I build multiple macros in parallel, then I can end up with two detailed routes running in parallel. This is a problem because then too many threads will be running in the system and the build will run much slower.

A similar problem exists w.r.t. memory usage. Most steps are single threaded and don't use too much memory, but some of the steps can use all the memory in the system and should not run in parallel with any other steps.

The project for reference... A RISC-V MegaBoom design.

https://github.com/The-OpenROAD-Project/megaboom

Here is a post with some information about builds, they can take quite a long time. 24 hours is nothing special for a design like MegaBoom, so Bazel artifacts as well as being able to specify dependencies carefully is very helpful.

https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts/discussions/1717

[Øyvind Harboe]

Ah. This is a long standing unsolved problem...

https://github.com/bazelbuild/bazel/issues/10443

[Øyvind Harboe]

In my case, it is the route targets that should be run in serial, so I can achieve what I want, run as much as possible in parallel, but no more, with a bash script.

- first run all subtargets up to and including CTS, these are all targets that use a single CPU(mostly)

- run all route targets in serial

- run the rest the top level target

#!/bin/bash

bazelisk build $(bazelisk query "attr('name', '.*cts', deps(DigitalTop_synth))" --noimplicit_deps)

# invoke bazelisk build once for each build target

for target in $(bazelisk query "attr('name', '.*route', deps(DigitalTop_synth))" --noimplicit_deps); do

bazelisk build $target

done

bazelisk build DigitalTop_generate_abstract