Resource auto-tuning via flags

[Julio Merino]

Hello,

Since its inception, Blaze has been developed and used on a fleet of rather-uniform high-end Linux workstations. This has had implications in how Blaze handles resources: the code assumes powerful machines in several places and hardcodes performance-related settings that only make sense in this environment. These static settings have worked fine in the Blaze case but don’t work well in the more heterogeneous landscape of the Bazel world.

The good thing is that we have a bunch of flags to control how Bazel uses these resources. The bad thing is that these flags are obscure and have default values that may not make any sense on the machine Bazel is run on. The ugly thing is that we have the flags altogether because Bazel should just behave in the best possible manner on any machine without manual tuning.

I've written a design document that explores which flags we currently have, what their semantics are, and proposes a mechanism to unify their interface and behavior. You can find it at:

https://docs.google.com/document/d/11N3SXtjPgGnlu0-cUU9gVegUMr5Wrjj_h8PCTGdQlB8/edit#

PTAL. Thanks!