RFC: experimental feature: Ninja weights

[David Turner]

Hello gn-dev@,

The Fuchsia build team has its own fork of Ninja, and would like to implement a new feature where individual Ninja actions (build statements, or even rules) could provide an explicit "weight" value, used to optimize the scheduling of long-pole actions.

This is a slight modification of an existing Ninja feature, where each action is assigned a default weight of 1, which is then propagated / added to its transitive dependents. This is used to determine a better build order, and we've seen improvements to our builds when it was merged into our tree.

The change is simply to allow a build statement to provide its own weight value > 1 for specific actions. For example something that could look like:

build some_output: some_rule some_input
    weight = 100

Of course, our Ninja build plan is generated by GN, so we'd like to allow GN action() targets to specify that value (an integer) through a new argument, which would be injected into the Ninja generated build plan (and otherwise ignored by other GN writers). E.g.:

action("some_target") {
  ...
  ninja_weight = 100
}

This feature being experimental would only be activated through a specific flag in .gn, as in:

experimental_ninja_weights = true

To ensure this doesn't modify the build plan of other projects, and would keep GN's behavior unmodified (e.g. making the "ninja_weight" assignment above an unused-variable error if the flag is not used in .gn).

We can provide CLs to implement this, but we'd appreciate your feedback on this idea.

- Digit

[Dirk Pranke]

Seems reasonable enough to me. I don't know how well it'll work in practice, but it seems at least worth being able to experiment with. I might be inclined to try and label it as "experimental" somehow in case we decide it's not worth it and want to drop it in the future, but I've no bright idea how to do that except via documentation.

-- Dirk

[Aaron Wood]

On Mon, Sep 29, 2025 at 1:59 PM Dirk Pranke <dpr...@chromium.org> wrote:

Seems reasonable enough to me.

Thanks, I'll start work on the CLs for it.

 

I don't know how well it'll work in practice, but it seems at least worth being able to experiment with.

We have some actions that are _very_ slow (multiple minutes), but few dependents, and so ninja ends up scheduling them late in the build because their computed weight is very low (say 5-10), vs. everything else in the build (which is 10s of 1000s of short, <5sec, or even <1 sec task), and then we end up with these slow actions being the sole things running in the build, sometimes for 30-60seconds, just because they were scheduled later in the build.  So for us it could help considerably (although it _is_ quite the corner case).

 

I might be inclined to try and label it as "experimental" somehow in case we decide it's not worth it and want to drop it in the future, but I've no bright idea how to do that except via documentation.

Do you think that feature name in .gn starting with `experimental_` wouldn't be enough to signal that?

[Dirk Pranke]

On Mon, Sep 29, 2025 at 2:06 PM Aaron Wood <aaro...@google.com> wrote:

On Mon, Sep 29, 2025 at 1:59 PM Dirk Pranke <dpr...@chromium.org> wrote:

Seems reasonable enough to me.

Thanks, I'll start work on the CLs for it.

 

I don't know how well it'll work in practice, but it seems at least worth being able to experiment with.

We have some actions that are _very_ slow (multiple minutes), but few dependents, and so ninja ends up scheduling them late in the build because their computed weight is very low (say 5-10), vs. everything else in the build (which is 10s of 1000s of short, <5sec, or even <1 sec task), and then we end up with these slow actions being the sole things running in the build, sometimes for 30-60seconds, just because they were scheduled later in the build.  So for us it could help considerably (although it _is_ quite the corner case).

 

I might be inclined to try and label it as "experimental" somehow in case we decide it's not worth it and want to drop it in the future, but I've no bright idea how to do that except via documentation.

Do you think that feature name in .gn starting with `experimental_` wouldn't be enough to signal that?

That might help, but then you'd have to change code down the road if you decided to make it not experimental, and I'm not sure I'd prefer making users have to do that.

-- Dirk

[Matt Stark]

Anything that ends up in the ninja files needs to be explicitly written into the build files. Would it not be better to, rather than doing that, have some kind of intelligence on the scheduler? When I attended bazelcon in 2023, I saw several talks about adding intelligence to that kind of thing.

Rather than relying on humans to write this explicitly, adding a simple neural network to the task scheduler might be another alternative we may want to consider?

To unsubscribe from this group and stop receiving emails from it, send an email to gn-dev+un...@chromium.org.

--

Thanks, Matt.

[Dirk Pranke]

I imagine a challenge to something like that is that you have to get the data to train on from somewhere. Most likely that would be earlier builds, but it would mean the first build would be out of luck. Kinda like PGO in compilers ...

Having some way to give hints up front therefore seems like it would be useful, and perhaps complementary to adding intelligence to a scheduler.

-- Dirk

[Matt Stark]

Good point about the hints, but I actually was thinking that maybe the neural network weights would be stored in version control or something like that. So that way the first build wouldn't be out of luck.

--

Thanks, Matt.

[Philipp Wollermann]

This sounds a lot like the go/fast-ninja design that Android implemented to solve the same problem they had in their Ninja builds.

I've asked the author of the doc if there is a public version of the doc that we can share on the list (or if I can make a copy that's public information only), but maybe Googlers could already have a look at that for some ideas, things that worked / didn't work, and to avoid accidental incompatibility of the approaches.

For my team it would be very useful if Android's build and GN could align on a compatible design, so that we can easily support both in Siso.

Best,

Philipp

[Jeongik Cha]

Thanks for sharing this thread, Philipp.

For fast ninja, I modified two components, Ninja and Soong.

For ninja, it should accept a kind of weight(priority) list, and use the list to build new build graph with accumulated weight.

For soong, it provides weight list to ninja, and it is based on either previous ninja log or heuristic logic(per-module or per-property(like srcs))

(and I tried ML/DL at the moment as well, but no luck, because one missed big rule causes big performance drop :()

I think GN has a similar role like Soong. So, IMHO, GN would better generate weight list instead of ninja because it has more information to inference the weight, and ninja'd better be general.

[Philipp Wollermann]

+Junji Watanabe +LaMont Jones +Cole Faust FYI as we discussed this topic a couple of times in our meetings

(Please note that if you would like to reply to this thread, you need to join https://groups.google.com/a/chromium.org/g/gn-dev first)

[Ben Boeckel]

On Mon, Sep 29, 2025 at 13:58:49 -0700, Dirk Pranke wrote:
> Seems reasonable enough to me. I don't know how well it'll work in
> practice, but it seems at least worth being able to experiment with. I
> might be inclined to try and label it as "experimental" somehow in case we
> decide it's not worth it and want to drop it in the future, but I've no
> bright idea how to do that except via documentation.

The way we do this in CMake is that we require a variable to be set to
a "magic" value (currently, we just generate a UUID). If the variable
for the experimental feature is not set (or set to the wrong value), the
feature, effectively, does not exist. We then recycle the UUID whenever
the feature is changed in a "meaningful" way. We also log a warning
(once) when the feature is used.

--Ben

[David Turner]

On Tue, Sep 30, 2025 at 3:03 AM Philipp Wollermann <phi...@google.com> wrote:

This sounds a lot like the go/fast-ninja design that Android implemented to solve the same problem they had in their Ninja builds.

I've asked the author of the doc if there is a public version of the doc that we can share on the list (or if I can make a copy that's public information only), but maybe Googlers could already have a look at that for some ideas, things that worked / didn't work, and to avoid accidental incompatibility of the approaches.

Thanks for the link. What we are asking here is akin to step 4-1 in this document where the custom weights are provided directly by GN action() declarations, which should be simple to implement (compared to schemes that rely on historical data, or more refined analysis of the build graph, like looking at target types). In our case, since our build is very skewed and we believe this would be both simple to implement, and enough to get noticeable improvements.

[David Turner]

Another way to do that would be to have a global "experimental_features = []" list in the top-level .gn file whose items are integer values, matching a GN bug database number, where the feature could be properly documented (and tracked if the feature ends up not working, or becomes officially supported through a different .gn variable later)

 

--Ben

[Ben Boeckel]

On Tue, Sep 30, 2025 at 16:00:54 +0200, David Turner wrote:
> Another way to do that would be to have a global "experimental_features =
> []" list in the top-level .gn file whose items are integer values, matching
> a GN bug database number, where the feature could be properly documented
> (and tracked if the feature ends up not working, or becomes officially
> supported through a different .gn variable later)

This can work if the feature doesn't change behavior in a way that
requires user code changes to adapt to (which feels like a possible
early usage ossification). Or a new ticket is opened for each revision
of the feature.

--Ben

[Aaron Wood]

On Mon, Sep 29, 2025 at 10:54 PM Jeongik Cha <jeo...@google.com> wrote:

Thanks for sharing this thread, Philipp.

For fast ninja, I modified two components, Ninja and Soong.

For ninja, it should accept a kind of weight(priority) list, and use the list to build new build graph with accumulated weight.

For soong, it provides weight list to ninja, and it is based on either previous ninja log or heuristic logic(per-module or per-property(like srcs))

(and I tried ML/DL at the moment as well, but no luck, because one missed big rule causes big performance drop :()

I think GN has a similar role like Soong. So, IMHO, GN would better generate weight list instead of ninja because it has more information to inference the weight, and ninja'd better be general.

Interesting, from reading the doc and looking at the CLs, you provided the weights on the side, in a separate file, vs. as part of the ninja rules / build plan files?

(I'm going to be the one implementing this in our fork, so I really appreciate the doc with links to CLs!)

[Jeongik Cha]

Ah, I added two options in ninja for Android: `usesninjalogasweightlist` and `usesweightlist`. With `usesninjalogasweightlist`, ninja uses .ninja_log file to gather weight. For `usesweightlist`, weight value is injected from outside(in this case soong)

Thanks

Jeongik

On Wed, Oct 1, 2025 at 6:13 AM Roland McGrath <mcgr...@google.com> wrote:

Having the weights outside the build.ninja files per se opens interesting possibilities for dynamically updating weights based on past runs without regenerating the build.ninja files.