Hi everyone,
As our build complexity grows, we’ve hit several limitations with the current GN extensibility model. I've written a proposal go/gn-extensibility-dd to address these concerns. The design doc is a long read (16 pages), so here's a TLDR for you: Heavily inspired by bazel
In fact, it looks suspiciously like we've very precisely and meticulously stolen the API...
We use rules, depsets, and providers to create a simple and powerful framework to pull all of the complex logic out of the build files.
As an example, GN currently does not specify inputs for C++ actions. This is a feature we need to address build flakiness and nondeterminism, for reasons I won't explain here. It would be extremely nontrivial to implement directly in GN, but could be implemented with the following code. You can see how we could use this to collect arbitrary metadata about the (transitive) dependencies of a target, and do whatever we wanted with it:
HeadersInfo = provider(["headers"])
def _impl(ctx):
public_headers_direct = ctx.gn.public()
public_headers = depset(
direct = public_headers_direct,
transitive = [dep[HeadersInfo].headers for dep in ctx.gn.public_deps() if HeadersInfo in dep],
)
required_inputs = depset(
direct = ctx.gn.sources(),
transitive = [dep[HeadersInfo].headers for dep in ctx.gn.deps() if HeadersInfo in dep] + [public_headers]
)
return [
HeadersInfo(headers = public_headers),
GnInputsInfo(files = required_inputs),
]
static_library = rule_extension(implementation = _impl)
shared_library = rule_extension(implementation = _impl)
source_set = rule_extension(implementation = _impl)
Current state: Prototype working!
I made a prototype.
Addressing potential concerns
- Value proposition: ~3 SWE months of work by the chrome build team. Likely to recoup time investment within a year.
- Backwards compatibility: This is strictly opt-in. This is an optional (but strongly recommended) replacement for templates.
- Performance: Benchmarks of the current prototype show that adding support for this feature has a statistically insignificant overhead to spin up the starlark interpreter for empty rules for every single build target. Writing more complex rules will of course have overhead, but:
- As it's opt-in, you can choose whether or not it's worth it for you.
- Implementing it with GN templates would also have overhead. I don't have metrics for how they compare yet
- Long-term Maintenance: This feature interacts very minimally with existing GN code. For the most part, this code is able to be tested independently of GN
--