Multiple entries in compile commands, label patterns, globbing

[Charles Nicholson]

Hi GN folks-

We're supporting 4-5 different hardware platforms in our single GN build; they're configured via templates and targets but share a single GN toolchain since they all use the same MCU and vendor C toolchain. This involves building the same .c files multiple times with different configs, making sure they're output to non-overlapping locations, etc. All of the targets for a specific platform "abc" will have the text "_abc_" in their labels, and "_abc_" is globally unique for this purpose. Sometimes these .c files are also compiled on the default (host) toolchain for unit testing.

We generate the compile_commands.json file via a naive "export_compile_commands = [ "//*" ]" line in our .gn file.

As you'd expect, the generated json file has multiple entries for each .c file that gets recompiled per hardware platform. This confuses our LSP clients, since some files are "disabled" based on hardware platforms, and the entire body of the file is unhelpfully greyed out. Which one gets picked is probably an implementation detail of the LSP server. Either way, it's generally not the one I want, because I can't control which one is picked :)

Ideally, we'd have the non-platform-specific targets explicitly listed in our export_compile_commands list in our .gn file, and then have some way for users to say "I'm explicitly working with _abc_ right now" and have the generated compile_commands.json file be the common files plus only the compilation lines for the _abc_ targets.

The build reference for "label patterns" doesn't have anything for internal globs- my naive attempt would be to tack "--add-export-compile-commands=//*_abc_*" onto our gn gen invocation, but of course that doesn't work.

Has anyone encountered and navigated this problem before? Any thoughts + advice are very welcome. Thanks in advance for reading!

Best,

Charles

[Roland McGrath]

I have encountered the issue but haven't done anything to address it.  In my case, all of the different compilations of each source file are interesting to me.  The LSP things I use (clangd, eglot) just seem to pick one for how they'll understand each file, and that's been tolerable enough.  What I'd really like is for the LSP client to give you the option to choose between different interpretations of the source file you're looking at, but I'm not aware of any LSP client (or server) setup that supports something like that.

For what you're looking for I think it's actually pretty easy.  The `gn help dotfile` text is not very clear about this.  But from reading the GN source code, I think the pattern actually just matches roots of deps graphs that you want.  It will then actually describe in the compilation database every target reachable from any of those, without filtering out those that don't themselves match the label patterns.  So I think you could just define `group("abc") { deps = ... }` to reach (only) all the things of interest when working on "abc", and then use `export_compile_commands = [ "//abc" ]` (or `--add-export-compile-commands=//abc` if the `.gn` list is empty or only includes always-want targets).

I haven't experimented with this and I might be reading the GN source wrong.

[Charles Nicholson]

Thanks for the response, Roland. Here's a (scrubbed) description of our build, grep-filtered for ABC, since those are the targets we'd want to only optionally be emitted into the compile_commands.json file. If I understand your suggestion correctly, I'd need to maintain one top-level GN group per board that contains these targets for the given board? That's probably not feasible, the maintenance would be unpleasant :)

❯ ./gn ls out/rel | grep ABC
//build:board_flags_ABC(//build/toolchains:xxx)
//xxx_sdk:libxxx_bootloader_ABC(//build/toolchains:xxx)
//xxx_sdk:libxxx_ABC(//build/toolchains:xxx)
//src/bsp:bsp_xxx_bootloader_ABC(//build/toolchains:xxx)
//src/bsp:bsp_xxx_ABC(//build/toolchains:xxx)
//src/lib:ABC(//build/toolchains:xxx)
//src/lib_aaa:ABC(//build/toolchains:xxx)
//src/lib_bbb:ABC(//build/toolchains:xxx)
//src/lib_ccc:ABC(//build/toolchains:xxx)
//src/lib_ddd:ABC(//build/toolchains:xxx)
//src/lib_eee:ABC(//build/toolchains:xxx)
//src/lib_fff:ABC(//build/toolchains:xxx)
//src/lib_ggg:ABC(//build/toolchains:xxx)
//src/lib_hhh:ABC(//build/toolchains:xxx)
//src/xxx/bootloader:ABC__elf(//build/toolchains:xxx)
//src/xxx/xxx:ABC__elf(//build/toolchains:xxx)

Having ~5 groups that duplicate all of this but substitute "DEF", "GHI", etc for "ABC" wouldn't be fun.

Charles

[Ben Boeckel]

On Tue, Jun 17, 2025 at 11:59:46 -0400, Charles Nicholson wrote:
> We're supporting 4-5 different hardware platforms in our single GN build;
> they're configured via templates and targets but share a single GN
> toolchain since they all use the same MCU and vendor C toolchain. This
> involves building the same .c files multiple times with different configs,
> making sure they're output to non-overlapping locations, etc. All of the
> targets for a specific platform "abc" will have the text "_abc_" in their
> labels, and "_abc_" is globally unique for this purpose. Sometimes these .c
> files are also compiled on the default (host) toolchain for unit testing.

Does GN provide the "output" for each source file/command? If so, you
might be able to teach/inform LSP tools which one to prefer based on
that.

https://clang.llvm.org/docs/JSONCompilationDatabase.html

--Ben

[Charles Nicholson]

Hi Ben- thanks for responding.

My conclusion is that the compilation database format simply isn't robust enough to represent multi-configuration builds, since it's just a flat list instead of having optional categories / hierarchy. I suspect that the path forward here involves post-processing the compilation database file by searching for the various preprocessor-definition and include-search-path flags that I know define a specific platform, and stripping out entries for all other platforms. :(

Best,

Charles

[Ben Boeckel]

On Thu, Jul 03, 2025 at 08:23:56 -0400, Charles Nicholson wrote:
> Hi Ben- thanks for responding.
>
> My conclusion is that the compilation database format simply isn't robust
> enough to represent multi-configuration builds, since it's just a flat list
> instead of having optional categories / hierarchy. I suspect that the path
> forward here involves post-processing the compilation database file by
> searching for the various preprocessor-definition and include-search-path
> flags that I know define a specific platform, and stripping out entries for
> all other platforms. :(

Indeed. This may be of interest:

https://isocpp.org/files/papers/P2977R2.html

My CppCon talk about it:

https://www.youtube.com/watch?v=GUqs_CM7K_0

It's designed for C++ modules support, but it acknowledges things like
"this target exists N times in the build graph" that this is.

--Ben