"Argument list too long" with many generated code dependencies

[Jeremy Meador]

I've created some automation to split up a monolithic set of protobuf files into a set of fine-grained proto_library and cpp_proto_library build rules in an attempt to get the most out of Bazel for our C++ builds. The automation produces build rules that work great when using clang. Unfortunately, this does not work on one of our target platforms. On ARM, gcc8 complains about argument lists being too long.

ERROR: /path/to/workspace/src/somelib/BUILD:14:11: Compiling src/somelib/somecode.cpp failed: (Exit 1): arm-linux-gnueabihf-gcc failed: error executing command (from target //src/somelib:lib) external/local_arm_cc_toolchains/wrappers/arm-linux-gnueabihf-gcc '--sysroot=external/arm_gcc_8_3_0/arm-linux-gnueabihf/libc' -Bexternal/arm_gcc_8_3_0/bin ... (remaining 3991 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging arm-linux-gnueabihf-gcc: error trying to exec 'external/arm_gcc_8_3_0/libexec/gcc/arm-linux-gnueabihf/8.3.0/cc1plus': execv: Argument list too long

This is because I now have hundreds of compiler flags for every cc_* rule that indirectly depends on the full set of generated protos and C++ headers:

-isystem external/my_defs_repo/my_defs/foo_message_cc_pb

-isystem bazel-out/armeabi-v7a-opt/bin/external/my_defs_repo/my_defs/foo_message_cc_pb

-isystem external/my_defs_repo/my_defs/bar_message_cc_pb

-isystem bazel-out/armeabi-v7a-opt/bin/external/my_defs_repo/my_defs/bar_message_cc_pb

... times 500 ...

Can we abbreviate these subcommands? I can use output_mode = "NO_PREFIX" in cpp_proto_library to get all headers in one output directory, but Bazel still tries to include some rule-specific paths instead of the flattened-out directory.
This is what my output directory looks like under bazel-out/armeabi-v7a-opt/bin/external/my_defs_repo/my_defs:

├── foo_message_cc_pb 

│ └── my_defs 

│  ├── foo_message.pb.cc 

│  └── foo_message.pb.h 

├── foo_message.pb.cc

├── foo_message.pb.h 

├── foo_message_proto-descriptor-set.proto.bin 

├── bar_message_cc_pb 

│ └── my_defs 

│  ├── bar_message.pb.cc 

│  └── bar_message.pb.h 

├── bar_message.pb.cc 

├── bar_message.pb.h 

├── bar_message_proto-descriptor-set.proto.bin

Thanks for taking a look!

[Filip Filmar]

Have you looked at response files?

(I've seen bazel using them for linker invocations FWIW)

F

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/efd51666-1767-4664-bd61-2f2d7fb1c125n%40googlegroups.com.

[Jeremy Meador]

I found a similar issue in another project. https://github.com/envoyproxy/envoy/issues/16196#issuecomment-846012165
It looks like they were forced to change their build in the end. I have yet to try upgrading gcc in our toolchain to see if that fixes the problem.

I want to clean up the compiler invocation if it is possible. I worry that it will impact tooling that uses these compiler invocations.

[David Turner]

On Fri, Jun 30, 2023 at 9:17 PM Filip Filmar <fil...@gmail.com> wrote:

Have you looked at response files?

(I've seen bazel using them for linker invocations FWIW)

FWIW, Bazel will use response files for linking when the cc_toolchain() definition sets supports_param_files to True [1], but there is nothing equivalent for compilation, unfortunately.

[1] https://bazel.build/reference/be/c-cpp#cc_toolchain_args

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAKaOXigHSwVG1qVdwmoGMZZtrqrffxew%2B1V1a-0Vx8XUjnEdmg%40mail.gmail.com.

[Jeremy Meador]

I've found a solution that works for my project. Changing the cpp_*_library rules to omit the line includes = [name_pb], from the underlying cc_library rule fixes the problem. I've opened a feature request with `rules_proto_grpc`: https://github.com/rules-proto-grpc/rules_proto_grpc/issues/267