macOS: 'ld: warning: ignoring duplicate libraries' for abseil-cpp libs

[Павел Олихвер]

Hello Dev,

I see the following warnings during the build protobuf v28.2 on macOS:

[ 90%] Linking CXX executable protoc-gen-upb
ld: warning: ignoring duplicate libraries: 'third_party/abseil-cpp/absl/base/libabsl_base.a', 'third_party/abseil-cpp/absl/base/libabsl_log_severity.a', 'third_party/abseil-cpp/absl/hash/libabsl_hash.a', 'third_party/abseil-cpp/absl/log/libabsl_log_globals.a', 'third_party/abseil-cpp/absl/random/libabsl_random_distributions.a', 'third_party/abseil-cpp/absl/status/libabsl_status.a', 'third_party/abseil-cpp/absl/strings/libabsl_cord.a', 'third_party/abseil-cpp/absl/strings/libabsl_strings.a', 'third_party/abseil-cpp/absl/synchronization/libabsl_synchronization.a', 'third_party/abseil-cpp/absl/time/libabsl_time.a'
[ 90%] Built target protoc-gen-upb
[ 91%] Building CXX object CMakeFiles/protoc-gen-upbdefs.dir/upb_generator/common.cc.o
[ 91%] Building CXX object CMakeFiles/protoc-gen-upbdefs.dir/upb_generator/file_layout.cc.o
[ 91%] Building CXX object CMakeFiles/protoc-gen-upbdefs.dir/upb_generator/mangle.cc.o
[ 91%] Building CXX object CMakeFiles/protoc-gen-upbdefs.dir/upb_generator/protoc-gen-upbdefs.cc.o
[ 91%] Building C object CMakeFiles/protoc-gen-upbdefs.dir/upb_generator/cmake/google/protobuf/compiler/plugin.upb_minitable.c.o
[ 92%] Linking CXX executable protoc-gen-upbdefs
ld: warning: ignoring duplicate libraries: 'third_party/abseil-cpp/absl/base/libabsl_base.a', 'third_party/abseil-cpp/absl/base/libabsl_log_severity.a', 'third_party/abseil-cpp/absl/hash/libabsl_hash.a', 'third_party/abseil-cpp/absl/log/libabsl_log_globals.a', 'third_party/abseil-cpp/absl/random/libabsl_random_distributions.a', 'third_party/abseil-cpp/absl/status/libabsl_status.a', 'third_party/abseil-cpp/absl/strings/libabsl_cord.a', 'third_party/abseil-cpp/absl/strings/libabsl_strings.a', 'third_party/abseil-cpp/absl/synchronization/libabsl_synchronization.a', 'third_party/abseil-cpp/absl/time/libabsl_time.a'
[ 92%] Built target protoc-gen-upbdefs
[ 92%] Building CXX object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/common.cc.o
[ 92%] Building CXX object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/file_layout.cc.o
[ 93%] Building CXX object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/mangle.cc.o
[ 93%] Building CXX object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/names.cc.o
[ 93%] Building CXX object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/protoc-gen-upb_minitable-main.cc.o
[ 93%] Building CXX object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/protoc-gen-upb_minitable.cc.o
[ 93%] Building C object CMakeFiles/protoc-gen-upb_minitable.dir/upb_generator/cmake/google/protobuf/compiler/plugin.upb_minitable.c.o
[ 93%] Linking CXX executable protoc-gen-upb_minitable
ld: warning: ignoring duplicate libraries: 'third_party/abseil-cpp/absl/base/libabsl_base.a', 'third_party/abseil-cpp/absl/base/libabsl_log_severity.a', 'third_party/abseil-cpp/absl/hash/libabsl_hash.a', 'third_party/abseil-cpp/absl/log/libabsl_log_globals.a', 'third_party/abseil-cpp/absl/random/libabsl_random_distributions.a', 'third_party/abseil-cpp/absl/status/libabsl_status.a', 'third_party/abseil-cpp/absl/strings/libabsl_cord.a', 'third_party/abseil-cpp/absl/strings/libabsl_strings.a', 'third_party/abseil-cpp/absl/synchronization/libabsl_synchronization.a', 'third_party/abseil-cpp/absl/time/libabsl_time.a'
[ 93%] Built target protoc-gen-upb_minitable
[ 94%] Building CXX object CMakeFiles/protoc.dir/src/google/protobuf/compiler/main.cc.o
[ 94%] Linking CXX executable protoc
ld: warning: ignoring duplicate libraries: 'third_party/abseil-cpp/absl/base/libabsl_base.a', 'third_party/abseil-cpp/absl/base/libabsl_log_severity.a', 'third_party/abseil-cpp/absl/hash/libabsl_hash.a', 'third_party/abseil-cpp/absl/log/libabsl_log_globals.a', 'third_party/abseil-cpp/absl/random/libabsl_random_distributions.a', 'third_party/abseil-cpp/absl/status/libabsl_status.a', 'third_party/abseil-cpp/absl/strings/libabsl_cord.a', 'third_party/abseil-cpp/absl/strings/libabsl_strings.a', 'third_party/abseil-cpp/absl/synchronization/libabsl_synchronization.a', 'third_party/abseil-cpp/absl/time/libabsl_time.a'


Are there any plan to fix these minor issues?

Thank you in advance!

[Tony Liao]

Hello!

In principle, we agree that our build should not be generating unnecessary warnings, but it is relatively speaking lower on our priority list. We generally try to reduce warnings where possible, incrementally, as we work on the codebase. If you're interested in helping to reduce noisy warnings, we'd welcome any PRs to that effect.

Thanks!

Tony

[Andrew Simpson]

This warning is appearing because of an interaction between CMake and a change in the linker on macOS. The old linker wouldn't rescan static libraries specified earlier in the command-line to resolve symbols needed by static libraries specified later in the command-line, so CMake inserted duplicates of static libraries to ensure all the dependencies were resolved. The new linker does rescan (so the duplicates aren't necessary) and complains about the duplication.

There's a discussion of this here: https://discourse.cmake.org/t/avoid-duplicate-linking-to-avoid-xcode-15-warnings/9084/9

Apparently setting cmake_minimum_required to version 3.29 or setting policy CMP0156 to NEW will make CMake automatically deduplicate static libraries when using the new macOS linker as well as any other platforms where the linker rescans.