Missing symbols when compiling with -c opt

[hal...@prenav.com]

Hey folks,

I've got a custom Skylark rule that generates some .o files, and I use those .o files as inputs to a cc_library rule:

special_library(
name = "lib_compile",
srcs = [
...
],
)

cc_library(
name = "lib",
srcs = [":lib_compile"],
hdrs = ["Lib.hpp"],
visibility = ["//visibility:public"],
)

lib_compile creates two object files: lib_compile.host.o and lib_compile.device.o. I want all of the symbols from both of these to end up in the cc_library. When compiling with -c fastbuild or -c dbg, two files are generated: liblib.a and liblib.so. Both of these contain the symbols from both of the object files (checked with nm).

When compiling with -c opt, though, I get three files: liblib.a, liblib.pic.a and liblib.so. Both liblib.pic.a and liblib.so contain the symbols I'd expect, but liblib.a contains no symbols at all. The command run to generate liblib.a is:

(cd /home/prenav/.cache/bazel/_bazel_prenav/498b20bdde769651f9658c988dcbe162/execroot/monorepo && \
exec env - \
/usr/bin/ar @bazel-out/local-opt/bin/lib/liblib.a-2.params)

And the contents of liblib.a-2.params:

$ cat bazel-out/local-opt/bin/lib/liblib.a-2.params
rcsD
bazel-out/local-opt/bin/lib/liblib.a

The contents of liblib.pic.a, for comparison, are what I would expect:

rcsD
bazel-out/local-opt/bin/lib/liblib.pic.a
bazel-out/local-opt/bin/lib/lib_compile.host.pic.o
bazel-out/local-opt/bin/lib/lib_compile.device.pic.o

Both of the objects are position-independent.

This wouldn't be a problem, except rules downstream of this rule seem to prefer the liblib.a archive. Both --force_pic and --dynamic_mode=fully cause other rules in our build to misbehave, so it's hard to know whether those would fix the problem.

Any idea what's going on? Thanks so much!
Haldean

[hal...@prenav.com]

Well, I figured out how to fix it (I only wish I had figured it out before I posted to the list!). I'm still interested in hearing whether this is intended behavior, and if it is, it'd be great to get it into the Skylark docs!

The fact that my object files ended with .pic.o has some significance; changing them to be named lib_compile.host.o and lib_compile.device.o makes the symbols show up in both liblib.a and liblib.pic.a.

[Lukács T. Berki]

I apologize for the confusing semantics of C++ rules :( We are working on it, but it'll take a while until they are sorted out.

I don't know off the bat why we treat .o and .pic.o files differently. We seem to handle the difference here (plus an extra skeleton in the closet):

https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/cpp/PrecompiledFiles.java

But this would indicate that it doesn't matter .pic.o and .o shoud be the same. I also didn't find any other relevant place where we check the extension.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/eea5f84c-f580-42f7-b2c1-d76d6a2c88db%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle | Registergericht und -nummer: Hamburg, HRB 86891

[Haldean Brown]

No worries! The majority of our switch to Bazel has been pretty painless, and now that we've switched it's great.

My read of that function is that, if usePic is false, it only accepts files that match CppFileTypes.OBJECT_FILE, and OBJECT_FILE:

https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/cpp/CppFileTypes.java#L122

...explicitly filters out CppFileTypes.PIC_OBJECT_FILE, which is .pic.o. I don't know where in the layers above it's deciding to build the two different archives, but that function looks like it's why my .pic.o files are getting filtered out, for sure.