glog is having a linking problem

[Chet Gnegy]

Hello, I'm a little uncertain of this problem or what changed to cause me to get it. I'm building multiple dynamic libraries that are both getting loaded into another application at runtime. Both of them rely transitively on glog which is built by bazel using the following rule:

cc_library (
  name = "GLog",
  hdrs = ["GLog.h"],
  deps = [
      "@com_github_gflags_gflags//:gflags",
      "@com_github_glog_glog//:glog",
  ],
  linkopts = ["-pthread"],
  alwayslink = 1,
)

When I load the second of two such libs, I get the following error message.

ERROR: something wrong with flag 'timestamp_in_logfile_name' in file 'external/com_github_glog_glog/src/logging.cc'.  One possibility: file 'external/com_github_glog_glog/src/logging.cc' is being linked both statically and dynamically into this executable.

I understand what it means at a basic level, but I'm not sure what to do to fix it, or for that matter why I'm not having issues with every other library that I'm relying on. 

Thanks for any tips,

Chet

[David Turner]

You have statically linked the same static C++ library into two different shared libraries, this is an ODR violation which can have undefined behavior at runtime.

To solve this, create a cc_shared_library(name = "GLog_shared") target that depends on your cc_library(name = "GLog") target, then have your other shared libraries depend on the former instead.
This will ensure that only one instance of GLog (and its global variables) exist in the process at runtime,

For historical reasons, cc_library() only produces static libraries (or even a collection of object files), except when they are dependencies of test binaries.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CACZLR%2BGgtu2s8zE9zQMMOmTwM44_AKrOroRaPbeTOR2f%2BaCHTQ%40mail.gmail.com.

[Chet Gnegy]

Even after all these years I didn’t know cc_shared_library existed. I’ll try that, thanks!

I’m sharing more than just glog, it would be nice to know what glog is exporting that is causing the collision. I guess I’m expecting that this would have been happening to me in many places if sharing a statically linked lib was all it took to have this problem. 

[David Turner]

On Wed, Jan 17, 2024 at 12:50 PM Chet Gnegy <chet...@gmail.com> wrote:

Even after all these years I didn’t know cc_shared_library existed. I’ll try that, thanks!

It's a pretty recent addition (it was experimental / undocumented for a long time). Before that, you could generate one DLL using `cc_binary()` with linkshared=True, which still works, but not recommended (read below).
 

I’m sharing more than just glog, it would be nice to know what glog is exporting that is causing the collision. I guess I’m expecting that this would have been happening to me in many places if sharing a statically linked lib was all it took to have this problem. 

The reason you noticed it here is because GLog is very defensive and printed an error message after detecting the problem. Most libraries do not.
In practice, any global variable (including `static const` ones) in libraries are potential for problematic ODR violation if they are statically linked to multiple executables and/or shared libraries running in the same process.
In practice, if these problems do not surface immediately at runtime, that's because they are harmless by accident, due to the nature of the code, or only trigger under rare conditions (that are usually _very_ hard to debug).

One of the reasons cc_shared_library() was written is to detect these types of problems at build time. And why the LINKABLE_MORE_THAN_ONCE tag to override the check (some libraries can be linked more than once if they only contain functions and constants whose addresses are never taken / used at runtime, but this is necessary for low-level runtime / system code only that must be written in a very specific way).
 
Hope this helps, and happy hacking :)

[Chet Gnegy]

That’s super helpful, but how do I decide what to use cc_shared_lib for. Consider something like Google’s mono repo. It contains some incredible number of targets that become binaries or shared libraries that go out into the world and do *something*, but a large percentage of them rely on base or absl and other common libs. Should those common libs be cc_shared_lib? Or should only the top level lib be 

cc_shared_lib? 

I wish I had a more specific example aside from my own. In my use case I make audio plugins so I build N plugins that all use glog, then I load them into a larger piece of software like Logic or ProTools, which is when I am seeing the clash. There are symbols inside the audio plugins that do need to be externally reachable (obviously). 

[David Turner]

On Wed, Jan 17, 2024 at 4:17 PM Chet Gnegy <chet...@gmail.com> wrote:

That’s super helpful, but how do I decide what to use cc_shared_lib for.

This depends on what you are trying to build and distribute outside of your own project. A general issue with shared libraries, that is not specific to Bazel or even C++. This is also why Rust and Go generate static binaries by default to avoid dealing with this kind of problem entirely.
 

Consider something like Google’s mono repo. It contains some incredible number of targets that become binaries or shared libraries that go out into the world and do *something*, but a large percentage of them rely on base or absl and other common libs. Should those common libs be cc_shared_lib? Or should only the top level lib be 

cc_shared_lib? 

Simple, there are, in general, no shared libraries that "go out into the world" at Google.  Instead, all production binaries are fully statically linked (so never use shared libraries, except a few system-provided ones like the C runtime). This fixes versioning / ABI mismatch problems. The exception are test binaries, which only use shared linking instead, as an optimization to avoid relinking all tests everytime a single dependency changes during development. Note that all these binaries are run in controlled environments managed by special infrastructure (the equivalent of `bazel run` / `bazel test` with remote execution).

So if you have a cc_binary() target, all its cc_library() dependencies are compiled as object files (or a static archive) and linked statically into the executable. If you have a cc_test() target, all its cc_library() dependencies are compiled as individual shared libraries, and dynamically linked into the test executable. cc_library() does not really decide what machine code to compile (or more accurately, it generates actions that can build plenty of object files, static archives and shared library files, but which ones are used is determined higher up in the build graph by its dependents).

 

I wish I had a more specific example aside from my own. In my use case I make audio plugins so I build N plugins that all use glog, then I load them into a larger piece of software like Logic or ProTools, which is when I am seeing the clash. There are symbols inside the audio plugins that do need to be externally reachable (obviously). 

The simplest solution would be to put GLog into a shared library that all your plugins link to at runtime. Even then, I am not sure this will work correctly because, as far as I know, GLog initialization should be called only once from the main executable, not from shared libraries that are dynamically loaded / unloaded. In other words, this library may not be a good fit for your use case. Try to see on the glog development mailing list how other people are dealing with this issue (it's very likely you're not the first to try to do something like that).

[Chet Gnegy]

I'm hitting this, I think:

https://github.com/bazelbuild/bazel/issues/18222

ERROR: /Users/chetgnegy/Neptune/Source/Neptune/BUILD:40:11: in deps attribute of cc_library rule //Neptune:Macros: cc_shared_library rule '//ThirdParty/Google/GLog:GLog' is misplaced here (expected genrule, cc_library, cc_inc_library, cc_embed_data, go_library, objc_library, cc_import, cc_proto_library, gentpl, gentplvars, genantlr, sh_library, cc_binary or cc_test) and '//ThirdParty/Google/GLog:GLog' does not have mandatory providers: 'CcInfo'. Since this rule was created by the macro 'cc_library', the error might have been caused by the macro implementation

 Just trying to swap out the rule for a new shared rule:

cc_library (
  name = "GLogLib",

  hdrs = ["GLog.h"],
  deps = [
      "@com_github_gflags_gflags//:gflags",
      "@com_github_glog_glog//:glog",
  ],
  linkopts = ["-pthread"],
  alwayslink = 1,
)

cc_shared_library (
  name = "GLog",
  deps = [":GLogLib"],
)

I'm running bazel-7.0.0-pre.20231018, and becoming concerned that I'm getting pigeonholed in my choice of bazel version.

https://github.com/bazelbuild/bazel/issues/18181#issuecomment-1826616294

[David Turner]

Looks like you have a cc_library() that depends on your cc_shared_library() through `deps`, try using `data` instead since this corresponds to a runtime library, not a build time one.

[Chet Gnegy]

I see. Now it says it can’t find the header. While I suppose I could work around that or fix it somehow, I feel like I’m going down a rabbit hole. Why can’t I just hide the symbols from glog so the different shared libraries simply can’t see each other’s statically linked glog copy? Can I?

Thanks for all of your help, by the way. I am learning things, for sure.