When do CC rules choose static libraries and when do they choose shared libraries

[m...@tweag.io]

Hi,

I'm currently debugging an issue in rules_haskell where if I have the following dependency graph,

cc_library(A)
^
|
haskell_library(B)
^
|
cc_binary(C)

then C links in B and A dynamically even when I set linkstatic = True on C. If instead I have,

cc_library(A)
^
|
cc_library(B)
^
|
cc_binary(C)

then C links B and A statically as expected. I use the C++ Starlark API to interoperate with the CC rules. If I do not include a dynamic library in the linking_context, then everything works fine, with static linking as expected. If I include *both* a static and a dynamic library in the linking_context for B, then C for some reasons prefers the dynamic library.

Any idea why this might happen? I've been trying to minimize my code but it's very difficult to reproduce the issue. What makes it harder still to debug is that LibraryToLink and other data structures are essentially opaque (no fields shown when I print them). So I don't know what providers I'm constructing differently than a cc_library rule would. I also get NullPointerException in some situations, though again, very hard to minimize.

What is the logic for selecting shared libraries from dependencies vs static libraries? Are static libraries always preferred unless linkstatic = False? Are there some circumstances in which dynamic libraries will be preferred?

[Laurent Le Brun]

> I also get NullPointerException in some situations, though again, very hard to minimize.

Can you file a bug? The Java stack trace might be enough to understand the problem.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/8d9e6611-9474-4d3d-91a5-5479b67a22a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Laurent

[Boespflug, Mathieu]

On Wed, 24 Apr 2019 at 14:03, Laurent Le Brun <laur...@google.com> wrote:
>
> > I also get NullPointerException in some situations, though again, very hard to minimize.
>
> Can you file a bug? The Java stack trace might be enough to understand the problem.

OK I'll start with that and get a minimal repro when I can. Here's the
ticket: https://github.com/bazelbuild/bazel/issues/8129.

[m...@tweag.io]

So I can now answer my own question:

> If I include *both* a static and a dynamic library in the linking_context for B, then C for some reasons prefers the dynamic library.
>
> Any idea why this might happen?

The reason seems to be that cc_binary uses "mostly static" linking by default, whereas surprisingly, cc_test uses dynamic linking by default. In a cc_test, unless you set `linkstatic = True`, then it will pick up the dynamic versions of its dependencies given a choice.

I came across this by pure chance, when grepping through the source code of Bazel looking for something else. I think that what I am observing is a consequence of this snippet, although I could be wrong: https://github.com/bazelbuild/bazel/blob/6ca40212f660c176086f53ec4548eba8f617e574/src/main/java/com/google/devtools/build/lib/rules/cpp/CcBinary.java#L364-L373.

Moreover, this difference does seem to be documented, although subtly: https://docs.bazel.build/versions/master/be/c-cpp.html#cc_binary.linkstatic.

So my question is now:

* What is the motivation for this difference in behaviour between cc_binary and cc_test?

[Laurent Le Brun]

cc Marcel

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/b50436aa-b5bf-4ca5-bce0-4aff0f0f1b3f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Laurent

[Marcel Hlopko]

Hi Mathieu,

originally cc_library created shared libraries only to speed up development cycle when working with tests by not statically relinking everything on every change. When a change doesn't affect the interface of the shared library of some cc_library, we don't have to relink the cc_test at all. With Bazel this could be improved - we're working with the llvm folks to add a tool that can really generate an interface shared library from an existing shared library (a shared library "with method bodies removed"). Then this path when we don't have to relink the test will be hit much more often.

So from that perspective it makes sense to change the behavior between cc_binary and cc_test. I've been replying to questions about this in the same way - do not depend on shared libraries created by cc_library, they are an implementation detail, an optimization, and they are generally not very useful since they only contain symbols from their sources, not from deps. For haskell library, it would be fine if you only provided static library, things should work just fine. 

Thanks! :)

--

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Boespflug, Mathieu]

Hi Marcel,

that's an interesting observation. In rules_haskell, we currently
build both static libs and dynamic libs because static libs are faster
and less of an operational hassle, but any metaprogramming requires
dynamic libraries for obscure technical reasons on Linux and macOS. We
were considering trying to get rid of the "obscure technical reasons"
and do static linking all the time only. In fact that is what Digital
Asset's DAML project[1] does. But faster incremental rebuilds might be
a nice property to have.

GHC has a flag to build both static and dynamic in one shot, which
saves a lot of build time if you will end up needing both (static libs
to create the deploy binary and dynamic libs for the tests). But
turning on that flag conversely wastes a lot of time if only one form
is needed. Whether both forms are needed or not is something that can
only be determined post-analysis time. What we'd want here for optimal
performance is the ability to express some kind of rewrite rules on
the action graph just before the execution phase.

Two questions:

* Do you use dynamic linking for tests in CI as well? If so doesn't
that mean that in practice CC rules will have to build each library
twice (once to test, once to link the deployed binary, both of which
happen as part of CI)?
* Is there a way to express rewrite rules mentioned above of the form
"if both libA.so and libA.a are wanted, then merge two actions into a
single action that builds both" today? If not, are the any plans for
that in the future? It seems that this would be useful in the CI case.

Best,

[1] https://github.com/digital-asset/daml

[Boespflug, Mathieu]

Oh and let me sneak in a third question:

* if you're using dynamic linking by default for CC tests, how are you
not hitting MACH-O header size limits on macOS? The limits are
extremely low and since Bazel adds LOAD entries in the binary itself
for all transitive dependencies rather than having each library LOAD
its direct dependencies, it makes exceeding the limit all the more
likely on medium to large projects.

[salvadorc...@gmail.com]