Start-end-lib and importing precompiled libraries

[Marcel Hlopko]

Hello gentlemen and the entire bazel world,

Last two undecided issues of go/bazel-importing-precompiled-cpp-libraries are:

1. --start-group --end-group support

2. Separate cc_static_library_import and cc_shared_library_import vs. unified cc_import

Regarding lib groups:

People don't seem to shout too loudly about this, but we do have a issue on github open for a while now, and there was a stack overflow question asked about this. We have 3 options there: 

* No support at all at the moment, only add it when it becomes a priority. Right now it's not.

* Only support lib groups for precompiled libraries. Lib groups are often a sign of suboptimal design that should be fixed in the new code, but tolerated in the legacy code. Since we are going to work on importing precompiled libraries soon, it'd be cost-effective to incorporate this into their design.

* Full support. That means having some way of telling bazel that some cc_libraries should form a lib group.

Personally, I'd vote against full support, for, well, the greater good. But I don't have more data and very little experience on this.

Regarding separate import rules vs single unified rule:

Ulf raised a concern that people might want to import both static and shared library from the same source, and pick one depending on the properties of the build. I'm not sure how often this happens in the real world (real world please tell me :) My naive assumption was that in the majority of cases people know up front if they want to depend on static or shared library, and in the remaining cases `select` picking one of two targets will work fine. I prefer forcing users to do a little bit of more setup work to make this decision explicit, than to teaching bazel to magically pick dependencies with very different linking semantics.

I'm aware that we have similar behavior with `cc_binary` and `cc_test` picking static or shared libraries for their dependencies. But that is a performance optimization, not a feature that should be used or depended on.

If you received this email it means that you're smarter than me and I hope for your advice.

Cheers!

--

-- 

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Lukács T. Berki]

On 2 October 2017 at 10:52, Marcel Hlopko <hlo...@google.com> wrote:

Hello gentlemen and the entire bazel world,

Last two undecided issues of go/bazel-importing-precompiled-cpp-libraries are:

1. --start-group --end-group support

2. Separate cc_static_library_import and cc_shared_library_import vs. unified cc_import

Regarding lib groups:

People don't seem to shout too loudly about this, but we do have a issue on github open for a while now, and there was a stack overflow question asked about this. We have 3 options there: 

* No support at all at the moment, only add it when it becomes a priority. Right now it's not.

* Only support lib groups for precompiled libraries. Lib groups are often a sign of suboptimal design that should be fixed in the new code, but tolerated in the legacy code. Since we are going to work on importing precompiled libraries soon, it'd be cost-effective to incorporate this into their design.

* Full support. That means having some way of telling bazel that some cc_libraries should form a lib group.

Personally, I'd vote against full support, for, well, the greater good. But I don't have more data and very little experience on this.

Agreed. It seems like if we need this later, it's possible to do this by adding the "these libraries form a group" rule without making incompatible changes.  

Regarding separate import rules vs single unified rule:

Ulf raised a concern that people might want to import both static and shared library from the same source, and pick one depending on the properties of the build. I'm not sure how often this happens in the real world (real world please tell me :) My naive assumption was that in the majority of cases people know up front if they want to depend on static or shared library, and in the remaining cases `select` picking one of two targets will work fine. I prefer forcing users to do a little bit of more setup work to make this decision explicit, than to teaching bazel to magically pick dependencies with very different linking semantics.

I'm aware that we have similar behavior with `cc_binary` and `cc_test` picking static or shared libraries for their dependencies. But that is a performance optimization, not a feature that should be used or depended on.

Actually, this goes into the heart of the "how should it be decided which libraries need to be linked statically and which ones dynamically" problem. I think it's better to have a single cc_import_library rule that can point to both a static and a dynamic library because that way, you don't have to construe a mechanism to tell Bazel that two rules are actually the static/dynamic versions of the same library.

If you received this email it means that you're smarter than me and I hope for your advice.

Cheers!

--

-- 

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Ian Lance Taylor]

On Mon, Oct 2, 2017 at 1:52 AM, Marcel Hlopko <hlo...@google.com> wrote:
>
> Regarding lib groups:
> People don't seem to shout too loudly about this, but we do have a issue on
> github open for a while now, and there was a stack overflow question asked
> about this. We have 3 options there:
>
> * No support at all at the moment, only add it when it becomes a priority.
> Right now it's not.
> * Only support lib groups for precompiled libraries. Lib groups are often a
> sign of suboptimal design that should be fixed in the new code, but
> tolerated in the legacy code. Since we are going to work on importing
> precompiled libraries soon, it'd be cost-effective to incorporate this into
> their design.
> * Full support. That means having some way of telling bazel that some
> cc_libraries should form a lib group.
>
> Personally, I'd vote against full support, for, well, the greater good. But
> I don't have more data and very little experience on this.

When I invented --start-group/--end-group it was to permit the
compiler support library to refer to the C library, and vice-versa.
The problem was that the C library and the compiler support library
had interdependencies, but were developed and maintained separately,
so there was no clean way to handle those interdependencies. I don't
know of any reason to use --start-group when you are actually in
control of your libraries, which is presumably the case when you are
writing your own BUILD files. I can't think of a case not involving
multiple interdependent prebuilt binary libraries where it would be
necessary to use --start-group/--end-group to group together
cc_library targets. And multiple interdependent prebuilt .a files can
probably be handled via linkopts anyhow.

I see that the Stack Overflow question
(https://stackoverflow.com/questions/45285327/build-c-code-with-circular-dependency-using-bazel-build-system/45299251)
is actually asking for circular dependencies for code they own, while
admitting that they could clean it up. As far as I can see their case
could be handled by setting alwayslink = 1 (which I assume that Bazel
supports). So it doesn't seem compelling.

> Regarding separate import rules vs single unified rule:
> Ulf raised a concern that people might want to import both static and shared
> library from the same source, and pick one depending on the properties of
> the build. I'm not sure how often this happens in the real world (real world
> please tell me :) My naive assumption was that in the majority of cases
> people know up front if they want to depend on static or shared library, and
> in the remaining cases `select` picking one of two targets will work fine. I
> prefer forcing users to do a little bit of more setup work to make this
> decision explicit, than to teaching bazel to magically pick dependencies
> with very different linking semantics.

Ulf is probably right that someone out there somewhere wants to do
that. I can't say that I've ever heard of that case, though.

Ian

[Tim Blakely]

+1 to deferring full support. A "these libraries form a group" rule would be nice if a new crosstool feature was added to support it down the road.

I only have an anecdotal example but the reason I was interested in start-/end-group support is that it is used in a few microcontroller SDKs that use circular dependencies during linking; namely the ESP8266 SDK and ESP32 SDK. It's been a while since I've toyed with it, so I don't know whether the circular dependencies were introduced at the chip level SDKs from Espressif or further down the software stack at the Xtensa core SDK level. Either way, the circular dependencies are not at a level that's possible to change without rewriting the entire SDK. A custom CROSSTOOL chain with a start-/end-group flag_set got around this, so it's not blocking by any means.

[Austin Schuh]

For one of my projects, I've got to build the standard C and C++ libraries for a microcontroller.  We are bug-fixing the standard C library enough that it made sense to spend a couple weeks and pull it into bazel.  Proper support would have saved me a couple days of pain, but it's done now.

I'm currently working around it by re-implementing the entire cc_binary and cc_library rules (so we can support multiple architectures in the same package), extracting the .a library files in a genrule, and then combining all of the .a's in a group back into a single .a with ar directly.  (libstdc++ has circular internal dependencies, along with newlib.  Sigh...)  Long term when dynamic configurations arrive, we should be able to drop our custom rules and will then need this support.

Austin 

[Manuel Klimek]

On Mon, Oct 2, 2017 at 10:52 AM Marcel Hlopko <hlo...@google.com> wrote:

Hello gentlemen and the entire bazel world,

Last two undecided issues of go/bazel-importing-precompiled-cpp-libraries are:

1. --start-group --end-group support

2. Separate cc_static_library_import and cc_shared_library_import vs. unified cc_import

Regarding lib groups:

People don't seem to shout too loudly about this, but we do have a issue on github open for a while now, and there was a stack overflow question asked about this. We have 3 options there: 

* No support at all at the moment, only add it when it becomes a priority. Right now it's not.

* Only support lib groups for precompiled libraries. Lib groups are often a sign of suboptimal design that should be fixed in the new code, but tolerated in the legacy code. Since we are going to work on importing precompiled libraries soon, it'd be cost-effective to incorporate this into their design.

* Full support. That means having some way of telling bazel that some cc_libraries should form a lib group.

Personally, I'd vote against full support, for, well, the greater good. But I don't have more data and very little experience on this.

Agree with what others have said - if supported for pre-compiled libs that seems fine, and it also is probably not a priority in general, as most of these will happen within the crosstool.

 

Regarding separate import rules vs single unified rule:

Ulf raised a concern that people might want to import both static and shared library from the same source, and pick one depending on the properties of the build. I'm not sure how often this happens in the real world (real world please tell me :) My naive assumption was that in the majority of cases people know up front if they want to depend on static or shared library, and in the remaining cases `select` picking one of two targets will work fine. I prefer forcing users to do a little bit of more setup work to make this decision explicit, than to teaching bazel to magically pick dependencies with very different linking semantics.

A single unified rule might also make sense from the perspective of simplicity:

As a user, I'll probably in the end want to write something like:

find_system_library("foo")

and then depend on "@foo//foo" 

Many system libs come with both a static and dynamic version. Forcing people to depend one manually requires one more decision to make.

One question is what our strategy for specifying which type of dependency I want is for other libraries. For example, if I want to build a

cc_library(name="bar", deps=["@bar//bar"], ...)

and I want this to be installed as /usr/lib/libfoo.so, linking dynamically against /usr/lib/libbar.so, what will I need to write?

Similarly, if I build a binary and I want to link statically, will I need to edit my dependencies?

[Lukács T. Berki]

On 4 October 2017 at 11:30, Manuel Klimek <kli...@google.com> wrote:

On Mon, Oct 2, 2017 at 10:52 AM Marcel Hlopko <hlo...@google.com> wrote:

Hello gentlemen and the entire bazel world,

Last two undecided issues of go/bazel-importing-precompiled-cpp-libraries are:

1. --start-group --end-group support

2. Separate cc_static_library_import and cc_shared_library_import vs. unified cc_import

Regarding lib groups:

People don't seem to shout too loudly about this, but we do have a issue on github open for a while now, and there was a stack overflow question asked about this. We have 3 options there: 

* No support at all at the moment, only add it when it becomes a priority. Right now it's not.

* Only support lib groups for precompiled libraries. Lib groups are often a sign of suboptimal design that should be fixed in the new code, but tolerated in the legacy code. Since we are going to work on importing precompiled libraries soon, it'd be cost-effective to incorporate this into their design.

* Full support. That means having some way of telling bazel that some cc_libraries should form a lib group.

Personally, I'd vote against full support, for, well, the greater good. But I don't have more data and very little experience on this.

Agree with what others have said - if supported for pre-compiled libs that seems fine, and it also is probably not a priority in general, as most of these will happen within the crosstool.

 

Regarding separate import rules vs single unified rule:

Ulf raised a concern that people might want to import both static and shared library from the same source, and pick one depending on the properties of the build. I'm not sure how often this happens in the real world (real world please tell me :) My naive assumption was that in the majority of cases people know up front if they want to depend on static or shared library, and in the remaining cases `select` picking one of two targets will work fine. I prefer forcing users to do a little bit of more setup work to make this decision explicit, than to teaching bazel to magically pick dependencies with very different linking semantics.

A single unified rule might also make sense from the perspective of simplicity:

As a user, I'll probably in the end want to write something like:

find_system_library("foo")

and then depend on "@foo//foo" 

Many system libs come with both a static and dynamic version. Forcing people to depend one manually requires one more decision to make.

One question is what our strategy for specifying which type of dependency I want is for other libraries. For example, if I want to build a

cc_library(name="bar", deps=["@bar//bar"], ...)

and I want this to be installed as /usr/lib/libfoo.so, linking dynamically against /usr/lib/libbar.so, what will I need to write?

I don't know... I specifcally wanted to sidestep this question for the purposes of this project because, well, it's complicated and mostly orthogonal.

Similarly, if I build a binary and I want to link statically, will I need to edit my dependencies?

I'm aware that we have similar behavior with `cc_binary` and `cc_test` picking static or shared libraries for their dependencies. But that is a performance optimization, not a feature that should be used or depended on.

If you received this email it means that you're smarter than me and I hope for your advice.

Cheers!

--

-- 

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Manuel Klimek]

I think what we plan to do there will influence what we want to do here?

If we plan to support annotating deps with the type of lib you want, it would seem natural to expect having a single import rule?

[James Y Knight]

On Mon, Oct 2, 2017 at 4:52 AM, Marcel Hlopko <hlo...@google.com> wrote:

Hello gentlemen and the entire bazel world,

Last two undecided issues of go/bazel-importing-precompiled-cpp-libraries are:

1. --start-group --end-group support

2. Separate cc_static_library_import and cc_shared_library_import vs. unified cc_import

Regarding lib groups:

People don't seem to shout too loudly about this, but we do have a issue on github open for a while now, and there was a stack overflow question asked about this. We have 3 options there: 

* No support at all at the moment, only add it when it becomes a priority. Right now it's not.

* Only support lib groups for precompiled libraries. Lib groups are often a sign of suboptimal design that should be fixed in the new code, but tolerated in the legacy code. Since we are going to work on importing precompiled libraries soon, it'd be cost-effective to incorporate this into their design.

* Full support. That means having some way of telling bazel that some cc_libraries should form a lib group.

Personally, I'd vote against full support, for, well, the greater good. But I don't have more data and very little experience on this.

I do know of one non-legacy reason why "lib groups" of cc_library rules might be useful.

Source files within a single library are often circularly-dependent, and that's generally accepted to be fine. The default behavior when searching an archive is to search all the object files within it (no ordering requirement), and thus users don't typically do anything special to get the desired linker behavior.

However, in bazel, this can be problematic, because copts can only be specified for an entire "cc_library" rule at a time. If you want to specify different compilation options for different source files within a single logical library (e.g. have some sources built with copts = ["-O3"] and some not), you must split into multiple cc_library rules. But, once you do that, you've lost the implicit circular-dependency allowance, and there's now no way to express the dependencies. Of course, you didn't really want these object files to end up in two separate archives in the first place, but every cc_library turns into a separate archive whether you like it or not, and now you're stuck.

Currently, people resort to using alwayslink=1 to workaround the issue, which is not ideal.

Now -- even knowing of this use case, I'm not sure it's worth it to implement a solution. Even if it is worth implementing a solution, I don't think it needs to be treated together with the support for importing prebuilt .a files -- I'd say the libgroup=1 attribute for the precompiled-libs case seems fine.

I could imagine supporting this use-case by introducing a new rule which takes the entire set of object-file outputs from multiple cc_library rules and turns that into one "library", or by introducing an independent "cc_source_group" rule which is just a container for sources that associates copts/features with them but doesn't actually do anything by itself.

[James Y Knight]

On Wed, Oct 4, 2017 at 9:40 AM, James Y Knight <jykn...@google.com> wrote:

On Mon, Oct 2, 2017 at 4:52 AM, Marcel Hlopko <hlo...@google.com> wrote:

Hello gentlemen and the entire bazel world,

Last two undecided issues of go/bazel-importing-precompiled-cpp-libraries are:

1. --start-group --end-group support

2. Separate cc_static_library_import and cc_shared_library_import vs. unified cc_import

Regarding lib groups:

People don't seem to shout too loudly about this, but we do have a issue on github open for a while now, and there was a stack overflow question asked about this. We have 3 options there: 

* No support at all at the moment, only add it when it becomes a priority. Right now it's not.

* Only support lib groups for precompiled libraries. Lib groups are often a sign of suboptimal design that should be fixed in the new code, but tolerated in the legacy code. Since we are going to work on importing precompiled libraries soon, it'd be cost-effective to incorporate this into their design.

* Full support. That means having some way of telling bazel that some cc_libraries should form a lib group.

Personally, I'd vote against full support, for, well, the greater good. But I don't have more data and very little experience on this.

I do know of one non-legacy reason why "lib groups" of cc_library rules might be useful.

Source files within a single library are often circularly-dependent, and that's generally accepted to be fine. The default behavior when searching an archive is to search all the object files within it (no ordering requirement), and thus users don't typically do anything special to get the desired linker behavior.

However, in bazel, this can be problematic, because copts can only be specified for an entire "cc_library" rule at a time. If you want to specify different compilation options for different source files within a single logical library (e.g. have some sources built with copts = ["-O3"] and some not), you must split into multiple cc_library rules. But, once you do that, you've lost the implicit circular-dependency allowance, and there's now no way to express the dependencies. Of course, you didn't really want these object files to end up in two separate archives in the first place, but every cc_library turns into a separate archive whether you like it or not, and now you're stuck.

Currently, people resort to using alwayslink=1 to workaround the issue, which is not ideal.

Now -- even knowing of this use case, I'm not sure it's worth it to implement a solution. Even if it is worth implementing a solution, I don't think it needs to be treated together with the support for importing prebuilt .a files -- I'd say the libgroup=1 attribute for the precompiled-libs case seems fine.

I could imagine supporting this use-case by introducing a new rule which takes the entire set of object-file outputs from multiple cc_library rules and turns that into one "library", or by introducing an independent "cc_source_group" rule which is just a container for sources that associates copts/features with them but doesn't actually do anything by itself.

At the risk of veering off the main topic of this thread...

I was just reminded of another current workaround for this issue which is hardcoded in blaze: the objc_library "non_arc_srcs" attribute. I don't know the history of this attribute, so I don't know if it was introduced for this reason or not. But at this point, the only thing it provides is the ability to pass the "-fno-objc-arc" to the compilation of some objects in the library and not others -- an ability which isn't generally available.

[Marcel Hlopko]

Thank you all for great advice. It's clear that we won't cover all the legitimate use cases of lib-groups if we only allow setting that for archives. I've rewritten go/bazel-importing-precompiled-cpp-libraries to go with a single cc_import rule, and will leave lib group work for future, right now it's not the top priority (as opposed to cc_import). Feel free to take a look at the updated doc. If all goes like is planned, we will start implementing this mid-November 2017.

Just one question to James: you mention that right now the only way for setting special copts to some sources is splitting cc_libraries. But there is also `per_file_copt` command line flag doing something similar. I have plans to re-investigate its usefulness, but I can imagine having something like per_file_features attributes on cc_* rules. Feel free to talk to me if you think this is important.

Thank you all again!

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAA2zVHp-25M2m7NYAxWqwoGwVUCZYtOWbYMV8ZGBtP7Vjwe2wQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Ryan Burn]

Did lib group support ever go forward?

MKL requires being linked together as a group: https://github.com/bazelbuild/bazel/issues/11593