"Configure-time configuration" for Abseil possible?

[guillaume papin]

The Abseil Compatibility Guidelines[1] state that users
must "not depend on a compiled representation of Abseil".

The description says:

> We do not promise any ABI compatibility
> -- we intend for Abseil to be built from source, hopefully from head.
> The internal layout of our types may change at any point, without notice.
> Building Abseil in the presence of different C++ standard library types
> may change Abseil types, especially for pre-adopted types
> (string_view, variant, etc)
> -- these will become typedefs and their ABI will change accordingly.
>
> -- https://abseil.io/about/compatibility

I'd like to discuss whether or not it is technically possible
to relax this guideline.
And if yes, whether it is something the Abseil maintainers will consider.


Why?
Because I think Abseil has the potential to be immensely useful,
something like an extension to the STL, something you can use in many projects.
It's so useful, it's something you would like to expose in your public API.

However, as immensely useful as it can be, it has big downside:
it does not integrate well with existing build strategies.
I understand that for Bazel and Bazel-like build system it fits,
but for the maybe "more traditional" build systems
this guideline imposes a huge burden.


There are parts of the guideline that I find ok:
- we do not promise any ABI compatibility
- live at head (was "hopefully from head")
- the internal layout of our types may change at any point, without notice
- building Abseil in the presence of different C++ standard library types
may change Abseil types, especially for pre-adopted types
(string_view, variant, etc)
-- these will become typedefs and their ABI will change accordingly.

And only one I don't like:
- we intend for Abseil to be built from source


My understanding for this rule, is that Abseil today,
does feature detection when a translation unit using Abseil is compiled.
So feature detection happens both when Abseil is built and when it used.
If you compile with different settings (C++ standard library types changes),
then you get different types for Abseil.

I would argue that this issue is solvable when not building from source.
Feature detection could happen when Abseil is built, but not when it is used.
The detection can be a configure/build/install output,
e.g. a good old "config.h" generated from a "config.h.in".


The way I see it, Abseil can keep all its current ideas, such as:

In C++17 you get std::string_view,
and in ealier C++ standard versions you get absl::string_view:
- if you build Abseil in C++11 mode, you can use it in C++17 mode,
it will use absl::string_view even if std::string_view exists
- if you build Abseil in C++17 mode, and you try to use it in C++11 mode,
you get a compilation error telling you this Abseil version require C++17

For macros like ABSL_HAVE_BUILTIN/ABSL_ASSERT,
I guess the current method of detecting these features
when a translation unit using Abseil is compiled is ok,
I would not put this one in a generated config.h.

Here I take only a few examples,
but I'm aware Abseil is not limited to these tricks.
and I would be interesting in knowing if there are some
that cannot be handled gracefully.


Generating a config.h would just impact Abseil build
instead of impacting the way people build their code.


What do you think,
is it technically possible to have a compiled representation
and still be correct?

[guillaume papin]

Hello again,

A friendly "up", in case this thread went unnoticed.

I'm still very interested to understand why this restriction is necessary.

Best regards,

Guillaume

---

From: abse...@googlegroups.com <abse...@googlegroups.com> on behalf of guillaume papin <guillau...@epitech.eu>
Sent: Wednesday, November 22, 2017 1:43:17 AM
To: abse...@googlegroups.com
Subject: [abseil-io] "Configure-time configuration" for Abseil possible?

 

--
You received this message because you are subscribed to the Google Groups "Abseil.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to abseil-io+...@googlegroups.com.
To post to this group, send email to abse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/abseil-io/HE1PR02MB11957E24ABB314C3345D9EC48D200%40HE1PR02MB1195.eurprd02.prod.outlook.com.
For more options, visit https://groups.google.com/d/optout.

[Titus Winters]

Sorry Guillaume - I was on vacation for a few weeks and everyone else seems to want me to chime in.  I've finally dug through my email sufficiently for this to be at the top ... I'll give you a proper response tomorrow if at all possible.

To view this discussion on the web visit https://groups.google.com/d/msgid/abseil-io/HE1PR02MB1195A4CB938F8E66AB8A5B618D3D0%40HE1PR02MB1195.eurprd02.prod.outlook.com.

[Titus Winters]

So I don't understand how you can get to "don't build from source" without ABI compatibility.

Yes, it's possible.  It isn't clear that it's helpful, or necessary, or that it doesn't bring in a host of other maintenance issues, but we can certainly discuss.

Lets flip the question: can you give a concrete example of two translation units where having a configure-generated .h *improves* the usability of Abseil?  Historically, when people bring this up, most of their examples are cases where auto-detection per-TU is *saving people* from ODR problems and corrupt builds.

On top of that: systems like autoconf that examine the build state and generate a config have an implicit maintenance problem - there's a disconnect between the test for a bug and the root cause analysis for where the bug is present.  The zero-config approach puts all of these things together: we know that there are thread_local issues in xcode prior to a particular version. Once that falls out of our support window, we can stop caring.  If we do everything via configuration, we lose (some, not all) visibility into whether those bugs / platform features / etc still exist. Historically, this means configuration is ever-increasing - not a property I like when talking about technical debts.

[guillaume papin]

Thank you for your answer...and question :)

> And only one I don't like:

> - we intend for Abseil to be built from source

So I don't understand how you can get to "don't build from source" without ABI compatibility.

I would just state that there are no ABI compatibility guarantee, and the library used at compile-time and runtime should be the same.

The runtime library should not be replaced by a more recent version.

I believe a lot of projects are already working that way, maybe not for the same reasons as Abseil, but it's not uncommon.

Lets flip the question: can you give a concrete example of two translation units where having a configure-generated .h *improves* the usability of Abseil? 

TBH, my concern is with build system integrations.

When I use CMake on a simple project, I sometime compile manually its few dependencies, build and install them in an install prefix.

Then, to build the main project, I just pass-in the this install prefix, and the dependencies are found.

I don't think there is anything wrong with this method, but it just does not seem possible with Abseil.

In my previous company I experienced 3 different build systems (inhouse tool, bitbake, catkin/ament) to build heterogeneous projects (autotools base, cmake based, custom-build tool based, ...).

All projects were built from source, but depended on a binary representation of the package they depends on.

It was possible that projects were built with different compile options, with the restriction that a project does not require an older standard that its dependencies.

I find this restriction..."less restrictive" than basically requiring the whole workspace to be built with the exact same flags.

Also, for CMake "build from source"  does not guarantee anything more than depending on a locally prebuilt target.

The compile options are specified on a per-target basis, so even within a same project, you can have multiple standards (not that I recommend it).

Another issue regarding build systems, is that they are quite a few build systems coming up, and they often capable of detecting pre-built stuff.

For example, meson (http://mesonbuild.com/).

If someone wants to use Abseil on a projet that uses this build system.

With the guideline, it has to rewrite the Abseil build in meson first, then use it.

By using a generated config.h and allowing an 'install' target to the CMake build, it should be easier.

First build and install Abseil locally with CMake, then point meson to it.

I would be curious to see how other projects (CMake-based or not) are using Abseil.

I found only one for now:

- https://github.com/facontidavide/ros_type_introspection

They forked Abseil (https://github.com/abseil/abseil-cpp), integrated cctz directly in the source (no live from head?), compile Abseil in C++11 mode, package the thing and depends on the binary representation.

On top of that: systems like autoconf that examine the build state and generate a config have an implicit maintenance problem - there's a disconnect between the test for a bug and the root cause analysis for where the bug is present.  The zero-config approach puts all of these things together: we know that there are thread_local issues in xcode prior to a particular version. Once that falls out of our support window, we can stop caring.  If we do everything via configuration, we lose (some, not all) visibility into whether those bugs / platform features / etc still exist. Historically, this means configuration is ever-increasing - not a property I like when talking about technical debts.

I understand the implication that it could be more difficult to maintain.

You raise the issue of the ever-increasing configuration.

And I'm thinking it may be some work to do the generation of a config.h for both Bazel and CMake.

If it's technically possible to achieve correct build by relaxing the rule,

then I assume it's a tradeoff between easier integration with build systems (potentially wider adoptions) VS higher maintenance cost.

As a potential user, I'd like Abseil to be easier to integrate into the tools I'm already using, because it looks like really useful library.

But I understand as maintainers you have to take into account the cost of these things.

Best regards,

Guillaume

---

From: 'Titus Winters' via Abseil.io <abse...@googlegroups.com>
Sent: Wednesday, December 6, 2017 4:32:58 PM
To: guillaume papin
Cc: abse...@googlegroups.com
Subject: Re: [abseil-io] "Configure-time configuration" for Abseil possible?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/abseil-io/CAEXQq2ZsCh%3Dc_ur3ieUHp35mu3qrDeJje%2BHWQ_LR9SbY3fubQw%40mail.gmail.com.

[Titus Winters]

I'm afraid "correctness" trumps "easier integration with build systems."  

One of the things that Abseil does that other libraries does not is specifically the standard-C++-type-preadoption trickery.  It's *critically* important for correctness that everything is built with exactly the same compiler and settings - you'll have link errors (at best) and ODR errors (most likely) if you mix build settings at all.  So the *safe* way to do that is to request that everything is built from source at once.

Obviously, it's fine to build everything as a library, *not update any source*, *not change build settings*, and install that.  

We cannot guarantee that any two versions will work together though. If build settings change at all, the installed version is almost certainly going to be harmful. If the library sources / headers are updated and not rebuilt, you're likely going to have a bad time.

For example: there's an optimization that I believe will land soon that changes this constant:

https://github.com/abseil/abseil-cpp/blob/master/absl/base/call_once.h#L137

into something smaller that can be done in fewer assembly instructions to reduce icache pressure on some architectures.  Practically speaking, I suspect there are many absl::call_once bits pre-compiled under the hood.  Changing that value in a header immediately invalidates all of the previously compiled code. I *happened* to spot it and realize it's gonna be an ABI break for people that are doing the wrong thing.

So here's an alternative: if you want to "precompile and install" Abseil, do it in a cronjob or something?  If you pull the daily version, rebuild it, and install that, then you're OK ... so long as only one set of build tools/flags are used that day.