[boost] Using Boost.Test header-only from CMake
Peter Dimov via Boost
use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp>
without linking to anything. This worked in the "legacy" structure where
all headers are always available, but doesn't work under CMake where
you have to link to header-only libraries in order to get the proper include
path.
Should we make an attempt to support this use by declaring a special
header-only target in Boost.Test (e.g. Boost::included_unit_test), or is
that a waste of time? One can as well link to the proper
Boost::unit_test_framework (and then even use it header-only, although
there's not much point in doing so.)
I'm inclined towards "waste of time".
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Glen Fernandes via Boost
<bo...@lists.boost.org> wrote:
>
> Some libraries (I encountered this when porting the tests of Signals2)
> use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp>
> without linking to anything. This worked in the "legacy" structure where
> all headers are always available, but doesn't work under CMake where
> you have to link to header-only libraries in order to get the proper include
> path.
>
> Should we make an attempt to support this use by declaring a special
> header-only target in Boost.Test (e.g. Boost::included_unit_test), or is
> that a waste of time? One can as well link to the proper
> Boost::unit_test_framework (and then even use it header-only, although
> there's not much point in doing so.)
>
> I'm inclined towards "waste of time".
What is the list of libraries that do this? I'm curious if most of
them are unmaintained or would otherwise be better served migrating to
LWT.
Glen
Peter Dimov via Boost
> On Sun, Jun 6, 2021 at 9:12 PM Peter Dimov via Boost
> <bo...@lists.boost.org> wrote:
> >
> > Some libraries (I encountered this when porting the tests of Signals2)
> > use Boost.Test in header-only mode, by including
> > <boost/test/included/unit_test.hpp>
> > without linking to anything. This worked in the "legacy" structure
> > where all headers are always available, but doesn't work under CMake
> > where you have to link to header-only libraries in order to get the
> > proper include path.
> >
> > Should we make an attempt to support this use by declaring a special
> > header-only target in Boost.Test (e.g. Boost::included_unit_test), or
> > is that a waste of time? One can as well link to the proper
> > Boost::unit_test_framework (and then even use it header-only, although
> > there's not much point in doing so.)
> >
> > I'm inclined towards "waste of time".
>
> Likewise.
>
> What is the list of libraries that do this? I'm curious if most of them are
> unmaintained or would otherwise be better served migrating to LWT.
actually the maintained ones, because the unmaintained ones are still
on the deprecated <boost/test/minimal.hpp>. :-)
The `included` ones are
algorithm
filesystem
geometry
heap
lockfree
math
mpi
numeric/conversion
numeric/odeint
numeric/ublas
process
ptr_container
random
range
signals2
sort
type_traits
uuid
yap
so this is widely used, and in fairness, it's a legitimate and documented
way to use Boost.Test.
I don't think we need to change anything in the libraries, but I also don't
think we need to duplicate the Boost.Test CMake targets to make them
have header-only versions. Linking to the normal target will also enable
the header-only use to compile.
For completeness, the libraries using the deprecated boost/test/minimal.hpp
are:
foreach
graph
graph_parallel
iterator
logic
numeric/conversion
numeric/interval
property_map
tokenizer
units
yap
These need to migrate to either LWT or the non-deprecated Boost.Test.
Edward Diener via Boost
> Some libraries (I encountered this when porting the tests of Signals2)
> use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp>
> without linking to anything. This worked in the "legacy" structure where
> all headers are always available, but doesn't work under CMake where
> you have to link to header-only libraries in order to get the proper include
> path.
>
> Should we make an attempt to support this use by declaring a special
> header-only target in Boost.Test (e.g. Boost::included_unit_test), or is
> that a waste of time? One can as well link to the proper
> Boost::unit_test_framework (and then even use it header-only, although
> there's not much point in doing so.)
>
> I'm inclined towards "waste of time".
and many libraries currently do this, it is hardly a "waste of time" to
support this under CMake.
Peter Dimov via Boost
> Since it is totally legitimate to use Boost.Test in header only mode, and many
> libraries currently do this, it is hardly a "waste of time" to support this under
> CMake.
So there's no real benefit from having a separate header-only target to link to;
you can just link to the ordinary target Boost::unit_test_framework.
John Maddock via Boost
> Edward Diener wrote:
>> Since it is totally legitimate to use Boost.Test in header only mode, and many
>> libraries currently do this, it is hardly a "waste of time" to support this under
>> CMake.
> Under CMake you can't use header-only libraries without linking to their targets.
> So there's no real benefit from having a separate header-only target to link to;
> you can just link to the ordinary target Boost::unit_test_framework.
to be, and b) add the resulting binary to library link list when it
doesn't need to be?
But I do appreciate the simplicity of a single target ;)
John.
--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Andrey Semashev via Boost
> On 07/06/2021 17:26, Peter Dimov via Boost wrote:
>> Edward Diener wrote:
>>> Since it is totally legitimate to use Boost.Test in header only mode,
>>> and many
>>> libraries currently do this, it is hardly a "waste of time" to
>>> support this under
>>> CMake.
>> Under CMake you can't use header-only libraries without linking to
>> their targets.
>> So there's no real benefit from having a separate header-only target
>> to link to;
>> you can just link to the ordinary target Boost::unit_test_framework.
>
> Doesn't that a) cause the unit test lib to be build when it doesn't need
> to be, and b) add the resulting binary to library link list when it
> doesn't need to be?
>
> But I do appreciate the simplicity of a single target ;)
symbols or ODR issues. I believe, you're not supposed to use header-only
Boost.Test and then link against it, too, as the same symbols would be
defined both in the test and the library.
Boost.Test could define two sets of targets - one for header-only and
the other for separately built library. One benefit of that is that you
could use different macros for these two targets, if needed (e.g. if the
library sources need to discriminate between the two configs).
Peter Dimov via Boost
> On 6/7/21 7:48 PM, John Maddock via Boost wrote:
> > On 07/06/2021 17:26, Peter Dimov via Boost wrote:
> >> Edward Diener wrote:
> >>> Since it is totally legitimate to use Boost.Test in header only
> >>> mode, and many libraries currently do this, it is hardly a "waste of
> >>> time" to support this under CMake.
> >> Under CMake you can't use header-only libraries without linking to
> >> their targets.
> >> So there's no real benefit from having a separate header-only target
> >> to link to; you can just link to the ordinary target
> >> Boost::unit_test_framework.
> >
> > Doesn't that a) cause the unit test lib to be build when it doesn't
> > need to be, and b) add the resulting binary to library link list when
> > it doesn't need to be?
> >
> > But I do appreciate the simplicity of a single target ;)
>
> Another question is whether unnecessarily linking would cause duplicate
> symbols or ODR issues. I believe, you're not supposed to use header-only
> Boost.Test and then link against it, too, as the same symbols would be defined
> both in the test and the library.
but it's indeed not quite right in principle.
> Boost.Test could define two sets of targets - one for header-only and the other
> for separately built library. One benefit of that is that you could use different
> macros for these two targets, if needed (e.g. if the library sources need to
> discriminate between the two configs).
Any input from the Boost.Test maintainer? Raffi?
Raffi Enficiaud via Boost
> Andrey Semashev wrote:
>> On 6/7/21 7:48 PM, John Maddock via Boost wrote:
>>> On 07/06/2021 17:26, Peter Dimov via Boost wrote:
>>>> Edward Diener wrote:
>>>>> Since it is totally legitimate to use Boost.Test in header only
>>>>> mode, and many libraries currently do this, it is hardly a "waste of
>>>>> time" to support this under CMake.
>
>>>> Under CMake you can't use header-only libraries without linking to
>>>> their targets.
>>>> So there's no real benefit from having a separate header-only target
>>>> to link to; you can just link to the ordinary target
>>>> Boost::unit_test_framework.
>>>
>>> Doesn't that a) cause the unit test lib to be build when it doesn't
>>> need to be, and b) add the resulting binary to library link list when
>>> it doesn't need to be?
>>>
>>> But I do appreciate the simplicity of a single target ;)
>>
>> Another question is whether unnecessarily linking would cause duplicate
>> symbols or ODR issues. I believe, you're not supposed to use header-only
>> Boost.Test and then link against it, too, as the same symbols would be defined
>> both in the test and the library.
>
> It works because the symbols in the executable override those in libraries,
> but it's indeed not quite right in principle.
I am skeptical about this:
* the overriding behaviour sounds to me system dependent,
* we may duplicate some of the singletons,
* some compilation information may be different between the build time
and the consumption time of the library. The 2 steps build (library
generation with one set of options, header only consumption with
potentially another set of options) may create difficult to debug
situations.
I would avoid this.
>
>> Boost.Test could define two sets of targets - one for header-only and the other
>> for separately built library. One benefit of that is that you could use different
>> macros for these two targets, if needed (e.g. if the library sources need to
>> discriminate between the two configs).
>
> That's probably what we'll have to do, yes.
>
> Any input from the Boost.Test maintainer? Raffi?
INTERFACE library, as it carries the include folder, compiler options,
defines etc together with the target cmake object, without actually
building anything, and that is what we need for header only:
https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#interface-libraries
I have not looked at the recent developments for CMake in Boost. If the
CMakeLists.txt I developed (under build/) is still there, it contained 2
targets: one for static and one for shared. Those were having different
and explicit names.
Having a 3rd target for header only sounds good to me. I can give a go
if you want.
Raffi
PS.: for those hitting the deprecated API... it's been deprecated since
almost a decade now :)
Peter Dimov via Boost
> I have not looked at the recent developments for CMake in Boost. If the
> CMakeLists.txt I developed (under build/) is still there, it contained 2
> targets: one for static and one for shared. Those were having different and
> explicit names.
on BUILD_SHARED_LIBS.
It causes problems in the more complicated cases where a compiled Boost::X
links to header-only Boost::Y which links to a compiled Boost::Z. Client code
can choose to link explicitly to Boost::static_X, and if X used Z directly, it could
link to Boost::static_Z, but it doesn't. And requiring all header-only libraries
to also provide triplicate targets so that static/shared-ness can be propagated
correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose
whether to link to the static or shared Z, but it doesn't know how.
(b2 does this "right" by having static/shared as a build feature, and
propagating it automatically. No need to split your targets by hand.)
Swimming upstream is sometimes justified when it results in something
objectively better than the accepted CMake way, but I don't believe that's
the case here, so BUILD_SHARED_LIBS will have to do. (Similarly, I tried to do
something b2-like with the tests, but at the end reverting to the way CMake
wants to do things proved better.)
Getting back to Boost.Test, I've duplicated the targets with an included_ prefix:
https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93af0a0
Edward Diener via Boost
> Raffi Enficiaud wrote:
>> I have not looked at the recent developments for CMake in Boost. If the
>> CMakeLists.txt I developed (under build/) is still there, it contained 2
>> targets: one for static and one for shared. Those were having different and
>> explicit names.
>
> I decided to not do this and instead use the "normal" CMake way of relying
> on BUILD_SHARED_LIBS.
>
> It causes problems in the more complicated cases where a compiled Boost::X
> links to header-only Boost::Y which links to a compiled Boost::Z. Client code
> can choose to link explicitly to Boost::static_X, and if X used Z directly, it could
> link to Boost::static_Z, but it doesn't. And requiring all header-only libraries
> to also provide triplicate targets so that static/shared-ness can be propagated
> correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose
> whether to link to the static or shared Z, but it doesn't know how.
natural to me that a header-only library would always choose to use some
other dependent library as header-only, even when that other library had
static or shared variants. In other words I applaud the decision of
Boost.Test to provide a header-only variant and can not even begin to
process the fact that CMake can not deal with Boost.Test as a
header-only library, if that is indeed the case.
If you are saying that the use of Boost.Test within Boost's creation of
CMakeLists.txt for its libraries can never be as a header-only only
library, please reconsider. If you are not saying that I apologize for
my noise.
Gavin Lambert via Boost
> I do not know CMake, so maybe my comment is irrelevant, but it seems
> natural to me that a header-only library would always choose to use some
> other dependent library as header-only, even when that other library had
> static or shared variants. In other words I applaud the decision of
> Boost.Test to provide a header-only variant and can not even begin to
> process the fact that CMake can not deal with Boost.Test as a
> header-only library, if that is indeed the case.
then wants to use header-only library B which also uses library A.
For this to work, one of the following must be true:
1. library B can figure out how program P chose to link to it and
links to it the exact same way.
2. library A is written such that in header-only mode it uses
entirely separate namespaces from its compiled modes; then B can link to
it in header-only mode without conflicting with other uses.
#2 is rarely true, because it's more work for both users and maintainers
(inline namespaces make it actually *feasible*, but still require people
to actually have considered the problem in the first place).
#1 is putting the cart before the horse, which can be problematic (but
nevertheless, is the "solution" most often taken, by "bubbling down"
settings from the parent projects) -- but it only works as long as
everyone respects them and everything gets compiled with matching settings.
(#1 also affect dependencies on the CRT itself, which is why most have
big global hammers to specify whether linking to the CRT statically or
dynamically, and why everything explodes when these settings don't match
between compilation units.)
When these conditions are not met, you end up with an ODR violation.
Which often works anyway by coincidence, especially in libraries without
singletons, but you're tap-dancing over a minefield. (And even doing #2
will land you with disjoint singleton "islands", but at least that way
it's presumably deliberate.)
(The safest way out of this minefield is to ensure that header-only
libraries either have no separate dependencies, or at least only ever
reference dependencies that do not have a compiled variant. This is of
course a scaling problem.)
TLDR: C++ is extremely ODR-prone and libraries are very fragile things.
Raffi Enficiaud via Boost
BUILD_SHARED_LIBS is a global property inherited by all targets, with
the drawback of not being able to explain all desirable configurations
(# exponential in the number of targets). One example where it is not
working: I need to run the unit tests of Boost.Test with shared, static
and header only, so those 3 targets must exist. But this is not an
isolated usage: as a project grow, it is sometimes needed that a
specific lib exists in both shared and static variant.
In any case, it is always possible (later, if needed) to define the
shared/static explicitly and make an alias on the target that mimics
BUILD_SHARED_LIBS.
I have to thank Peter and all other boosters that participated to this
effort.
On a side note: I believe the "democratic" way of doing it does not work
inside the current boost setup, and yield endless discussions not
producing any artifacts.
OTOH, the person doing that should have enough established authority &
being respected within Boost for doing such a job. Peter has this, I do not.
Coming back to Boost.Test, Edward + Peter: your call.
Raffi
Gavin Lambert via Boost
between static/shared are mostly fine these days, because the tools have
some blunt hammers that mostly do the right things with them. (But
still bite people on occasion.)
But libraries that can be configured between compiled and header-only
are inherently a "here be dragons" and unless done *really* carefully
(or sufficiently small that they can exist without dependencies of their
own), are best avoided.
Pick one of header-only or compiled, and commit to it. Don't try to
hedge it to configuration, as this will only lead to pain and suffering.
And don't try to be header-only unless all of your dependencies are also
header-only, or you're *really darn sure* that the build tools can unify
the static/shared configuration across the whole program.
Edward Diener via Boost
Peter has done to bring CMake to Boost, but I was simply arguing that if
Boost's implementation of CMake can not deal with the header-only
variation of Boost.Test then conceptually this appears to be a failure.
However if whatever Peter has done allows other code using CMake to
treat Boost.Test as header-only, as well as compiled as either a static
or shared library when chosen, I think that is optimal.
I know that in a project on which I am working I use Boost.Test in
header-only mode and it works fine when using b2 to run tests for the
project. I would like to think that if I try to use CMake to run tests
for the project using the header-only variant of Boost.Test it will
continue to work fine. That's all really.
Alexander Grund via Boost
> I know that in a project on which I am working I use Boost.Test in
> header-only mode and it works fine when using b2 to run tests for the
> project. I would like to think that if I try to use CMake to run tests
> for the project using the header-only variant of Boost.Test it will
> continue to work fine. That's all really.
https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93af0a0
So all you got to do is "link" to `Boost::included_unit_test_framework`
and you can use it header-only. "Link" in CMake terms, no actual linking
done, only declaring a dependency, and getting the headers into the
include path.
Although I would have used `header_only` as a suffix instead, but well,
naming ;)
Andrey Semashev via Boost
scheme would be preferable? I'm thinking something like Boost::x names
the default config, Boost::x::header_only means header-only variant,
Boost::x::static means static library, and so on. This makes it possible
to add more configurations in the future, including the library-specific
ones, if needed.
Peter Dimov via Boost
> > Getting back to Boost.Test, I've duplicated the targets with an included_
> > prefix:
> >
> > https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93af0a0
>
> Not to start a bikeshedding argument, but maybe a more generic naming
> scheme would be preferable?
#include <boost/test/unit_test.hpp>
and the "included" one is used with
#include <boost/test/included/unit_test.hpp>
it seemed natural to call the target Boost::included_unit_test_framework
in the second case. (Similarly with the execution monitors.)
Glen Fernandes via Boost
> Glen Fernandes wrote:
> >
> > What is the list of libraries that do this? I'm curious if most of them are
> > unmaintained or would otherwise be better served migrating to LWT.
>
> The libraries that use the "included/" versions of the Test libraries are
> actually the maintained ones, because the unmaintained ones are still
> on the deprecated <boost/test/minimal.hpp>. :-)
>
> are:
>
> foreach
> graph
> graph_parallel
> iterator
> logic
> numeric/conversion
> numeric/interval
> property_map
> tokenizer
> units
> yap
>
> These need to migrate to either LWT or the non-deprecated Boost.Test.
Almost finished migrating Units to LWT too, and will do the others afterwards.
Glen
Glen Fernandes via Boost
> For completeness, the libraries using the deprecated boost/test/minimal.hpp are:
>
> foreach
> graph
> graph_parallel
> iterator
> logic
> numeric/conversion
> numeric/interval
> property_map
> tokenizer
> units
> yap
>
> These need to migrate to either LWT or the non-deprecated Boost.Test.
to be merged.
Rainer Deyke via Boost
> On 8/06/2021 4:53 pm, Edward Diener wrote:
>> I do not know CMake, so maybe my comment is irrelevant, but it seems
>> natural to me that a header-only library would always choose to use
>> some other dependent library as header-only, even when that other
>> library had static or shared variants. In other words I applaud the
>> decision of Boost.Test to provide a header-only variant and can not
>> even begin to process the fact that CMake can not deal with Boost.Test
>> as a header-only library, if that is indeed the case.
>
> A problem occurs when program P uses library A as static or shared and
> then wants to use header-only library B which also uses library A.
>
> For this to work, one of the following must be true:
> 1. library B can figure out how program P chose to link to it and
> links to it the exact same way.
A header-only library cannot *link* to anything, or it is not header-only.
A header-only library B can depend on symbols being provided by the
calling program P, which can delegate the task to library A, or to
another library C which provides the same interface as A. There should
be no direct connection from B to A. Depending on the design of B, this
may involve #including a header from A before #including a header from
B, or it can involve making the headers from A available to B so that B
can #include them directly. Where these symbols and/or headers come
from is the business of P, not of B.
--
Rainer Deyke (rai...@eldwood.com)
Andrey Semashev via Boost
> On 08.06.21 09:33, Gavin Lambert via Boost wrote:
>> On 8/06/2021 4:53 pm, Edward Diener wrote:
>>> I do not know CMake, so maybe my comment is irrelevant, but it seems
>>> natural to me that a header-only library would always choose to use
>>> some other dependent library as header-only, even when that other
>>> library had static or shared variants. In other words I applaud the
>>> decision of Boost.Test to provide a header-only variant and can not
>>> even begin to process the fact that CMake can not deal with
>>> Boost.Test as a header-only library, if that is indeed the case.
>>
>> A problem occurs when program P uses library A as static or shared and
>> then wants to use header-only library B which also uses library A.
>>
>> For this to work, one of the following must be true:
>> 1. library B can figure out how program P chose to link to it and
>> links to it the exact same way.
>
> A header-only library cannot *link* to anything, or it is not header-only.
CMake defines targets for header-only libraries, which can have
dependencies on other targets, including on static or shared libraries.
So the program that "links" to the header-only library picks up its
dependencies recursively. As a result, the linker is invoked with all
(binary) libraries in the dependency tree.
Whether you can consider such library header-only in the first place is
a philosophical question. I do, because otherwise any library that calls
the standard library cannot be called header-only.
> A header-only library B can depend on symbols being provided by the
> calling program P, which can delegate the task to library A, or to
> another library C which provides the same interface as A. There should
> be no direct connection from B to A. Depending on the design of B, this
> may involve #including a header from A before #including a header from
> B, or it can involve making the headers from A available to B so that B
> can #include them directly. Where these symbols and/or headers come
> from is the business of P, not of B.
Reverse dependencies are rather uncommon, and it's not what is being
discussed here. The question is how the header-only library selects the
targets to depend on.
I think, it should not select any specific target unless it absolutely
must (i.e. it won't work otherwise). In terms of the earlier example,
the library B should depend on the generic target A. Now, that target A
can be defined as a header-only, shared or static variant, depending on
the user's choice or some default.
Gavin Lambert via Boost
> Another way of putting all this is that libraries that can be configured
> between static/shared are mostly fine these days, because the tools have
> some blunt hammers that mostly do the right things with them. (But
> still bite people on occasion.)
FWIW, the most common biting issue is when mixing static/shared.
All-static and all-shared are usually pretty safe.
But when libraries A and B both use dependency C, and either of A and B
is shared, then C must be shared as well -- things will often break
horribly if C is static.
(Having said that, what actually happens is highly dependent on whether
using Windows or something UNIXy, and for the latter what symbol
visibility level is being used. And for some libraries and some
platforms/settings then there will be no apparent problems.)
But this sort of thing is usually why tools like CMake prefer to offer
global settings to configure static/shared rather than letting it be
configured on a target-by-target basis. You *can* override that, but it
is at your own peril. (Or that of the downstream app.)
Alexander Grund via Boost
> FWIW, the most common biting issue is when mixing static/shared.
>
> All-static and all-shared are usually pretty safe.
>
> But when libraries A and B both use dependency C, and either of A and
> B is shared, then C must be shared as well -- things will often break
> horribly if C is static.
>
> (Having said that, what actually happens is highly dependent on
> whether using Windows or something UNIXy, and for the latter what
> symbol visibility level is being used. And for some libraries and
> some platforms/settings then there will be no apparent problems.)
>
> But this sort of thing is usually why tools like CMake prefer to offer
> global settings to configure static/shared rather than letting it be
> configured on a target-by-target basis. You *can* override that, but
> it is at your own peril. (Or that of the downstream app.)
usually(!) an anti-pattern. You should just have foo (i.e. Boost::foo)
and let CMake decide what that should be based on BUILD_SHARED (the one
variable whose naming is bad)
Of course this still allows to build your Boost::foo explicitely as
static or shared in which case the problem is also avoided as then only
one version of it exists anyway.
> A header-only library cannot *link* to anything, or it is not
header-only.
"depends on", which IMO is correct. So e.g. a header-only threading
library will depend on pthreads, so it will link to pthreads.
Of course you could omit linking to pthread at the cost of the user
having to find out on what this "header-only" library depends on and
having to link all those manually.
Gavin Lambert via Boost
> Of course this still allows to build your Boost::foo explicitely as
> static or shared in which case the problem is also avoided as then only
> one version of it exists anyway.
used by at least two different consumers (at least one of which is
shared) that end up as part of a single final application, that's
exactly the problem case that I was talking about.
This is because that can end up with multiple copies of the library in
the same application (but in separate binaries), subject to the whims of
the platform and linker.
Alexander Grund via Boost
Am 11.06.21 um 09:33 schrieb Gavin Lambert via Boost:
>> Of course this still allows to build your Boost::foo explicitely as
>> static or shared in which case the problem is also avoided as then
>> only one version of it exists anyway.
>
> No, if you choose to build Boost::foo as static and it happens to be
> used by at least two different consumers (at least one of which is
> shared) that end up as part of a single final application, that's
> exactly the problem case that I was talking about.
>
> This is because that can end up with multiple copies of the library in
> the same application (but in separate binaries), subject to the whims
> of the platform and linker.
`add_library(Boost::Foo STATIC source.cpp)` as opposed to
`add_library(Boost::Foo source.cpp)` in CMake terms.
So there is only static Boost::Foo and no shared version EVER.
This would work, wouldn't it?
Rainer Deyke via Boost
> On 6/9/21 2:05 PM, Rainer Deyke via Boost wrote:
>> On 08.06.21 09:33, Gavin Lambert via Boost wrote:
>>> For this to work, one of the following must be true:
>>> 1. library B can figure out how program P chose to link to it and
>>> links to it the exact same way.
>>
>> A header-only library cannot *link* to anything, or it is not
>> header-only.
>
> CMake defines targets for header-only libraries, which can have
> dependencies on other targets, including on static or shared libraries.
> So the program that "links" to the header-only library picks up its
> dependencies recursively. As a result, the linker is invoked with all
> (binary) libraries in the dependency tree.
If this functionality works the way I think does, then I think it should
not be used.
> Whether you can consider such library header-only in the first place is
> a philosophical question. I do, because otherwise any library that calls
> the standard library cannot be called header-only.
Actually, the standard library is a great example of what I am talking
about. A header-only library (B) should not care about the specific
implementation of the standard library (A) being used by the program
(P). There is no link dependency from B to A. There is a requirement
from B to P that P must provide /a/ implementation of the C++ standard
library, but B doesn't care if it is libstdc++ or libc++ or even a
custom standard library implementation that is part of P and not a
separate library at all.
Again, to make my point absolutely clear: header-only library B "using"
library A does not imply a dependency from B to A. Instead, it implies
a requirement on a program P that uses B to also link to A. Header-only
libraries use interfaces. Programs link to implementations.
--
Rainer Deyke (rai...@eldwood.com)
Deniz Bahadir via Boost
> On 10.06.21 23:36, Andrey Semashev via Boost wrote:
>> On 6/9/21 2:05 PM, Rainer Deyke via Boost wrote:
>>> On 08.06.21 09:33, Gavin Lambert via Boost wrote:
>>>> For this to work, one of the following must be true:
>>>> 1. library B can figure out how program P chose to link to it
>>>> and links to it the exact same way.
>>>
>>> A header-only library cannot *link* to anything, or it is not
>>> header-only.
>>
>> CMake defines targets for header-only libraries, which can have
>> dependencies on other targets, including on static or shared
>> libraries. So the program that "links" to the header-only library
>> picks up its dependencies recursively. As a result, the linker is
>> invoked with all (binary) libraries in the dependency tree.
>
> If this functionality works the way I think does, then I think it
> should not be used.
And in what way do you think it works? (Honest question.)
>
>> Whether you can consider such library header-only in the first place
>> is a philosophical question. I do, because otherwise any library that
>> calls the standard library cannot be called header-only.
>
> Actually, the standard library is a great example of what I am talking
> about. A header-only library (B) should not care about the specific
> implementation of the standard library (A) being used by the program
> (P). There is no link dependency from B to A. There is a requirement
> from B to P that P must provide /a/ implementation of the C++ standard
> library, but B doesn't care if it is libstdc++ or libc++ or even a
> custom standard library implementation that is part of P and not a
> separate library at all.
That works for the standard-library, because you can be sure that one is
available. (Otherwise your C++ compiler would be broken.)
However, for other libraries A there is no way to guarantee that.
Of course, you need to make sure that (any) dependency A is available on
your build machine, otherwise building could never succeed. But the
extra burden, to prepare the build-files (aka CMakeLists.txt) of your
program P to find and explicitly configure even indirect dependencies
(using `find_package`, `target_link_libraries`,
`target_include_directories` etc.) is not to be underestimated.
For a complex dependency-hierarchy this can become a nightmare, in
particular if you need to find out in what order to process the
individual build-files (`add_subdirectories`, `find_package` etc.).
Instead, program P should only be expected to declare its direct
dependencies (using `target_link_libraries`) and automatically get all
the usage-requirements (aka dependencies) of these. This only requires,
that every dependency (CMake target) carries its own usage-requirements.
And, IMO, this should be true for header-only CMake targets, too.
>
> Again, to make my point absolutely clear: header-only library B
> "using" library A does not imply a dependency from B to A. Instead, it
> implies a requirement on a program P that uses B to also link to A.
> Header-only libraries use interfaces. Programs link to implementations.
>
I just want to emphasize that even such interfaces are part of the
usage-requirements I was talking about.
Therefore B has a **usage-dependency** on A. And the CMake target for B
should communicate this (through its usage-requirements) to its user,
the program P (or more specific its CMake target).
The usage-requirements of the CMake target for B will become the
build-requirements of the CMake target for P (and could possibly even
become its usage-requirements as well).
And whether A is header-only or not is irrelevant. This (indirect)
dependency from P to A can either mean linking against A or just having
the correct include-search-path for the headers of A.
The CMakeLists.txt file for building P needs to have this information
either way. And this can be provided explicitly (as you prefer) which
can become quite complex and tedious, or automatically and indirectly
through B.
Deniz
PS: Maybe watching the first 11 minutes of my CMake talk [1] from
Meeting C++ 2018 might help to better understand this target-centric
approach CMake took.
[1] https://youtu.be/y7ndUhdQuU8
--
BENOCS GmbH
Dipl.-Inform. Deniz Bahadir
Reuchlinstr. 10 D
10553 Berlin
Germany
Phone: +49 - 30 / 577 0004-22
Email: deniz....@benocs.com
www.benocs.com
Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese
Commercial Register: Amtsgericht Bonn HRB 19378
Peter Dimov via Boost
> On 10.06.21 23:36, Andrey Semashev via Boost wrote:
> > CMake defines targets for header-only libraries, which can have
> > dependencies on other targets, including on static or shared libraries.
> > So the program that "links" to the header-only library picks up its
> > dependencies recursively. As a result, the linker is invoked with all
> > (binary) libraries in the dependency tree.
>
> If this functionality works the way I think does, then I think it should not be
> used.
headers are dumped into a central location, e.g. /usr/include or, in our
case, $BOOST_ROOT/boost. That it works with our b2 setup is an artifact
of this physical organization (and requires our `b2 headers` step).
If the libraries are properly separated, which should really be required
by any sane package management scheme, you have to "link" to
header-only libraries in order to get the proper directory into your
include path.
As a side effect, this also makes the compiled -> header-only -> compiled
case work without any effort from the end user (who would otherwise be
forced to change his link line if an update to 'header-only' changes the list
of compiled libraries that need to be linked).
Rainer Deyke via Boost
> Am 11.06.21 um 11:17 schrieb Rainer Deyke via Boost:
>> should not be used.
>
> And in what way do you think it works? (Honest question.)
I think it forces the users of library B to (indirectly) link to library
A, as opposed any other library that implements the interface of A. In
the worst case, it might even unnecessarily force a specific version of A.
>> Actually, the standard library is a great example of what I am talking
>> about. A header-only library (B) should not care about the specific
>> implementation of the standard library (A) being used by the program
>> (P). There is no link dependency from B to A. There is a requirement
>> from B to P that P must provide /a/ implementation of the C++ standard
>> library, but B doesn't care if it is libstdc++ or libc++ or even a
>> custom standard library implementation that is part of P and not a
>> separate library at all.
>
> That works for the standard-library, because you can be sure that one is
> available. (Otherwise your C++ compiler would be broken.)
> However, for other libraries A there is no way to guarantee that.
You guarantee it by documenting it as a requirement for the users of B.
The human users, not CMake.
> Of course, you need to make sure that (any) dependency A is available on
> your build machine, otherwise building could never succeed. But the
> extra burden, to prepare the build-files (aka CMakeLists.txt) of your
> program P to find and explicitly configure even indirect dependencies
> (using `find_package`, `target_link_libraries`,
> `target_include_directories` etc.) is not to be underestimated.
> For a complex dependency-hierarchy this can become a nightmare, in
> particular if you need to find out in what order to process the
> individual build-files (`add_subdirectories`, `find_package` etc.).
In order to build P, you need to build the specific versions of all of
the libraries that P requires, with the specific build settings that P
requires. The build process should never pick up random library
versions found in system directories. Program P must be able to say
"use this modified version of libpng instead of the upstream version",
"use LibreSSL instead of OpenSSL", and "don't use zlib at all, I'll
provide my own version of the zlib functions".
--
Rainer Deyke (rai...@eldwood.com)
Deniz Bahadir via Boost
If library B is a good citizen it uses `find_package` to look for its
dependencies (more specifically, for the CMake target of its
dependencies) and provides customization points (CMake options or simple
variables).
So if Program P wants to force a specific version of the indirect
dependency A upon B it can set these customization points beforehand or
it can just call `find_package` for that indirect dependency explicitly
beforehand. The later call to `find_package` from B's CMakeLists.txt
should then just return the same target (without even bothering searching).
Of course, this requires careful crafting of B's CMakeLists.txt. (But it
really can be worth it.)
In case the indirect dependency A is header-only itself, it becomes even
simpler because you do not have to cope with a library that might have
been compiled using different/incompatible compiler-flags.
IMO, CMake is quite powerful and one can achieve some really "funky stuff".
But, as always, one should remember to not make it too complex or one
will be unable to find anyone else who is willing to maintain it later.
Deniz
--
BENOCS GmbH
Dipl.-Inform. Deniz Bahadir
Reuchlinstr. 10 D
10553 Berlin
Germany
Phone: +49 - 30 / 577 0004-22
Email: deniz....@benocs.com
www.benocs.com
Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese
Commercial Register: Amtsgericht Bonn HRB 19378
Gavin Lambert via Boost
> I meant the case where Boost::foo is ONLY ever build as static. I.e.
> `add_library(Boost::Foo STATIC source.cpp)` as opposed to
> `add_library(Boost::Foo source.cpp)` in CMake terms.
>
> So there is only static Boost::Foo and no shared version EVER.
> This would work, wouldn't it?
with multiple independent "islands".
Gavin Lambert via Boost
> Actually, the standard library is a great example of what I am talking
> about. A header-only library (B) should not care about the specific
> implementation of the standard library (A) being used by the program
> (P). There is no link dependency from B to A. There is a requirement
> from B to P that P must provide /a/ implementation of the C++ standard
> library, but B doesn't care if it is libstdc++ or libc++ or even a
> custom standard library implementation that is part of P and not a
> separate library at all.
It does, however, need to be sure that it is using the same standard
library as P (and with the same settings). Mixing multiple standard
libraries into the same P will only end poorly -- which is usually also
true (but less immediately obvious) when mixing multiple variants of
other single libraries into P.
You're assuming that by not specifying an explicit link from B to the
standard library that it will inherently have the same settings as P.
You're even mostly correct, because the standard library has a bit of
special handling in the build tools.
But this issue is not unique to the standard library, but applies to any
dependency, including ones that you do have to explicitly tell the build
tools about.
Rainer Deyke via Boost
> On 11/06/2021 9:17 pm, Rainer Deyke wrote:
>> Actually, the standard library is a great example of what I am talking
>> about. A header-only library (B) should not care about the specific
>> implementation of the standard library (A) being used by the program
>> (P). There is no link dependency from B to A. There is a requirement
>> from B to P that P must provide /a/ implementation of the C++ standard
>> library, but B doesn't care if it is libstdc++ or libc++ or even a
>> custom standard library implementation that is part of P and not a
>> separate library at all.
>
> It does, however, need to be sure that it is using the same standard
> library as P (and with the same settings). Mixing multiple standard
> libraries into the same P will only end poorly -- which is usually also
> true (but less immediately obvious) when mixing multiple variants of
> other single libraries into P.
>
> You're assuming that by not specifying an explicit link from B to the
> standard library that it will inherently have the same settings as P.
> You're even mostly correct, because the standard library has a bit of
> special handling in the build tools.
No, I'm saying that by not having a separate compilation step, there is
literally no way for B to have different settings than P, because there
is nothing that could be compiled with different settings. This is true
regardless of the build system, because it is true at the compile level.
There is no set of arguments that the build system can pass to the
compiler that would cause B to be built with different settings than P.
B is a header-only library. To use B, the C++ preprocessor literally
pastes the code from B into P. Therefore, B /has/ no build settings.
There are only the build settings from P, which are applied
indiscriminately to P's own code and the code from B that P #includes.
--
Rainer Deyke (rai...@eldwood.com)
Gavin Lambert via Boost
>> You're assuming that by not specifying an explicit link from B to the
>> standard library that it will inherently have the same settings as P.
>> You're even mostly correct, because the standard library has a bit of
>> special handling in the build tools.
>
> No, I'm saying that by not having a separate compilation step, there is
> literally no way for B to have different settings than P, because there
> is nothing that could be compiled with different settings. This is true
> regardless of the build system, because it is true at the compile level.
> There is no set of arguments that the build system can pass to the
> compiler that would cause B to be built with different settings than P.
>
> B is a header-only library. To use B, the C++ preprocessor literally
> pastes the code from B into P. Therefore, B /has/ no build settings.
> There are only the build settings from P, which are applied
> indiscriminately to P's own code and the code from B that P #includes.
That's actually completely false. The preprocessor itself *is* build
settings -- defines can be defined differently (along with other
compiler options) between the two builds.
And yes, there may still be two builds. There is the build of P itself
(which includes A) and the build of another library C that is linked
into P but also includes A. Both of these can be via B or separately,
and both can cause different code for B to be generated if the settings
differ, which is an ODR violation.
(Or cause two distinct copies of B to be generated, where C is a shared
library with private visibility, for example.)
For some examples, there's usually a compiler option to choose whether
the standard library is linked as static or as shared, which in turn
causes different #defines to be defined that will affect compilation.
Another example is things that can vary due to the C++ language
selected, or whether or not you've defined various "posix extension"
flags, or library-specific configuration defines. Or something as
simple as compiling one with exception support and one without.
Having a header-only library is not a silver bullet to avoid these kinds
of compilation incompatibilities. In fact it makes some of them worse.
Rainer Deyke via Boost
> On 14/06/2021 5:34 pm, Rainer Deyke wrote:
>> B is a header-only library. To use B, the C++ preprocessor literally
>> pastes the code from B into P. Therefore, B /has/ no build settings.
>> There are only the build settings from P, which are applied
>> indiscriminately to P's own code and the code from B that P #includes.
>
> That's actually completely false. The preprocessor itself *is* build
> settings -- defines can be defined differently (along with other
> compiler options) between the two builds.
There are no two builds in my example. There is only P. Yes, using B
modifies the build settings of P, but B has no build settings of its own.
By the same token, linking P to separately compiled library A also
modifies the build settings of P, but this is separate from the build
settings that are actually used to /build/ A.
> And yes, there may still be two builds. There is the build of P itself
> (which includes A) and the build of another library C that is linked
> into P but also includes A. Both of these can be via B or separately,
> and both can cause different code for B to be generated if the settings
> differ, which is an ODR violation.
That's a different situation. If you add a separately compiled library
C to the equation, then C does have its own build settings, which need
to be compatible with P's build settings. This is true whether C uses B
or not, and whether B is header-only or not.
Or you could, if C allows it, use C as a header-only library, which
completely solves the problem.
--
Rainer Deyke (rai...@eldwood.com)