[boost] Boost modular: Build without version number suffixes

[Robert Dailey]

When I build modular boost on Linux Mint 17, I get SO files named like so:

libboost_date_time.so.1.57.0

with a corresponding symlink without the version number:

libboost_date_time.so

I'd like it to build without the version number because when I rename
the base file to remove the version, code links fine but during
runtime it tries to load the SO with the version number at the end. I
don't know how to fix that, so I assume the fix is to build boost
without the version numbers.

Any help is appreciated.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Robert Dailey]

On Sun, Jan 25, 2015 at 10:48 PM, Robert Dailey
<rcdaile...@gmail.com> wrote:
> When I build modular boost on Linux Mint 17, I get SO files named like so:
>
> libboost_date_time.so.1.57.0
>
> with a corresponding symlink without the version number:
>
> libboost_date_time.so
>
> I'd like it to build without the version number because when I rename
> the base file to remove the version, code links fine but during
> runtime it tries to load the SO with the version number at the end. I
> don't know how to fix that, so I assume the fix is to build boost
> without the version numbers.
>
> Any help is appreciated.

I figured out what I had to do. I made the following change to
boostcpp.jam (around line 158):


# Optionally add version suffix. On NT, library with version suffix will
# not be recognized by linkers. On CYGWIN, we get strage duplicate
# symbol errors when library is generated with version suffix. On OSX,
# version suffix is not needed -- the linker expects the
# libFoo.1.2.3.dylib format. AIX linkers do not accept version suffixes
# either. Pgi compilers can not accept a library with version suffix.
if $(type) = SHARED_LIB &&
! [ $(property-set).get <target-os> ] in windows cygwin darwin aix &&
! [ $(property-set).get <toolset> ] in pgi
{
result = $(result) ; #.$(BOOST_VERSION) ;
}


The BOOST_VERSION was being added to the end with no apparent option
to turn it off. Would love to see an option to turn this off.

[Steve M. Robbins]

On January 25, 2015 10:48:12 PM Robert Dailey wrote:
> When I build modular boost on Linux Mint 17, I get SO files named like so:
>
> libboost_date_time.so.1.57.0
>
> with a corresponding symlink without the version number:
>
> libboost_date_time.so

That's standard practice on linux. The SO versioning means you can install,
say, Boost 1.60 alongside 1.57 and files linked against the older version
continue to work. (Boost does not maintain ABI compatibility across versions)

So ...

> I'd like it to build without the version number because when I rename
> the base file to remove the version, code links fine but during
> runtime it tries to load the SO with the version number at the end.

.... really curious why you want to do that.


-Steve

[Robert Dailey]

On Mon, Jan 26, 2015 at 12:07 AM, Steve M. Robbins <st...@sumost.ca> wrote:
> On January 25, 2015 10:48:12 PM Robert Dailey wrote:
>> When I build modular boost on Linux Mint 17, I get SO files named like so:
>>
>> libboost_date_time.so.1.57.0
>>
>> with a corresponding symlink without the version number:
>>
>> libboost_date_time.so
>
> That's standard practice on linux. The SO versioning means you can install,
> say, Boost 1.60 alongside 1.57 and files linked against the older version
> continue to work. (Boost does not maintain ABI compatibility across versions)
>
> So ...
>
>> I'd like it to build without the version number because when I rename
>> the base file to remove the version, code links fine but during
>> runtime it tries to load the SO with the version number at the end.
>
> .... really curious why you want to do that.
>
>
> -Steve
>

I have an application that I build on both Windows and Linux. For each
platform, I compile the dependent libraries (Boost, OpenSSL, etc) on
each platform and store the binaries in a git submodule (only binaries
and third party libs are stored in this submodule). I use CMake to
build this project and having the third party dependencies handled the
same way on both platforms simplifies my build scripts and, most
importantly, makes my project build out of the box without having to
install stuff via package manager (Windows has no such concept
anyway). It also guarantees that the libraries are always available
(for example if a version of OpenSSL falls off the edge of the planet,
I won't be in a crisis since I maintain it myself. This is important
to an enterprise).

Having explained all of that, I do not need the version numbering in
boost because different versions do not coexist. My build script looks
for all *.so files and performs 2 steps:

* For each SO file found, pass that as a link target to projects in
CMake that depend on it
* For each SO file found, copy that to the binary output directory so
that the executables that depend on those SO files may find them
(instead of delegating the search to PATH or whatnot).

it seems to me that the way boost builds SO files implies that the SO
file is meant to be used for linking only and the library that is
actually dynamically loaded at runtime is not the SO file but the ones
ending in a version. This inconsistency breaks for me as I've
explained above.

Hope that helps clarify.

[Rob Stewart]

On January 26, 2015 10:22:52 AM EST, Robert Dailey <rcdaile...@gmail.com> wrote:
> On Mon, Jan 26, 2015 at 12:07 AM, Steve M. Robbins <st...@sumost.ca>
> wrote:
> > On January 25, 2015 10:48:12 PM Robert Dailey wrote:
> >> When I build modular boost on Linux Mint 17, I get SO files named
> like so:
> >>
> >> libboost_date_time.so.1.57.0
> >>
> >> with a corresponding symlink without the version number:
> >>
> >> libboost_date_time.so
> >
> > That's standard practice on linux. The SO versioning means you can
> install,
> > say, Boost 1.60 alongside 1.57 and files linked against the older
> version
> > continue to work. (Boost does not maintain ABI compatibility across
> versions)

Right

> > So ...
> >
> >> I'd like it to build without the version number because when I
> rename
> >> the base file to remove the version, code links fine but during
> >> runtime it tries to load the SO with the version number at the end.
> >
> > .... really curious why you want to do that.
>

> I have an application that I build on both Windows and Linux. For each
> platform, I compile the dependent libraries (Boost, OpenSSL, etc) on
> each platform and store the binaries in a git submodule (only binaries
> and third party libs are stored in this submodule). I use CMake to
> build this project and having the third party dependencies handled the
> same way on both platforms simplifies my build scripts and, most
> importantly, makes my project build out of the box without having to
> install stuff via package manager (Windows has no such concept
> anyway). It also guarantees that the libraries are always available
> (for example if a version of OpenSSL falls off the edge of the planet,
> I won't be in a crisis since I maintain it myself. This is important
> to an enterprise).

We maintain our own collections of packages, too, but we do so in the filesystem.

> Having explained all of that, I do not need the version numbering in
> boost because different versions do not coexist. My build script looks
> for all *.so files and performs 2 steps:
>
> * For each SO file found, pass that as a link target to projects in
> CMake that depend on it
> * For each SO file found, copy that to the binary output directory so
> that the executables that depend on those SO files may find them
> (instead of delegating the search to PATH or whatnot).

We do that on Windows and use the binary's RUNPATH to find shared objects on Linux. However, we do reference shared objects in the usual Linux way: unversioned symlink to versioned shared object.

> it seems to me that the way boost builds SO files implies that the SO
> file is meant to be used for linking only and the library that is
> actually dynamically loaded at runtime is not the SO file but the ones
> ending in a version.

The symlink is unversioned so you implicitly get the version it references without having to link against the versioned shared object. (Your build always mentions the unversioned name.) The Linux linker knows to resolve the symlink and links against the versioned shared object for you. That leads to the runtime reference to the versioned shared object. Two applications linked that way, at different times, can reference different versions of the shared object.

> This inconsistency breaks for me as I've
> explained above.

I understand that you want consistency between platforms. Just beware that fighting the norms of a platform will cause difficulties. That said, ISTR an option for Boost Build to build things as you want, but I don't know what it is. (I added "[build]" to the subject in hopes that will attract the right attention to your query.

___
Rob

(Sent from my portable computation engine)

[Vladimir Prus]

On 01/27/2015 12:39 PM, Rob Stewart wrote:
>> it seems to me that the way boost builds SO files implies that the SO
>>> file is meant to be used for linking only and the library that is actually dynamically loaded at runtime is not the SO file but the
>>> ones ending in a version.
> The symlink is unversioned so you implicitly get the version it references without having to link against the versioned shared object.
> (Your build always mentions the unversioned name.) The Linux linker knows to resolve the symlink and links against the versioned shared
> object for you. That leads to the runtime reference to the versioned shared object. Two applications linked that way, at different times,
> can reference different versions of the shared object.

And I would imagine that the script that copies Boost libraries into the final location can be modified to also copy
so.NNN files.

>>> This inconsistency breaks for me as I've explained above.
> I understand that you want consistency between platforms. Just beware that fighting the norms of a platform will cause difficulties. That
> said, ISTR an option for Boost Build to build things as you want, but I don't know what it is. (I added "[build]" to the subject in hopes
> that will attract the right attention to your query.

I don't believe there is such an option. I will be open to a new value for --layout option, say 'bare', that suppresses version
number completely. Robert, since you've already identified the code in question, and it's not particularly specific to Boost.Build,
would you be interested in submitting a pull request?

Thanks,

--
Vladimir Prus
CodeSourcery / Mentor Embedded
http://vladimirprus.com

[Beman Dawes]

On Tue, Jan 27, 2015 at 5:26 AM, Vladimir Prus
<vlad...@codesourcery.com> wrote:
> On 01/27/2015 12:39 PM, Rob Stewart wrote:
>>>
>>
>> I understand that you want consistency between platforms. Just beware that
>> fighting the norms of a platform will cause difficulties. That
>> said, ISTR an option for Boost Build to build things as you want, but I
>> don't know what it is. (I added "[build]" to the subject in hopes
>> that will attract the right attention to your query.
>
>
> I don't believe there is such an option. I will be open to a new value for
> --layout option, say 'bare', that suppresses version
> number completely. Robert, since you've already identified the code in
> question, and it's not particularly specific to Boost.Build,
> would you be interested in submitting a pull request?

We may also want to add something to the "Getting started..." docs
mentioning the option and its uses.

--Beman

[Robert Dailey]

On Tue, Jan 27, 2015 at 7:09 AM, Beman Dawes <bda...@acm.org> wrote:
> On Tue, Jan 27, 2015 at 5:26 AM, Vladimir Prus
> <vlad...@codesourcery.com> wrote:
>> On 01/27/2015 12:39 PM, Rob Stewart wrote:
>>>>
>>>
>>> I understand that you want consistency between platforms. Just beware that
>>> fighting the norms of a platform will cause difficulties. That
>>> said, ISTR an option for Boost Build to build things as you want, but I
>>> don't know what it is. (I added "[build]" to the subject in hopes
>>> that will attract the right attention to your query.
>>
>>
>> I don't believe there is such an option. I will be open to a new value for
>> --layout option, say 'bare', that suppresses version
>> number completely. Robert, since you've already identified the code in
>> question, and it's not particularly specific to Boost.Build,
>> would you be interested in submitting a pull request?
>
> We may also want to add something to the "Getting started..." docs
> mentioning the option and its uses.

I'd be happy to do a pull request for this.

I realize I could edit the copy script to perform symlink creation or
copy files with the version number at the end. The problem is this
introduces conditional logic based on platform (which makes this more
difficult for me) and also it's much easier to look for *.so. There is
no clear pattern when a version number is at the end, I'd have to use
a regular expression to support version number changes later.

Again I understand this is a platform norm but the function of the
version number at the end IMHO is to resolve conflicts when installed
through package manager (so that multiple boost versions can live side
by side). When I have no need for that, I don't see myself as
"breaking the norms" by using the name without the version. In fact,
it's perfectly common to see libs with just *.so at the end. Every
other library I've built does this out of the box except for Boost.

I am not familiar with contribution best practices for Boost. Is there
recommended reading material? I'll make sure I understand everything
so I don't waste anyone's time. I don't like doing patches via email
because it's overly complicated, but if you guys use github pull
requests I'm more than happy to use that. Thanks again everyone :)

[Steve M. Robbins]

On January 27, 2015 10:34:39 AM Robert Dailey wrote:

> >> I don't believe there is such an option.

I seem to recall that unversioned was to be the only option in the distant
past. But several of us pushed for matching the system norms. :-)

> >> I will be open to a new value
> >> for
> >> --layout option, say 'bare', that suppresses version
> >> number completely. Robert, since you've already identified the code in
> >> question, and it's not particularly specific to Boost.Build,
> >> would you be interested in submitting a pull request?

> I'd be happy to do a pull request for this.

Be sure not to change only the file name, but also suppress the SOVERSION
embedded in the file itself.

> I realize I could edit the copy script to perform symlink creation or
> copy files with the version number at the end. The problem is this
> introduces conditional logic based on platform (which makes this more
> difficult for me) and also it's much easier to look for *.so. There is
> no clear pattern when a version number is at the end, I'd have to use
> a regular expression to support version number changes later.

If you are currently copying "*.so", wouldn't it be quite easy to change that
to "*.so*"?

> Again I understand this is a platform norm but the function of the
> version number at the end IMHO is to resolve conflicts when installed
> through package manager (so that multiple boost versions can live side
> by side). When I have no need for that, I don't see myself as
> "breaking the norms" by using the name without the version. In fact,
> it's perfectly common to see libs with just *.so at the end. Every
> other library I've built does this out of the box except for Boost.

In fact, that's quite rare. If you look at /usr/lib on most linux
distributions (e.g. Debian) you will see the vast majority are versioned, with
an unversioned .so symlink for link-time.

Also note that unless you take precautions (e.g. using rpath), the run time
loader will in fact look first in the system locations such as /usr/lib and
often including /usr/local/lib for the library. If you don't control the
content of /usr, this leaves you open to obscure bugs where you are running
against something other than what you shipped.


-Steve