[boost] Using both boost 64 and 32 bit on Windows with CMake

[Klaim - Joël Lamotte]

It appear that this mailing list don't accept cross messages so I'm
posting a link to a discussion related to both boost and cmake.

Basically I'm looking for guidance to help improving development of
one application using boost in 64 and 32 bit modes
by finding a way to make the FindBoost cmake module find the
right binaries depending on either mode.
Currently you have to specify an undocumented CMake variable manually
to do this.

See the full discussion:

http://cmake.3232098.n2.nabble.com/boost-b2-Differenciate-32bit-and-64-bit-binaries-when-using-CMake-td7581982.html

Joel Lamotte

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Mathias Gaunard]

On 15/04/13 17:20, Klaim - Joël Lamotte wrote:
> It appear that this mailing list don't accept cross messages so I'm
> posting a link to a discussion related to both boost and cmake.
>
> Basically I'm looking for guidance to help improving development of
> one application using boost in 64 and 32 bit modes
> by finding a way to make the FindBoost cmake module find the
> right binaries depending on either mode.

If you're building 32 and 64 bit on the same machine, then at least of
one these is cross-compilation.
With CMake (and other build systems), cross-compilation cannot happen
automatically, you have to specify manually which directories to use to
find libraries. Using the default may easily result in conflicts with
the native libraries installed.

This is not a Boost-specific problem, this will affect any use of a find
module and even find_library.

> Currently you have to specify an undocumented CMake variable manually
> to do this.

Boost_DIR, BOOST_ROOT, BOOSTROOT, BOOST_INCLUDEDIR and BOOST_LIBRARYDIR
are all documented.

[Klaim - Joël Lamotte]

On Tue, Apr 16, 2013 at 8:13 AM, Mathias Gaunard <
mathias...@ens-lyon.org> wrote:

>
> If you're building 32 and 64 bit on the same machine, then at least of one
> these is cross-compilation.
> With CMake (and other build systems), cross-compilation cannot happen
> automatically, you have to specify manually which directories to use to
> find libraries. Using the default may easily result in conflicts with the
> native libraries installed.
>
>

With CMake on windows you just have to use the "Visual Studio x64" to
compile for 64 bit and the non x64 to compile in 32 bit. (see
http://cmake.org/cmake/help/v2.8.10/cmake.html#section_Generators)
The CMake scripts can take into account in which mode libraries have to be
used and it works well for libraries that take this into account, not like
boost.

You can consider this problem as Windows/VS-specific if you will. I have no
idea if the same problem is true on linux or with gcc/mingw/clang.

> This is not a Boost-specific problem, this will affect any use of a find
> module and even find_library.
>
>

Obviously it IS a boost specific problem as it is one of the few libraries
which don't have any basic way of differenciate 32 and 64bit binaries.
The CMake module can't rely on any such knowledge (on Windows?) to
correctly find the right binaries depending on both the current mode
selected on the CMake configuration level.
All other libraries I use provide both 32 and 64 bit binaries at specific
emplacements (relative to where they are installed) and the associated
CMake module will chose one
or the other depending on the mode choosen on configuration.

Boost just makes things complicated by not specifying a way to
differenciate these binaries (again, on Windows).

I understand the default lib directory should contain the native binaries.
Now, would there be another solution to help with CMake modules
implementation to get the right binaries
depending on the generator choice?

>
> Currently you have to specify an undocumented CMake variable manually
>> to do this.
>>
>
> Boost_DIR, BOOST_ROOT, BOOSTROOT, BOOST_INCLUDEDIR and BOOST_LIBRARYDIR
> are all documented.
>

If you read the link you would know I'm talking about the CMake help
documentation which don't describe BOOST_LIBRARYDIR at all.
When you're using CMake to configure your libraries, that's where you look.
I filed a ticket for this.

Joel Lamotte

[Mathias Gaunard]

On 16/04/13 12:00, Klaim - Joël Lamotte wrote:

> Boost just makes things complicated by not specifying a way to
> differenciate these binaries (again, on Windows).

The Boost naming scheme does not encode 32/64 bit, nor does the build
system build both for you.
There is nothing FindBoost can do.

Just put 32-bit and 64-bit variants in different directories, and
configure your build to use the right directory for your build.

> If you read the link you would know I'm talking about the CMake help
> documentation which don't describe BOOST_LIBRARYDIR at all.
> When you're using CMake to configure your libraries, that's where you look.
> I filed a ticket for this.

The documentation of a find module is whatever is written at the
beginning of the file in question.

[Klaim - Joël Lamotte]

On Tue, Apr 16, 2013 at 2:45 PM, Mathias Gaunard <
mathias...@ens-lyon.org> wrote:

>
> The Boost naming scheme does not encode 32/64 bit, nor does the build
> system build both for you.
> There is nothing FindBoost can do.
>
>

That's exactly what I am saying.

> Just put 32-bit and 64-bit variants in different directories, and
> configure your build to use the right directory for your build.
>
>

That's what I am doing but I am not satisfied with having to re-define the
BOOST_LIBRARYDIR each time I have to configure
a cmake project, which I do a lot when working with CMake because I need to
make sure the CMake scripts works when
generating from scratch.
I'm also not satisfied to have co-workers spending time on setting up a
build system just to start working on my project,
which is why I put a lot of effort into minimizing the time to start
working on actual code.

>
>
> If you read the link you would know I'm talking about the CMake help
>> documentation which don't describe BOOST_LIBRARYDIR at all.
>> When you're using CMake to configure your libraries, that's where you
>> look.
>> I filed a ticket for this.
>>
>
> The documentation of a find module is whatever is written at the beginning
> of the file in question.
>

The HELP documentation is whatever is written when you use "cmake
--help-module <module-name>" and once again that variable is not
documented in the "cmake --help-module FindBoost" command.

Not everybody like to have to dive in the source files of their tools to
understand them, or have time to do so.

Joel Lamotte

[Andrey Semashev]

On Tuesday 16 April 2013 17:42:28 Klaim - Joël Lamotte wrote:
> On Tue, Apr 16, 2013 at 2:45 PM, Mathias Gaunard <
>

> > Just put 32-bit and 64-bit variants in different directories, and
> > configure your build to use the right directory for your build.
>
> That's what I am doing but I am not satisfied with having to re-define the
> BOOST_LIBRARYDIR each time I have to configure
> a cmake project, which I do a lot when working with CMake because I need to
> make sure the CMake scripts works when
> generating from scratch.
> I'm also not satisfied to have co-workers spending time on setting up a
> build system just to start working on my project,
> which is why I put a lot of effort into minimizing the time to start
> working on actual code.

You're not bound to these particular variables. CMake scripts can use
environment varables and command line switches, which covers different
configurations quite well. Just define a couple of your configuration
variables for different targets and be done with it. You can even create your
own CMake module that handles all the configuration specifics you have.

> > The documentation of a find module is whatever is written at the beginning
> > of the file in question.
>
> The HELP documentation is whatever is written when you use "cmake
> --help-module <module-name>" and once again that variable is not
> documented in the "cmake --help-module FindBoost" command.
>
> Not everybody like to have to dive in the source files of their tools to
> understand them, or have time to do so.

I think CMake documentation issues are not relevant to Boost per se.

[Michael Marcin]

FWIW when using boost with CMake projects I use a slightly modified
FindBoost.cmake module.


*** /Users/jaeger/Documents/FindBoostOrig.cmake 2012-11-27
07:26:32.000000000 -0600
--- /Volumes/BOOTCAMP/code/cmake-utils/FindBoost.cmake 2013-02-15
12:40:21.000000000 -0600
***************
*** 929,934 ****
--- 929,940 ----
endif()
endif()

+ if(CMAKE_CL_64)
+ set(BOOST_ARCH x64)
+ else()
+ set(BOOST_ARCH x86)
+ endif()
+
foreach(COMPONENT ${Boost_FIND_COMPONENTS})
string(TOUPPER ${COMPONENT} UPPERCOMPONENT)
set( Boost_${UPPERCOMPONENT}_LIBRARY
"Boost_${UPPERCOMPONENT}_LIBRARY-NOTFOUND" )
***************
*** 967,972 ****
--- 973,979 ----
HINTS ${_boost_LIBRARY_SEARCH_DIRS}
${_boost_FIND_OPTIONS}
DOC "${_boost_docstring_release}"
+ PATH_SUFFIXES /${BOOST_ARCH}
)

#
***************
*** 999,1004 ****
--- 1006,1012 ----
HINTS ${_boost_LIBRARY_SEARCH_DIRS}
${_boost_FIND_OPTIONS}
DOC "${_boost_docstring_debug}"
+ PATH_SUFFIXES /${BOOST_ARCH}
)

if(Boost_REALPATH)







After building boost with this batch file


@echo off
ECHO build start

CALL bootstrap.bat

rmdir lib /S /Q
mkdir lib

set BUILD_OPTIONS=--build-dir=c:\boost\boost_build --layout=versioned
--stagedir=.\lib toolset=msvc-11.0 link=static threading=multi
-sNO_ZLIB=0 -sZLIB_SOURCE=c:\zlib\zlib-1.2.7

ECHO x86 START
b2 %BUILD_OPTIONS%

ren lib\lib x86
ECHO x86 END

ECHO x64 START
b2 %BUILD_OPTIONS% address-model=64
ren lib\lib x64
ECHO x64 END

ECHO build done

[Klaim - Joël Lamotte]

On Tue, Apr 16, 2013 at 7:36 PM, Andrey Semashev
<andrey....@gmail.com>wrote:

>
> You're not bound to these particular variables. CMake scripts can use
> environment varables and command line switches, which covers different
> configurations quite well. Just define a couple of your configuration
> variables for different targets and be done with it. You can even create
> your
> own CMake module that handles all the configuration specifics you have.
>
>

Yes, but it don't solve the problem "in general as there is no general way
to identify boost binaries other than the
native ones. Which means even if I provide a FindBoost version which is
modified (like Michael's one)
it will only work if the developer have put the binaries the way I decided
they should be, not in a "conventional" way,
which is my problem.

> > > The documentation of a find module is whatever is written at the
> beginning
> > > of the file in question.
> >
> > The HELP documentation is whatever is written when you use "cmake
> > --help-module <module-name>" and once again that variable is not
> > documented in the "cmake --help-module FindBoost" command.
> >
> > Not everybody like to have to dive in the source files of their tools to
> > understand them, or have time to do so.
>
> I think CMake documentation issues are not relevant to Boost per se.
>
>

Yes, as said before I made an issue ticket in their tracker so it's
recorded.

Joel Lamotte

[Andrey Semashev]

On Tuesday 16 April 2013 22:05:34 Klaim - Joël Lamotte wrote:
> On Tue, Apr 16, 2013 at 7:36 PM, Andrey Semashev
>
> <andrey....@gmail.com>wrote:
> > You're not bound to these particular variables. CMake scripts can use
> > environment varables and command line switches, which covers different
> > configurations quite well. Just define a couple of your configuration
> > variables for different targets and be done with it. You can even create
> > your
> > own CMake module that handles all the configuration specifics you have.
>
> Yes, but it don't solve the problem "in general as there is no general way
> to identify boost binaries other than the
> native ones. Which means even if I provide a FindBoost version which is
> modified (like Michael's one)
> it will only work if the developer have put the binaries the way I decided
> they should be, not in a "conventional" way,
> which is my problem.

There is no "conventional" way on Windows, except for System32 and SysWOW64
folders for runtime libraries (which, I believe, is not your problem at this
point, and should not be used directly anyway). Basically, you have to choose
directory layout for every project you build. Whether or not you choose a
particular layout as your convention is up to you.

[Michael Marcin]

On 4/16/13 3:46 PM, Andrey Semashev wrote:
>
> There is no "conventional" way on Windows, except for System32 and SysWOW64
> folders for runtime libraries (which, I believe, is not your problem at this
> point, and should not be used directly anyway). Basically, you have to choose
> directory layout for every project you build. Whether or not you choose a
> particular layout as your convention is up to you.
>

Sort of, most libraries choose one, at least as a default.
Which can help tools like CMake make educated guesses that help users.

For example there is no conventional place to install boost on windows
but CMake's find module by default searches:

set(_boost_INCLUDE_SEARCH_DIRS_SYSTEM
C:/boost/include
C:/boost
"$ENV{ProgramFiles}/boost/include"
"$ENV{ProgramFiles}/boost"
/sw/local/include
)

TBB puts its libs in intel64 or ia32 sub directories which isn't all
that common of a naming scheme in my experience but is certainly usable.

[Mathias Gaunard]

On 16/04/13 22:06, Michael Marcin wrote:

> Sort of, most libraries choose one, at least as a default.
> Which can help tools like CMake make educated guesses that help users.

Most libraries do not encode half as much data as what Boost encodes in
its versioned naming scheme.
Most libraries certainly do not care about 32 or 64 bits, it's two
configuratons you set up and compile separately, like building for two
different architectures.

> For example there is no conventional place to install boost on windows

There is, C:\Boost.
That's the default installation directory of the Boost windows installers.
Of course you cannot put both 32 or 64 bit in there without making
subfolders.

> TBB puts its libs in intel64 or ia32 sub directories which isn't all
> that common of a naming scheme in my experience but is certainly usable.

It's a naming scheme that is used by all Intel libraries (and it's not
even that consistent, some versions of TBB also have intel64/gc4.4 for
example).
It's not used by all libraries everywhere.

I know of many pieces of software which use an approach like that but
with different directory names.
Some will use a full triplet like x86_64-linux-gnu some will use weird
names like glnxa64 etc.

[Klaim - Joël Lamotte]

On Wed, Apr 17, 2013 at 12:50 PM, Mathias Gaunard <
mathias...@ens-lyon.org> wrote:

>
> Most libraries do not encode half as much data as what Boost encodes in
> its versioned naming scheme.
>

I don't see the relationship with the point I'm making.

> Most libraries certainly do not care about 32 or 64 bits, it's two
> configuratons you set up and compile separately, like building for two
> different architectures.
>
>

Well all other libraries I use so far do care. Also, I don't see what you
want to mean here because I agree it's two different configurations, it is
not at all the point here.
The point is to simplify

>
>
> For example there is no conventional place to install boost on windows
>>
>
> There is, C:\Boost.
> That's the default installation directory of the Boost windows installers.
>

I know, it's one of the locations, with BOOST_DIR, where CMake's FindBoost
module will look in.
This is not the problem.

> Of course you cannot put both 32 or 64 bit in there without making
> subfolders.
>
>

This is the problem exactly that I'm pointing. I can obviously set the
library dir using BOOST_LIBRARYDIR
but I then have to do it each time I need to work on CMake files of the
projects and need to delete cache and
configure from scratch.

>
>
> TBB puts its libs in intel64 or ia32 sub directories which isn't all
>> that common of a naming scheme in my experience but is certainly usable.
>>
>
> It's a naming scheme that is used by all Intel libraries (and it's not
> even that consistent, some versions of TBB also have intel64/gc4.4 for
> example).
> It's not used by all libraries everywhere.
>
> I know of many pieces of software which use an approach like that but with
> different directory names.
> Some will use a full triplet like x86_64-linux-gnu some will use weird
> names like glnxa64 etc.
>
>

I don't understand what is your point here?
CMake's FindTBB knows that convention which helps it find the right
binaries depending on the configuration/architecture choosen
which is what I point is lacking with Boost, because of the lack of
convention.

I'm not saying boost should use one convention in particular, just that it
should provide a convention so that I or someone can patch the FindBoost
module.

Joel Lamotte

[Mathias Gaunard]

On 17/04/13 13:50, Klaim - Joël Lamotte wrote:
> On Wed, Apr 17, 2013 at 12:50 PM, Mathias Gaunard <
> mathias...@ens-lyon.org> wrote:
>
>>
>> Most libraries do not encode half as much data as what Boost encodes in
>> its versioned naming scheme.
>>
>
> I don't see the relationship with the point I'm making.

Every paragraph in my previous email was related to the text that I
quoted, written by Michael Marcin.
Unless you're the same person, this wasn't particularly directed to what
you were saying.

[Michael Marcin]

On 4/17/2013 5:50 AM, Mathias Gaunard wrote:
> On 16/04/13 22:06, Michael Marcin wrote:
>
>> For example there is no conventional place to install boost on windows
>
> There is, C:\Boost.
> That's the default installation directory of the Boost windows installers.
> Of course you cannot put both 32 or 64 bit in there without making
> subfolders.
>

Really? Because the getting started guide says it's going to put it in:

C:\Program Files\boost\boost_1_53_0\lib\

http://www.boost.org/doc/libs/1_53_0/more/getting_started/windows.html#install-visual-studio-binaries

also:

The path to the boost root directory (often C:\Program
Files\boost\boost_1_53_0) is sometimes referred to as $BOOST_ROOT in
documentation and mailing lists .

http://www.boost.org/doc/libs/1_53_0/more/getting_started/windows.html#the-boost-distribution

[Rob Stewart]

On Apr 17, 2013, at 8:50 AM, Klaim - Joël Lamotte <mjk...@gmail.com> wrote:

> On Wed, Apr 17, 2013 at 12:50 PM, Mathias Gaunard <
> mathias...@ens-lyon.org> wrote:

[snipped to the essence]

>> Of course you cannot put both 32 or 64 bit in there without making subfolders.
>>
> This is the problem exactly that I'm pointing. I can obviously set the library dir using BOOST_LIBRARYDIR
> but I then have to do it each time I need to work on CMake files of the projects and need to delete cache and configure from scratch.
>
>> TBB puts its libs in intel64 or ia32 sub directories which isn't all that common of a naming scheme in my experience but is certainly usable.
>

> CMake's FindTBB knows that convention which helps it find the right binaries depending on the configuration/architecture choosen which is what I point is lacking with Boost, because of the lack of convention.
>
> I'm not saying boost should use one convention in particular, just that it should provide a convention so that I or someone can patch the FindBoost module.

I see no reason against choosing a subdirectory structure in which to place 32 and 64 bit libraries. However, if Program Files is the install location, Windows already distinguishes between them: Program Files and Program Files (x86). IOW, if that's the install location, the subdirectories aren't needed (though one can still be used in each location).

I vote for simple, straightforward names, like 32 and 64, for the subdirectories, BTW.

Having said that much, naming the directories with much of the platform and architecture information, while simplifying the library names, would be nicer IME.

___
Rob

(Sent from my portable computation engine)

[Klaim - Joël Lamotte]

On Thu, Apr 18, 2013 at 11:36 AM, Rob Stewart <robert...@comcast.net>wrote:

> I see no reason against choosing a subdirectory structure in which to
> place 32 and 64 bit libraries. However, if Program Files is the install
> location, Windows already distinguishes between them: Program Files and
> Program Files (x86). IOW, if that's the install location, the
> subdirectories aren't needed (though one can still be used in each
> location).

The location of the boost install is solved by specifying Boost_DIR (or
BOOST_DIR?) as an environment variable (using FindBoost module)
It's indeed the directory of binaries inside that install location which is
found automatically by the module only for one architecture.

> I vote for simple, straightforward names, like 32 and 64, for the
subdirectories, BTW.

I agree it would make things straightforward.
The changes necessary would just be to set these as default directories for
output binaries into b2 (on windows only?) and the windows installer I
think.
Then a patch to the FindBoost module can finally happen.

Joel Lamotte