[boost] Problem cross-compiling boost.context for Raspberry Pi (2)

[Biddiscombe, John A.]

Hello list,

I am able to compile the full set of boost libraries on the Raspberry Pi, and I get all the libraries successfully built, including context and coroutine which I need. However, when I cross compile on an Ubuntu virtual machine setup for the job, all the libraries except context/coroutine are built.

The problem seems to be that when cross compiling, the file "unsupported.cpp" is being pulled in (here is a verbose output showing flag I'm using)

"/home/biddisco/pi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -pthread -fPIC -D__arm__ -std=c++11 -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard -DBOOST_ALL_NO_LIB=1 -DBOOST_CHRONO_DYN_LINK=1 -DBOOST_CONTEXT_DYN_LINK=1 -DBOOST_CONTEXT_SOURCE -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_BUILD_DLL=1 -DBOOST_THREAD_POSIX -DBOOST_THREAD_USE_DLL=1 -DNDEBUG -I"." -c -o "bin.v2/libs/context/build/gcc-arm/release/threading-multi/unsupported.o" "libs/context/src/unsupported.cpp"

which emanates from the jamfile boost_1_58_0/libs/context/build/Jamfile.v2 where the following
alias asm_context_sources
: unsupported.cpp
;
is in a section marked #COMBINED

Now, when I build on the pi itself, this file is not being added to the build, so it seems to me that during the bootstrap phase on the pi, different flags are being used which trigger the correct behaviour of bjam

my user-config.jam looks like this

using gcc : : /home/biddisco/pi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ : <compile-flags>-D__arm__ -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard ;

(NB. I added -D__arm__ to see if it helps, but it does not make a difference without it).

I have no knowledge of how bjam / boost.build do their things, so I do not know what to try next. It seems that I need to bootstrap the build using the flags for the pi, but built for the host so that it can generate the correct makefiles and this looks like where the problem is arising.
the build process seems good apart from this wrinkle

The bootstrap command I'm using is
./bootstrap.sh --with-toolset=gcc

and the build appears sensible when I use
./b2 cxxflags="-D__arm__ -std=c++11 -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard" --layout=versioned threading=multi link=shared variant=release -j8

Performing configuration checks
- 32-bit : yes
- arm : yes

.... etc etc

Can anyone see what I'm doing wrong?

(Note : With another project which depends on libpthread, I had to copy libpthread from the pi to the host and force the linker to use the pthread lib from the pi and not the host one, does bjam do something internally which might require a trick like this? - note2 - the binaries generated via this process run without error)

Many thanks

JB

--
John Biddiscombe, email:biddisco @.at.@ cscs.ch
http://www.cscs.ch/
CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07
Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Oliver Kowalke]

for cross compiling you have to tell boost.build some properties
(architecture, address-model, abi, binary-format) because
boost.build can't detect it correctly + you have to use the correct
assembler tool.

I assume you want to compile for:

architecture: ARM
address-model: 32
api: AAPCS
binary-format: ELF

the command line should be: b2 toolset=gcc architecture=arm
address-model=32 api=aapcs binary-format=elf

[Vladimir Prus]

On 6/16/2015 9:22 AM, Biddiscombe, John A. wrote:

> and the build appears sensible when I use
> ./b2 cxxflags="-D__arm__ -std=c++11 -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard" --layout=versioned threading=multi link=shared variant=release -j8

Hi John,

you specify cxxflags, but you don't apear to specify linkflags. That's quite dangerous - the three -m options
above affect multilib selection, and if multilib differs between compilation and linking, it's bad.

> using gcc : : /home/biddisco/pi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ : <compile-flags>-D__arm__ -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard ;

The same problem here. Also, the value of "<compile-flags>" option probably should be quoted? Is this
user-config.jam actually read - I would expect a syntax error here. You can use --debug-configuration
to make sure it's read, and please quote the value in any case.

> (Note : With another project which depends on libpthread, I had to copy libpthread from the pi to the host and force the linker to use the pthread lib from the pi and not the host one, does bjam do something internally which might require a trick like this? - note2 - the binaries generated via this process run without error)

Copying libpthread would suggest the sysroot you have on host is different from sysroot on target - and if
that's that case, everything will likely explode sooner or later. It would be good to veryfy that
your host toolchain matches exactly what's on target.

With the above issues fixed, I would expect this alternative in libs/context/build/Jamfile.v2 to be taken:

alias asm_context_sources
: asm/make_arm_aapcs_elf_gas.S
asm/jump_arm_aapcs_elf_gas.S
: <abi>aapcs
<address-model>32
<architecture>arm
<binary-format>elf
<toolset>gcc
;

Could you try forcing it, by:

b2 abi=aapcs binary-format=elf address-model=32 architecture=arm toolset=gcc <further-options>

If this works, it would mean some of these options end up defaulted incorrectly, with abi and binary-format
particularly suspect. What's your host system, not OSX per chance?

Thanks,
Volodya

[Oliver Kowalke]

2015-06-16 8:49 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:

> If this works, it would mean some of these options end up defaulted
> incorrectly, with abi and binary-format
> particularly suspect. What's your host system, not OSX per chance?
>

Is boost.build able to determine the correct value for property <abi> while
cross compiling boost.context?
Property <abi> is determined by evaluating os.platform - in John's case the
Jamfile expects
that os.platform returns 'ARM' (abi=aapcs).

[Vladimir Prus]

On 6/16/2015 9:58 AM, Oliver Kowalke wrote:
> 2015-06-16 8:49 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:
>
>> If this works, it would mean some of these options end up defaulted
>> incorrectly, with abi and binary-format
>> particularly suspect. What's your host system, not OSX per chance?
>>
>
> Is boost.build able to determine the correct value for property <abi> while
> cross compiling boost.context?

Depends on what value is correct. If the check can be formulated as compiling some
source, or running some tools, then it can be done.

However, unless I am mistaken, all ARM variants use <abi>aapcs - so maybe it
does not need to be detected at all, and can be removed from conditions?

- Volodya

[Biddiscombe, John A.]

Oliver, Vladimir

Thank you both for the fast replies. The build appears to be fixed now. I wish I has posted my question last night instead of staying up till 1am trying to work out what was wrong!

For future googlers and readers of the list, I amended my user-config.jam to read as follows, (it appears that <compile-flags> is incorrect and I should have used <compileflags>, so now I can omit them from the b2 options and they are picked up correctly).

contents of ~/user-config.jam

using gcc : pi :
/home/biddisco/pi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ :
<compileflags>"-D__arm__ -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard"
;

Then after bootstrap.sh

./b2 -d+2 toolset=gcc-pi architecture=arm address-model=32 abi=aapcs binary-format=elf --debug-configuration --without-python -s NO_BZIP2=1 --layout=versioned threading=multi link=shared variant=release --prefix=/home/biddisco/apps/boost-1.58.0 -j8

and I have a complete set of libraries including context and coroutine.

>
you specify cxxflags, but you don't apear to specify linkflags. That's quite dangerous - the three -m options above affect multilib selection, and if multilib differs between compilation and linking, it's bad.
<

Quite right. the forcing of libpthread.a I mentioned was for a CMake try_compile step which resulted in fail as the link step was incorrect. Looking in the compiler toolchain, I see that there are the basic libs required in there, so I shall amend my procedure accordingly and make sure that CMake looks in the correct place.

Many thanks again.

JB

[Oliver Kowalke]

2015-06-16 9:28 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:

Depends on what value is correct. If the check can be formulated as
> compiling some
> source, or running some tools, then it can be done.
>

AFAIK it is very hard or nearly impossible for code to detect for which ABI
and/or binary format it is compiled for.
ABI/binary format are set/specified by flags in the compiler command line.
The address model can be detected over pointer size - but ABI ...?!

> However, unless I am mistaken, all ARM variants use <abi>aapcs - so maybe
> it
> does not need to be detected at all, and can be removed from conditions?
>

For ARM this is true but not for the other architectures (MIPS for instance
has 4 different ABIs).
boost.context determines the default values for ABI/binary-format on the
values returned by 'os.name' and 'os.platform'.
I don't know another way for reliable ABI/binary-format detection in
boost.build.

[Vladimir Prus]

On 6/16/2015 10:48 AM, Oliver Kowalke wrote:
> 2015-06-16 9:28 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:
>
> Depends on what value is correct. If the check can be formulated as
>> compiling some
>> source, or running some tools, then it can be done.
>>
>
> AFAIK it is very hard or nearly impossible for code to detect for which ABI
> and/or binary format it is compiled for.
> ABI/binary format are set/specified by flags in the compiler command line.
> The address model can be detected over pointer size - but ABI ...?!

Right, at least GDB appear to have APCS vs AAPCS as a user option.

>> However, unless I am mistaken, all ARM variants use <abi>aapcs - so maybe
>> it
>> does not need to be detected at all, and can be removed from conditions?
>>
>
> For ARM this is true but not for the other architectures (MIPS for instance
> has 4 different ABIs).

Correct, but we're talking about ARM right now - there, adding <abi> to condition
appears unnecessary and problematic. There is no chance of ambiguity between ARM
and MIPs variants. Do you see any issues if <abi> property is removed from
condition for all ARM variants?

> boost.context determines the default values for ABI/binary-format on the
> values returned by 'os.name' and 'os.platform'.
> I don't know another way for reliable ABI/binary-format detection in
> boost.build.

For avoidance of doubt, do you mean "boost.build" specifically? From your first
observation, it appears that it's hard to detect ABI in any build system not
using mind reading technology?

- Volodya

[Oliver Kowalke]

2015-06-16 10:12 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:

Correct, but we're talking about ARM right now - there, adding <abi> to
> condition
> appears unnecessary and problematic. There is no chance of ambiguity
> between ARM
> and MIPs variants. Do you see any issues if <abi> property is removed from
> condition for all ARM variants?
>

At the moment the rules in context/build/Jamfile.v2 for ARM are over
specified,
e.g. architecture and address-mode are sufficient. I keep <abi> for ARM in
order to be
consistent with the rules for the other architectures in the Jamfile.
That means it is always a problem to determine the correct ABI and binary
format for
a certain compilation.

>
> boost.context determines the default values for ABI/binary-format on the
>> values returned by 'os.name' and 'os.platform'.
>> I don't know another way for reliable ABI/binary-format detection in
>> boost.build.
>>
>
> For avoidance of doubt, do you mean "boost.build" specifically? From your
> first
> observation, it appears that it's hard to detect ABI in any build system
> not
> using mind reading technology?
>

I use features os.name and os.platform from boost.build (boost.config?) to
set
ABI and binary format (e.g. the default values).

[Vladimir Prus]

On 6/16/2015 11:30 AM, Oliver Kowalke wrote:
> 2015-06-16 10:12 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:
>
> Correct, but we're talking about ARM right now - there, adding <abi> to
>> condition
>> appears unnecessary and problematic. There is no chance of ambiguity
>> between ARM
>> and MIPs variants. Do you see any issues if <abi> property is removed from
>> condition for all ARM variants?
>>
>
> At the moment the rules in context/build/Jamfile.v2 for ARM are over
> specified,
> e.g. architecture and address-mode are sufficient. I keep <abi> for ARM in
> order to be
> consistent with the rules for the other architectures in the Jamfile.
> That means it is always a problem to determine the correct ABI and binary
> format for
> a certain compilation.

It seems to me that consistency between different architectures forces users
of ARM to specify abi, which is not great. I would recommend dropping abi
requirements there.

>> boost.context determines the default values for ABI/binary-format on the
>>> values returned by 'os.name' and 'os.platform'.
>>> I don't know another way for reliable ABI/binary-format detection in
>>> boost.build.
>>>
>>
>> For avoidance of doubt, do you mean "boost.build" specifically? From your
>> first
>> observation, it appears that it's hard to detect ABI in any build system
>> not
>> using mind reading technology?
>>
>
> I use features os.name and os.platform from boost.build (boost.config?) to
> set
> ABI and binary format (e.g. the default values).

These tell you host os. I doubt that could be helpful to determine ABI really.

- Volodya

[Oliver Kowalke]

2015-06-16 15:26 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:

It seems to me that consistency between different architectures forces users
> of ARM to specify abi,
>

only in the case of cross compiling

> These tell you host os. I doubt that could be helpful to determine ABI
> really.
>

As I tried to explain - there is no way. boost.context defines defaults for
<abi> and
<binary-format> depending on architecture + operating system etc., for
instance in the Jamfile

binary-format:
default ELF
os == MACOSX => MACH-O
os == Windows => PE
os == AIX => XCOFF

abi:
default SYSV
os == Windows => MS
platform == ARM => AAPCS
platform == MIPS => O32

boost.build/boost.config do not provide other infos

[Vladimir Prus]

On 6/16/2015 4:40 PM, Oliver Kowalke wrote:
> 2015-06-16 15:26 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:
>
> It seems to me that consistency between different architectures forces users
>> of ARM to specify abi,
>>
>
> only in the case of cross compiling

I would guess cross-compiling for ARM is more common than native compiling for ARM,
and removing this requirement should have no adverse effects in the native case,
so what's the downside to removing it?

- Volodya

[Oliver Kowalke]

2015-06-16 16:44 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:

I would guess cross-compiling for ARM is more common than native compiling
> for ARM,
> and removing this requirement should have no adverse effects in the native
> case,
> so what's the downside to removing it?
>

For cross compiling you have always to specify <abi> and <binary-format>
because both
properties can not be correctly determined in the case of cross compiling.

[Vladimir Prus]

On 6/16/2015 6:56 PM, Oliver Kowalke wrote:
> 2015-06-16 16:44 GMT+02:00 Vladimir Prus <vladim...@gmail.com>:
>
> I would guess cross-compiling for ARM is more common than native compiling
>> for ARM,
>> and removing this requirement should have no adverse effects in the native
>> case,
>> so what's the downside to removing it?
>>
>
> For cross compiling you have always to specify <abi> and <binary-format>
> because both
> properties can not be correctly determined in the case of cross compiling.

I don't follow. If I do not specify abi when cross-compiling to ARM, then:

- Compiler does not care much, it just uses the default one
- Boost.Build does not care either, since all ARM alternatives in boost.context
have the same ABI

What am I missing? Maybe I should submit a pull request, and you'll point out
in which cases it will result in compilation failure or wrong compilation options?

Thanks,
Volodya

[Oliver Kowalke]

> - Compiler does not care much, it just uses the default one
> - Boost.Build does not care either, since all ARM alternatives in
boost.context
> have the same ABI
>
> What am I missing

boost.context has to compile assemblerfiles/code. For this purpose it has
asm files for combinations of architecture, abi, binary-format and
address-model. The Jamfile of boost.context has rules for selecting the
correct ASM file. In the case of cross compiling the rules get the wrong
data. Theuser has to overwrite abi and the other properties guessed by
context's Jamfile.

[Gavin Lambert]

On 17/06/2015 18:30, Oliver Kowalke wrote:
>> - Compiler does not care much, it just uses the default one
>> - Boost.Build does not care either, since all ARM alternatives in
> boost.context
>> have the same ABI
>>
>> What am I missing
>
> boost.context has to compile assemblerfiles/code. For this purpose it has
> asm files for combinations of architecture, abi, binary-format and
> address-model. The Jamfile of boost.context has rules for selecting the
> correct ASM file. In the case of cross compiling the rules get the wrong
> data. Theuser has to overwrite abi and the other properties guessed by
> context's Jamfile.

I think that Vladimir's suggestion is that:

1. The initial defaults for all properties should be auto-detected from
the host OS as normal.
2. Specifying "architecture=arm" explicitly, also changes the default
for "abi" to "aapcs" (but it can still be overruled by the command
line), along with any other properties for which knowing the
architecture is sufficient to overrule other auto-detection.

(Or, for another way to look at it: specifying the architecture makes it
assume a certain abi without having to do any auto-detection, unless
otherwise specified.)

I don't how easy it would be to convince B2 to do such a thing, however,
or whether it's sufficiently worthwhile. :)

[Oliver Kowalke]

2015-07-09 8:59 GMT+02:00 Gavin Lambert <gav...@compacsort.com>:

> 1. The initial defaults for all properties should be auto-detected from
> the host OS as normal.
>

you can not auto-detect properties like binary-format - the OS might
support different binary formats on a certain architecture -> MIPS + LINUX
+ O32/N32/N64 ...

> 2. Specifying "architecture=arm" explicitly, also changes the default for
> "abi" to "aapcs" (but it can still be overruled by the command line),

that's what context/build/Jamfile.v2 already does (local rule default_abi (
) )

> along with any other properties for which knowing the architecture is
> sufficient to overrule other auto-detection.
>

binary-format isn't easily to detect

[Oliver Kowalke]

2015-07-09 9:13 GMT+02:00 Oliver Kowalke <oliver....@gmail.com>:

>
>>
>> 2. Specifying "architecture=arm" explicitly, also changes the default for
>> "abi" to "aapcs" (but it can still be overruled by the command line),
>
>
> that's what context/build/Jamfile.v2 already does (local rule default_abi
> ( ) )
>

rules default_abi() and default_binary_format() guess what should be the
default values for features <abi> and <binary-format>.
In the rules itself you have no access to the property value of
<architecture>