[PATCH 0/2] Enable cross compilation on multiarch distros

[Punit Agrawal]

Hi,

Here's a couple of fixes to rely on the compiler to resolve library
locations during the build rather than using hard coded paths in the
build/Makefile when cross compiling. With these, I am able to cross
build OSv for arm64 on Debian using -

$ export CROSS_PREFIX="aarch64-linux-gnu-"
$ ./scripts/build -j8 arch=aarch64 fs=rofs image=native-example

The following steps can be used to setup the environment -

# dpkg --add-architecture arm64
# apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libboost-system-dev:arm64

As an additional benefit of the patches, it is now also possible to
use CROSS_PREFIX to speed up compilation for both native and cross
scenarios using ccache without messing with symlinks to compiler
binaries -

$ export CROSS_PREFIX="ccache " # native
or
$ export CROSS_PREFIX="ccache aarch64-linux-gnu-

I've tested this on Debian both native and cross. Any additional
testing especially on Fedora / CentOS which do not provide multiarch
would be appreciated.

Thanks,
Punit

Punit Agrawal (2):
scripts/build: Simplify libgcc_s.so.1 detection
Makefile: Rework boost detection to enable cross compilation

Makefile | 38 ++++++++++++++++----------------------
scripts/build | 17 ++++++++---------
2 files changed, 24 insertions(+), 31 deletions(-)

--
2.29.2

[Punit Agrawal]

When cross compiling, it would be good to support the following two
cases -

* Distro provided cross compiler + libraries - This is the case when a
distro with multi-arch support (e.g., Debian derivaties)

* Local libraries installation - Needed when the distro doesn't
support multi-arch or there's a need to use local installs

Update the libgcc detection to query the compiler regarding the
location of the libraries - provided the correct CROSS_PREFIX is
specified. If this fails to find the library, fall back to the hard
coded location. Also, add a message to inform the user about the
fallback usage.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---
scripts/build | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/scripts/build b/scripts/build
index b494141ce16a..1296c4855498 100755
--- a/scripts/build
+++ b/scripts/build
@@ -263,15 +263,14 @@ kernel_end=$(($loader_size+2097151 & ~2097151))
cd $OUT

host_arch=$(uname -m)
-if [[ "$host_arch" == "x86_64" && "$arch" == 'aarch64' ]]; then
- libgcc_s_path=$(${CROSS_PREFIX:-aarch64-linux-gnu-}gcc -print-file-name=libgcc_s.so.1)
- if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then
- libgcc_s_dir=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)
- else
- libgcc_s_dir=$(dirname $(readlink -f $libgcc_s_path))
- fi
-else
- libgcc_s_dir=$(dirname $(readlink -f $(gcc -print-file-name=libgcc_s.so.1)))
+CC=${CROSS_PREFIX}gcc
+libgcc_s_dir=$(dirname $(readlink -f $(${CC} -print-file-name=libgcc_s.so.1)))
+if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then
+ cat <<-EOF
+ Unable to resolve libgcc_s.so.1 using "${CC}".
+ Looking in ../downloaded_packages/aarch64/gcc/install/lib64
+ EOF
+ libgcc_s_dir=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)
fi

if [ "$export" != "none" ]; then
--
2.29.2

[Punit Agrawal]

The boost detection mechanism assumes that if a CROSS_PREFIX is
specified the library has to be installed in
"build/downloaded_packages/aarch64/boost/install" folder in the OSv
sources folder.

This is not the case when cross compiling on a distro that supports
multiarch and the boost libraries are installed using distro provided
packages.

Fix this by relying on the compiler (native or cross) to provide the
location of boost libraries in the first instance. If this fails to
find the library location, then fallback to the hard coded path.

With the changes, it is not possible to cross compile OSv using -

CROSS_PREFIX="aarch64-linux-gnu-"

./scripts/build -j8 arch=aarch64 fs=rofs image=native-example

provided libboost-system-dev for arm64 is installed in the system.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---

Makefile | 38 ++++++++++++++++----------------------
1 file changed, 16 insertions(+), 22 deletions(-)

diff --git a/Makefile b/Makefile
index 66448a9be982..502e019a2f3b 100644
--- a/Makefile
+++ b/Makefile
@@ -1845,34 +1845,28 @@ else
endif
endif

-ifeq ($(CROSS_PREFIX),)
- # link with -mt if present, else the base version (and hope it is multithreaded)
- boost-mt := -mt
+# link with -mt if present, else the base version (and hope it is multithreaded)
+boost-mt := -mt
+boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
+ifeq ($(filter /%,$(boost-lib-dir)),)
+ boost-mt :=
boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
ifeq ($(filter /%,$(boost-lib-dir)),)
- boost-mt :=
- boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
- ifeq ($(filter /%,$(boost-lib-dir)),)
+ $(warning Unable to resolve libboost_system.a using $(CC))
+ ifeq ($(arch),aarch64)
+ $(warning Falling back to build/downloaded_packages/aarch64/boost/install)
+ aarch64_boostbase = build/downloaded_packages/aarch64/boost/install
+ ifeq (,$(wildcard $(aarch64_boostbase)))
+ $(error Missing $(aarch64_boostbase) directory. Please run "./scripts/download_aarch64_packages.py")
+ endif
+ boost-lib-dir := $(firstword $(dir $(shell find $(aarch64_boostbase)/ -name libboost_system*.a)))
+ boost-mt := $(if $(filter %-mt.a, $(wildcard $(boost-lib-dir)/*.a)),-mt)
+ INCLUDES += -isystem $(aarch64_boostbase)/usr/include
+ else
$(error Error: libboost_system.a needs to be installed.)
endif
endif
- # When boost_env=host, we won't use "-nostdinc", so the build machine's
- # header files will be used normally. So we don't need to add anything
- # special for Boost.
- boost-includes =
-else
-ifeq ($(arch),aarch64)
- aarch64_boostbase = build/downloaded_packages/aarch64/boost/install
- ifeq (,$(wildcard $(aarch64_boostbase)))
- $(error Missing $(aarch64_boostbase) directory. Please run "./scripts/download_aarch64_packages.py")
- endif
-
- boost-lib-dir := $(firstword $(dir $(shell find $(aarch64_boostbase)/ -name libboost_system*.a)))
- boost-mt := $(if $(filter %-mt.a, $(wildcard $(boost-lib-dir)/*.a)),-mt)
- boost-includes = -isystem $(aarch64_boostbase)/usr/include
-endif
endif
-
boost-libs := $(boost-lib-dir)/libboost_system$(boost-mt).a

objects += fs/nfs/nfs_null_vfsops.o
--
2.29.2

[Nadav Har'El]

This patch seems reasonable to me, but I'd better leave it to Waldek to test that it doesn't break his own cross-compilation setup and commit it.

Thanks.

--
Nadav Har'El

n...@scylladb.com

[Punit Agrawal]

Turns out this patch needs rebasing onto latest changes from Waldek to
support user specified boost install. Not sure how I managed to miss
those - sorry for the noise.

With those patches, there's no need for the special casing of aarch64 -
the user can specify boost_dir. Let me send a new version with the
updated changes.

[Punit Agrawal]

Hi,

This is the second version of the patch to enable cross building OSv
in multiarch distros. This version updates the second patch to rely on
the newly added boost_base support.

Cover letter from previous version -

Here's a couple of fixes to rely on the compiler to resolve library
locations during the build rather than using hard coded paths in the
build/Makefile when cross compiling. With these, I am able to cross
build OSv for arm64 on Debian using -

$ export CROSS_PREFIX="aarch64-linux-gnu-"

$ ./scripts/build -j8 arch=aarch64 fs=rofs image=native-example

The following steps can be used to setup the environment -

# dpkg --add-architecture arm64
# apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libboost-system-dev:arm64

As an additional benefit of the patches, it is now also possible to
use CROSS_PREFIX to speed up compilation for both native and cross
scenarios using ccache without messing with symlinks to compiler
binaries -

$ export CROSS_PREFIX="ccache " # native
or
$ export CROSS_PREFIX="ccache aarch64-linux-gnu-

I've tested this on Debian both native and cross. Any additional
testing especially on Fedora / CentOS which do not provide multiarch
would be appreciated.

Thanks,
Punit

Punit Agrawal (2):
scripts/build: Simplify libgcc_s.so.1 detection

Simplify boost library detection logic

Makefile | 33 ++++++++++-----------------------
modules/common.gmk | 27 +++++++--------------------
scripts/build | 17 ++++++++---------
3 files changed, 25 insertions(+), 52 deletions(-)

--
2.29.2

[Punit Agrawal]

When cross compiling, it would be good to support the following two
cases -

* Distro provided cross compiler + libraries - This is the case when a
distro with multi-arch support (e.g., Debian derivaties)

* Local libraries installation - Needed when the distro doesn't
support multi-arch or there's a need to use local installs

Update the libgcc detection to query the compiler regarding the
location of the libraries - provided the correct CROSS_PREFIX is
specified. If this fails to find the library, fall back to the hard
coded location. Also, add a message to inform the user about the
fallback usage.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---

scripts/build | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/scripts/build b/scripts/build

index 5fdcc7a88214..88f48f5e0582 100755
--- a/scripts/build
+++ b/scripts/build
@@ -271,15 +271,14 @@ kernel_end=$(($loader_size+2097151 & ~2097151))

[Punit Agrawal]

Commits 5a9be20243fd ("boost: enhance main makefile to build against
arbitrary boost") and 765fa172ab41 ("boost: enhance modules/common.gmk
to build against arbitrary boost") added support for allowing the user
to specify arbitrary boost_base but left in place the special casing
when building for aarch64.

As boost_base can also be used to specify the location for aarch64
boost libraries when required, drop the extra logic that hard codes
paths for arm64 in favour of the more flexible approach.

As an added benefit the change also enables cross building in
environments where the compiler can be trusted to correctly resolve
the library paths such as on Debian or Ubuntu.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---

Makefile | 33 ++++++++++-----------------------
modules/common.gmk | 27 +++++++--------------------

2 files changed, 17 insertions(+), 43 deletions(-)

diff --git a/Makefile b/Makefile
index 9d80f095a3c7..b5d3f2cb0892 100644
--- a/Makefile
+++ b/Makefile
@@ -1847,33 +1847,20 @@ endif

#Allow user specify non-default location of boost
ifeq ($(boost_base),)
- ifeq ($(CROSS_PREFIX),)

- # link with -mt if present, else the base version (and hope it is multithreaded)
- boost-mt := -mt

+ # link with -mt if present, else the base version (and hope it is multithreaded)
+ boost-mt := -mt
+ boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
+ ifeq ($(filter /%,$(boost-lib-dir)),)

+ boost-mt :=
boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
ifeq ($(filter /%,$(boost-lib-dir)),)
- boost-mt :=
- boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
- ifeq ($(filter /%,$(boost-lib-dir)),)

- $(error Error: libboost_system.a needs to be installed.)
- endif
- endif

- # When boost_env=host, we won't use "-nostdinc", so the build machine's
- # header files will be used normally. So we don't need to add anything
- # special for Boost.
- boost-includes =
- else

- ifeq ($(arch),aarch64)

- aarch64_boostbase = build/downloaded_packages/aarch64/boost/install
- ifeq (,$(wildcard $(aarch64_boostbase)))
- $(error Missing $(aarch64_boostbase) directory. Please run "./scripts/download_aarch64_packages.py")
- endif
-
- boost-lib-dir := $(firstword $(dir $(shell find $(aarch64_boostbase)/ -name libboost_system*.a)))
- boost-mt := $(if $(filter %-mt.a, $(wildcard $(boost-lib-dir)/*.a)),-mt)
- boost-includes = -isystem $(aarch64_boostbase)/usr/include

+ $(error Error: libboost_system.a needs to be installed.)
endif
endif
+ # When boost_env=host, we won't use "-nostdinc", so the build machine's
+ # header files will be used normally. So we don't need to add anything
+ # special for Boost.
+ boost-includes =
else
# Use boost specified by the user
boost-lib-dir := $(firstword $(dir $(shell find $(boost_base)/ -name libboost_system*.a)))
diff --git a/modules/common.gmk b/modules/common.gmk
index e0814b42414e..6ae5870ae365 100644
--- a/modules/common.gmk
+++ b/modules/common.gmk
@@ -123,27 +123,14 @@ endif
# Let us detect presence of boost headers and library
# Allow user specify non-default location of boost
ifeq ($(boost_base),)
- ifeq ($(CROSS_PREFIX),)
- boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system.so))
- ifeq ($(filter /%,$(boost-lib-dir)),)
- $(error Error: libboost_system.so needs to be installed.)
- endif

- # When boost_env=host, we won't use "-nostdinc", so the build machine's
- # header files will be used normally. So we don't need to add anything
- # special for Boost.
- boost-includes =
- else
- ifeq ($(arch),aarch64)

- aarch64_boostbase = $(src)/build/downloaded_packages/aarch64/boost/install

- ifeq (,$(wildcard $(aarch64_boostbase)))
- $(error Missing $(aarch64_boostbase) directory. Please run "./scripts/download_aarch64_packages.py")
- endif
-

- boost-lib-dir := $(firstword $(dir $(shell find $(aarch64_boostbase)/ -name libboost_system.so)))
- LDFLAGS += -L$(boost-lib-dir)

- boost-includes = -isystem $(aarch64_boostbase)/usr/include
- endif

+ boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system.so))
+ ifeq ($(filter /%,$(boost-lib-dir)),)
+ $(error Error: libboost_system.so needs to be installed.)
endif
+ # When boost_env=host, we won't use "-nostdinc", so the build machine's
+ # header files will be used normally. So we don't need to add anything
+ # special for Boost.
+ boost-includes =
else
ifeq ($(filter /%,$(boost_base)),)
absolute_boost_base := $(src)/$(boost_base)# It must be relative OSv root
--
2.29.2

[Punit Agrawal]

Hi Nadav,

"Nadav Har'El" <n...@scylladb.com> writes:

> This patch seems reasonable to me, but I'd better leave it to Waldek
> to test that it doesn't break his own cross-compilation setup and
> commit it.

Thanks for taking a look. I just sent out an updated version - I'd
missed a few commits from the master branch and this set needed
updating.

I'll wait to hear back from Waldek regarding testing in Fedora/CentOS.

Thanks,
Punit

[...]

[Waldek Kozaczuk]

On Fedora I cross-compile like this:

./scripts/build mode=release image=tests fs=rofs -j$(nproc) arch=aarch64 --create-disk

So in my case the CROSS_PREFIX is not set at all this would use x86_64 gcc and fix x86_64 version og libgcc_s.so

 

+libgcc_s_dir=$(dirname $(readlink -f $(${CC} -print-file-name=libgcc_s.so.1)))
+if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then

Did you mean  libgcc_s_dir? This also probably would not be correct.

+       cat <<-EOF
+       Unable to resolve libgcc_s.so.1 using "${CC}".
+       Looking in ../downloaded_packages/aarch64/gcc/install/lib64
+       EOF
+       libgcc_s_dir=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)
 fi 

All in all, this does not work on Fedora. 

 

I think this is a more correct version:

diff --git a/scripts/build b/scripts/build

index 5fdcc7a8..b07dfd6e 100755

--- a/scripts/build

+++ b/scripts/build

@@ -272,14 +272,20 @@ cd $OUT

 

 host_arch=$(uname -m)

 if [[ "$host_arch" == "x86_64" && "$arch" == 'aarch64' ]]; then

-       libgcc_s_path=$(${CROSS_PREFIX:-aarch64-linux-gnu-}gcc -print-file-name=libgcc_s.so.1)

-       if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then

-               libgcc_s_dir=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)

-       else

-               libgcc_s_dir=$(dirname $(readlink -f $libgcc_s_path))

-       fi

+       CC=${CROSS_PREFIX:-aarch64-linux-gnu-}gcc

+else

+       CC=gcc

+fi

+

+libgcc_s_path=$(${CC} -print-file-name=libgcc_s.so.1)

+if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then

+       cat <<-EOF

+       Unable to resolve libgcc_s.so.1 using "${CC}".

+       Looking in ../downloaded_packages/aarch64/gcc/install/lib64

+       EOF

+       libgcc_s_dir=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)

 else

-       libgcc_s_dir=$(dirname $(readlink -f $(gcc -print-file-name=libgcc_s.so.1)))

+       libgcc_s_dir=$(dirname $(readlink -f $libgcc_s_path))

 fi

 

 if [ "$export" != "none" ]; then

[Waldek Kozaczuk]

This also does not work on Fedora:

./scripts/build mode=release image=tests fs=rofs -j$(nproc) arch=aarch64 --create-disk

Building into build/release.aarch64

Makefile:1857: *** Error: libboost_system.a needs to be installed..  Stop.

make failed. Exiting from build script

On Mon, Feb 15, 2021 at 7:49 AM Punit Agrawal <punita...@gmail.com> wrote:

[Punit Agrawal]

Hi Waldek,

Would you be averse to setting the CROSS_PREFIX before calling the build
script? Something like -

CROSS_PREFIX=aarch64-linux-gnu- ./scripts/build mode=release ...

I was hesitating to drop the aarch64-linux-gnu- prefix but dropping the
hard-coded defaults regarding paths for compiler prefixes / library
locations will allow simplifying the build scripts and Makefiles. Think
about all the places we've got aarch64 hard coded in the build system.

Looking ahead, dropping the default will also reduce the churn when
porting OSv to other architectures (RISC-V for example).

What do you think?

> +libgcc_s_dir=$(dirname $(readlink -f $(${CC} -print-file-name=libgcc_s.so.1)))
> +if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then
>
> Did you mean libgcc_s_dir? This also probably would not be correct.

Indeed - it should've been libgcc_s_dir here.

Not sure what happened there but your patch shows up with extra newlines.

If you think that it is better to leave the CROSS_PREFIX defaults in
place for aarch64 then this patch works too.

Regards,
Punit

[Punit Agrawal]

Waldek Kozaczuk <jwkoz...@gmail.com> writes:

> This also does not work on Fedora:
>
> ./scripts/build mode=release image=tests fs=rofs -j$(nproc) arch=aarch64 --create-disk
>
> Building into build/release.aarch64
>
> Makefile:1857: *** Error: libboost_system.a needs to be installed.. Stop.
>
> make failed. Exiting from build script

Yup - continuing with the theme from previous patch, the changes here
are expecting the user to use the recently introduced "boost_base" to
specify the location of the boost library.

> On Mon, Feb 15, 2021 at 7:49 AM Punit Agrawal <punita...@gmail.com> wrote:
>
> Commits 5a9be20243fd ("boost: enhance main makefile to build against
> arbitrary boost") and 765fa172ab41 ("boost: enhance modules/common.gmk
> to build against arbitrary boost") added support for allowing the user
> to specify arbitrary boost_base but left in place the special casing
> when building for aarch64.
>
> As boost_base can also be used to specify the location for aarch64
> boost libraries when required, drop the extra logic that hard codes
> paths for arm64 in favour of the more flexible approach.
>
> As an added benefit the change also enables cross building in
> environments where the compiler can be trusted to correctly resolve
> the library paths such as on Debian or Ubuntu.
>
> Signed-off-by: Punit Agrawal <punita...@gmail.com>
> ---
> Makefile | 33 ++++++++++-----------------------
> modules/common.gmk | 27 +++++++--------------------
> 2 files changed, 17 insertions(+), 43 deletions(-)

I get excited by negative diffstat. But let me know if it's too much to
expect the user to specify boost_base - I'll rework the patches to
revert to the hard coded paths.

Thanks for taking the patches for a spin.

Punit

[...]

[Waldek Kozaczuk]

Would that not require arch anymore in this case? So how would it work on fedora? 

I was hesitating to drop the aarch64-linux-gnu- prefix but dropping the
hard-coded defaults regarding paths for compiler prefixes / library
locations will allow simplifying the build scripts and Makefiles. Think
about all the places we've got aarch64 hard coded in the build system.

Looking ahead, dropping the default will also reduce the churn when
porting OSv to other architectures (RISC-V for example).

What do you think?

Not sure. On one hand, it would be nice for the changes to be backwards compatible so that we do not have to update the docs. On other hand, I agree with some of your points that it would make it easier to support RISC-V in future. 

I wonder what others think. I guest a lot of that applies to other patch.

[Punit Agrawal]

Waldek Kozaczuk <jwkoz...@gmail.com> writes:

> On Wednesday, February 17, 2021 at 7:56:52 AM UTC-5 punit wrote:

[...]

> > On Fedora I cross-compile like this:
> > ./scripts/build mode=release image=tests fs=rofs -j$(nproc) arch=aarch64 --create-disk
> > So in my case the CROSS_PREFIX is not set at all this would use x86_64
> > gcc and fix x86_64 version og libgcc_s.so
>
> Would you be averse to setting the CROSS_PREFIX before calling the build
> script? Something like -
>
> CROSS_PREFIX=aarch64-linux-gnu- ./scripts/build mode=release ...
>
> Would that not require arch anymore in this case? So how would it work
> on fedora?

You would still need to pass the "arch=" to the build. The '...' at the
end were meant to indicate that the rest of the invocation stays
unmodified.

export CROSS_PREFIX=aarch-linux-gnu-

./scripts/build mode=release image=tests fs=rofs -j$(nproc) arch=aarch64 --create-disk

> I was hesitating to drop the aarch64-linux-gnu- prefix but dropping the
> hard-coded defaults regarding paths for compiler prefixes / library
> locations will allow simplifying the build scripts and Makefiles. Think
> about all the places we've got aarch64 hard coded in the build system.
>
> Looking ahead, dropping the default will also reduce the churn when
> porting OSv to other architectures (RISC-V for example).
>
> What do you think?
>
> Not sure. On one hand, it would be nice for the changes to be
> backwards compatible so that we do not have to update the docs. On
> other hand, I agree with some of your points that it would make it
> easier to support RISC-V in future.

Good point - I am not updating the docs in the patches.

Thinking about it a bit, it would be better to split the changes and
focus this series on enabling cross-compilation on Debian like
distros. Let's do the backward incompatible changes to the build system
in a separate series.

I'll send the updated patches shortly.

Thanks,
Punit

[Punit Agrawal]

Hi,

This is the third version of the patch to enable cross building OSv in
multiarch distros. This version updates the patches to use the
compiler to resolve the library locations before falling back to the
hard coded locations when cross compiling.

The changes enable cross compiling on multiarch distros while leaving
existing usage unchanged in other supported environments.

With these, I am able to cross build OSv for arm64 on Debian using -

$ export CROSS_PREFIX="aarch64-linux-gnu-"
$ ./scripts/build -j8 arch=aarch64 fs=rofs image=native-example

The following steps can be used to setup the cross environment on x86
installation -

# dpkg --add-architecture arm64
# apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libboost-system-dev:arm64

As an additional benefit of the patches, it is now also possible to
use CROSS_PREFIX to speed up compilation for both native and cross
scenarios using ccache without messing with symlinks to compiler
binaries -

$ export CROSS_PREFIX="ccache " # native
or
$ export CROSS_PREFIX="ccache aarch64-linux-gnu-

The patches have been tested on Debian both native and cross. Any

[Punit Agrawal]

When cross compiling, it would be good to support the following two
cases -

* Distro provided cross compiler + libraries - This is the case when a
distro with multi-arch support (e.g., Debian derivaties)

* Local libraries installation - Needed when the distro doesn't
support multi-arch or there's a need to use local installs

Update the libgcc detection to query the compiler regarding the
location of the libraries - provided the correct CROSS_PREFIX is
specified. If this fails to find the library, fall back to the hard
coded location. Also, add a message to inform the user about the
fallback usage.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---

scripts/build | 20 ++++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/scripts/build b/scripts/build
index 5fdcc7a88214..fd03af658fb6 100755
--- a/scripts/build
+++ b/scripts/build
@@ -270,17 +270,21 @@ kernel_end=$(($loader_size+2097151 & ~2097151))
# the case in our old build.mk).
cd $OUT

+CC=gcc

host_arch=$(uname -m)
if [[ "$host_arch" == "x86_64" && "$arch" == 'aarch64' ]]; then
- libgcc_s_path=$(${CROSS_PREFIX:-aarch64-linux-gnu-}gcc -print-file-name=libgcc_s.so.1)
- if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then
- libgcc_s_dir=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)
- else
- libgcc_s_dir=$(dirname $(readlink -f $libgcc_s_path))
- fi

-else

- libgcc_s_dir=$(dirname $(readlink -f $(gcc -print-file-name=libgcc_s.so.1)))

+ CC=${CROSS_PREFIX:-aarch64-linux-gnu-}gcc

+fi
+
+libgcc_s_path=$(${CC} -print-file-name=libgcc_s.so.1)
+if [[ "$libgcc_s_path" == "libgcc_s.so.1" ]]; then
+ cat <<-EOF
+ Unable to resolve libgcc_s.so.1 using "${CC}".
+ Looking in ../downloaded_packages/aarch64/gcc/install/lib64
+ EOF

+ libgcc_s_path=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64)
fi
+libgcc_s_dir=$(dirname $(readlink -f ${libgcc_s_path}))

if [ "$export" != "none" ]; then

export_dir=${vars[export_dir]-$SRC/build/export}
--
2.29.2

[Punit Agrawal]

When cross compiling for aarch64 on an x86_64 host, the build system
assumes that the boost library is placed in
"build/downloaded_packages/aarch64/boost/install". This assumption
works well on systems where cross libraries are not provided and need
to be manually installed but does not support using multiarch distros
that use standard location for cross libraries that the cross compiler
can resolve.

Update the boost library detection logic to first use the compiler to
resolve the boost libraries (both during native and cross-builds) and
failing that for aarch64 falling back to looking in the hardcoded path.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---

Makefile | 30 +++++++++++++++---------------
modules/common.gmk | 20 ++++++++++----------
2 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/Makefile b/Makefile
index 9d80f095a3c7..c4f08e4fb31d 100644
--- a/Makefile
+++ b/Makefile
@@ -1847,22 +1847,20 @@ endif

#Allow user specify non-default location of boost
ifeq ($(boost_base),)
- ifeq ($(CROSS_PREFIX),)
- # link with -mt if present, else the base version (and hope it is multithreaded)
- boost-mt := -mt
+ # link with -mt if present, else the base version (and hope it is multithreaded)
+ boost-mt := -mt
+ boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
+ ifeq ($(filter /%,$(boost-lib-dir)),)
+ boost-mt :=
boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))

- ifeq ($(filter /%,$(boost-lib-dir)),)
- boost-mt :=

- boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system$(boost-mt).a))
- ifeq ($(filter /%,$(boost-lib-dir)),)

- $(error Error: libboost_system.a needs to be installed.)

- endif
- endif
- # When boost_env=host, we won't use "-nostdinc", so the build machine's
- # header files will be used normally. So we don't need to add anything
- # special for Boost.
- boost-includes =
- else

+ endif

+ # When boost_env=host, we won't use "-nostdinc", so the build machine's
+ # header files will be used normally. So we don't need to add anything
+ # special for Boost.
+ boost-includes =

+ ifeq ($(filter /%,$(boost-lib-dir)),)
+ # If the compiler cannot find the boost library, for aarch64 we look in a
+ # special location before giving up.
ifeq ($(arch),aarch64)
aarch64_boostbase = build/downloaded_packages/aarch64/boost/install
ifeq (,$(wildcard $(aarch64_boostbase)))
@@ -1872,6 +1870,8 @@ ifeq ($(boost_base),)

boost-lib-dir := $(firstword $(dir $(shell find $(aarch64_boostbase)/ -name libboost_system*.a)))

boost-mt := $(if $(filter %-mt.a, $(wildcard $(boost-lib-dir)/*.a)),-mt)

boost-includes = -isystem $(aarch64_boostbase)/usr/include
+ else
+ $(error Error: libboost_system.a needs to be installed.)
endif
endif
else
diff --git a/modules/common.gmk b/modules/common.gmk
index e0814b42414e..39c4c7ed2827 100644
--- a/modules/common.gmk
+++ b/modules/common.gmk
@@ -123,16 +123,14 @@ endif

# Let us detect presence of boost headers and library
# Allow user specify non-default location of boost
ifeq ($(boost_base),)
- ifeq ($(CROSS_PREFIX),)
- boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system.so))
- ifeq ($(filter /%,$(boost-lib-dir)),)
- $(error Error: libboost_system.so needs to be installed.)
- endif
- # When boost_env=host, we won't use "-nostdinc", so the build machine's
- # header files will be used normally. So we don't need to add anything
- # special for Boost.
- boost-includes =
- else

+ boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system.so))

+ # When boost_env=host, we won't use "-nostdinc", so the build machine's

+ # header files will be used normally. So we don't need to add anything
+ # special for Boost.
+ boost-includes =

+ ifeq ($(filter /%,$(boost-lib-dir)),)
+ # If the compiler cannot find the boost library, for aarch64 we look in a
+ # special location before giving up.
ifeq ($(arch),aarch64)
aarch64_boostbase = $(src)/build/downloaded_packages/aarch64/boost/install
ifeq (,$(wildcard $(aarch64_boostbase)))
@@ -142,6 +140,8 @@ ifeq ($(boost_base),)

boost-lib-dir := $(firstword $(dir $(shell find $(aarch64_boostbase)/ -name libboost_system.so)))

LDFLAGS += -L$(boost-lib-dir)
boost-includes = -isystem $(aarch64_boostbase)/usr/include
+ else

+ $(error Error: libboost_system.so needs to be installed.)
endif

endif
else
--
2.29.2

[Waldek Kozaczuk]

Sorry for the delay in reviewing it.

This part does not work on Fedora. I think it should be:

libgcc_s_path=$(readlink -f ../downloaded_packages/aarch64/gcc/install/lib64/libgcc_s.so.1) 

[Waldek Kozaczuk]

Thanks. I have tested it for both x64 and aarch64 with cross-compiling on Fedora, Ubuntu, and CentOS 7. t works. I probably have not exhausted all combinations though.

So I am committing it.

Waldek

[Commit Bot]

From: Punit Agrawal <punita...@gmail.com>
Committer: Waldemar Kozaczuk <jwkoz...@gmail.com>
Branch: master

Update boost library detection logic for cross compiling

When cross compiling for aarch64 on an x86_64 host, the build system
assumes that the boost library is placed in
"build/downloaded_packages/aarch64/boost/install". This assumption
works well on systems where cross libraries are not provided and need
to be manually installed but does not support using multiarch distros
that use standard location for cross libraries that the cross compiler
can resolve.

Update the boost library detection logic to first use the compiler to
resolve the boost libraries (both during native and cross-builds) and
failing that for aarch64 falling back to looking in the hardcoded path.

Signed-off-by: Punit Agrawal <punita...@gmail.com>

Message-Id: <20210219223813.842...@gmail.com>

---
diff --git a/Makefile b/Makefile

[Punit Agrawal]

You're right. Let me send an updated patch fixing this.

[...]

[Punit Agrawal]

Waldek Kozaczuk <jwkoz...@gmail.com> writes:

> Thanks. I have tested it for both x64 and aarch64 with cross-compiling on Fedora, Ubuntu, and CentOS 7. t works. I probably have not exhausted all combinations though.
>
> So I am committing it.

Thanks for pushing this out.

[...]

[Punit Agrawal]

When cross compiling, it would be good to support the following two
cases -

* Distro provided cross compiler + libraries - This is the case when a
distro with multi-arch support (e.g., Debian derivaties)

* Local libraries installation - Needed when the distro doesn't
support multi-arch or there's a need to use local installs

Update the libgcc detection to query the compiler regarding the
location of the libraries - provided the correct CROSS_PREFIX is
specified. If this fails to find the library, fall back to the hard
coded location. Also, add a message to inform the user about the
fallback usage.

Signed-off-by: Punit Agrawal <punita...@gmail.com>
---

Hi Waldek,

Here's a fixed up version of the patch. I also dropped the extra call
to readlink as it's going to be applied further on anyways.

Thanks,
Punit

scripts/build | 20 ++++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/scripts/build b/scripts/build

index 5fdcc7a88214..9576f40d6a1a 100755

+ libgcc_s_path="../downloaded_packages/aarch64/gcc/install/lib64/libgcc_s.so.1"

fi
+libgcc_s_dir=$(dirname $(readlink -f ${libgcc_s_path}))

if [ "$export" != "none" ]; then
export_dir=${vars[export_dir]-$SRC/build/export}
--

2.30.0

[Commit Bot]

From: Punit Agrawal <punita...@gmail.com>
Committer: Waldemar Kozaczuk <jwkoz...@gmail.com>
Branch: master

scripts/build: Simplify libgcc_s.so.1 detection

When cross compiling, it would be good to support the following two
cases -

* Distro provided cross compiler + libraries - This is the case when a
distro with multi-arch support (e.g., Debian derivaties)

* Local libraries installation - Needed when the distro doesn't
support multi-arch or there's a need to use local installs

Update the libgcc detection to query the compiler regarding the
location of the libraries - provided the correct CROSS_PREFIX is
specified. If this fails to find the library, fall back to the hard
coded location. Also, add a message to inform the user about the
fallback usage.

Signed-off-by: Punit Agrawal <punita...@gmail.com>

Message-Id: <20210225115258.89...@gmail.com>

---
diff --git a/scripts/build b/scripts/build