[PATCH 1/2] build: set arch earlier

[Stewart Hildebrand]

Add a case in the initial argument parsing loop to both set the variable and
append it to the args array.

While we're here, use a cleaner syntax for appending to an array.

If arch= isn't given on the build command line, explicitly set it to the
default x64.

Signed-off-by: Stewart Hildebrand <stewart.h...@dornerworks.com>
---
scripts/build | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/scripts/build b/scripts/build
index b494141c..d859202a 100755
--- a/scripts/build
+++ b/scripts/build
@@ -80,11 +80,16 @@ do
image=*|modules=*|fs=*|usrskel=*|check|--append-manifest|--create-disk) ;;
clean)
stage1_args=clean ;;
- *) # yuck... Is there a prettier way to append to array?
- args[${#args[@]}]="$i" ;;
+ arch=*)
+ eval ${i}
+ args+=("$i") ;;
+ *)
+ args+=("$i") ;;
esac
done

+arch=${arch:-x64}
+
make "${args[@]}" ${stage1_args} | tee build.out
# check exit status of make
status=${PIPESTATUS[0]}
@@ -162,7 +167,6 @@ fs_size=${vars[fs_size]-$(($fs_size_mb*1024*1024))}
fs_size=$((fs_size - (fs_size & 511)))

SRC=`pwd`
-arch=`expr $OUT : '.*\.\(.*\)'`
mode=`expr $OUT : '.*/\(.*\)\..*'`

# Set "modules" according to the image= or modules= paramters, or some
--
2.30.0

[Stewart Hildebrand]

This is a source-able script that downloads the aarch64 toolchain from
developer.arm.com, extracts it, and, when being sourced, adds the path to PATH.
If ccache is installed, it will also add the ccache directory to the front of
PATH.

Add a check in the OSv build script to source the toolchain if arch=aarch64 and
the toolchain has been downloaded to the build directory.

Add "centos" as an option in download_aarch64_packages.py to download the
toolchain. I did not change the logic for Ubuntu here due to the possibility of
ending up with unvetted combinations of this GCC version with various Ubuntu
system libraries that could be built with different versions of GCC. This
toolchain can still be used on Ubuntu by sourcing it directly, but in this case
I recommend building Boost and other dependent libraries from source to ensure
they are built with the same toolchain that is being used to build OSv and your
apps/modules.

This toolchain should work on any distribution that ARM supports, which is
Ubuntu 16.04 LTS or later, or RHEL 7 or later. I've tested this on CentOS 7 and
Ubuntu 18.04.

My primary motivation for this is to support the aarch64 build on CentOS 7.
While the EPEL repository for CentOS 7 does have an aarch64 cross compiler
toolchain available, it is quite an old version (4.8.5). On Ubuntu, the aarch64
cross compiler toolchain conflicts with the gcc-multilib package. Using the ARM
toolchain allows us to avoid these particular issues.

This toolchain does not provide Boost, so Boost will need to be built. Here are
instructions I used for building boost from source. Ensure that the current
working directory is the root of the OSv repository, then run the following
commands:

$ source scripts/download_aarch64_toolchain.sh
$ mkdir -p build/downloaded_packages/aarch64/boost
$ pushd build/downloaded_packages/aarch64/boost
$ wget https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.gz
$ tar -xf boost_1_70_0.tar.gz
$ cd boost_1_70_0/
$ BOOST_DIR=$(pwd)
$ ./bootstrap.sh
$ echo "using gcc : arm : aarch64-none-linux-gnu-g++ ;" > user-config.jam
$ ./b2 --user-config=${BOOST_DIR}/user-config.jam toolset=gcc-arm architecture=arm address-model=64 -j$(nproc)

After building boost, we need to set up some symlinks so that OSv can find the
boost headers and libraries. Ensure your current working directory is the boost
source directory BOOST_DIR (it should be already), then run the following
commands:

$ ln -s boost_1_70_0/stage ../install
$ mkdir -p stage/usr/include
$ ln -s ../../../boost stage/usr/include/boost
$ popd

After building boost and setting up the symlinks, we should be ready to build
OSv with the aarch64 toolchain:

$ scripts/build arch=aarch64 fs=ramfs --create-disk image=native-example -j$(nproc)

Signed-off-by: Stewart Hildebrand <stewart.h...@dornerworks.com>
---

scripts/build | 5 +++
scripts/download_aarch64_packages.py | 2 +
scripts/download_aarch64_toolchain.sh | 55 +++++++++++++++++++++++++++
3 files changed, 62 insertions(+)
create mode 100644 scripts/download_aarch64_toolchain.sh

diff --git a/scripts/build b/scripts/build
index d859202a..6070ba0f 100755
--- a/scripts/build
+++ b/scripts/build
@@ -90,6 +90,11 @@ done

arch=${arch:-x64}

+if [[ "$arch" == "aarch64" && -d "build/downloaded_packages/aarch64/toolchain/" ]]; then
+ . scripts/download_aarch64_toolchain.sh
+ export CROSS_PREFIX=aarch64-none-linux-gnu-
+fi

+
make "${args[@]}" ${stage1_args} | tee build.out
# check exit status of make
status=${PIPESTATUS[0]}

diff --git a/scripts/download_aarch64_packages.py b/scripts/download_aarch64_packages.py
index 58a3cf7d..931decf9 100755
--- a/scripts/download_aarch64_packages.py
+++ b/scripts/download_aarch64_packages.py
@@ -78,6 +78,8 @@ elif name.lower() == 'ubuntu':
print("Cound not find boost version from neither main nor universe ports index!")
sys.exit(1)
commands_to_download = ubuntu_download_commands(boost_version)
+elif name.lower() == "centos":
+ commands_to_download = [ 'bash -eu %s/scripts/download_aarch64_toolchain.sh' % osv_root ]
else:
print("The distribution %s is not supported for cross-compiling aarch64 version of OSv" % name)
sys.exit(1)
diff --git a/scripts/download_aarch64_toolchain.sh b/scripts/download_aarch64_toolchain.sh
new file mode 100644
index 00000000..0403712c
--- /dev/null
+++ b/scripts/download_aarch64_toolchain.sh
@@ -0,0 +1,55 @@
+#!/bin/bash -eu
+
+# Copyright (c) 2021, DornerWorks, Ltd.
+# Author: Stewart Hildebrand
+# SPDX-License-Identifier: BSD-3-Clause OR MIT
+
+# Usage:
+# $ source download_aarch64_toolchain.sh
+# or
+# $ . download_aarch64_toolchain.sh
+
+# https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads
+
+ARM64_TOOLCHAIN_SCRIPTDIR="$(cd $(dirname ${BASH_SOURCE}) && pwd)/"
+ARM64_TOOLCHAIN_DESTINATION="${ARM64_TOOLCHAIN_SCRIPTDIR}../build/downloaded_packages/aarch64/toolchain/"
+ARM64_TOOLCHAIN_VERSION=9.2-2019.12
+ARM64_TOOLCHAIN_FILENAME=gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
+
+if [ ! -d ${ARM64_TOOLCHAIN_DESTINATION}${ARM64_TOOLCHAIN_FILENAME} ]; then
+ mkdir -p "${ARM64_TOOLCHAIN_DESTINATION}"
+ pushd "${ARM64_TOOLCHAIN_DESTINATION}" >/dev/null
+ if [ ! -s ${ARM64_TOOLCHAIN_FILENAME}.tar.xz ]; then
+ wget https://developer.arm.com/-/media/Files/downloads/gnu-a/${ARM64_TOOLCHAIN_VERSION}/binrel/${ARM64_TOOLCHAIN_FILENAME}.tar.xz
+ fi
+ tar -xf ${ARM64_TOOLCHAIN_FILENAME}.tar.xz
+ popd >/dev/null
+fi
+
+if [ -d /usr/lib64/ccache ]; then
+ CCACHE_SYMLINK_DIR=/usr/lib64/ccache
+fi
+if [ -d /usr/lib/ccache ]; then
+ CCACHE_SYMLINK_DIR=/usr/lib/ccache
+fi
+
+PATH=${ARM64_TOOLCHAIN_DESTINATION}${ARM64_TOOLCHAIN_FILENAME}/bin:${PATH}
+if [ ! -z "${CCACHE_SYMLINK_DIR-}" ]; then
+ PATH=${CCACHE_SYMLINK_DIR}:$(echo ${PATH} | sed "s|${CCACHE_SYMLINK_DIR}:||g")
+fi
+
+if [ -z "${CCACHE_SYMLINK_DIR-}" ]; then
+ echo "It is recommended to install ccache and create symlinks for the"
+ echo "aarch64 toolchain in order to significantly speed up the build."
+else
+ if [ ! -h "${CCACHE_SYMLINK_DIR}/aarch64-none-linux-gnu-g++" ] || [ ! -h "${CCACHE_SYMLINK_DIR}/aarch64-none-linux-gnu-gcc" ]; then
+ echo "It is recommended to create ccache symlinks for the aarch64"
+ echo "toolchain in order to significantly speed up the build. Run the"
+ echo "following commands to create the appropriate symlinks:"
+ echo ""
+ echo " sudo ln -s ../../bin/ccache \"${CCACHE_SYMLINK_DIR}/aarch64-none-linux-gnu-gcc\""
+ echo " sudo ln -s ../../bin/ccache \"${CCACHE_SYMLINK_DIR}/aarch64-none-linux-gnu-g++\""
+ echo ""
+ echo "After this, please re-source the toolchain script to enable ccache."
+ fi
+fi
--
2.30.0

[Stewart Hildebrand]

The 2nd patch should have the subject "[PATCH 2/2] Add script to download aarch64 cross compiler toolchain", and I believe the original email sent in fact has the correct subject. However, this 2nd email is In-Reply-To the first one, and due to the way google groups displays it appears the subject is hidden. Sorry about that. What's the recommended way to send a patch series with multiple patches?

[Waldek Kozaczuk]

Sorry I missed your question. This wiki page should explain all this.

Waldek

--
You received this message because you are subscribed to the Google Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/4d0a4acf-d7f2-4ec0-ae2f-ecb72a0e988bn%40googlegroups.com.

[Waldek Kozaczuk]

Should we really default to x64 or the host arch (uname -m)? I build often on my Odroid machine natively.

In reality, it probably should be like that:

arch=$(uname -m)

if [ "${arch}" == "x86_64" ]; then

        arch="x64"

fi

 

+
make "${args[@]}" ${stage1_args} | tee build.out
# check exit status of make
status=${PIPESTATUS[0]}
@@ -162,7 +167,6 @@ fs_size=${vars[fs_size]-$(($fs_size_mb*1024*1024))}
fs_size=$((fs_size - (fs_size & 511)))

SRC=`pwd`
-arch=`expr $OUT : '.*\.\(.*\)'`

So I guess we are deciding to default to the host arch (or x64)  instead of whatever the last build was, are we?

[Waldek Kozaczuk]

I guess once I sent my patch that adds ./scripts/download_and_build_boost.sh we will

not need to do that manually and most of the below will be much simpler.

[Stewart Hildebrand]

I forgot to consider this case. I was misled by a comment further above in the build script that says "default is x64" (I'll update the comment in v2). Taking a step back, I don't actually need to set arch earlier. I only need to know if it's a cross build.

Assuming these are the supported build configs:

1. build==x64, target == x64 (arch may be omitted/unset)

2. build==x64, target == aarch64 (arch=aarch64)

3. build==aarch64, target==aarch64 (arch may be omitted/unset)

The only time we'd want to use the cross compiler toolchain is if build == x64 and target == aarch64. In this case, specifying arch=aarch64 is required, and it'll be set in the case I added to the arg-parsing loop above. For v2, I'll simply omit the default-setting line here.

+
make "${args[@]}" ${stage1_args} | tee build.out
# check exit status of make
status=${PIPESTATUS[0]}
@@ -162,7 +167,6 @@ fs_size=${vars[fs_size]-$(($fs_size_mb*1024*1024))}
fs_size=$((fs_size - (fs_size & 511)))

SRC=`pwd`
-arch=`expr $OUT : '.*\.\(.*\)'`

So I guess we are deciding to default to the host arch (or x64)  instead of whatever the last build was, are we?

I'll leave this line in for v2.

[Stewart Hildebrand]

Stewart Hildebrand (2):
build: set arch in arg-parsing loop

Add script to download aarch64 cross compiler toolchain

scripts/build | 14 +++++--

scripts/download_aarch64_packages.py | 2 +
scripts/download_aarch64_toolchain.sh | 55 +++++++++++++++++++++++++++

3 files changed, 68 insertions(+), 3 deletions(-)
create mode 100644 scripts/download_aarch64_toolchain.sh

--
2.30.0

[Stewart Hildebrand]

This is a source-able script that downloads the aarch64 toolchain from
developer.arm.com, extracts it, and, when being sourced, adds the path to PATH.
If ccache is installed, it will also add the ccache directory to the front of
PATH.

Add a check in the OSv build script to source the toolchain if arch=aarch64 and
the toolchain has been downloaded to the build directory.

Add "centos" as an option in download_aarch64_packages.py to download the
toolchain. I did not change the logic for Ubuntu here due to the possibility of
ending up with unvetted combinations of this GCC version with various Ubuntu
system libraries that could be built with different versions of GCC. This
toolchain can still be used on Ubuntu by sourcing it directly, but in this case
I recommend building Boost and other dependent libraries from source to ensure
they are built with the same toolchain that is being used to build OSv and your
apps/modules.

This toolchain should work on any distribution that ARM supports, which is
Ubuntu 16.04 LTS or later, or RHEL 7 or later. I've tested this on CentOS 7 and
Ubuntu 18.04.

My primary motivation for this is to support the aarch64 build on CentOS 7.
While the EPEL repository for CentOS 7 does have an aarch64 cross compiler
toolchain available, it is quite an old version (4.8.5). On Ubuntu, the aarch64
cross compiler toolchain conflicts with the gcc-multilib package. Using the ARM
toolchain allows us to avoid these particular issues.

This toolchain does not provide Boost, so Boost will need to be built. Boost can
be built with the script download_and_build_boost.sh in the scripts/ directory,
or it can be built manually. Here are instructions I used for building Boost

from source. Ensure that the current working directory is the root of the OSv
repository, then run the following commands:

$ source scripts/download_aarch64_toolchain.sh
$ mkdir -p build/downloaded_packages/aarch64/boost
$ pushd build/downloaded_packages/aarch64/boost
$ wget https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.gz
$ tar -xf boost_1_70_0.tar.gz
$ cd boost_1_70_0/
$ BOOST_DIR=$(pwd)
$ ./bootstrap.sh
$ echo "using gcc : arm : aarch64-none-linux-gnu-g++ ;" > user-config.jam
$ ./b2 --user-config=${BOOST_DIR}/user-config.jam toolset=gcc-arm architecture=arm address-model=64 -j$(nproc)

After building Boost, we need to set up some symlinks so that OSv can find the
Boost headers and libraries. Ensure your current working directory is the Boost

source directory BOOST_DIR (it should be already), then run the following
commands:

$ ln -s boost_1_70_0/stage ../install
$ mkdir -p stage/usr/include
$ ln -s ../../../boost stage/usr/include/boost
$ popd

After building Boost and setting up the symlinks, we should be ready to build

OSv with the aarch64 toolchain:

$ scripts/build arch=aarch64 fs=ramfs --create-disk image=native-example -j$(nproc)

Signed-off-by: Stewart Hildebrand <stewart.h...@dornerworks.com>

---

v2:
expand arch with a trailing dash in recognition that it may be unset
update patch description to mention Waldek's boost build script
---
scripts/build | 5 +++

scripts/download_aarch64_packages.py | 2 +
scripts/download_aarch64_toolchain.sh | 55 +++++++++++++++++++++++++++

3 files changed, 62 insertions(+)
create mode 100644 scripts/download_aarch64_toolchain.sh

diff --git a/scripts/build b/scripts/build
index 250d5360..5fdcc7a8 100755
--- a/scripts/build
+++ b/scripts/build
@@ -88,6 +88,11 @@ do
esac
done

+if [[ "${arch-}" == "aarch64" && -d "build/downloaded_packages/aarch64/toolchain/" ]]; then

+ . scripts/download_aarch64_toolchain.sh
+ export CROSS_PREFIX=aarch64-none-linux-gnu-
+fi

+
make "${args[@]}" ${stage1_args} | tee build.out
# check exit status of make
status=${PIPESTATUS[0]}

[Commit Bot]

From: Stewart Hildebrand <stewart.h...@dornerworks.com>
Committer: Waldemar Kozaczuk <jwkoz...@gmail.com>
Branch: master

Add script to download aarch64 cross compiler toolchain

Message-Id: <20210208184545.97466...@dornerworks.com>

---
diff --git a/scripts/build b/scripts/build

--- a/scripts/build
+++ b/scripts/build
@@ -88,6 +88,11 @@ do
esac
done

+if [[ "${arch-}" == "aarch64" && -d "build/downloaded_packages/aarch64/toolchain/" ]]; then
+ . scripts/download_aarch64_toolchain.sh
+ export CROSS_PREFIX=aarch64-none-linux-gnu-
+fi
+
make "${args[@]}" ${stage1_args} | tee build.out
# check exit status of make
status=${PIPESTATUS[0]}
diff --git a/scripts/download_aarch64_packages.py b/scripts/download_aarch64_packages.py

--- a/scripts/download_aarch64_packages.py
+++ b/scripts/download_aarch64_packages.py
@@ -78,6 +78,8 @@ def ubuntu_identify_boost_version(codename, index):

print("Cound not find boost version from neither main nor universe ports index!")
sys.exit(1)
commands_to_download = ubuntu_download_commands(boost_version)
+elif name.lower() == "centos":
+ commands_to_download = [ 'bash -eu %s/scripts/download_aarch64_toolchain.sh' % osv_root ]
else:
print("The distribution %s is not supported for cross-compiling aarch64 version of OSv" % name)
sys.exit(1)
diff --git a/scripts/download_aarch64_toolchain.sh b/scripts/download_aarch64_toolchain.sh

--- a/scripts/download_aarch64_toolchain.sh