[PATCH 0/8] Provide infrastructure and examples for custom kernels and modules

235 views
Skip to first unread message

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
This is now finally the promised patches to provide simple patterns for
building custom kernels for Isar images that replace the distro kernels.
It also provids the same pattern for custom out-of-tree kernel modules.

Along come two fixes, one to respect proxies during dpkg builds, the
other to adjust isar-apt repo prio.

Furthermore, I'm proposing to rename "SRC_DIR" to OE-standard "S".

This series is also available at

https://github.com/siemens/isar jan/kernel-build

Jan

Jan Kiszka (8):
Forward proxy settings to dpkg build
Prioritize isar-apt repo over all others
Replace SRC_DIR with S
Install kernel via replaceable recipe
Provide include file for easy custom kernel builds
Add custom kernel examples
Provide include file for easy custom module builds
Add exemplary kernel module

doc/user_manual.md | 6 +-
meta-isar/conf/distro/debian-jessie.conf | 2 +
meta-isar/conf/distro/debian-stretch.conf | 2 +
meta-isar/conf/distro/debian-wheezy.conf | 2 +
meta-isar/conf/distro/raspbian-jessie.conf | 2 +
meta-isar/conf/local.conf.sample | 2 +-
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 3 +-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 3 +-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 3 +-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 +-
.../recipes-app/example-hello/example-hello.bb | 2 +-
meta-isar/recipes-app/libhello/libhello.bb | 2 +-
.../recipes-core/images/files/multistrap.conf.in | 1 +
meta-isar/recipes-core/images/isar-image-base.bb | 6 +-
.../example-module/example-module.bb | 5 +
.../example-module/files/src/Makefile | 11 +
.../example-module/files/src/example-module.c | 19 ++
meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 10 +
.../recipes-kernel/linux/linux-mainline_4.14.17.bb | 8 +
meta/classes/dpkg.bbclass | 3 +-
meta/recipes-devtools/buildchroot/buildchroot.bb | 4 +-
.../buildchroot/files/isar-apt-prefs | 3 +
.../buildchroot/files/multistrap.conf.in | 1 +
.../linux-module/files/debian/changelog | 5 +
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 +
.../recipes-kernel/linux-module/files/debian/rules | 8 +
meta/recipes-kernel/linux-module/module.inc | 17 ++
meta/recipes-kernel/linux/linux-custom.inc | 91 ++++++
meta/recipes-kernel/linux/linux-distro.bb | 14 +
35 files changed, 549 insertions(+), 17 deletions(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c
create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.17.bb
create mode 100644 meta/recipes-devtools/buildchroot/files/isar-apt-prefs
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc
create mode 100644 meta/recipes-kernel/linux/linux-custom.inc
create mode 100644 meta/recipes-kernel/linux/linux-distro.bb

--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This simplifies the common task of using a custom kernel instead of the
pre-selected disto variant: Move the kernel installation into a dummy
dpkg-raw recipe that only has the kernel package as dependency.

Which recipe is used for providing the kernel can now be selected via
the well-know PREFERRED_PROVIDER_virtual/kernel, just like in OE.

The kernel package name is derived from the KERNEL_ARCH variable which
is communicated from the target multiconfig file to the linux-distro
recipe.

Based on an idea of Henning Schild.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/distro/debian-jessie.conf | 2 ++
meta-isar/conf/distro/debian-stretch.conf | 2 ++
meta-isar/conf/distro/debian-wheezy.conf | 2 ++
meta-isar/conf/distro/raspbian-jessie.conf | 2 ++
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 ++-
meta-isar/recipes-core/images/isar-image-base.bb | 5 ++++-
meta/recipes-kernel/linux/linux-distro.bb | 14 ++++++++++++++
14 files changed, 38 insertions(+), 9 deletions(-)
create mode 100644 meta/recipes-kernel/linux/linux-distro.bb

diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta-isar/conf/distro/debian-jessie.conf
index d4bc4b1..4494c91 100644
--- a/meta-isar/conf/distro/debian-jessie.conf
+++ b/meta-isar/conf/distro/debian-jessie.conf
@@ -10,3 +10,5 @@ DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security"
DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
+
+PREFERRED_PROVIDER_virtual/kernel ?= "linux-distro"
diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta-isar/conf/distro/debian-stretch.conf
index 74177fd..1375c1d 100644
--- a/meta-isar/conf/distro/debian-stretch.conf
+++ b/meta-isar/conf/distro/debian-stretch.conf
@@ -8,3 +8,5 @@ DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security"
DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
+
+PREFERRED_PROVIDER_virtual/kernel ?= "linux-distro"
diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta-isar/conf/distro/debian-wheezy.conf
index 017316e..1e99ddd 100644
--- a/meta-isar/conf/distro/debian-wheezy.conf
+++ b/meta-isar/conf/distro/debian-wheezy.conf
@@ -10,3 +10,5 @@ DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security"
DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
+
+PREFERRED_PROVIDER_virtual/kernel ?= "linux-distro"
diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-isar/conf/distro/raspbian-jessie.conf
index bd3c280..6a6c568 100644
--- a/meta-isar/conf/distro/raspbian-jessie.conf
+++ b/meta-isar/conf/distro/raspbian-jessie.conf
@@ -10,3 +10,5 @@ DISTRO_APT_SOURCE_SEC ?= ""
DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh"
DISTRO_MULTICONF_BOOTSTRAP ?= "base"
DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
+
+PREFERRED_PROVIDER_virtual/kernel ?= "linux-distro"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
index 51d39b6..c80a936 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
@@ -6,7 +6,7 @@ MACHINE ?= "qemuamd64"
DISTRO ?= "debian-jessie"
DISTRO_ARCH ?= "amd64"

-IMAGE_PREINSTALL += "linux-image-amd64"
+KERNEL_ARCH ?= "amd64"

ROOTFS_DEV ?= "sda"

diff --git a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
index c59876a..28ab3ae 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
@@ -6,9 +6,10 @@ MACHINE ?= "qemuamd64"
DISTRO ?= "debian-stretch"
DISTRO_ARCH ?= "amd64"

+KERNEL_ARCH ?= "amd64"
+
IMAGE_PREINSTALL += " \
init \
- linux-image-amd64 \
"

ROOTFS_DEV ?= "sda"
diff --git a/meta-isar/conf/multiconfig/qemuarm-jessie.conf b/meta-isar/conf/multiconfig/qemuarm-jessie.conf
index ed84c6c..48244eb 100644
--- a/meta-isar/conf/multiconfig/qemuarm-jessie.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-jessie.conf
@@ -13,7 +13,7 @@ MACHINE ?= "qemuarm"
DISTRO ?= "debian-jessie"
DISTRO_ARCH ?= "armhf"

-IMAGE_PREINSTALL += "linux-image-armmp"
+KERNEL_ARCH ?= "armmp"

ROOTFS_DEV ?= "vda"

diff --git a/meta-isar/conf/multiconfig/qemuarm-stretch.conf b/meta-isar/conf/multiconfig/qemuarm-stretch.conf
index 1cce97a..5cf24f2 100644
--- a/meta-isar/conf/multiconfig/qemuarm-stretch.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-stretch.conf
@@ -6,9 +6,10 @@ MACHINE ?= "qemuarm"
DISTRO ?= "debian-stretch"
DISTRO_ARCH ?= "armhf"

+KERNEL_ARCH ?= "armmp"
+
IMAGE_PREINSTALL += " \
init \
- linux-image-armmp \
"

ROOTFS_DEV ?= "vda"
diff --git a/meta-isar/conf/multiconfig/qemuarm-wheezy.conf b/meta-isar/conf/multiconfig/qemuarm-wheezy.conf
index 38ea191..695cdce 100644
--- a/meta-isar/conf/multiconfig/qemuarm-wheezy.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-wheezy.conf
@@ -13,7 +13,7 @@ MACHINE ?= "qemuarm"
DISTRO ?= "debian-wheezy"
DISTRO_ARCH ?= "armhf"

-IMAGE_PREINSTALL += "linux-image-vexpress"
+KERNEL_ARCH ?= "vexpress"

ROOTFS_DEV ?= "mmcblk0"

diff --git a/meta-isar/conf/multiconfig/qemui386-jessie.conf b/meta-isar/conf/multiconfig/qemui386-jessie.conf
index d589661..f73ad98 100644
--- a/meta-isar/conf/multiconfig/qemui386-jessie.conf
+++ b/meta-isar/conf/multiconfig/qemui386-jessie.conf
@@ -6,7 +6,7 @@ MACHINE ?= "qemui386"
DISTRO ?= "debian-jessie"
DISTRO_ARCH ?= "i386"

-IMAGE_PREINSTALL += "linux-image-686-pae"
+KERNEL_ARCH ?= "686-pae"

ROOTFS_DEV ?= "sda"

diff --git a/meta-isar/conf/multiconfig/qemui386-stretch.conf b/meta-isar/conf/multiconfig/qemui386-stretch.conf
index 40239e4..60f8ccb 100644
--- a/meta-isar/conf/multiconfig/qemui386-stretch.conf
+++ b/meta-isar/conf/multiconfig/qemui386-stretch.conf
@@ -6,9 +6,10 @@ MACHINE ?= "qemui386"
DISTRO ?= "debian-stretch"
DISTRO_ARCH ?= "i386"

+KERNEL_ARCH ?= "686-pae"
+
IMAGE_PREINSTALL += " \
init \
- linux-image-686-pae \
"

ROOTFS_DEV ?= "sda"
diff --git a/meta-isar/conf/multiconfig/rpi-jessie.conf b/meta-isar/conf/multiconfig/rpi-jessie.conf
index d1dd9d9..c7c7fae 100644
--- a/meta-isar/conf/multiconfig/rpi-jessie.conf
+++ b/meta-isar/conf/multiconfig/rpi-jessie.conf
@@ -13,13 +13,14 @@ MACHINE ?= "rpi"
DISTRO ?= "raspbian-jessie"
DISTRO_ARCH ?= "armhf"

+KERNEL_ARCH ?= "rpi-rpfv"
+
IMAGE_PREINSTALL += " \
bind9-host \
dnsutils \
iproute2 \
iputils-ping \
isc-dhcp-client \
- linux-image-rpi-rpfv \
lrzsz \
net-tools \
netcat-openbsd \
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index e54b9a6..5340767 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -18,12 +18,15 @@ PV = "1.0"

inherit image

-DEPENDS += "${IMAGE_INSTALL}"
+DEPENDS += "${IMAGE_INSTALL} \
+ virtual/kernel"

IMAGE_PREINSTALL += "apt \
dbus \
localepurge"

+IMAGE_INSTALL += "${@d.getVar("PREFERRED_PROVIDER_virtual/kernel", True)}"
+
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"

do_rootfs[dirs] = "${WORKDIR}/hooks_multistrap"
diff --git a/meta/recipes-kernel/linux/linux-distro.bb b/meta/recipes-kernel/linux/linux-distro.bb
new file mode 100644
index 0000000..a0700e8
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-distro.bb
@@ -0,0 +1,14 @@
+# Debian kernel pseudo package
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+
+DESCRIPTION = "Meta package for distribution kernel"
+MAINTAINER = "ISAR project <isar-...@googlegroups.com>"
+PROVIDES = "virtual/kernel"
+
+KERNEL_ARCH ?= "${DISTRO_ARCH}"
+
+DEBIAN_DEPENDS = "linux-image-${KERNEL_ARCH}"
+
+inherit dpkg-raw
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Analogously to linux-custom.inc, this provides an include for custom
kernel modules. The user just needs to specify the module SRC_URI, and
those sources must be then reside in a separate directory which is - as
usual - defined via S.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/recipes-kernel/linux-module/files/debian/changelog | 5 +++++
meta/recipes-kernel/linux-module/files/debian/compat | 1 +
meta/recipes-kernel/linux-module/files/debian/control | 11 +++++++++++
meta/recipes-kernel/linux-module/files/debian/rules | 8 ++++++++
meta/recipes-kernel/linux-module/module.inc | 17 +++++++++++++++++
5 files changed, 42 insertions(+)
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog
new file mode 100644
index 0000000..c1c3516
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog
@@ -0,0 +1,5 @@
+@PN@ (@PV@) unstable; urgency=low
+
+ * Generated package.
+
+ -- ISAR project <isar-...@googlegroups.com> Tue, 6 Feb 2018 00:00:00 +0000
diff --git a/meta/recipes-kernel/linux-module/files/debian/compat b/meta/recipes-kernel/linux-module/files/debian/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
new file mode 100644
index 0000000..d8fbcaa
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control
@@ -0,0 +1,11 @@
+Source: @PN@
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-@KERNEL_ARCH@
+Maintainer: ISAR project <isar-...@googlegroups.com>
+
+Package: @PN@
+Architecture: any
+Depends: linux-image-@KERNEL_ARCH@
+Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules
new file mode 100755
index 0000000..51d3dc5
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/rules
@@ -0,0 +1,8 @@
+#!/usr/bin/make -f
+
+export KDIR=$(shell ls -d /lib/modules/*/build)
+
+export DEB_BUILD_OPTIONS=parallel=$(shell nproc)
+
+%:
+ dh $@ --parallel
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
new file mode 100644
index 0000000..a17a14f
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -0,0 +1,17 @@
+FILESPATH =. "${LAYERDIR_core}/recipes-kernel/linux-module/files:"
+
+DESCRIPTION ?= "Custom kernel module ${PN}"
+
+DEPENDS = "virtual/kernel"
+
+inherit dpkg
+
+SRC_URI = "file://debian/"
+
+dpkg_runbuild_prepend() {
+ cp -r ${WORKDIR}/debian ${WORKDIR}/${S}/
+ sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
+ -e 's/@KERNEL_ARCH@/${KERNEL_ARCH}/g' \
+ -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
+ ${WORKDIR}/${S}/debian/changelog ${WORKDIR}/${S}/debian/control
+}
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This aligns us with OE/bitbake style.

Along updating the manual, also fix SRC_REV -> SRCREV.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
doc/user_manual.md | 6 +++---
meta-isar/recipes-app/example-hello/example-hello.bb | 2 +-
meta-isar/recipes-app/libhello/libhello.bb | 2 +-
meta/classes/dpkg.bbclass | 2 +-
4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 969f6d2..0771d1e 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -513,7 +513,7 @@ PV = "1.0"
SRC_URI = "git://github.com/ilbers/hello.git"
SRCREV = "ad7065ecc4840cc436bfcdac427386dbba4ea719"

-SRC_DIR = "git"
+S = "git"

inherit dpkg
```
@@ -531,8 +531,8 @@ This approach prevents duplication of the license files in different packages.

- `PV` - Package version.
- `SRC_URI` - The link where to fetch application source. Please check the BitBake user manual for supported download formats.
- - `SRC_DIR` - The directory name where application sources will be unpacked. For `git` repositories, it should be set to `git`. Please check the BitBake user manual for supported download formats.
- - `SRC_REV` - Source code revision to fetch. Please check the BitBake user manual for supported download formats.
+ - `S` - The directory name where application sources will be unpacked. For `git` repositories, it should be set to `git`. Please check the BitBake user manual for supported download formats.
+ - `SRCREV` - Source code revision to fetch. Please check the BitBake user manual for supported download formats.

The last line in the example above adds recipe to the Isar work chain.

diff --git a/meta-isar/recipes-app/example-hello/example-hello.bb b/meta-isar/recipes-app/example-hello/example-hello.bb
index 9d83e4c..73cca0a 100644
--- a/meta-isar/recipes-app/example-hello/example-hello.bb
+++ b/meta-isar/recipes-app/example-hello/example-hello.bb
@@ -17,6 +17,6 @@ DEPENDS += "libhello"
SRC_URI = "git://github.com/ilbers/hello.git;protocol=https"
SRCREV = "86cc719b3359adc3c4e243387feba50360a860f3"

-SRC_DIR = "git"
+S = "git"

inherit dpkg
diff --git a/meta-isar/recipes-app/libhello/libhello.bb b/meta-isar/recipes-app/libhello/libhello.bb
index df5793e..1875831 100644
--- a/meta-isar/recipes-app/libhello/libhello.bb
+++ b/meta-isar/recipes-app/libhello/libhello.bb
@@ -13,6 +13,6 @@ PV = "0.1-98f2e41"
SRC_URI = "git://github.com/ilbers/libhello.git;protocol=https"
SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93"

-SRC_DIR = "git"
+S = "git"

inherit dpkg
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 9cd4403..06f0579 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -6,5 +6,5 @@ inherit dpkg-base
# Build package from sources using build script
dpkg_runbuild() {
E="${@ bb.utils.export_proxies(d)}"
- sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
+ sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${S}
}
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Required when the package needs to install build dependencies.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/classes/dpkg.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index c10f5ba..9cd4403 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -5,5 +5,6 @@ inherit dpkg-base

# Build package from sources using build script
dpkg_runbuild() {
- sudo chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
+ E="${@ bb.utils.export_proxies(d)}"
+ sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
}
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Provide two examples for describing custom kernel builds. Both are for
x86_64, simply using upstream x86_64_defconfig to define the required
defconfig. One example is demoing a build from a tarball, the other
pulling from git.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 10 +
.../recipes-kernel/linux/linux-mainline_4.14.17.bb | 8 +
3 files changed, 326 insertions(+)
create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.17.bb

diff --git a/meta-isar/recipes-kernel/linux/files/defconfig b/meta-isar/recipes-kernel/linux/files/defconfig
new file mode 100644
index 0000000..e32fc1f
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/files/defconfig
@@ -0,0 +1,308 @@
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_TASKSTATS=y
+CONFIG_TASK_DELAY_ACCT=y
+CONFIG_TASK_XACCT=y
+CONFIG_TASK_IO_ACCOUNTING=y
+CONFIG_FHANDLE=y
+CONFIG_AUDIT=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_LOG_BUF_SHIFT=18
+CONFIG_CGROUPS=y
+CONFIG_CGROUP_FREEZER=y
+CONFIG_CPUSETS=y
+CONFIG_CGROUP_CPUACCT=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_BLK_DEV_INITRD=y
+# CONFIG_COMPAT_BRK is not set
+CONFIG_PROFILING=y
+CONFIG_KPROBES=y
+CONFIG_JUMP_LABEL=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+CONFIG_MODULE_FORCE_UNLOAD=y
+CONFIG_PARTITION_ADVANCED=y
+CONFIG_OSF_PARTITION=y
+CONFIG_AMIGA_PARTITION=y
+CONFIG_MAC_PARTITION=y
+CONFIG_BSD_DISKLABEL=y
+CONFIG_MINIX_SUBPARTITION=y
+CONFIG_SOLARIS_X86_PARTITION=y
+CONFIG_UNIXWARE_DISKLABEL=y
+CONFIG_SGI_PARTITION=y
+CONFIG_SUN_PARTITION=y
+CONFIG_KARMA_PARTITION=y
+CONFIG_EFI_PARTITION=y
+CONFIG_SMP=y
+CONFIG_CALGARY_IOMMU=y
+CONFIG_NR_CPUS=64
+CONFIG_SCHED_SMT=y
+CONFIG_PREEMPT_VOLUNTARY=y
+CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
+CONFIG_X86_MCE=y
+CONFIG_MICROCODE=y
+CONFIG_MICROCODE_AMD=y
+CONFIG_X86_MSR=y
+CONFIG_X86_CPUID=y
+CONFIG_NUMA=y
+CONFIG_X86_CHECK_BIOS_CORRUPTION=y
+# CONFIG_MTRR_SANITIZER is not set
+CONFIG_EFI=y
+CONFIG_HZ_1000=y
+CONFIG_KEXEC=y
+CONFIG_CRASH_DUMP=y
+CONFIG_RANDOMIZE_BASE=y
+CONFIG_RANDOMIZE_MEMORY=y
+# CONFIG_COMPAT_VDSO is not set
+CONFIG_HIBERNATION=y
+CONFIG_PM_DEBUG=y
+CONFIG_PM_TRACE_RTC=y
+CONFIG_ACPI_DOCK=y
+CONFIG_CPU_FREQ=y
+# CONFIG_CPU_FREQ_STAT is not set
+CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
+CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
+CONFIG_CPU_FREQ_GOV_ONDEMAND=y
+CONFIG_X86_ACPI_CPUFREQ=y
+CONFIG_PCI_MMCONFIG=y
+CONFIG_PCIEPORTBUS=y
+CONFIG_PCCARD=y
+CONFIG_YENTA=y
+CONFIG_HOTPLUG_PCI=y
+CONFIG_BINFMT_MISC=y
+CONFIG_IA32_EMULATION=y
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_XFRM_USER=y
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+CONFIG_IP_ROUTE_VERBOSE=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_IP_PNP_BOOTP=y
+CONFIG_IP_PNP_RARP=y
+CONFIG_IP_MROUTE=y
+CONFIG_IP_PIMSM_V1=y
+CONFIG_IP_PIMSM_V2=y
+CONFIG_SYN_COOKIES=y
+# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_BEET is not set
+# CONFIG_INET_DIAG is not set
+CONFIG_TCP_CONG_ADVANCED=y
+# CONFIG_TCP_CONG_BIC is not set
+# CONFIG_TCP_CONG_WESTWOOD is not set
+# CONFIG_TCP_CONG_HTCP is not set
+CONFIG_TCP_MD5SIG=y
+CONFIG_IPV6=y
+CONFIG_INET6_AH=y
+CONFIG_INET6_ESP=y
+CONFIG_NETLABEL=y
+CONFIG_NETFILTER=y
+# CONFIG_NETFILTER_ADVANCED is not set
+CONFIG_NF_CONNTRACK=y
+CONFIG_NF_CONNTRACK_FTP=y
+CONFIG_NF_CONNTRACK_IRC=y
+CONFIG_NF_CONNTRACK_SIP=y
+CONFIG_NF_CT_NETLINK=y
+CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=y
+CONFIG_NETFILTER_XT_TARGET_NFLOG=y
+CONFIG_NETFILTER_XT_TARGET_SECMARK=y
+CONFIG_NETFILTER_XT_TARGET_TCPMSS=y
+CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
+CONFIG_NETFILTER_XT_MATCH_POLICY=y
+CONFIG_NETFILTER_XT_MATCH_STATE=y
+CONFIG_NF_CONNTRACK_IPV4=y
+CONFIG_IP_NF_IPTABLES=y
+CONFIG_IP_NF_FILTER=y
+CONFIG_IP_NF_TARGET_REJECT=y
+CONFIG_NF_NAT=y
+CONFIG_IP_NF_TARGET_MASQUERADE=y
+CONFIG_IP_NF_MANGLE=y
+CONFIG_NF_CONNTRACK_IPV6=y
+CONFIG_IP6_NF_IPTABLES=y
+CONFIG_IP6_NF_MATCH_IPV6HEADER=y
+CONFIG_IP6_NF_FILTER=y
+CONFIG_IP6_NF_TARGET_REJECT=y
+CONFIG_IP6_NF_MANGLE=y
+CONFIG_NET_SCHED=y
+CONFIG_NET_EMATCH=y
+CONFIG_NET_CLS_ACT=y
+CONFIG_HAMRADIO=y
+CONFIG_CFG80211=y
+CONFIG_MAC80211=y
+CONFIG_MAC80211_LEDS=y
+CONFIG_RFKILL=y
+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_DEBUG_DEVRES=y
+CONFIG_CONNECTOR=y
+CONFIG_BLK_DEV_LOOP=y
+CONFIG_BLK_DEV_SD=y
+CONFIG_BLK_DEV_SR=y
+CONFIG_BLK_DEV_SR_VENDOR=y
+CONFIG_CHR_DEV_SG=y
+CONFIG_SCSI_CONSTANTS=y
+CONFIG_SCSI_SPI_ATTRS=y
+# CONFIG_SCSI_LOWLEVEL is not set
+CONFIG_ATA=y
+CONFIG_SATA_AHCI=y
+CONFIG_ATA_PIIX=y
+CONFIG_PATA_AMD=y
+CONFIG_PATA_OLDPIIX=y
+CONFIG_PATA_SCH=y
+CONFIG_MD=y
+CONFIG_BLK_DEV_MD=y
+CONFIG_BLK_DEV_DM=y
+CONFIG_DM_MIRROR=y
+CONFIG_DM_ZERO=y
+CONFIG_MACINTOSH_DRIVERS=y
+CONFIG_MAC_EMUMOUSEBTN=y
+CONFIG_NETDEVICES=y
+CONFIG_NETCONSOLE=y
+CONFIG_TIGON3=y
+CONFIG_NET_TULIP=y
+CONFIG_E100=y
+CONFIG_E1000=y
+CONFIG_E1000E=y
+CONFIG_SKY2=y
+CONFIG_FORCEDETH=y
+CONFIG_8139TOO=y
+CONFIG_R8169=y
+CONFIG_FDDI=y
+CONFIG_INPUT_POLLDEV=y
+# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
+CONFIG_INPUT_EVDEV=y
+CONFIG_INPUT_JOYSTICK=y
+CONFIG_INPUT_TABLET=y
+CONFIG_INPUT_TOUCHSCREEN=y
+CONFIG_INPUT_MISC=y
+CONFIG_VT_HW_CONSOLE_BINDING=y
+# CONFIG_LEGACY_PTYS is not set
+CONFIG_SERIAL_NONSTANDARD=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_NR_UARTS=32
+CONFIG_SERIAL_8250_EXTENDED=y
+CONFIG_SERIAL_8250_MANY_PORTS=y
+CONFIG_SERIAL_8250_SHARE_IRQ=y
+CONFIG_SERIAL_8250_DETECT_IRQ=y
+CONFIG_SERIAL_8250_RSA=y
+CONFIG_HW_RANDOM=y
+# CONFIG_HW_RANDOM_INTEL is not set
+# CONFIG_HW_RANDOM_AMD is not set
+CONFIG_NVRAM=y
+CONFIG_HPET=y
+# CONFIG_HPET_MMAP is not set
+CONFIG_I2C_I801=y
+CONFIG_WATCHDOG=y
+CONFIG_AGP=y
+CONFIG_AGP_AMD64=y
+CONFIG_AGP_INTEL=y
+CONFIG_DRM=y
+CONFIG_DRM_I915=y
+CONFIG_FB_MODE_HELPERS=y
+CONFIG_FB_TILEBLITTING=y
+CONFIG_FB_EFI=y
+# CONFIG_LCD_CLASS_DEVICE is not set
+CONFIG_VGACON_SOFT_SCROLLBACK=y
+CONFIG_LOGO=y
+# CONFIG_LOGO_LINUX_MONO is not set
+# CONFIG_LOGO_LINUX_VGA16 is not set
+CONFIG_SOUND=y
+CONFIG_SND=y
+CONFIG_SND_SEQUENCER=y
+CONFIG_SND_SEQ_DUMMY=y
+CONFIG_SND_MIXER_OSS=y
+CONFIG_SND_PCM_OSS=y
+CONFIG_SND_SEQUENCER_OSS=y
+CONFIG_SND_HRTIMER=y
+CONFIG_SND_HDA_INTEL=y
+CONFIG_SND_HDA_HWDEP=y
+CONFIG_HIDRAW=y
+CONFIG_HID_GYRATION=y
+CONFIG_LOGITECH_FF=y
+CONFIG_HID_NTRIG=y
+CONFIG_HID_PANTHERLORD=y
+CONFIG_PANTHERLORD_FF=y
+CONFIG_HID_PETALYNX=y
+CONFIG_HID_SAMSUNG=y
+CONFIG_HID_SONY=y
+CONFIG_HID_SUNPLUS=y
+CONFIG_HID_TOPSEED=y
+CONFIG_HID_PID=y
+CONFIG_USB_HIDDEV=y
+CONFIG_USB=y
+CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
+CONFIG_USB_MON=y
+CONFIG_USB_EHCI_HCD=y
+CONFIG_USB_EHCI_TT_NEWSCHED=y
+CONFIG_USB_OHCI_HCD=y
+CONFIG_USB_UHCI_HCD=y
+CONFIG_USB_PRINTER=y
+CONFIG_USB_STORAGE=y
+CONFIG_EDAC=y
+CONFIG_RTC_CLASS=y
+# CONFIG_RTC_HCTOSYS is not set
+CONFIG_DMADEVICES=y
+CONFIG_EEEPC_LAPTOP=y
+CONFIG_AMD_IOMMU=y
+CONFIG_INTEL_IOMMU=y
+# CONFIG_INTEL_IOMMU_DEFAULT_ON is not set
+CONFIG_EFI_VARS=y
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+CONFIG_QUOTA=y
+CONFIG_QUOTA_NETLINK_INTERFACE=y
+# CONFIG_PRINT_QUOTA_WARNING is not set
+CONFIG_QFMT_V2=y
+CONFIG_AUTOFS4_FS=y
+CONFIG_ISO9660_FS=y
+CONFIG_JOLIET=y
+CONFIG_ZISOFS=y
+CONFIG_MSDOS_FS=y
+CONFIG_VFAT_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS_POSIX_ACL=y
+CONFIG_HUGETLBFS=y
+CONFIG_NFS_FS=y
+CONFIG_NFS_V3_ACL=y
+CONFIG_NFS_V4=y
+CONFIG_ROOT_NFS=y
+CONFIG_NLS_DEFAULT="utf8"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ASCII=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_UTF8=y
+CONFIG_PRINTK_TIME=y
+# CONFIG_ENABLE_WARN_DEPRECATED is not set
+CONFIG_MAGIC_SYSRQ=y
+# CONFIG_UNUSED_SYMBOLS is not set
+CONFIG_DEBUG_KERNEL=y
+# CONFIG_SCHED_DEBUG is not set
+CONFIG_SCHEDSTATS=y
+CONFIG_TIMER_STATS=y
+CONFIG_DEBUG_STACK_USAGE=y
+CONFIG_BLK_DEV_IO_TRACE=y
+CONFIG_PROVIDE_OHCI1394_DMA_INIT=y
+CONFIG_EARLY_PRINTK_DBGP=y
+CONFIG_DEBUG_STACKOVERFLOW=y
+# CONFIG_DEBUG_RODATA_TEST is not set
+CONFIG_DEBUG_BOOT_PARAMS=y
+CONFIG_OPTIMIZE_INLINING=y
+CONFIG_UNWINDER_ORC=y
+CONFIG_SECURITY=y
+CONFIG_SECURITY_NETWORK=y
+CONFIG_SECURITY_SELINUX=y
+CONFIG_SECURITY_SELINUX_BOOTPARAM=y
+CONFIG_SECURITY_SELINUX_DISABLE=y
+# CONFIG_CRYPTO_ANSI_CPRNG is not set
diff --git a/meta-isar/recipes-kernel/linux/linux-cip_4.4.bb b/meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
new file mode 100644
index 0000000..d4fdda4
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
@@ -0,0 +1,10 @@
+require recipes-kernel/linux/linux-custom.inc
+
+SRC_URI += " \
+ git://git.kernel.org/pub/scm/linux/kernel/git/bwh/linux-cip.git;branch=linux-4.4.y-cip \
+ file://defconfig"
+
+SRCREV = "4e52cc5f668c4666e31a8485725b5f4e897b3baf"
+PV = "4.4.112-cip18"
+
+S = "git"
diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.14.17.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.17.bb
new file mode 100644
index 0000000..6c0b0ab
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.17.bb
@@ -0,0 +1,8 @@
+require recipes-kernel/linux/linux-custom.inc
+
+SRC_URI += " \
+ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${PV}.tar.xz \
+ file://defconfig"
+SRC_URI[sha256sum] = "27c69487f17b3c4e6576dbc9fc89a94525c4825c75ffbd4006fb536b97410a4b"
+
+S = "linux-${PV}"
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:43 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

With this include file, it becomes almost trivial to replace the distro
kernel with a custom build. You just need to include linux-custom.inc,
specify the source URI, define via S where the source is unpacked to and
provide a defconfig. To switch to a custom kernel recipe,
PREFERRED_PROVIDER_virtual/kernel has to be adjusted in local.conf or
the distro conf.

The approach works internally by first running "make deb-pkg" on the
kernel and the repackages the output to make the binary linux-image and
linux-header debs act as replacement of their distro packages. This
results in a suboptimal technical implementation which may eventually be
replaced by an isar-implemented deb-pkg build process. However, this is
not expected to affect the user-visible interface of this include file.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/recipes-kernel/linux/linux-custom.inc | 91 ++++++++++++++++++++++++++++++
1 file changed, 91 insertions(+)
create mode 100644 meta/recipes-kernel/linux/linux-custom.inc

diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
new file mode 100644
index 0000000..d487515
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -0,0 +1,91 @@
+DESCRIPTION ?= "Custom kernel"
+PROVIDES = "virtual/kernel"
+
+inherit dpkg-base
+
+KERNEL_DEBIAN_DEPENDS = "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)"
+KERNEL_HEADERS_DEBIAN_DEPENDS = "libc6, libssl1.1, gcc"
+KBUILD_DEPENDS = "libssl-dev libelf-dev bc"
+
+REPACK_DIR = "${PP}/repack"
+REPACK_LINUX_IMAGE_DIR = "${REPACK_DIR}/linux-image"
+REPACK_LINUX_HEADERS_DIR = "${REPACK_DIR}/linux-headers"
+
+dpkg_runbuild() {
+ E="${@ bb.utils.export_proxies(d)}"
+ sudo -E chroot ${BUILDCHROOT_DIR} sh -c '
+ set -e
+
+ apt-get install -y -o Debug::pkgProblemResolver=yes \
+ --no-install-recommends ${KBUILD_DEPENDS}
+
+ cd ${PP}/${S}
+ cp ../defconfig .config
+ make olddefconfig
+
+ make -j ${@bb.utils.cpu_count() * 2} deb-pkg
+
+ rm -rf ${REPACK_DIR}
+ mkdir -p ${REPACK_DIR}
+ mkdir -p ${REPACK_LINUX_IMAGE_DIR}
+ mkdir -p ${REPACK_LINUX_HEADERS_DIR}
+
+ cd ${PP}
+ tar xzf linux-${PV}_${PV}-1.debian.tar.gz -C ${REPACK_DIR}
+ dpkg-deb -R linux-image-${PV}_${PV}-1_${KERNEL_ARCH}.deb \
+ ${REPACK_LINUX_IMAGE_DIR}
+ dpkg-deb -R linux-headers-${PV}_${PV}-1_${KERNEL_ARCH}.deb \
+ ${REPACK_LINUX_HEADERS_DIR}
+
+ dpkg-gencontrol -crepack/debian/control \
+ -lrepack/debian/changelog \
+ -frepack/debian/files \
+ -plinux-image-${PV} \
+ -P${REPACK_LINUX_IMAGE_DIR} \
+ -DPackage="linux-image-${KERNEL_ARCH}" \
+ -DSection=kernel \
+ -DPriority=required \
+ -DProvides="${PN}" \
+ -DDepends="${KERNEL_DEBIAN_DEPENDS}"
+
+ # Add Debian-like link installation to postinst
+ touch ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install
+ sed -i ${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst \
+ -e "/^set -e$/a\\
+\\
+if [ -f /lib/modules/${PV}/.fresh-install ]; then\\
+ change=install\\
+else\\
+ change=upgrade\\
+fi\\
+linux-update-symlinks \$change ${PV} /boot/vmlinuz-${PV}\\
+rm -f /lib/modules/${PV}/.fresh-install"
+
+ # Add Debian-like link removal to postrm
+ sed -i ${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm \
+ -e "/^set -e$/a\\
+\\
+rm -f /lib/modules/${PV}/.fresh-install\\
+\\
+if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\
+ linux-update-symlinks remove ${PV} /boot/vmlinuz-${PV}\\
+fi"
+
+ dpkg-gencontrol -crepack/debian/control \
+ -lrepack/debian/changelog \
+ -frepack/debian/files \
+ -plinux-headers-${PV} \
+ -P${REPACK_LINUX_HEADERS_DIR} \
+ -Vkernel:debarch="${KERNEL_ARCH}" \
+ -DPackage="linux-headers-${KERNEL_ARCH}" \
+ -DSection=kernel \
+ -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}"
+
+ dpkg-deb -b ${REPACK_LINUX_IMAGE_DIR} \
+ linux-image-${KERNEL_ARCH}_${PV}-1_${KERNEL_ARCH}.deb
+ rm -f linux-image-${PV}_${PV}-1_${KERNEL_ARCH}.deb
+ dpkg-deb -b ${REPACK_LINUX_HEADERS_DIR} \
+ linux-headers-${KERNEL_ARCH}_${PV}-1_${KERNEL_ARCH}.deb
+ rm -f linux-headers-${PV}_${PV}-1_${KERNEL_ARCH}.deb
+ '
+}
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:44 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This ensures that we can override packages from upstream Debian or other
external sources with our self-built versions. We achieve this for now
by asking multistrap to drop a preferences file for the buildchroot so
that dependency installations use the right priority. For the image
build, this does not work because all packages are pull during the
bootstrap. Therefore, we set aptdefaultrelease to isar to ensure that
our repo gets the higher priority.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/recipes-core/images/files/multistrap.conf.in | 1 +
meta-isar/recipes-core/images/isar-image-base.bb | 1 +
meta/recipes-devtools/buildchroot/buildchroot.bb | 4 +++-
meta/recipes-devtools/buildchroot/files/isar-apt-prefs | 3 +++
meta/recipes-devtools/buildchroot/files/multistrap.conf.in | 1 +
5 files changed, 9 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-devtools/buildchroot/files/isar-apt-prefs

diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in
index 432b6af..2ab7eab 100644
--- a/meta-isar/recipes-core/images/files/multistrap.conf.in
+++ b/meta-isar/recipes-core/images/files/multistrap.conf.in
@@ -10,6 +10,7 @@ aptsources=##DISTRO_MULTICONF_APTSOURCES##
configscript=##CONFIG_SCRIPT##
setupscript=##SETUP_SCRIPT##
hookdir=##DIR_HOOKS##
+aptdefaultrelease=isar

[base]
source=##DISTRO_APT_SOURCE##
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 8ddbabb..e54b9a6 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -53,6 +53,7 @@ do_rootfs() {
-e 's|##IMAGE_INSTALL##|${IMAGE_INSTALL}|g' \
-e 's|##DEPLOY_DIR_APT##|copy:///${DEPLOY_DIR_APT}/${DISTRO}|g' \
-e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \
+ -e 's|##APT_PREFS##|./'"$WORKDIR_REL"'/isar-apt-prefs|g' \
"${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"

# Do not use bitbake flag [dirs] here because this folder should have
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index df9df19..bf80114 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -12,7 +12,8 @@ FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:"
SRC_URI = "file://multistrap.conf.in \
file://configscript.sh \
file://setup.sh \
- file://build.sh"
+ file://build.sh \
+ file://isar-apt-prefs"
PV = "1.0"

BUILDCHROOT_PREINSTALL ?= "gcc \
@@ -58,6 +59,7 @@ do_build() {
-e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
-e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
-e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
+ -e 's|##APT_PREFS##|./'"$WORKDIR_REL"'/isar-apt-prefs|g' \
"${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"

do_setup_mounts
diff --git a/meta/recipes-devtools/buildchroot/files/isar-apt-prefs b/meta/recipes-devtools/buildchroot/files/isar-apt-prefs
new file mode 100644
index 0000000..2db40e5
--- /dev/null
+++ b/meta/recipes-devtools/buildchroot/files/isar-apt-prefs
@@ -0,0 +1,3 @@
+Package: *
+Pin: release n=isar,c=main
+Pin-Priority: 1001
diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
index 480a4b8..89c4968 100644
--- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
+++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
@@ -10,6 +10,7 @@ aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES##
configscript=##CONFIG_SCRIPT##
setupscript=##SETUP_SCRIPT##
hookdir=##DIR_HOOKS##
+aptpreferences=##APT_PREFS##

[base]
source=##DISTRO_APT_SOURCE##
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 10:05:44 AM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This demonstrates the usage of the new module.inc via a primitive
hello world module.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/local.conf.sample | 2 +-
.../recipes-kernel/example-module/example-module.bb | 5 +++++
.../recipes-kernel/example-module/files/src/Makefile | 11 +++++++++++
.../example-module/files/src/example-module.c | 19 +++++++++++++++++++
4 files changed, 36 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 2ae43e7..92dea46 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -153,7 +153,7 @@ CONF_VERSION = "1"

#
# The default list of extra packages to be installed.
-IMAGE_INSTALL = "example-hello example-raw"
+IMAGE_INSTALL = "example-hello example-raw example-module"

#
# Default parallel jobs for bitbake:
diff --git a/meta-isar/recipes-kernel/example-module/example-module.bb b/meta-isar/recipes-kernel/example-module/example-module.bb
new file mode 100644
index 0000000..86dbf18
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/example-module.bb
@@ -0,0 +1,5 @@
+include recipes-kernel/linux-module/module.inc
+
+SRC_URI += "file://src"
+
+S = "src"
diff --git a/meta-isar/recipes-kernel/example-module/files/src/Makefile b/meta-isar/recipes-kernel/example-module/files/src/Makefile
new file mode 100644
index 0000000..59a9703
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/Makefile
@@ -0,0 +1,11 @@
+obj-m := example-module.o
+
+INSTALL_MOD_PATH ?= $(DESTDIR)
+export INSTALL_MOD_PATH
+
+modules modules_install clean:
+ $(MAKE) -C $(KDIR) M=$(PWD) $@
+
+install: modules_install
+
+.PHONY: modules modules_install install clean
diff --git a/meta-isar/recipes-kernel/example-module/files/src/example-module.c b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
new file mode 100644
index 0000000..2099e16
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
@@ -0,0 +1,19 @@
+/*
+ * Example modules
+ *
+ * Copyright (c) Siemens AG, 2018
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
+ */
+
+#include <linux/module.h>
+
+static int __init example_module_init(void)
+{
+ printk("Just an example\n");
+ return -ENOANO;
+}
+
+module_init(example_module_init);
+
+MODULE_LICENSE("GPL");
--
2.13.6

Jan Kiszka

unread,
Feb 8, 2018, 12:23:00 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Required when the package needs to install build dependencies.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/classes/dpkg.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index c10f5ba..9cd4403 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -5,5 +5,6 @@ inherit dpkg-base

# Build package from sources using build script
dpkg_runbuild() {
- sudo chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
+ E="${@ bb.utils.export_proxies(d)}"

Jan Kiszka

unread,
Feb 8, 2018, 12:23:00 PM2/8/18
to isar-users
This is now finally the promised patches to provide simple patterns for
building custom kernels for Isar images that replace the distro kernels.
It also provids the same pattern for custom out-of-tree kernel modules.

Along come two fixes, one to respect proxies during dpkg builds, the
other to adjust isar-apt repo prio.

Furthermore, I'm proposing to rename "SRC_DIR" to OE-standard "S".

This series is also available at

https://github.com/siemens/isar jan/kernel-build

Changes in v2:
- added copyright headers and license statements
- "update" to kernel 4.14.18

Jan

Jan Kiszka (8):
Forward proxy settings to dpkg build
Prioritize isar-apt repo over all others
Replace SRC_DIR with S
Install kernel via replaceable recipe
Provide include file for easy custom kernel builds
Add custom kernel examples
Provide include file for easy custom module builds
Add exemplary kernel module

doc/user_manual.md | 6 +-
meta-isar/conf/distro/debian-jessie.conf | 2 +
meta-isar/conf/distro/debian-stretch.conf | 2 +
meta-isar/conf/distro/debian-wheezy.conf | 2 +
meta-isar/conf/distro/raspbian-jessie.conf | 2 +
meta-isar/conf/local.conf.sample | 2 +-
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 3 +-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 3 +-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 3 +-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 +-
.../recipes-app/example-hello/example-hello.bb | 2 +-
meta-isar/recipes-app/libhello/libhello.bb | 2 +-
.../recipes-core/images/files/multistrap.conf.in | 1 +
meta-isar/recipes-core/images/isar-image-base.bb | 6 +-
.../example-module/example-module.bb | 12 +
.../example-module/files/src/Makefile | 18 ++
.../example-module/files/src/example-module.c | 19 ++
meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 17 ++
.../recipes-kernel/linux/linux-mainline_4.14.18.bb | 15 +
meta/classes/dpkg.bbclass | 3 +-
meta/recipes-devtools/buildchroot/buildchroot.bb | 4 +-
.../buildchroot/files/isar-apt-prefs | 3 +
.../buildchroot/files/multistrap.conf.in | 1 +
.../linux-module/files/debian/changelog | 5 +
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 +
.../recipes-kernel/linux-module/files/debian/rules | 15 +
meta/recipes-kernel/linux-module/module.inc | 24 ++
meta/recipes-kernel/linux/linux-custom.inc | 98 +++++++
meta/recipes-kernel/linux/linux-distro.bb | 16 ++
35 files changed, 600 insertions(+), 17 deletions(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c
create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
create mode 100644 meta/recipes-devtools/buildchroot/files/isar-apt-prefs
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Provide two examples for describing custom kernel builds. Both are for
x86_64, simply using upstream x86_64_defconfig to define the required
defconfig. One example is demoing a build from a tarball, the other
pulling from git.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 17 ++
.../recipes-kernel/linux/linux-mainline_4.14.18.bb | 15 +
3 files changed, 340 insertions(+)
create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb

diff --git a/meta-isar/recipes-kernel/linux/files/defconfig b/meta-isar/recipes-kernel/linux/files/defconfig
new file mode 100644
new file mode 100644
index 0000000..0af4444
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
@@ -0,0 +1,17 @@
+# Example recipe for building the CIP 4.4 kernel
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+require recipes-kernel/linux/linux-custom.inc
+
+SRC_URI += " \
+ git://git.kernel.org/pub/scm/linux/kernel/git/bwh/linux-cip.git;branch=linux-4.4.y-cip \
+ file://defconfig"
+
+SRCREV = "4e52cc5f668c4666e31a8485725b5f4e897b3baf"
+PV = "4.4.112-cip18"
+
+S = "git"
diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
new file mode 100644
index 0000000..dc4b4ce
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
@@ -0,0 +1,15 @@
+# Example recipe for building the mainline kernel
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+require recipes-kernel/linux/linux-custom.inc
+
+SRC_URI += " \
+ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${PV}.tar.xz \
+ file://defconfig"
+SRC_URI[sha256sum] = "866a94c1c38d923ae18e74b683d7a8a79b674ebdfe7f40f1a3be9a27d39fe354"

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

With this include file, it becomes almost trivial to replace the distro
kernel with a custom build. You just need to include linux-custom.inc,
specify the source URI, define via S where the source is unpacked to and
provide a defconfig. To switch to a custom kernel recipe,
PREFERRED_PROVIDER_virtual/kernel has to be adjusted in local.conf or
the distro conf.

The approach works internally by first running "make deb-pkg" on the
kernel and the repackages the output to make the binary linux-image and
linux-header debs act as replacement of their distro packages. This
results in a suboptimal technical implementation which may eventually be
replaced by an isar-implemented deb-pkg build process. However, this is
not expected to affect the user-visible interface of this include file.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/recipes-kernel/linux/linux-custom.inc | 98 ++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
create mode 100644 meta/recipes-kernel/linux/linux-custom.inc

diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
new file mode 100644
index 0000000..feb96a1
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -0,0 +1,98 @@
+# Custom kernel recipe include
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+DESCRIPTION ?= "Custom kernel"
+PROVIDES = "virtual/kernel"
+
+inherit dpkg-base
+
+KERNEL_DEBIAN_DEPENDS = "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)"
+KERNEL_HEADERS_DEBIAN_DEPENDS = "libc6, libssl1.1, gcc"
+KBUILD_DEPENDS = "libssl-dev libelf-dev bc"
+
+REPACK_DIR = "${PP}/repack"
+REPACK_LINUX_IMAGE_DIR = "${REPACK_DIR}/linux-image"
+REPACK_LINUX_HEADERS_DIR = "${REPACK_DIR}/linux-headers"
+
+dpkg_runbuild() {
+ E="${@ bb.utils.export_proxies(d)}"

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This demonstrates the usage of the new module.inc via a primitive
hello world module.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/local.conf.sample | 2 +-
.../recipes-kernel/example-module/example-module.bb | 12 ++++++++++++
.../recipes-kernel/example-module/files/src/Makefile | 18 ++++++++++++++++++
.../example-module/files/src/example-module.c | 19 +++++++++++++++++++
4 files changed, 50 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 2ae43e7..92dea46 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -153,7 +153,7 @@ CONF_VERSION = "1"

#
# The default list of extra packages to be installed.
-IMAGE_INSTALL = "example-hello example-raw"
+IMAGE_INSTALL = "example-hello example-raw example-module"

#
# Default parallel jobs for bitbake:
diff --git a/meta-isar/recipes-kernel/example-module/example-module.bb b/meta-isar/recipes-kernel/example-module/example-module.bb
new file mode 100644
index 0000000..c7042cf
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/example-module.bb
@@ -0,0 +1,12 @@
+# Example recipe for building a custom module
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+include recipes-kernel/linux-module/module.inc
+
+SRC_URI += "file://src"
+
+S = "src"
diff --git a/meta-isar/recipes-kernel/example-module/files/src/Makefile b/meta-isar/recipes-kernel/example-module/files/src/Makefile
new file mode 100644
index 0000000..1625b28
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/Makefile
@@ -0,0 +1,18 @@
+# Example module
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: GPL-2.0
+
+obj-m := example-module.o
+
+INSTALL_MOD_PATH ?= $(DESTDIR)
+export INSTALL_MOD_PATH
+
+modules modules_install clean:
+ $(MAKE) -C $(KDIR) M=$(PWD) $@
+
+install: modules_install
+
+.PHONY: modules modules_install install clean
diff --git a/meta-isar/recipes-kernel/example-module/files/src/example-module.c b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
new file mode 100644
index 0000000..07cb214
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
@@ -0,0 +1,19 @@
+/*
+ * Example modules
+ *
+ * Copyright (c) Siemens AG, 2018
+ *
+ * SPDX-License-Identifier: GPL-2.0

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This simplifies the common task of using a custom kernel instead of the
pre-selected disto variant: Move the kernel installation into a dummy
dpkg-raw recipe that only has the kernel package as dependency.

Which recipe is used for providing the kernel can now be selected via
the well-know PREFERRED_PROVIDER_virtual/kernel, just like in OE.

The kernel package name is derived from the KERNEL_ARCH variable which
is communicated from the target multiconfig file to the linux-distro
recipe.

Based on an idea of Henning Schild.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/distro/debian-jessie.conf | 2 ++
meta-isar/conf/distro/debian-stretch.conf | 2 ++
meta-isar/conf/distro/debian-wheezy.conf | 2 ++
meta-isar/conf/distro/raspbian-jessie.conf | 2 ++
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 ++-
meta-isar/recipes-core/images/isar-image-base.bb | 5 ++++-
meta/recipes-kernel/linux/linux-distro.bb | 16 ++++++++++++++++
14 files changed, 40 insertions(+), 9 deletions(-)
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index e54b9a6..5340767 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -18,12 +18,15 @@ PV = "1.0"

inherit image

-DEPENDS += "${IMAGE_INSTALL}"
+DEPENDS += "${IMAGE_INSTALL} \
+ virtual/kernel"

IMAGE_PREINSTALL += "apt \
dbus \
localepurge"

+IMAGE_INSTALL += "${@d.getVar("PREFERRED_PROVIDER_virtual/kernel", True)}"
+
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"

do_rootfs[dirs] = "${WORKDIR}/hooks_multistrap"
diff --git a/meta/recipes-kernel/linux/linux-distro.bb b/meta/recipes-kernel/linux/linux-distro.bb
new file mode 100644
index 0000000..1af93ed
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-distro.bb
@@ -0,0 +1,16 @@
+# Debian kernel pseudo package
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This aligns us with OE/bitbake style.

Along updating the manual, also fix SRC_REV -> SRCREV.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
doc/user_manual.md | 6 +++---
meta-isar/recipes-app/example-hello/example-hello.bb | 2 +-
meta-isar/recipes-app/libhello/libhello.bb | 2 +-
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 9cd4403..06f0579 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -6,5 +6,5 @@ inherit dpkg-base
# Build package from sources using build script
dpkg_runbuild() {

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Analogously to linux-custom.inc, this provides an include for custom
kernel modules. The user just needs to specify the module SRC_URI, and
those sources must be then reside in a separate directory which is - as
usual - defined via S.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
.../linux-module/files/debian/changelog | 5 +++++
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 ++++++++++
.../recipes-kernel/linux-module/files/debian/rules | 15 ++++++++++++++
meta/recipes-kernel/linux-module/module.inc | 24 ++++++++++++++++++++++
5 files changed, 56 insertions(+)
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog
new file mode 100644
index 0000000..c1c3516
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog
@@ -0,0 +1,5 @@
+@PN@ (@PV@) unstable; urgency=low
+
+ * Generated package.
+
+ -- ISAR project <isar-...@googlegroups.com> Tue, 6 Feb 2018 00:00:00 +0000
diff --git a/meta/recipes-kernel/linux-module/files/debian/compat b/meta/recipes-kernel/linux-module/files/debian/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
new file mode 100644
index 0000000..d8fbcaa
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control
@@ -0,0 +1,11 @@
+Source: @PN@
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-@KERNEL_ARCH@
+Maintainer: ISAR project <isar-...@googlegroups.com>
+
+Package: @PN@
+Architecture: any
+Depends: linux-image-@KERNEL_ARCH@
+Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules
new file mode 100755
index 0000000..4723cd8
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/rules
@@ -0,0 +1,15 @@
+#!/usr/bin/make -f
+
+# Debian rules for custom kernel module build
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+export KDIR=$(shell ls -d /lib/modules/*/build)
+
+export DEB_BUILD_OPTIONS=parallel=$(shell nproc)
+
+%:
+ dh $@ --parallel
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
new file mode 100644
index 0000000..ecac75f
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -0,0 +1,24 @@
+# Custom kernel module recipe include
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+FILESPATH =. "${LAYERDIR_core}/recipes-kernel/linux-module/files:"
+

Jan Kiszka

unread,
Feb 8, 2018, 12:23:01 PM2/8/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This ensures that we can override packages from upstream Debian or other
external sources with our self-built versions. We achieve this for now
by asking multistrap to drop a preferences file for the buildchroot so
that dependency installations use the right priority. For the image
build, this does not work because all packages are pull during the
bootstrap. Therefore, we set aptdefaultrelease to isar to ensure that
our repo gets the higher priority.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/recipes-core/images/files/multistrap.conf.in | 1 +
meta-isar/recipes-core/images/isar-image-base.bb | 1 +
meta/recipes-devtools/buildchroot/buildchroot.bb | 4 +++-
meta/recipes-devtools/buildchroot/files/isar-apt-prefs | 3 +++
meta/recipes-devtools/buildchroot/files/multistrap.conf.in | 1 +
5 files changed, 9 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-devtools/buildchroot/files/isar-apt-prefs

diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in
index 432b6af..2ab7eab 100644
--- a/meta-isar/recipes-core/images/files/multistrap.conf.in
+++ b/meta-isar/recipes-core/images/files/multistrap.conf.in
@@ -10,6 +10,7 @@ aptsources=##DISTRO_MULTICONF_APTSOURCES##
configscript=##CONFIG_SCRIPT##
setupscript=##SETUP_SCRIPT##
hookdir=##DIR_HOOKS##
+aptdefaultrelease=isar

[base]
source=##DISTRO_APT_SOURCE##
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 8ddbabb..e54b9a6 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
new file mode 100644

Henning Schild

unread,
Feb 9, 2018, 11:46:37 AM2/9/18
to [ext] Jan Kiszka, isar-users
Am Thu, 8 Feb 2018 18:22:53 +0100
schrieb "[ext] Jan Kiszka" <jan.k...@siemens.com>:
This is a hack for the bootstrap environment, we probably need the
prevs here as well. I guess an apt-get update/upgrade in the image
would get me the debian kernel again.

Henning

Jan Kiszka

unread,
Feb 9, 2018, 11:52:22 AM2/9/18
to Henning Schild, isar-users
It is not a bullet-proof approach until we fully control apt prios and
pinnings on the target for all of out packages.

We either need packet pinning rules on the target for all packages that
come from the then unavailable isar-apt repo. Or the user makes that
repo available via a server so that we can prioritize it as well.

But none of that exists yet, and until then this should be good enough
to at least generate consistent images.

Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Henning Schild

unread,
Feb 9, 2018, 11:59:58 AM2/9/18
to [ext] Jan Kiszka, isar-users
This is where the (nasty) magic happens ;). I would suggest pulling the
shell script out like dpkg.bbclass is calling its build.sh, that should
make it more readable.

On which kernel versions have you actually tested this? I just built
the jailhouse example and saw it was 4.14. Will that work on 4.4 or
even before?

Henning

Am Thu, 8 Feb 2018 18:22:56 +0100
schrieb "[ext] Jan Kiszka" <jan.k...@siemens.com>:

Henning Schild

unread,
Feb 9, 2018, 12:01:11 PM2/9/18
to [ext] Jan Kiszka, isar-users
Am Thu, 8 Feb 2018 18:22:57 +0100
schrieb "[ext] Jan Kiszka" <jan.k...@siemens.com>:

> From: Jan Kiszka <jan.k...@siemens.com>
>
> Provide two examples for describing custom kernel builds. Both are for
> x86_64, simply using upstream x86_64_defconfig to define the required
> defconfig. One example is demoing a build from a tarball, the other
> pulling from git.
>
> Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
> ---
> meta-isar/recipes-kernel/linux/files/defconfig | 308
> +++++++++++++++++++++
> meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 17
> ++ .../recipes-kernel/linux/linux-mainline_4.14.18.bb | 15 + 3 files
> changed, 340 insertions(+) create mode 100644
> meta-isar/recipes-kernel/linux/files/defconfig create mode 100644
> meta-isar/recipes-kernel/linux/linux-cip_4.4.bb create mode 100644

I think my question has just been answered ;).

Henning

Henning Schild

unread,
Feb 9, 2018, 12:15:33 PM2/9/18
to Jan Kiszka, isar-users
Am Fri, 9 Feb 2018 17:52:20 +0100
schrieb Jan Kiszka <jan.k...@siemens.com>:
Right, i forgot about the absence of isar-apt when the image has been
deployed.

Henning

Jan Kiszka

unread,
Feb 9, 2018, 12:24:20 PM2/9/18
to Henning Schild, isar-users
On 2018-02-09 17:59, Henning Schild wrote:
> This is where the (nasty) magic happens ;). I would suggest pulling the
> shell script out like dpkg.bbclass is calling its build.sh, that should
> make it more readable.

Hmm, not sure. I would have to export lots of variables for that....

>
> On which kernel versions have you actually tested this? I just built
> the jailhouse example and saw it was 4.14. Will that work on 4.4 or
> even before?

See patch 6: tested down to 4.4 so far.

Jan

Henning Schild

unread,
Feb 9, 2018, 12:30:41 PM2/9/18
to [ext] Jan Kiszka, isar-users
Am Thu, 8 Feb 2018 18:22:56 +0100
schrieb "[ext] Jan Kiszka" <jan.k...@siemens.com>:

These guys and the repacking are here because someone wrote a
build-script that does not allow customization, how about not repeating
that mistake and using "?=".

Henning

Henning Schild

unread,
Feb 9, 2018, 12:31:17 PM2/9/18
to Jan Kiszka, isar-users
Am Fri, 9 Feb 2018 18:24:18 +0100
schrieb Jan Kiszka <jan.k...@siemens.com>:

> On 2018-02-09 17:59, Henning Schild wrote:
> > This is where the (nasty) magic happens ;). I would suggest pulling
> > the shell script out like dpkg.bbclass is calling its build.sh,
> > that should make it more readable.
>
> Hmm, not sure. I would have to export lots of variables for that....

Yes, i see.

Henning

Jan Kiszka

unread,
Feb 9, 2018, 12:54:15 PM2/9/18
to Henning Schild, isar-users
As this is an include, you can already easily adjust after including.
Granted, defining those as default assignment would permit project-wide
overrides in local.conf & Co. OK, will send v3.

Jan

Jan Kiszka

unread,
Feb 9, 2018, 12:57:23 PM2/9/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

With this include file, it becomes almost trivial to replace the distro
kernel with a custom build. You just need to include linux-custom.inc,
specify the source URI, define via S where the source is unpacked to and
provide a defconfig. To switch to a custom kernel recipe,
PREFERRED_PROVIDER_virtual/kernel has to be adjusted in local.conf or
the distro conf.

The approach works internally by first running "make deb-pkg" on the
kernel and the repackages the output to make the binary linux-image and
linux-header debs act as replacement of their distro packages. This
results in a suboptimal technical implementation which may eventually be
replaced by an isar-implemented deb-pkg build process. However, this is
not expected to affect the user-visible interface of this include file.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

Changes in v3:
- allow central overriding of kernel dependencies

meta/recipes-kernel/linux/linux-custom.inc | 98 ++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
create mode 100644 meta/recipes-kernel/linux/linux-custom.inc

diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
new file mode 100644
index 0000000..201f1c2
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -0,0 +1,98 @@
+# Custom kernel recipe include
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+DESCRIPTION ?= "Custom kernel"
+PROVIDES = "virtual/kernel"
+
+inherit dpkg-base
+
+KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)"
+KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1, gcc"
+KBUILD_DEPENDS ?= "libssl-dev libelf-dev bc"

Henning Schild

unread,
Feb 9, 2018, 2:04:19 PM2/9/18
to Jan Kiszka, isar-users
Am Fri, 9 Feb 2018 18:54:12 +0100
schrieb Jan Kiszka <jan.k...@siemens.com>:
Maybe i missed that so far, but could you explain why this is an
include and not a bbclass. Not that i prefer one over the other but so
far we do not seem to use .incs in Isar.

Henning

Jan Kiszka

unread,
Feb 11, 2018, 10:16:23 AM2/11/18
to Henning Schild, isar-users
I have no preference either. When I started this, I looked at
linux-yocto, and that was an include at the top level. If you dig
deeper, there are also kernel classes in OE.

I still have a hard time finding differences between both extension
methods. The only difference I see right now is that "inherit
linux-kernel" would be shorter than "require
recipes-kernel/linux/linux-custom.inc"...

Anyway, I've remodeled this as class and will send out v3 soon. The
module should stay an include as it comes with additional files.

Jan

Jan Kiszka

unread,
Feb 11, 2018, 10:25:21 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Required when the package needs to install build dependencies.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/classes/dpkg.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index c10f5ba..9cd4403 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -5,5 +5,6 @@ inherit dpkg-base

# Build package from sources using build script
dpkg_runbuild() {
- sudo chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
+ E="${@ bb.utils.export_proxies(d)}"
+ sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
}
--
2.13.6

Jan Kiszka

unread,
Feb 11, 2018, 10:25:21 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This ensures that we can override packages from upstream Debian or other
external sources with our self-built versions. We achieve this for now
by asking multistrap to drop a preferences file for the buildchroot so
that dependency installations use the right priority. For the image
build, this does not work because all packages are pull during the
bootstrap. Therefore, we set aptdefaultrelease to isar to ensure that
our repo gets the higher priority.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/recipes-core/images/files/multistrap.conf.in | 1 +
meta-isar/recipes-core/images/isar-image-base.bb | 1 +
meta/recipes-devtools/buildchroot/buildchroot.bb | 4 +++-
meta/recipes-devtools/buildchroot/files/isar-apt-prefs | 3 +++
meta/recipes-devtools/buildchroot/files/multistrap.conf.in | 1 +
5 files changed, 9 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-devtools/buildchroot/files/isar-apt-prefs

diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in
index 432b6af..2ab7eab 100644
--- a/meta-isar/recipes-core/images/files/multistrap.conf.in
+++ b/meta-isar/recipes-core/images/files/multistrap.conf.in
@@ -10,6 +10,7 @@ aptsources=##DISTRO_MULTICONF_APTSOURCES##
configscript=##CONFIG_SCRIPT##
setupscript=##SETUP_SCRIPT##
hookdir=##DIR_HOOKS##
+aptdefaultrelease=isar

[base]
source=##DISTRO_APT_SOURCE##
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 8ddbabb..e54b9a6 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -53,6 +53,7 @@ do_rootfs() {
-e 's|##IMAGE_INSTALL##|${IMAGE_INSTALL}|g' \
-e 's|##DEPLOY_DIR_APT##|copy:///${DEPLOY_DIR_APT}/${DISTRO}|g' \
-e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \
+ -e 's|##APT_PREFS##|./'"$WORKDIR_REL"'/isar-apt-prefs|g' \
"${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"

# Do not use bitbake flag [dirs] here because this folder should have
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index 570c0ad..eb6a744 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -12,7 +12,8 @@ FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:"
SRC_URI = "file://multistrap.conf.in \
file://configscript.sh \
file://setup.sh \
- file://build.sh"
+ file://build.sh \
+ file://isar-apt-prefs"
PV = "1.0"

BUILDCHROOT_PREINSTALL ?= "gcc \
@@ -58,6 +59,7 @@ do_build() {
-e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
-e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
-e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
+ -e 's|##APT_PREFS##|./'"$WORKDIR_REL"'/isar-apt-prefs|g' \
"${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"

do_setup_mounts
diff --git a/meta/recipes-devtools/buildchroot/files/isar-apt-prefs b/meta/recipes-devtools/buildchroot/files/isar-apt-prefs
new file mode 100644
index 0000000..2db40e5
--- /dev/null
+++ b/meta/recipes-devtools/buildchroot/files/isar-apt-prefs
@@ -0,0 +1,3 @@
+Package: *
+Pin: release n=isar,c=main
+Pin-Priority: 1001
diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
index 480a4b8..89c4968 100644
--- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
+++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
@@ -10,6 +10,7 @@ aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES##
configscript=##CONFIG_SCRIPT##
setupscript=##SETUP_SCRIPT##
hookdir=##DIR_HOOKS##
+aptpreferences=##APT_PREFS##

[base]
source=##DISTRO_APT_SOURCE##
--
2.13.6

Jan Kiszka

unread,
Feb 11, 2018, 10:25:21 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This aligns us with OE/bitbake style.

Along updating the manual, also fix SRC_REV -> SRCREV.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 9cd4403..06f0579 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -6,5 +6,5 @@ inherit dpkg-base
# Build package from sources using build script
dpkg_runbuild() {
E="${@ bb.utils.export_proxies(d)}"
- sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
+ sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${S}
}
--
2.13.6

Jan Kiszka

unread,
Feb 11, 2018, 10:25:22 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

With this class, it becomes almost trivial to replace the distro kernel
with a custom build. You just need to inherit linux-kernel, specify the
source URI, define via S where the source is unpacked to and provide a
defconfig. To switch to a custom kernel recipe,
PREFERRED_PROVIDER_virtual/kernel has to be adjusted in local.conf or
the distro conf.

The approach works internally by first running "make deb-pkg" on the
kernel and the repackages the output to make the binary linux-image and
linux-header debs act as replacement of their distro packages. This
results in a suboptimal technical implementation which may eventually be
replaced by an isar-implemented deb-pkg build process. However, this is
not expected to affect the user-visible interface of this class.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta/classes/linux-kernel.bbclass | 98 +++++++++++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
create mode 100644 meta/classes/linux-kernel.bbclass

diff --git a/meta/classes/linux-kernel.bbclass b/meta/classes/linux-kernel.bbclass
new file mode 100644
index 0000000..5f4df3f
--- /dev/null
+++ b/meta/classes/linux-kernel.bbclass
@@ -0,0 +1,98 @@
+# Custom kernel build
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+DESCRIPTION ?= "Custom kernel"
+PROVIDES = "virtual/kernel"
+
+inherit dpkg-base
+
+KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)"
+KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1, gcc"
+KBUILD_DEPENDS ?= "libssl-dev libelf-dev bc"
+
+REPACK_DIR = "${PP}/repack"
+REPACK_LINUX_IMAGE_DIR = "${REPACK_DIR}/linux-image"
+REPACK_LINUX_HEADERS_DIR = "${REPACK_DIR}/linux-headers"
+
+dpkg_runbuild() {
+ E="${@ bb.utils.export_proxies(d)}"
--
2.13.6

Jan Kiszka

unread,
Feb 11, 2018, 10:25:22 AM2/11/18
to isar-users
This is now finally the promised patches to provide simple patterns for
building custom kernels for Isar images that replace the distro kernels.
It also provids the same pattern for custom out-of-tree kernel modules.

Along come two fixes, one to respect proxies during dpkg builds, the
other to adjust isar-apt repo prio.

Furthermore, I'm proposing to rename "SRC_DIR" to OE-standard "S".

Changes in v3 (only resent patch 5):
- allow central overriding of kernel dependencies

Changes in v4
- simplified changes to isar-image-base.bb by reordering code
- modeled custom kernel build as class

Jan

Jan Kiszka (8):
Forward proxy settings to dpkg build
Prioritize isar-apt repo over all others
Replace SRC_DIR with S
Install kernel via replaceable recipe
Provide class for easy custom kernel builds
Add custom kernel examples
Provide include file for easy custom module builds
Add exemplary kernel module

doc/user_manual.md | 6 +-
meta-isar/conf/distro/debian-jessie.conf | 2 +
meta-isar/conf/distro/debian-stretch.conf | 2 +
meta-isar/conf/distro/debian-wheezy.conf | 2 +
meta-isar/conf/distro/raspbian-jessie.conf | 2 +
meta-isar/conf/local.conf.sample | 2 +-
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 3 +-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 3 +-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 3 +-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 +-
.../recipes-app/example-hello/example-hello.bb | 2 +-
meta-isar/recipes-app/libhello/libhello.bb | 2 +-
.../recipes-core/images/files/multistrap.conf.in | 1 +
meta-isar/recipes-core/images/isar-image-base.bb | 3 +
.../example-module/example-module.bb | 12 +
.../example-module/files/src/Makefile | 18 ++
.../example-module/files/src/example-module.c | 19 ++
meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 17 ++
.../recipes-kernel/linux/linux-mainline_4.14.18.bb | 15 +
meta/classes/dpkg.bbclass | 3 +-
meta/classes/linux-kernel.bbclass | 98 +++++++
meta/recipes-devtools/buildchroot/buildchroot.bb | 4 +-
.../buildchroot/files/isar-apt-prefs | 3 +
.../buildchroot/files/multistrap.conf.in | 1 +
.../linux-module/files/debian/changelog | 5 +
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 +
.../recipes-kernel/linux-module/files/debian/rules | 15 +
meta/recipes-kernel/linux-module/module.inc | 24 ++
meta/recipes-kernel/linux/linux-distro.bb | 16 ++
35 files changed, 598 insertions(+), 16 deletions(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c
create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
create mode 100644 meta/classes/linux-kernel.bbclass
create mode 100644 meta/recipes-devtools/buildchroot/files/isar-apt-prefs
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

Jan Kiszka

unread,
Feb 11, 2018, 10:25:22 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Provide two examples for describing custom kernel builds. Both are for
x86_64, simply using upstream x86_64_defconfig to define the required
defconfig. One example is demoing a build from a tarball, the other
pulling from git.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 17 ++
.../recipes-kernel/linux/linux-mainline_4.14.18.bb | 15 +
3 files changed, 340 insertions(+)
create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb

diff --git a/meta-isar/recipes-kernel/linux/files/defconfig b/meta-isar/recipes-kernel/linux/files/defconfig
new file mode 100644
new file mode 100644
index 0000000..dc04a02
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
@@ -0,0 +1,17 @@
+# Example recipe for building the CIP 4.4 kernel
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+inherit linux-kernel
+
+SRC_URI += " \
+ git://git.kernel.org/pub/scm/linux/kernel/git/bwh/linux-cip.git;branch=linux-4.4.y-cip \
+ file://defconfig"
+
+SRCREV = "4e52cc5f668c4666e31a8485725b5f4e897b3baf"
+PV = "4.4.112-cip18"
+
+S = "git"
diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
new file mode 100644
index 0000000..f6b820f
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
@@ -0,0 +1,15 @@
+# Example recipe for building the mainline kernel
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+inherit linux-kernel
+
+SRC_URI += " \
+ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${PV}.tar.xz \
+ file://defconfig"
+SRC_URI[sha256sum] = "866a94c1c38d923ae18e74b683d7a8a79b674ebdfe7f40f1a3be9a27d39fe354"
+
+S = "linux-${PV}"
--
2.13.6

Jan Kiszka

unread,
Feb 11, 2018, 10:25:22 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This simplifies the common task of using a custom kernel instead of the
pre-selected disto variant: Move the kernel installation into a dummy
dpkg-raw recipe that only has the kernel package as dependency.

Which recipe is used for providing the kernel can now be selected via
the well-know PREFERRED_PROVIDER_virtual/kernel, just like in OE.

The kernel package name is derived from the KERNEL_ARCH variable which
is communicated from the target multiconfig file to the linux-distro
recipe.

Based on an idea of Henning Schild.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/distro/debian-jessie.conf | 2 ++
meta-isar/conf/distro/debian-stretch.conf | 2 ++
meta-isar/conf/distro/debian-wheezy.conf | 2 ++
meta-isar/conf/distro/raspbian-jessie.conf | 2 ++
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 3 ++-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 ++-
meta-isar/recipes-core/images/isar-image-base.bb | 2 ++
meta/recipes-kernel/linux/linux-distro.bb | 16 ++++++++++++++++
14 files changed, 38 insertions(+), 8 deletions(-)
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index e54b9a6..2d16f8f 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -18,6 +18,8 @@ PV = "1.0"

inherit image

+IMAGE_INSTALL += "${@d.getVar("PREFERRED_PROVIDER_virtual/kernel", True)}"
+
DEPENDS += "${IMAGE_INSTALL}"

IMAGE_PREINSTALL += "apt \
diff --git a/meta/recipes-kernel/linux/linux-distro.bb b/meta/recipes-kernel/linux/linux-distro.bb
new file mode 100644
index 0000000..1af93ed
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-distro.bb
@@ -0,0 +1,16 @@
+# Debian kernel pseudo package
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+

Jan Kiszka

unread,
Feb 11, 2018, 10:25:23 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This provides an include for custom kernel modules. We choose an include
over a class as we carry a couple of files to debianize the module. The
user just needs to specify the module SRC_URI, and those sources must be
then reside in a separate directory which is - as usual - defined via S.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
.../linux-module/files/debian/changelog | 5 +++++
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 ++++++++++
.../recipes-kernel/linux-module/files/debian/rules | 15 ++++++++++++++
meta/recipes-kernel/linux-module/module.inc | 24 ++++++++++++++++++++++
5 files changed, 56 insertions(+)
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog
new file mode 100644
index 0000000..c1c3516
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog
@@ -0,0 +1,5 @@
+@PN@ (@PV@) unstable; urgency=low
+
+ * Generated package.
+
+ -- ISAR project <isar-...@googlegroups.com> Tue, 6 Feb 2018 00:00:00 +0000
diff --git a/meta/recipes-kernel/linux-module/files/debian/compat b/meta/recipes-kernel/linux-module/files/debian/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
new file mode 100644
index 0000000..d8fbcaa
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control
@@ -0,0 +1,11 @@
+Source: @PN@
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-@KERNEL_ARCH@
+Maintainer: ISAR project <isar-...@googlegroups.com>
+
+Package: @PN@
+Architecture: any
+Depends: linux-image-@KERNEL_ARCH@
+Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules
new file mode 100755
index 0000000..4723cd8
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/rules
@@ -0,0 +1,15 @@
+#!/usr/bin/make -f
+
+# Debian rules for custom kernel module build
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+export KDIR=$(shell ls -d /lib/modules/*/build)
+
+export DEB_BUILD_OPTIONS=parallel=$(shell nproc)
+
+%:
+ dh $@ --parallel
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
new file mode 100644
index 0000000..ecac75f
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -0,0 +1,24 @@
+# Custom kernel module recipe include
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+FILESPATH =. "${LAYERDIR_core}/recipes-kernel/linux-module/files:"
+

Jan Kiszka

unread,
Feb 11, 2018, 10:25:23 AM2/11/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This demonstrates the usage of the new module.inc via a primitive
hello world module.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/local.conf.sample | 2 +-
.../recipes-kernel/example-module/example-module.bb | 12 ++++++++++++
.../recipes-kernel/example-module/files/src/Makefile | 18 ++++++++++++++++++
.../example-module/files/src/example-module.c | 19 +++++++++++++++++++
4 files changed, 50 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 2ae43e7..92dea46 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -153,7 +153,7 @@ CONF_VERSION = "1"

#
# The default list of extra packages to be installed.
-IMAGE_INSTALL = "example-hello example-raw"
+IMAGE_INSTALL = "example-hello example-raw example-module"

#
# Default parallel jobs for bitbake:
diff --git a/meta-isar/recipes-kernel/example-module/example-module.bb b/meta-isar/recipes-kernel/example-module/example-module.bb
new file mode 100644
index 0000000..c7042cf
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/example-module.bb
@@ -0,0 +1,12 @@
+# Example recipe for building a custom module
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+include recipes-kernel/linux-module/module.inc
+
+SRC_URI += "file://src"
+
+S = "src"
diff --git a/meta-isar/recipes-kernel/example-module/files/src/Makefile b/meta-isar/recipes-kernel/example-module/files/src/Makefile
new file mode 100644
index 0000000..1625b28
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/Makefile
@@ -0,0 +1,18 @@
+# Example module
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: GPL-2.0
+
+obj-m := example-module.o
+
+INSTALL_MOD_PATH ?= $(DESTDIR)
+export INSTALL_MOD_PATH
+
+modules modules_install clean:
+ $(MAKE) -C $(KDIR) M=$(PWD) $@
+
+install: modules_install
+
+.PHONY: modules modules_install install clean
diff --git a/meta-isar/recipes-kernel/example-module/files/src/example-module.c b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
new file mode 100644

Alexander Smirnov

unread,
Feb 12, 2018, 4:33:41 AM2/12/18
to Jan Kiszka, isar-users
On 02/11/2018 06:25 PM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.k...@siemens.com>
>
> This ensures that we can override packages from upstream Debian or other
> external sources with our self-built versions. We achieve this for now
> by asking multistrap to drop a preferences file for the buildchroot so
> that dependency installations use the right priority. For the image
> build, this does not work because all packages are pull during the
> bootstrap. Therefore, we set aptdefaultrelease to isar to ensure that
> our repo gets the higher priority.
>

I'm not sure that I completely understand this. What is the use-case for
this? If you build your own copy of upstream package, this
prioritization could be resolved by specifying suffix to version.

For me this approach looks like a good way to have unexpected troubles
like the following:

1. You pick current glibc from upstream distro XYZ, let's say v1.0.
2. Patch it and put glibc-v1.0-isar to isar-apt.
3. Wait for some time until glibc is updated in upstream, let's say to v1.1.
4. Due to glibc-v1.0-isar is set to be preferred, it's going to be
installed. But glibc is the core library in distro, and lots of packages
in XYZ now require glibc-v1.1. So even if you will be able to generate
image, you can't guarantee it works correctly...

Alex

Jan Kiszka

unread,
Feb 12, 2018, 4:41:20 AM2/12/18
to Alexander Smirnov, isar-users
On 2018-02-12 10:33, Alexander Smirnov wrote:
> On 02/11/2018 06:25 PM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.k...@siemens.com>
>>
>> This ensures that we can override packages from upstream Debian or other
>> external sources with our self-built versions. We achieve this for now
>> by asking multistrap to drop a preferences file for the buildchroot so
>> that dependency installations use the right priority. For the image
>> build, this does not work because all packages are pull during the
>> bootstrap. Therefore, we set aptdefaultrelease to isar to ensure that
>> our repo gets the higher priority.
>>
>
> I'm not sure that I completely understand this. What is the use-case for
> this? If you build your own copy of upstream package, this
> prioritization could be resolved by specifying suffix to version.

Simple example: custom kernel build of linux-cip

This generates a package called linux-image-amd64 (for amd64 as target),
and that may have a version < debian's kernel. Now, if we do not
prioritize isar-apt over upstream, the upstream kernel is taken, even
when you switched PREFERRED_PROVIDER (in fact, the build will break when
that virtual package linux-cip is selected but can't be installed due to
the version preference "newest").

>
> For me this approach looks like a good way to have unexpected troubles
> like the following:
>
> 1. You pick current glibc from upstream distro XYZ, let's say v1.0.
> 2. Patch it and put glibc-v1.0-isar to isar-apt.
> 3. Wait for some time until glibc is updated in upstream, let's say to
> v1.1.
> 4. Due to glibc-v1.0-isar is set to be preferred, it's going to be
> installed. But glibc is the core library in distro, and lots of packages
> in XYZ now require glibc-v1.1. So even if you will be able to generate
> image, you can't guarantee it works correctly...

Well, sure, no one takes away the task from you of properly maintaining
the recipes when upstream updates happen (and you pull them into your
reproducible build).

Jan

Alexander Smirnov

unread,
Feb 12, 2018, 5:08:17 AM2/12/18
to Jan Kiszka, isar-users
On 02/12/2018 12:41 PM, Jan Kiszka wrote:
> On 2018-02-12 10:33, Alexander Smirnov wrote:
>> On 02/11/2018 06:25 PM, Jan Kiszka wrote:
>>> From: Jan Kiszka <jan.k...@siemens.com>
>>>
>>> This ensures that we can override packages from upstream Debian or other
>>> external sources with our self-built versions. We achieve this for now
>>> by asking multistrap to drop a preferences file for the buildchroot so
>>> that dependency installations use the right priority. For the image
>>> build, this does not work because all packages are pull during the
>>> bootstrap. Therefore, we set aptdefaultrelease to isar to ensure that
>>> our repo gets the higher priority.
>>>
>>
>> I'm not sure that I completely understand this. What is the use-case for
>> this? If you build your own copy of upstream package, this
>> prioritization could be resolved by specifying suffix to version.
>
> Simple example: custom kernel build of linux-cip
>
> This generates a package called linux-image-amd64 (for amd64 as target),
> and that may have a version < debian's kernel. Now, if we do not
> prioritize isar-apt over upstream, the upstream kernel is taken, even
> when you switched PREFERRED_PROVIDER (in fact, the build will break when
> that virtual package linux-cip is selected but can't be installed due to
> the version preference "newest").

Ok, but why you can't name you kernel 'linux-cip-image-amd64'? Having
the same name as with original Debian one could be confusing.


Moreover 'man 1 multistrap' states the same:
...
The combination of default release, explicit suite and apt preferences
can quickly become complex and bugs can be very hard to identify.
...

So, from two options:

- Rename custom kernel package
- Prioritize apt

the first one looks more easy and robust.

>
>>
>> For me this approach looks like a good way to have unexpected troubles
>> like the following:
>>
>> 1. You pick current glibc from upstream distro XYZ, let's say v1.0.
>> 2. Patch it and put glibc-v1.0-isar to isar-apt.
>> 3. Wait for some time until glibc is updated in upstream, let's say to
>> v1.1.
>> 4. Due to glibc-v1.0-isar is set to be preferred, it's going to be
>> installed. But glibc is the core library in distro, and lots of packages
>> in XYZ now require glibc-v1.1. So even if you will be able to generate
>> image, you can't guarantee it works correctly...
>
> Well, sure, no one takes away the task from you of properly maintaining
> the recipes when upstream updates happen (and you pull them into your
> reproducible build).

If this is only about specific package[s], I'd suggest to propose this
feature as an option to *overwrite* the specific package and do not do
this by default for the whole isar-apt.

Alex

Jan Kiszka

unread,
Feb 12, 2018, 5:11:36 AM2/12/18
to Alexander Smirnov, isar-users
That is mandatory, in order to write packages that depend on THE kernel,
instead of "my-custom-kernel-in-version-4.4.112".

Jan

Jan Kiszka

unread,
Feb 12, 2018, 5:27:25 AM2/12/18
to Alexander Smirnov, isar-users
To make it clearer, just look at the example-module case: The recipe can
build both against the distro as well as all custom kernels, without
touching it. And that is a pattern not limited to kernel<->module
relationships.

Alexander Smirnov

unread,
Feb 12, 2018, 8:10:07 AM2/12/18
to Jan Kiszka, isar-users
kernel<->module pair are built from the same source tree, so here no
problem with ${PN}-${PV} substitution as dependency. Also I've parsed
pure Debian packages, nobody depends from 'linux-image-*', so no
correlation with upstream packages.

What are the other use-cases for this pattern? As I mentioned above with
'glibc' example, this is unclean approach, distro could become broken at
anytime. This could become robust *only* after build reproducibility
integration, when you cache the upstream content. But with build-rep you
could manually manage the content of the cache, so prioritization is not
needed, you could just drop duplications from the cache.

In general having packages with the same name in 'source.list' and
overriding version policies in apt looks very dangerous for me.

If the current series requires prioritization for kernel only, it would
be nice to specify only this package in apt preferences.

Alex

Jan Kiszka

unread,
Feb 12, 2018, 8:45:12 AM2/12/18
to Alexander Smirnov, isar-users
No, they can be independent, just like in the example I submitted. It's
an external kernel module vs. some kernel. Please re-read.

For Debian, this specific problem does not exist because Debian only
ships in-tree modules or those that are built on the target on
installation. But you will find the exact same pattern when browsing
external projects that come with modules.

>
> What are the other use-cases for this pattern? As I mentioned above with
> 'glibc' example, this is unclean approach, distro could become broken at
> anytime. This could become robust *only* after build reproducibility
> integration, when you cache the upstream content. But with build-rep you
> could manually manage the content of the cache, so prioritization is not
> needed, you could just drop duplications from the cache.

Every lib-dependency is another pattern for the special kernel/module
case: Every distro package that come with a dependencies that you want
to replace with a custom build AND you have good reason to downgrade
that package, then you want your own packages to have higher prior.
That's why this hard-coding is the right thing to do, in general: If you
are building a package that competes with upstream, there must be a
reason...

>
> In general having packages with the same name in 'source.list' and
> overriding version policies in apt looks very dangerous for me.
>
> If the current series requires prioritization for kernel only, it would
> be nice to specify only this package in apt preferences.

See above, the kernel is just the first case. There will be more, I'm
absolutely sure.

Alexander Smirnov

unread,
Feb 12, 2018, 11:31:47 AM2/12/18
to Jan Kiszka, isar-users
Ok, I see that you define KERNEL_ARCH variable in machine config and
this variable is used to bind kernel<->module build process. Sometimes
KERNEL_ARCH has to be changed according to your kernel configuration.
It's not enough to change preferred_provider from "linux-distro" to
"linux-cip" if you want to use LPAE for your ARM kernel. You also need
to change KERNEL_ARCH from "armmp" to "armmp-lpae" to be correct.

How you see the overall procedure to switch to custom kernel? Why
KERNEL_ARCH can't be "cip-armmp" in this case?

For example armbian uses branch names and machines to distinguish images:

https://www.armbian.com/kernel/

- linux-image-dev-sun8i
- linux-image-next-odroidc2

So if define KERNEL_ARCH as ${ORIGIN}-${MACHINE}, you never will get
conflicts with the upstream. Also this doesn't affect modules building
at all.

>
>>
>> What are the other use-cases for this pattern? As I mentioned above with
>> 'glibc' example, this is unclean approach, distro could become broken at
>> anytime. This could become robust *only* after build reproducibility
>> integration, when you cache the upstream content. But with build-rep you
>> could manually manage the content of the cache, so prioritization is not
>> needed, you could just drop duplications from the cache.
>
> Every lib-dependency is another pattern for the special kernel/module
> case: Every distro package that come with a dependencies that you want
> to replace with a custom build AND you have good reason to downgrade
> that package, then you want your own packages to have higher prior.
> That's why this hard-coding is the right thing to do, in general: If you
> are building a package that competes with upstream, there must be a
> reason...
>
>>
>> In general having packages with the same name in 'source.list' and
>> overriding version policies in apt looks very dangerous for me.
>>
>> If the current series requires prioritization for kernel only, it would
>> be nice to specify only this package in apt preferences.
>
> See above, the kernel is just the first case. There will be more, I'm
> absolutely sure.

I've asked especially about these cases. :-( At the moment there is only
kernel case, which could be solved without this prioritization.

Alex

Jan Kiszka

unread,
Feb 12, 2018, 12:00:52 PM2/12/18
to Alexander Smirnov, isar-users
OK, but this cannot be defined by the kernel recipe. It has to be
defined by the machine setting. A kernel recipe could then select a
config that matches or reject the KERNEL_ARCH. Currently, we do nothing
in this direction, that is a valid point for future improvements.

>
> How you see the overall procedure to switch to custom kernel? Why

PREFERRED_PROVIDER_virtual/kernel = "my-nice-kernel"

See also
https://github.com/siemens/jailhouse-images/blob/master/conf/multiconfig/qemuamd64-jailhouse.conf

> KERNEL_ARCH can't be "cip-armmp" in this case?

Because the KERNEL_ARCH remains amd64, armmp, or whatever you
configured. Changing the kernel source doesn't change that in any way.

>
> For example armbian uses branch names and machines to distinguish images:
>
> https://www.armbian.com/kernel/
>
>  - linux-image-dev-sun8i
>  - linux-image-next-odroidc2
>
> So if define KERNEL_ARCH as ${ORIGIN}-${MACHINE}, you never will get
> conflicts with the upstream. Also this doesn't affect modules building
> at all.

This is about conflict: We are *replacing* the distro kernel. But when
doing this, we do not want to change all its consumers. Armbian likely
never ran into the use case of having to design such a full replacement,
also regarding dependencies (I do not see them shipping any out-of-tree
module packages).

And the same is true for all the non-kernel cases we will face when
replacing library-like packages. Please do not focus all your thoughts
on the kernel. This issue is absolutely generic.

>
>>
>>>
>>> What are the other use-cases for this pattern? As I mentioned above with
>>> 'glibc' example, this is unclean approach, distro could become broken at
>>> anytime. This could become robust *only* after build reproducibility
>>> integration, when you cache the upstream content. But with build-rep you
>>> could manually manage the content of the cache, so prioritization is not
>>> needed, you could just drop duplications from the cache.
>>
>> Every lib-dependency is another pattern for the special kernel/module
>> case: Every distro package that come with a dependencies that you want
>> to replace with a custom build AND you have good reason to downgrade
>> that package, then you want your own packages to have higher prior.
>> That's why this hard-coding is the right thing to do, in general: If you
>> are building a package that competes with upstream, there must be a
>> reason...
>>
>>>
>>> In general having packages with the same name in 'source.list' and
>>> overriding version policies in apt looks very dangerous for me.
>>>
>>> If the current series requires prioritization for kernel only, it would
>>> be nice to specify only this package in apt preferences.
>>
>> See above, the kernel is just the first case. There will be more, I'm
>> absolutely sure.
>
> I've asked especially about these cases. :-( At the moment there is only
> kernel case, which could be solved without this prioritization.

Use your imagination: enforced replacement of ANY package we rebuild,
but with a lower version. We rebuild packages for a reason. That implies
we want them to be actually used. And that means our packages require a
higher priority.

Jan Kiszka

unread,
Feb 12, 2018, 12:27:24 PM2/12/18
to Alexander Smirnov, isar-users
On 2018-02-12 18:00, Jan Kiszka wrote:
>> Ok, I see that you define KERNEL_ARCH variable in machine config and
>> this variable is used to bind kernel<->module build process. Sometimes
>> KERNEL_ARCH has to be changed according to your kernel configuration.
>> It's not enough to change preferred_provider from "linux-distro" to
>> "linux-cip" if you want to use LPAE for your ARM kernel. You also need
>> to change KERNEL_ARCH from "armmp" to "armmp-lpae" to be correct.
>
> OK, but this cannot be defined by the kernel recipe. It has to be
> defined by the machine setting. A kernel recipe could then select a
> config that matches or reject the KERNEL_ARCH. Currently, we do nothing
> in this direction, that is a valid point for future improvements.

Actually, this is rather simple to address in the own recipes that want
multi-arch kernel support, and I will likely use this pattern in
jailhouse-images:

FILESPATH =. "config-${KERNEL_ARCH}:"

and then put config-amd64/defconfig, config-armmp/defconfig etc. into
your recipe folder.

Alexander Smirnov

unread,
Feb 12, 2018, 1:51:14 PM2/12/18
to Jan Kiszka, isar-users
It doesn't remain for all the cases. ARM debian provides two kernel images:

- linux-image-armmp
- linux-image-armmp-lpae

Current Isar doesn't handle this, but if you build your kernel with
LPAE, I assume you should change KERNEL_ARCH respectively, otherwise
there will be some inaccuracy. So the same machine could have several
KERNEL_ARCHs.

Then, what should I write in KERNEL_ARCH if I add new machine, let's say
Cubieboard?

In general, KERNEL_ARCH is not exactly what it's used here for.
According to the:

https://kernel-handbook.alioth.debian.org/ch-packaging.html

the linux image name looks like the following:

- linux-image[-featureset]-flavour

Also chapter 3.3 stays:

8<--
Each architecture usually has multiple flavours of the binary kernel
images. Different flavours correspond to different kernel configuration
files, used to build the binary images from the same kernel tree.
8<--

So having KERNEL_ARCH as the constant is deviation from Debian flow.

In addition:

8<--
Again, multiple flavours of binary images may be built from the
featureset tree. For example, the i386 architecture has a number of
different flavours, such as 686 and 686-pae, built from the common
Debian kernel source. It also contains the rt featureset. The source
tree for building the kernels for each of these featuresets is obtained
by applying additional patches to the Debian kernel source. It may be
used to build the rt-686-pae binary image flavours.
8<--

So in my opinion, CIP is a feature set.

I don't really think, that replacing upstream Debian kernel package by
custom one and claiming that is the same will make Debian community happy...


Ok, I see that we have different vision on this topic, I could stay with
this implementation for now, but in general in my opinion this looks
like a big hack and should be reworked.
My imagination told me how it will be difficult to debug segfaults on
the target, because I pinned old libA in Isar-apt, while upstream
dependents already have been compiled according the new version of libA.
It wouldn't be so funny if nobody had faced with such problems in the
past...

Alex

Alexander Smirnov

unread,
Feb 12, 2018, 1:56:24 PM2/12/18
to Jan Kiszka, isar-users
On 02/11/2018 06:25 PM, Jan Kiszka wrote:
What is the benefit of having this script in the class? I see that we
have separate files for:

- buildchroot
- kernel modules

Why this chroot script should be a part of bitbake class? This makes
debugging much more complicated, because I have no possibility to run
this script manually from buildchroot (like I could do with build.sh)

Also formatting style looks nasty here.

Alex

Alexander Smirnov

unread,
Feb 12, 2018, 2:03:05 PM2/12/18
to Jan Kiszka, isar-users
On 02/11/2018 06:25 PM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.k...@siemens.com>
>
> Provide two examples for describing custom kernel builds. Both are for
> x86_64, simply using upstream x86_64_defconfig to define the required
> defconfig. One example is demoing a build from a tarball, the other
> pulling from git.
>
> Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
> ---
> meta-isar/recipes-kernel/linux/files/defconfig | 308 +++++++++++++++++++++
> meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 17 ++
> .../recipes-kernel/linux/linux-mainline_4.14.18.bb | 15 +
> 3 files changed, 340 insertions(+)
> create mode 100644 meta-isar/recipes-kernel/linux/files/defconfig
> create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
> create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
>
> diff --git a/meta-isar/recipes-kernel/linux/files/defconfig b/meta-isar/recipes-kernel/linux/files/defconfig
> new file mode 100644
> index 0000000..e32fc1f
> --- /dev/null
> +++ b/meta-isar/recipes-kernel/linux/files/defconfig
> @@ -0,0 +1,308 @@

Do you have any concerns if I add a comment here from where this config
was taken? Sometimes it would be very helpful to understand what this
config contains, at least which arch.
With best regards,
Alexander Smirnov

ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov

Jan Kiszka

unread,
Feb 13, 2018, 2:03:16 AM2/13/18
to Alexander Smirnov, isar-users
I played with it, and it "just" took 6 exports to set up the interface
to script-based build. That will likely not be convenient for debugging
as well (and I never had to call things manually that way, I got all
information from logs), but I can keep that pattern. Will also mean:
back to an include, rather than using a class (no big issue).

>
> Also formatting style looks nasty here.

That's not going to change unless you suggest some alternative for the
sed-based code injection into postinst/postrm.

Jan Kiszka

unread,
Feb 13, 2018, 2:03:36 AM2/13/18
to Alexander Smirnov, isar-users
Done.

Jan Kiszka

unread,
Feb 13, 2018, 2:22:28 AM2/13/18
to Alexander Smirnov, isar-users
In fact, one machine would have to have multiple configs, selecting the
kernel flavor. In OE terms, that would make different machines, though
targeting the same hardware.
Debian has a strong word here, though it complicates things
significantly. But let me look into expressing the kernel selection via
KERNEL_WHATEVER. It will mean that PREFERRED_PROVIDER can no longer be
used, though. And we will have to generate / parameterize the control
files of all module packages. Maybe that's Debian intention as well.
Again: One guy shooting himself into the foot by explicitly stating this
incorrect setup doesn't mean all others users have to give up on this
valid and required default property of self-build packages. It remains a
fact that self-built packages are there to customize and overwrite
upstream packages, independent of how the kernel will look like. That's
one key purpose of Isar.

If I manage to model the kernel differently, I will probably be able to
drop this patch from /this/ series. But the topic will only come up
again when Claudius proposes his multistrap rework which will include
shipping aptpreferences as well - and prioritizing our own repo properly.

Alexander Smirnov

unread,
Feb 13, 2018, 3:09:28 AM2/13/18
to Jan Kiszka, isar-users
I'm not against package overwriting. I mean that there is *not the only
one way* to do this. Apt provides way to overwrite packages via versions.

How I see the creation of robust derivatives:

1. Create local copy for upstream apt (let's name it base-apt) with
packages you are going to use for your build.
2. Choose package you want to modify, let's say libA.
3. Create recipe for it and update version to be ">" than original one.
4. Work with this environment.

This approach at least somehow guarantees that you will have
reproducible build. The packages in your base-apt are built according to
the libA version you've customized. So my opinion: user could overwrite
only package in local mirrors, i.e. mirrors that could not be updated by
3rd parties.


What could happen bad with pinning over upstream (libA is from upstream):

1. libA.h v1.0 had:

#define BUFFER_SIZE 32

2. libA.h v1.1 has:

#define BUFFER_SIZE 16

3. Upstream contains packages that were already rebuilt according to the
libA v1.1

4. You are still using libA v1.0 derivative.

5. Buffer overrun could occur now in any libA consumer from the upstream.

So, there is *no* robust way to overwrite package from live upstream,
you are always in the risk.

I could imaging that some business units may require this apt
prioritization for their specific product's needs, so lets have this
feature in Isar. But I think it shouldn't be enabled by default. Also
the user documentation for this feature should describe all the risks
that user takes if this option is enabled.

> If I manage to model the kernel differently, I will probably be able to
> drop this patch from /this/ series. But the topic will only come up
> again when Claudius proposes his multistrap rework which will include
> shipping aptpreferences as well - and prioritizing our own repo properly.
>

As I said, I could stay with this now to move forward. So the things
discussed above could be fixed later.

Alex

Alexander Smirnov

unread,
Feb 13, 2018, 3:16:11 AM2/13/18
to Jan Kiszka, isar-users
What do you mean by "done"? :-)

BTW: in future it would nice to implement configurable defconfig like
it's done in Yocto via variable, for example:

https://github.com/Freescale/meta-freescale/blob/master/conf/machine/t1042d4rdb-64b.conf#L22

otherwise this recipe could be used for single machine only.

Alex

Alexander Smirnov

unread,
Feb 13, 2018, 3:19:51 AM2/13/18
to Jan Kiszka, isar-users


On 02/13/2018 10:03 AM, Jan Kiszka wrote:
Just for sure that I understand the issue, why no sed injections? You
already do this for kernel module by "sedding" control file.

Alex

Jan Kiszka

unread,
Feb 13, 2018, 3:22:52 AM2/13/18
to Alexander Smirnov, isar-users
This doesn't solve intentional downgrades, like the kernel example would
have been one.

>
> This approach at least somehow guarantees that you will have
> reproducible build. The packages in your base-apt are built according to
> the libA version you've customized. So my opinion: user could overwrite
> only package in local mirrors, i.e. mirrors that could not be updated by
> 3rd parties.
>
>
> What could happen bad with pinning over upstream (libA is from upstream):
>
> 1. libA.h v1.0 had:
>
> #define BUFFER_SIZE 32
>
> 2. libA.h v1.1 has:
>
> #define BUFFER_SIZE 16
>
> 3. Upstream contains packages that were already rebuilt according to the
> libA v1.1
>
> 4. You are still using libA v1.0 derivative.
>
> 5. Buffer overrun could occur now in any libA consumer from the upstream.
>
> So, there is *no* robust way to overwrite package from live upstream,
> you are always in the risk.

Of course, if you overwrite something in an incompatible way, you have
to ensure updating all dependencies as well. That is the mistake done in
your example.

>
> I could imaging that some business units may require this apt
> prioritization for their specific product's needs, so lets have this
> feature in Isar. But I think it shouldn't be enabled by default. Also
> the user documentation for this feature should describe all the risks
> that user takes if this option is enabled.

Let's see how flexible the repo and package pinning configuration will
become with Claudius approach, and then we may switch to an explicit per
package enabling as well. More intuitive is the default I propose.

>
>> If I manage to model the kernel differently, I will probably be able to
>> drop this patch from /this/ series. But the topic will only come up
>> again when Claudius proposes his multistrap rework which will include
>> shipping aptpreferences as well - and prioritizing our own repo properly.
>>
>
> As I said, I could stay with this now to move forward. So the things
> discussed above could be fixed later.

User-visible changes should not be lightly merged. So this discussion is
valuable when it helps to come up with consistent and sustainable
interfaces. So far, the user was not yet enough in our focus, and that
is hopefully changing now.

Jan Kiszka

unread,
Feb 13, 2018, 3:24:08 AM2/13/18
to Alexander Smirnov, isar-users
The sed code is visually the most ugliest part of my build script.

Jan Kiszka

unread,
Feb 13, 2018, 3:25:00 AM2/13/18
to Alexander Smirnov, isar-users

Alexander Smirnov

unread,
Feb 13, 2018, 3:46:03 AM2/13/18
to Jan Kiszka, isar-users
What are the next steps with this series?
Anyway I want at least patches 1 and 3 from it.

Alex

Jan Kiszka

unread,
Feb 13, 2018, 3:51:04 AM2/13/18
to Alexander Smirnov, isar-users
Yes, feel free to pick them ahead of the rest! I will work on the others
and rebase on next when ready.

Alexander Smirnov

unread,
Feb 13, 2018, 4:41:43 AM2/13/18
to Jan Kiszka, isar-users


On 02/11/2018 06:25 PM, Jan Kiszka wrote:
> This is now finally the promised patches to provide simple patterns for
> building custom kernels for Isar images that replace the distro kernels.
> It also provids the same pattern for custom out-of-tree kernel modules.
>
> Along come two fixes, one to respect proxies during dpkg builds, the
> other to adjust isar-apt repo prio.
>
> Furthermore, I'm proposing to rename "SRC_DIR" to OE-standard "S".
>
> Changes in v3 (only resent patch 5):
> - allow central overriding of kernel dependencies
>
> Changes in v4
> - simplified changes to isar-image-base.bb by reordering code
> - modeled custom kernel build as class
>

Applied 1 and 3. Thanks!

Alex

Henning Schild

unread,
Feb 13, 2018, 4:53:51 AM2/13/18
to [ext] Jan Kiszka, Alexander Smirnov, isar-users
Am Tue, 13 Feb 2018 08:03:12 +0100
schrieb "[ext] Jan Kiszka" <jan.k...@siemens.com>:
You could write all these exports into a file that the scripts sources.
Given that file exists, one could run manually and the recipe would not
contain all that logic.

> >
> > Also formatting style looks nasty here.
>
> That's not going to change unless you suggest some alternative for the
> sed-based code injection into postinst/postrm.

Can that code be appended with echo >> ? The patching looks pretty
unconditional already.

Henning

> Jan
>

Jan Kiszka

unread,
Feb 13, 2018, 5:21:02 AM2/13/18
to Henning Schild, Alexander Smirnov, isar-users
That sounds like overkill. And there is already a file containing that
information (run.do_build).

I'm still looking for a concrete case where local invocation is needed,
even more as we are now able to restart builds easily.

>
>>>
>>> Also formatting style looks nasty here.
>>
>> That's not going to change unless you suggest some alternative for the
>> sed-based code injection into postinst/postrm.
>
> Can that code be appended with echo >> ? The patching looks pretty
> unconditional already.

I need to inject the code at a specific place in to the script, so it's
not about appending or prepending. At the same time, I'm concerned that
a real patch would break too often.

Jan Kiszka

unread,
Feb 14, 2018, 4:13:19 AM2/14/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This provides an include for custom kernel modules. We choose an include
over a class as we carry a couple of files to debianize the module. The
user just needs to specify the module SRC_URI, and those sources must be
then reside in a separate directory which is - as usual - defined via S.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
.../linux-module/files/debian/changelog | 5 +++++
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 ++++++++++
.../recipes-kernel/linux-module/files/debian/rules | 15 ++++++++++++++
meta/recipes-kernel/linux-module/module.inc | 24 ++++++++++++++++++++++
5 files changed, 56 insertions(+)
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog
new file mode 100644
index 0000000..c1c3516
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog
@@ -0,0 +1,5 @@
+@PN@ (@PV@) unstable; urgency=low
+
+ * Generated package.
+
+ -- ISAR project <isar-...@googlegroups.com> Tue, 6 Feb 2018 00:00:00 +0000
diff --git a/meta/recipes-kernel/linux-module/files/debian/compat b/meta/recipes-kernel/linux-module/files/debian/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
new file mode 100644
index 0000000..707f857
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control
@@ -0,0 +1,11 @@
+Source: @PN@
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-@KERNEL_NAME@
+Maintainer: ISAR project <isar-...@googlegroups.com>
+
+Package: @PN@
+Architecture: any
+Depends: linux-image-@KERNEL_NAME@
+Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules
new file mode 100755
index 0000000..4723cd8
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/rules
@@ -0,0 +1,15 @@
+#!/usr/bin/make -f
+
+# Debian rules for custom kernel module build
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+export KDIR=$(shell ls -d /lib/modules/*/build)
+
+export DEB_BUILD_OPTIONS=parallel=$(shell nproc)
+
+%:
+ dh $@ --parallel
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
new file mode 100644
index 0000000..aa232f7
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -0,0 +1,24 @@
+# Custom kernel module recipe include
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+FILESPATH =. "${LAYERDIR_core}/recipes-kernel/linux-module/files:"
+
+DESCRIPTION ?= "Custom kernel module ${PN}"
+
+DEPENDS += "linux-headers-${KERNEL_NAME}"
+
+SRC_URI += "file://debian/"
+
+inherit dpkg
+
+dpkg_runbuild_prepend() {
+ cp -r ${WORKDIR}/debian ${WORKDIR}/${S}/
+ sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
+ -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \
+ -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
+ ${WORKDIR}/${S}/debian/changelog ${WORKDIR}/${S}/debian/control
+}
--
2.13.6

Jan Kiszka

unread,
Feb 14, 2018, 4:13:19 AM2/14/18
to isar-users
This is now finally the promised patches to provide simple patterns for
building custom kernels for Isar images that replace the distro kernels.
It also provids the same pattern for custom out-of-tree kernel modules.

Changes in v5:
- dropped merged patches
- reworked kernel packages naming and selection mechanism to use
flavor/arch rather them overwriting upstream names
- fixed rebuilds of the kernel (.version was causing problems)
- moved kernel build into script - and switched back to include pattern
- provided note on origin of exemplary defconfig
- add KERNEL_DEFCONFIG variable

Jan

Jan Kiszka (5):
Make distro kernel replaceable
Provide include for easy custom kernel builds
Add custom kernel examples
Provide include file for easy custom module builds
Add exemplary kernel module

meta-isar/conf/distro/debian-jessie.conf | 4 +
meta-isar/conf/distro/debian-stretch.conf | 3 +
meta-isar/conf/distro/debian-wheezy.conf | 2 +
meta-isar/conf/distro/raspbian-jessie.conf | 1 +
meta-isar/conf/local.conf.sample | 2 +-
meta-isar/conf/multiconfig/qemuamd64-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 7 +-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-stretch.conf | 7 +-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemui386-stretch.conf | 7 +-
meta-isar/conf/multiconfig/rpi-jessie.conf | 3 +-
meta-isar/recipes-core/images/isar-image-base.bb | 2 +
.../example-module/example-module.bb | 12 +
.../example-module/files/src/Makefile | 18 ++
.../example-module/files/src/example-module.c | 19 ++
.../recipes-kernel/linux/files/x86_64_defconfig | 310 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 19 ++
.../recipes-kernel/linux/linux-mainline_4.14.18.bb | 17 ++
.../linux-module/files/debian/changelog | 5 +
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 +
.../recipes-kernel/linux-module/files/debian/rules | 15 +
meta/recipes-kernel/linux-module/module.inc | 24 ++
meta/recipes-kernel/linux/files/build-kernel.sh | 75 +++++
meta/recipes-kernel/linux/linux-custom.inc | 47 ++++
meta/recipes-kernel/linux/linux-distro.bb | 13 +
28 files changed, 614 insertions(+), 18 deletions(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c
create mode 100644 meta-isar/recipes-kernel/linux/files/x86_64_defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc
create mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh
create mode 100644 meta/recipes-kernel/linux/linux-custom.inc
create mode 100644 meta/recipes-kernel/linux/linux-distro.bb

--
2.13.6

Jan Kiszka

unread,
Feb 14, 2018, 4:13:19 AM2/14/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

Provide two examples for describing custom kernel builds. Both are for
x86_64, simply using upstream x86_64_defconfig to define the required
defconfig. One example is demoing a build from a tarball, the other
pulling from git.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
.../recipes-kernel/linux/files/x86_64_defconfig | 310 +++++++++++++++++++++
meta-isar/recipes-kernel/linux/linux-cip_4.4.bb | 19 ++
.../recipes-kernel/linux/linux-mainline_4.14.18.bb | 17 ++
3 files changed, 346 insertions(+)
create mode 100644 meta-isar/recipes-kernel/linux/files/x86_64_defconfig
create mode 100644 meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
create mode 100644 meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb

diff --git a/meta-isar/recipes-kernel/linux/files/x86_64_defconfig b/meta-isar/recipes-kernel/linux/files/x86_64_defconfig
new file mode 100644
index 0000000..f3bafe0
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/files/x86_64_defconfig
@@ -0,0 +1,310 @@
+# Copy of linux/arch/x86/configs/x86_64_defconfig for demonstration purposes
+
new file mode 100644
index 0000000..7502f70
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-cip_4.4.bb
@@ -0,0 +1,19 @@
+# Example recipe for building the CIP 4.4 kernel
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+require recipes-kernel/linux/linux-custom.inc
+ file://x86_64_defconfig"
+
+SRCREV = "4e52cc5f668c4666e31a8485725b5f4e897b3baf"
+PV = "4.4.112-cip18"
+
+S = "git"
+
+KERNEL_DEFCONFIG_qemuamd64 = "x86_64_defconfig"
diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
new file mode 100644
index 0000000..2c93d40
--- /dev/null
+++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.14.18.bb
@@ -0,0 +1,17 @@
+# Example recipe for building the mainline kernel
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+require recipes-kernel/linux/linux-custom.inc
+ file://x86_64_defconfig"
+SRC_URI[sha256sum] = "866a94c1c38d923ae18e74b683d7a8a79b674ebdfe7f40f1a3be9a27d39fe354"
+
+S = "linux-${PV}"
+
+KERNEL_DEFCONFIG_qemuamd64 = "x86_64_defconfig"
--
2.13.6

Jan Kiszka

unread,
Feb 14, 2018, 4:13:20 AM2/14/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This demonstrates the usage of the new module.inc via a primitive
hello world module.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
meta-isar/conf/local.conf.sample | 2 +-
.../recipes-kernel/example-module/example-module.bb | 12 ++++++++++++
.../recipes-kernel/example-module/files/src/Makefile | 18 ++++++++++++++++++
.../example-module/files/src/example-module.c | 19 +++++++++++++++++++
4 files changed, 50 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 2ae43e7..92dea46 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -153,7 +153,7 @@ CONF_VERSION = "1"

#
# The default list of extra packages to be installed.
-IMAGE_INSTALL = "example-hello example-raw"
+IMAGE_INSTALL = "example-hello example-raw example-module"

#
# Default parallel jobs for bitbake:
diff --git a/meta-isar/recipes-kernel/example-module/example-module.bb b/meta-isar/recipes-kernel/example-module/example-module.bb
new file mode 100644
index 0000000..c7042cf
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/example-module.bb
@@ -0,0 +1,12 @@
+# Example recipe for building a custom module
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+include recipes-kernel/linux-module/module.inc
+
+SRC_URI += "file://src"
+
+S = "src"
diff --git a/meta-isar/recipes-kernel/example-module/files/src/Makefile b/meta-isar/recipes-kernel/example-module/files/src/Makefile
new file mode 100644
index 0000000..1625b28
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/Makefile
@@ -0,0 +1,18 @@
+# Example module
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: GPL-2.0
+
+obj-m := example-module.o
+
+INSTALL_MOD_PATH ?= $(DESTDIR)
+export INSTALL_MOD_PATH
+
+modules modules_install clean:
+ $(MAKE) -C $(KDIR) M=$(PWD) $@
+
+install: modules_install
+
+.PHONY: modules modules_install install clean
diff --git a/meta-isar/recipes-kernel/example-module/files/src/example-module.c b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
new file mode 100644
index 0000000..07cb214
--- /dev/null
+++ b/meta-isar/recipes-kernel/example-module/files/src/example-module.c
@@ -0,0 +1,19 @@
+/*
+ * Example modules
+ *
+ * Copyright (c) Siemens AG, 2018
+ *
+ * SPDX-License-Identifier: GPL-2.0
+ */
+
+#include <linux/module.h>
+
+static int __init example_module_init(void)
+{
+ printk("Just an example\n");
+ return -ENOANO;
+}
+
+module_init(example_module_init);
+
+MODULE_LICENSE("GPL");
--
2.13.6

Jan Kiszka

unread,
Feb 14, 2018, 10:21:32 AM2/14/18
to isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This demonstrates the usage of the new module.inc via a primitive
hello world module.

For wheezy:vexpress, module builds unfortunately fail with
/usr/bin/ld: unrecognized option '-Wl,-z,relro'

So disable the example on in the related multiconfig.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

Changes in v6:
- wheezy:vexpress exception

meta-isar/conf/local.conf.sample | 2 +-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 3 +++
.../recipes-kernel/example-module/example-module.bb | 12 ++++++++++++
.../recipes-kernel/example-module/files/src/Makefile | 18 ++++++++++++++++++
.../example-module/files/src/example-module.c | 19 +++++++++++++++++++
5 files changed, 53 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/recipes-kernel/example-module/example-module.bb
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/Makefile
create mode 100644 meta-isar/recipes-kernel/example-module/files/src/example-module.c

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 2ae43e7..92dea46 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -153,7 +153,7 @@ CONF_VERSION = "1"

#
# The default list of extra packages to be installed.
-IMAGE_INSTALL = "example-hello example-raw"
+IMAGE_INSTALL = "example-hello example-raw example-module"

#
# Default parallel jobs for bitbake:
diff --git a/meta-isar/conf/multiconfig/qemuarm-wheezy.conf b/meta-isar/conf/multiconfig/qemuarm-wheezy.conf
index 11355a4..90e337a 100644
--- a/meta-isar/conf/multiconfig/qemuarm-wheezy.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-wheezy.conf
@@ -21,3 +21,6 @@ QEMU_ARCH ?= "arm"
QEMU_MACHINE ?= "vexpress-a9"
QEMU_CPU ?= ""
QEMU_ROOTFS_DEV ?= "-sd ##ROOTFS_IMAGE##"
+
+# module build does not work on wheezy for vexpress
+IMAGE_INSTALL_remove = "example-module"

Alexander Smirnov

unread,
Feb 15, 2018, 2:15:56 AM2/15/18
to Jan Kiszka, isar-users
On 02/14/2018 06:21 PM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.k...@siemens.com>
>
> This demonstrates the usage of the new module.inc via a primitive
> hello world module.
>
> For wheezy:vexpress, module builds unfortunately fail with
> /usr/bin/ld: unrecognized option '-Wl,-z,relro'
>
> So disable the example on in the related multiconfig.
>

Any plans to fix this?

Alex

Jan Kiszka

unread,
Feb 15, 2018, 2:35:19 AM2/15/18
to Alexander Smirnov, isar-users
From: Jan Kiszka <jan.k...@siemens.com>

This provides an include for custom kernel modules. We choose an include
over a class as we carry a couple of files to debianize the module. The
user just needs to specify the module SRC_URI, and those sources must be
then reside in a separate directory which is - as usual - defined via S.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

Changes in v6:
- fix module build on wheezy-arm and (upcoming) stretch-arm64

This allows to keep v5 of patch 5. Sorry for the mess, but I initially
thought it was an issue with old wheezy on special ARM hardware only.
But when it also popped up on stretch-arm64, I finally analyzed it.

.../linux-module/files/debian/changelog | 5 +++++
.../linux-module/files/debian/compat | 1 +
.../linux-module/files/debian/control | 11 ++++++++++
.../recipes-kernel/linux-module/files/debian/rules | 15 ++++++++++++++
meta/recipes-kernel/linux-module/module.inc | 24 ++++++++++++++++++++++
5 files changed, 56 insertions(+)
create mode 100644 meta/recipes-kernel/linux-module/files/debian/changelog
create mode 100644 meta/recipes-kernel/linux-module/files/debian/compat
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
create mode 100644 meta/recipes-kernel/linux-module/module.inc

diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog
new file mode 100644
index 0000000..c1c3516
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog
@@ -0,0 +1,5 @@
+@PN@ (@PV@) unstable; urgency=low
+
+ * Generated package.
+
+ -- ISAR project <isar-...@googlegroups.com> Tue, 6 Feb 2018 00:00:00 +0000
diff --git a/meta/recipes-kernel/linux-module/files/debian/compat b/meta/recipes-kernel/linux-module/files/debian/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
new file mode 100644
index 0000000..707f857
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control
@@ -0,0 +1,11 @@
+Source: @PN@
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-@KERNEL_NAME@
+Maintainer: ISAR project <isar-...@googlegroups.com>
+
+Package: @PN@
+Architecture: any
+Depends: linux-image-@KERNEL_NAME@
+Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules
new file mode 100755
index 0000000..46cf977
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/rules
@@ -0,0 +1,15 @@
+#!/usr/bin/make -f
+
+# Debian rules for custom kernel module build
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+export KDIR=$(shell ls -d /lib/modules/*/build)
+
+export DEB_BUILD_OPTIONS=parallel=$(shell nproc)
+
+%:
+ CFLAGS= LDFLAGS= dh $@ --parallel
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
new file mode 100644
index 0000000..aa232f7
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -0,0 +1,24 @@
+# Custom kernel module recipe include
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+

Alexander Smirnov

unread,
Feb 15, 2018, 2:54:09 AM2/15/18
to Jan Kiszka, isar-users

On 02/15/2018 10:35 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.k...@siemens.com>
>
> This provides an include for custom kernel modules. We choose an include
> over a class as we carry a couple of files to debianize the module. The
> user just needs to specify the module SRC_URI, and those sources must be
> then reside in a separate directory which is - as usual - defined via S.
>

Should I take then [v5 5/5]?

Alex

Jan Kiszka

unread,
Feb 15, 2018, 2:54:48 AM2/15/18
to Alexander Smirnov, isar-users
On 2018-02-15 08:54, Alexander Smirnov wrote:
>
> On 02/15/2018 10:35 AM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.k...@siemens.com>
>>
>> This provides an include for custom kernel modules. We choose an include
>> over a class as we carry a couple of files to debianize the module. The
>> user just needs to specify the module SRC_URI, and those sources must be
>> then reside in a separate directory which is - as usual - defined via S.
>>
>
> Should I take then [v5 5/5]?

Yes, that's what I meant below.

Jan

Alexander Smirnov

unread,
Feb 15, 2018, 3:01:10 AM2/15/18
to Jan Kiszka, isar-users
On 02/15/2018 10:54 AM, Jan Kiszka wrote:
> On 2018-02-15 08:54, Alexander Smirnov wrote:
>>
>> On 02/15/2018 10:35 AM, Jan Kiszka wrote:
>>> From: Jan Kiszka <jan.k...@siemens.com>
>>>
>>> This provides an include for custom kernel modules. We choose an include
>>> over a class as we carry a couple of files to debianize the module. The
>>> user just needs to specify the module SRC_URI, and those sources must be
>>> then reside in a separate directory which is - as usual - defined via S.
>>>
>>
>> Should I take then [v5 5/5]?
>
> Yes, that's what I meant below.
>

Damn, sorry, thanks :-)

Alex

Alexander Smirnov

unread,
Feb 15, 2018, 6:24:42 AM2/15/18
to Jan Kiszka, isar-users
On 02/14/2018 12:13 PM, Jan Kiszka wrote:
> This is now finally the promised patches to provide simple patterns for
> building custom kernels for Isar images that replace the distro kernels.
> It also provids the same pattern for custom out-of-tree kernel modules.
>
> Changes in v5:
> - dropped merged patches
> - reworked kernel packages naming and selection mechanism to use
> flavor/arch rather them overwriting upstream names
> - fixed rebuilds of the kernel (.version was causing problems)
> - moved kernel build into script - and switched back to include pattern
> - provided note on origin of exemplary defconfig
> - add KERNEL_DEFCONFIG variable
>

Looks good to me, applied to next. Should I wait documentation for this
feature?

One small issue: you use @XYZ@ pattern for substitution, while ##XYZ##
is used in multistrap configs. I'm with unified style in Isar. I'd
prefer your variant, so just as TODO for somebody - replace remaining
##XYZ## patterns in Isar to @XYZ@.

Alex

Alexander Smirnov

unread,
Feb 15, 2018, 9:43:44 AM2/15/18
to Jan Kiszka, isar-users
On 02/14/2018 12:13 PM, Jan Kiszka wrote:
> This is now finally the promised patches to provide simple patterns for
> building custom kernels for Isar images that replace the distro kernels.
> It also provids the same pattern for custom out-of-tree kernel modules.
>
> Changes in v5:
> - dropped merged patches
> - reworked kernel packages naming and selection mechanism to use
> flavor/arch rather them overwriting upstream names
> - fixed rebuilds of the kernel (.version was causing problems)
> - moved kernel build into script - and switched back to include pattern
> - provided note on origin of exemplary defconfig
> - add KERNEL_DEFCONFIG variable
>
> Jan

$ start_vm -a arm -d jessie

...

root@isar:~# modprobe example-module
[ 110.444107] Just an example
modprobe: ERROR: could not insert 'example_module': No anode


The same for amd64:


root@isar:~# modprobe example-module
[ 38.343926] example_module: loading out-of-tree module taints kernel.
[ 38.353140] Just an example
modprobe: ERROR: could not insert 'example_module': No anode

Please take a look.

Alex

Jan Kiszka

unread,
Feb 15, 2018, 10:00:10 AM2/15/18
to Alexander Smirnov, isar-users
static int __init example_module_init(void)
{
printk("Just an example\n");
return -ENOANO;
}

module_init(example_module_init);

:)

Jan

Alexander Smirnov

unread,
Feb 15, 2018, 10:05:05 AM2/15/18
to Jan Kiszka, isar-users
Ouch, didn't notice this. :-(

Do you have any specific reason to have module_init failed? Could we
have zero return code to test its probe-ability and loading-by-default
for CI?

Alex

Jan Kiszka

unread,
Feb 15, 2018, 10:13:48 AM2/15/18
to Alexander Smirnov, isar-users
I didn't think of CI for this module yet, but that may make more sense
then. Do you plan to write some CI script for that? Then just change the
module code to your needs. It's "just an example". :)

Jan Kiszka

unread,
Feb 15, 2018, 10:14:30 AM2/15/18
to Alexander Smirnov, isar-users
If the multistrap cases will go anyway, this consolidation would come
for free.

Alexander Smirnov

unread,
Feb 15, 2018, 10:27:35 AM2/15/18
to Jan Kiszka, isar-users
Ok, thanks!

Now there is simple automated which checks that each virtual machine has
reached login prompt. You could see this in the bottom of console output:

http://isar-build.org:8080/job/isar_next/105/console

To be honest, I don't want to check this manually after each merge, so
I'm thinking about test automation for everything new appears in Isar :-)

Alex
Reply all
Reply to author
Forward
0 new messages