[PATCH v4 0/8] imagetypes

57 views
Skip to first unread message

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:39 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
This series introduces image types and conversions as known from OE.
The main motivation is to provide a more flexible way of generating
and processing images, without the need to insert custom tasks (e.g.,
currently we have custom classes adding tasks for things like bz2 or xz
compression of images in several downstream layers).

These patches have been tested on some of our layers, and also
work with some more "challenging" use cases like image-in-image recipes
and custom image types brought by layers.

This only has a bare minimum of conversions (gz, xz), so there should
probably be follow-ups for additional compressions, or things
like checksums, which can also be nicely done with this.

- p1 prepares by removing image-type-suffix from image build directories,
which enables sharing of the completed rootfs across image types
- p2 refactors the SDK, and makes it an image feature instead of
a separate recipe (has been discussed here a while ago)
- p3 does the actual work, but keeps all original files
- p4 moves code to different files
- p5 adapts meta-isar
- p6 removes IMAGE_FSTYPES as a dimension in multiconfig. Now that
we can build multiple types from one rootfs, multconfig is
just the wrong way of doing this (also been discussed here).
- p7 documents
- p8 adapts start_vm tests

changes since v3:
- better handling of xz compression options (taken from OE)
- fix IMAGE_FSTYPES append in qemuamd64-bullseye.conf
- add patch to adapt start_vm to new image type and file names

changes since v2:
- fix bug where construction of IMAGE_BASETYPES was not recursive
- add comment explaining IMAGE_BASETYPES
- fix ISAR_CROSS_COMPILE exception for ubifs+armhf
- use "IMAGE_CLASSES +=" instead of "INHERIT +=" in machine configs
- fix whitespace errors in documentation

changes since v1:
- rebase
- fixed naming issue in wic.bmap files
- more reasonable default settings for xz
- add documentation to user manual

Adriaan Schmidt (7):
image: remove IMAGE_SUFFIX
image: refactor SDK
meta: introduce IMAGE_CMD_*
imagetypes: restructure files
imagetypes: switch meta-isar to the new scheme
meta-isar: remove IMAGE_FSTYPES from multiconfig definitions
docs: add new imagetypes to user manual

Anton Mikanovich (1):
start_vm: update image types checking

doc/user_manual.md | 86 ++++---
meta-isar/classes/rpi-sdimg.bbclass | 2 +-
meta-isar/classes/ubi-ubifs-img.bbclass | 11 +-
meta-isar/conf/local.conf.sample | 4 -
meta-isar/conf/machine/bananapi.conf | 2 +-
meta-isar/conf/machine/container-amd64.conf | 2 +-
meta-isar/conf/machine/de0-nano-soc.conf | 2 +-
meta-isar/conf/machine/hikey.conf | 2 +-
meta-isar/conf/machine/imx6-sabrelite.conf | 3 +-
meta-isar/conf/machine/nanopi-neo.conf | 2 +-
meta-isar/conf/machine/phyboard-mira.conf | 3 +-
meta-isar/conf/machine/qemuamd64.conf | 2 +-
meta-isar/conf/machine/qemuarm.conf | 2 +-
meta-isar/conf/machine/qemuarm64.conf | 2 +-
meta-isar/conf/machine/qemui386.conf | 2 +-
meta-isar/conf/machine/qemumipsel.conf | 2 +-
meta-isar/conf/machine/qemuriscv64.conf | 2 +-
meta-isar/conf/machine/rpi-common.conf | 2 +-
meta-isar/conf/machine/rpi.conf | 2 +-
meta-isar/conf/machine/sifive-fu540.conf | 2 +-
meta-isar/conf/machine/stm32mp15x.conf | 2 +-
meta-isar/conf/machine/virtualbox.conf | 2 +-
meta-isar/conf/machine/vmware.conf | 2 +-
.../qemuamd64-bullseye-cpiogz.conf | 7 -
.../multiconfig/qemuamd64-bullseye-tgz.conf | 7 -
.../conf/multiconfig/qemuamd64-bullseye.conf | 2 +
.../multiconfig/qemuamd64-buster-cpiogz.conf | 9 -
.../multiconfig/qemuamd64-buster-tgz.conf | 9 -
.../conf/multiconfig/qemuamd64-buster.conf | 2 +-
.../recipes-core/images/isar-image-ubi.bb | 4 +-
meta/classes/container-img.bbclass | 17 --
meta/classes/cpiogz-img.bbclass | 22 --
meta/classes/ext4-img.bbclass | 24 --
meta/classes/fit-img.bbclass | 29 ---
.../classes/image-container-extension.bbclass | 83 -------
meta/classes/image-sdk-extension.bbclass | 87 -------
meta/classes/image.bbclass | 212 ++++++++++++++++--
meta/classes/imagetypes.bbclass | 93 ++++++++
meta/classes/imagetypes_container.bbclass | 95 ++++++++
.../{vm-img.bbclass => imagetypes_vm.bbclass} | 24 +-
...wic-img.bbclass => imagetypes_wic.bbclass} | 45 ++--
meta/classes/rootfs.bbclass | 1 +
meta/classes/sdk.bbclass | 137 +++++++++++
meta/classes/targz-img.bbclass | 15 --
meta/classes/ubi-img.bbclass | 32 ---
meta/classes/ubifs-img.bbclass | 32 ---
.../{sdkchroot => sdk-files}/files/README.sdk | 0
.../files/configscript.sh | 0
.../files/gcc-sysroot-wrapper.sh | 0
.../files/relocate-sdk.sh | 0
meta/recipes-devtools/sdk-files/sdk-files.bb | 26 +++
meta/recipes-devtools/sdkchroot/sdkchroot.bb | 78 -------
scripts/start_vm | 8 +-
testsuite/cibase.py | 4 +-
testsuite/citest.py | 2 -
testsuite/start_vm.py | 8 +-
56 files changed, 673 insertions(+), 584 deletions(-)
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
delete mode 100644 meta/classes/container-img.bbclass
delete mode 100644 meta/classes/cpiogz-img.bbclass
delete mode 100644 meta/classes/ext4-img.bbclass
delete mode 100644 meta/classes/fit-img.bbclass
delete mode 100644 meta/classes/image-container-extension.bbclass
delete mode 100644 meta/classes/image-sdk-extension.bbclass
create mode 100644 meta/classes/imagetypes.bbclass
create mode 100644 meta/classes/imagetypes_container.bbclass
rename meta/classes/{vm-img.bbclass => imagetypes_vm.bbclass} (89%)
rename meta/classes/{wic-img.bbclass => imagetypes_wic.bbclass} (86%)
create mode 100644 meta/classes/sdk.bbclass
delete mode 100644 meta/classes/targz-img.bbclass
delete mode 100644 meta/classes/ubi-img.bbclass
delete mode 100644 meta/classes/ubifs-img.bbclass
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/README.sdk (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/configscript.sh (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/gcc-sysroot-wrapper.sh (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/relocate-sdk.sh (100%)
create mode 100644 meta/recipes-devtools/sdk-files/sdk-files.bb
delete mode 100644 meta/recipes-devtools/sdkchroot/sdkchroot.bb

--
2.30.2

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:39 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
It is no longer required, and removing it enables sharing of rootfs
data between image types.

Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
meta/classes/image.bbclass | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index eb879ffe..8899d1e9 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -1,15 +1,13 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2017 ilbers GmbH

-# Replace possible multiple spaces with single underscores
-IMAGE_SUFFIX = "${@'_'.join(d.getVar("IMAGE_FSTYPES", True).split())}"
# Make workdir and stamps machine-specific without changing common PN target
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_SUFFIX}/${PV}-${PR}"
-STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_SUFFIX}/${PV}-${PR}"
-STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_SUFFIX}/*-*"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
+STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
+STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"

# Sstate also needs to be machine-specific
-SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}-${IMAGE_SUFFIX}"
+SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"

IMAGE_INSTALL ?= ""
IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4-img"}"
@@ -29,7 +27,7 @@ INITRD_IMAGE ?= "${IMAGE_FULLNAME}-initrd.img"
DTB_FILES ?= ""

# Useful variables for imager implementations:
-PP = "/home/builder/${PN}-${MACHINE}-${IMAGE_SUFFIX}"
+PP = "/home/builder/${PN}-${MACHINE}"
PP_DEPLOY = "${PP}/deploy"
PP_ROOTFS = "${PP}/rootfs"
PP_WORK = "${PP}/work"
--
2.30.2

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:39 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
- turn the image-sdk-extension into an image recipe variant (requires
renaming of the class to `sdk` to use BBCLASSEXTEND).
- put the logic from sdkchroot into that variant
- turn the remainder of sdkcroot into the package sdk-files that
that is then installed into SDKs

API change: SDK_FORMATS now has the same semantics (and knows the same
formats) as IMAGE_FSTYPES. Because there is no image type to generate
tar.xz images, for now the default SDK_FORMAT is `targz-img`.

NOTE: this patch does not work stand-alone. It requires changes to the
way image types are handled.

Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
meta/classes/image-sdk-extension.bbclass | 87 -----------
meta/classes/image.bbclass | 2 +-
meta/classes/sdk.bbclass | 137 ++++++++++++++++++
.../{sdkchroot => sdk-files}/files/README.sdk | 0
.../files/configscript.sh | 0
.../files/gcc-sysroot-wrapper.sh | 0
.../files/relocate-sdk.sh | 0
meta/recipes-devtools/sdk-files/sdk-files.bb | 26 ++++
meta/recipes-devtools/sdkchroot/sdkchroot.bb | 78 ----------
9 files changed, 164 insertions(+), 166 deletions(-)
delete mode 100644 meta/classes/image-sdk-extension.bbclass
create mode 100644 meta/classes/sdk.bbclass
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/README.sdk (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/configscript.sh (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/gcc-sysroot-wrapper.sh (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/relocate-sdk.sh (100%)
create mode 100644 meta/recipes-devtools/sdk-files/sdk-files.bb
delete mode 100644 meta/recipes-devtools/sdkchroot/sdkchroot.bb

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
deleted file mode 100644
index 052c1b58..00000000
--- a/meta/classes/image-sdk-extension.bbclass
+++ /dev/null
@@ -1,87 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-#
-# This class extends the image.bbclass to supply the creation of a sdk
-
-SDK_INCLUDE_ISAR_APT ?= "0"
-SDK_FORMATS ?= "tar-xz"
-
-sdk_tar_xz() {
- # Copy mount_chroot.sh for convenience
- sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
-
- # Create SDK archive
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
- bbdebug 1 "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz"
-}
-
-do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-do_populate_sdk[depends] = "sdkchroot:do_build"
-do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS"
-do_populate_sdk() {
- local sdk_container_formats=""
-
- if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then
- # Copy isar-apt with deployed Isar packages
- sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt
- else
- # Remove isar-apt repo entry
- sudo rm -f ${SDKCHROOT_DIR}/etc/apt/sources.list.d/isar-apt.list
- fi
-
- sudo umount -R ${SDKCHROOT_DIR}/dev || true
- sudo umount ${SDKCHROOT_DIR}/proc || true
- sudo umount -R ${SDKCHROOT_DIR}/sys || true
-
- # Remove setup scripts
- sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh ${SDKCHROOT_DIR}/configscript.sh
-
- # Make all links relative
- for link in $(find ${SDKCHROOT_DIR}/ -type l); do
- target=$(readlink $link)
-
- if [ "${target#/}" != "${target}" ]; then
- basedir=$(dirname $link)
- new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${SDKCHROOT_DIR}/${target})
-
- # remove first to allow rewriting directory links
- sudo rm $link
- sudo ln -s $new_target $link
- fi
- done
-
- # Set up sysroot wrapper
- for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
- for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
- sudo mv "${tool}" "${tool}.bin"
- sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
- done
- done
-
- # separate SDK formats: TAR and container formats
- for sdk_format in ${SDK_FORMATS} ; do
- case ${sdk_format} in
- "tar-xz")
- sdk_tar_xz
- ;;
- "docker-archive" | "oci" | "oci-archive" | "docker-daemon" | "containers-storage")
- sdk_container_formats="${sdk_container_formats} ${sdk_format}"
- ;;
- *)
- die "unsupported SDK format specified: ${sdk_format}"
- ;;
- esac
- done
-
- # generate the SDK in all the desired container formats
- if [ -n "${sdk_container_formats}" ] ; then
- bbnote "Generating SDK container in ${sdk_container_formats} format"
- containerize_rootfs "${SDKCHROOT_DIR}" "${sdk_container_formats}" "sdk-"
- fi
-}
-
-addtask populate_sdk after do_rootfs
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8899d1e9..f87b76e7 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -78,7 +78,7 @@ ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
ROOTFS_POSTPROCESS_COMMAND_prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "

inherit rootfs
-inherit image-sdk-extension
+inherit sdk
inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
new file mode 100644
index 00000000..adf9a1fe
--- /dev/null
+++ b/meta/classes/sdk.bbclass
@@ -0,0 +1,137 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass to supply the creation of a sdk
+
+# hook up the -sdk image variant
+BBCLASSEXTEND = "sdk"
+BPN = "${PN}"
+
+python sdk_virtclass_handler() {
+ pn = e.data.getVar('PN')
+ if pn.endswith('-sdk'):
+ e.data.setVar('BPN', pn[:-len('-sdk')])
+ e.data.appendVar('OVERRIDES', ':class-sdk')
+ # sdkchroot deploy only for sdk image
+ bb.build.addtask('deploy_sdkchroot', 'do_build', 'do_rootfs', d)
+ else:
+ # add do_populate_sdk only to the non-sdk variant
+ # it only exists to preserve the interface...
+ bb.build.addtask('populate_sdk', '', '', e.data)
+ e.data.appendVarFlag('do_populate_sdk', 'depends', '${BPN}-sdk:do_build')
+ e.data.appendVarFlag('do_clean', 'depends', '${BPN}-sdk:do_clean')
+}
+addhandler sdk_virtclass_handler
+sdk_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
+
+# SDK is image-specific:
+SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${MACHINE}"
+
+# SDK settings
+SDK_INCLUDE_ISAR_APT ?= "0"
+SDK_FORMATS ?= "targz-img"
+SDK_INSTALL ?= ""
+SDK_PREINSTALL += " \
+ debhelper \
+ autotools-dev \
+ dpkg \
+ locales \
+ docbook-to-man \
+ apt \
+ automake \
+ devscripts \
+ equivs"
+
+TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
+TOOLCHAIN_${HOST_ARCH} = "build-essential"
+TOOLCHAIN_i386 = "build-essential"
+TOOLCHAIN_append_compat-arch = " crossbuild-essential-${COMPAT_DISTRO_ARCH}"
+
+# rootfs/image overrides for the SDK
+ROOTFS_ARCH_class-sdk = "${HOST_ARCH}"
+ROOTFS_DISTRO_class-sdk = "${HOST_DISTRO}"
+ROOTFS_PACKAGES_class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${SDK_INSTALL}"
+ROOTFS_FEATURES_append_class-sdk = " clean-package-cache generate-manifest export-dpkg-status"
+ROOTFS_MANIFEST_DEPLOY_DIR_class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
+ROOTFS_DPKGSTATUS_DEPLOY_DIR_class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
+
+IMAGE_FSTYPES_class-sdk = "${SDK_FORMATS}"
+
+# bitbake dependencies
+SDKDEPENDS += "sdk-files ${SDK_INSTALL}"
+SDKDEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
+DEPENDS_class-sdk = "${SDKDEPENDS}"
+
+SDKROOTFSDEPENDS = ""
+SDKROOTFSDEPENDS_class-sdk = "${BPN}:do_rootfs"
+do_rootfs_install[depends] += "${SDKROOTFSDEPENDS}"
+
+SDKROOTFSVARDEPS = ""
+SDKROOTFSVARDEPS_class-sdk = "SDK_INCLUDE_ISAR_APT"
+do_rootfs_install[vardeps] += "${SDKROOTFSVARDEPS}"
+
+# additional SDK steps
+ROOTFS_CONFIGURE_COMMAND_append_class-sdk = " ${@'rootfs_configure_isar_apt_dir' if d.getVar('SDK_INCLUDE_ISAR_APT') == '1' else ''}"
+rootfs_configure_isar_apt_dir() {
+ # Copy isar-apt instead of mounting:
+ sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
+}
+
+ROOTFS_POSTPROCESS_COMMAND_prepend_class-sdk = "sdkchroot_configscript "
+sdkchroot_configscript () {
+ sudo chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+}
+
+ROOTFS_POSTPROCESS_COMMAND_append_class-sdk = " sdkchroot_finalize"
+sdkchroot_finalize() {
+ if [ "${SDK_INCLUDE_ISAR_APT}" = "0" ]; then
+ # Remove isar-apt repo entry
+ sudo rm -f ${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list
+ fi
+
+ sudo umount -R ${ROOTFSDIR}/dev || true
+ sudo umount ${ROOTFSDIR}/proc || true
+ sudo umount -R ${ROOTFSDIR}/sys || true
+
+ # Remove setup scripts
+ sudo rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
+
+ # Make all links relative
+ for link in $(find ${ROOTFSDIR}/ -type l); do
+ target=$(readlink $link)
+
+ if [ "${target#/}" != "${target}" ]; then
+ basedir=$(dirname $link)
+ new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${ROOTFSDIR}/${target})
+
+ # remove first to allow rewriting directory links
+ sudo rm $link
+ sudo ln -s $new_target $link
+ fi
+ done
+
+ # Set up sysroot wrapper
+ for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
+ for tool in $(find ${ROOTFSDIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
+ sudo mv "${tool}" "${tool}.bin"
+ sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+ done
+ done
+}
+
+do_deploy_sdkchroot[dirs] = "${DEPLOY_DIR_SDKCHROOT}"
+do_deploy_sdkchroot() {
+ ln -Tfsr "${ROOTFSDIR}" "${SDKCHROOT_DIR}"
+}
+
+CLEANFUNCS_class-sdk = "clean_deploy"
+clean_deploy() {
+ rm -f "${SDKCHROOT_DIR}"
+}
+
+do_populate_sdk[noexec] = "1"
+do_populate_sdk() {
+ :
+}
diff --git a/meta/recipes-devtools/sdkchroot/files/README.sdk b/meta/recipes-devtools/sdk-files/files/README.sdk
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/README.sdk
rename to meta/recipes-devtools/sdk-files/files/README.sdk
diff --git a/meta/recipes-devtools/sdkchroot/files/configscript.sh b/meta/recipes-devtools/sdk-files/files/configscript.sh
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/configscript.sh
rename to meta/recipes-devtools/sdk-files/files/configscript.sh
diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdk-files/files/gcc-sysroot-wrapper.sh
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
rename to meta/recipes-devtools/sdk-files/files/gcc-sysroot-wrapper.sh
diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdk-files/files/relocate-sdk.sh
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
rename to meta/recipes-devtools/sdk-files/files/relocate-sdk.sh
diff --git a/meta/recipes-devtools/sdk-files/sdk-files.bb b/meta/recipes-devtools/sdk-files/sdk-files.bb
new file mode 100644
index 00000000..36cdb319
--- /dev/null
+++ b/meta/recipes-devtools/sdk-files/sdk-files.bb
@@ -0,0 +1,26 @@
+# SDK Root filesystem
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2018 ilbers GmbH
+
+DESCRIPTION = "Isar SDK Root filesystem"
+
+LICENSE = "gpl-2.0"
+LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
+
+inherit dpkg-raw
+
+SRC_URI = " \
+ file://configscript.sh \
+ file://relocate-sdk.sh \
+ file://gcc-sysroot-wrapper.sh \
+ file://README.sdk"
+PV = "0.1"
+
+do_install() {
+ install -m 644 ${WORKDIR}/README.sdk ${D}
+ install -m 755 ${WORKDIR}/relocate-sdk.sh ${D}
+ install -m 755 -d ${D}/usr/bin
+ install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${D}/usr/bin
+ install -m 755 ${WORKDIR}/configscript.sh ${D}
+}
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
deleted file mode 100644
index e367eae2..00000000
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ /dev/null
@@ -1,78 +0,0 @@
-# SDK Root filesystem
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2018 ilbers GmbH
-
-DESCRIPTION = "Isar SDK Root filesystem"
-
-LICENSE = "gpl-2.0"
-LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
-
-SRC_URI = " \
- file://configscript.sh \
- file://relocate-sdk.sh \
- file://gcc-sysroot-wrapper.sh \
- file://README.sdk"
-PV = "0.1"
-
-SDK_INSTALL ?= ""
-
-DEPENDS += "${SDK_INSTALL}"
-
-DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
-
-TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
-TOOLCHAIN_${HOST_ARCH} = "build-essential"
-TOOLCHAIN_i386 = "build-essential"
-TOOLCHAIN_append_compat-arch = " crossbuild-essential-${COMPAT_DISTRO_ARCH}"
-
-inherit rootfs
-ROOTFS_ARCH = "${HOST_ARCH}"
-ROOTFS_DISTRO = "${HOST_DISTRO}"
-ROOTFSDIR = "${S}"
-ROOTFS_PACKAGES = "${SDK_PREINSTALL} ${SDK_INSTALL} ${TOOLCHAIN}"
-ROOTFS_FEATURES += "clean-package-cache generate-manifest export-dpkg-status"
-ROOTFS_MANIFEST_DEPLOY_DIR = "${DEPLOY_DIR_SDKCHROOT}"
-ROOTFS_DPKGSTATUS_DEPLOY_DIR = "${DEPLOY_DIR_SDKCHROOT}"
-
-SDK_PREINSTALL += " \
- debhelper \
- autotools-dev \
- dpkg \
- locales \
- docbook-to-man \
- apt \
- automake \
- devscripts \
- equivs"
-
-SDK_INCLUDE_ISAR_APT ?= "0"
-
-S = "${WORKDIR}/rootfs"
-
-ROOTFS_CONFIGURE_COMMAND += "${@'rootfs_configure_isar_apt_dir' if d.getVar('SDK_INCLUDE_ISAR_APT') == '1' else ''}"
-rootfs_configure_isar_apt_dir() {
- # Copy isar-apt instead of mounting:
- sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
-}
-
-ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
-sdkchroot_install_files() {
- # Configure root filesystem
- sudo install -m 644 ${WORKDIR}/README.sdk ${S}
- sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
- sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
- sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
- sudo chroot ${S} /configscript.sh ${DISTRO_ARCH}
-}
-
-do_sdkchroot_deploy[dirs] = "${DEPLOY_DIR_SDKCHROOT}"
-do_sdkchroot_deploy() {
- ln -Tfsr "${ROOTFSDIR}" "${SDKCHROOT_DIR}"
-}
-addtask sdkchroot_deploy before do_build after do_rootfs
-
-CLEANFUNCS = "clean_deploy"
-clean_deploy() {
- rm -f "${SDKCHROOT_DIR}"
-}
--
2.30.2

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:42 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
Instead of per-type classes *-img.bbclass, we move to
- imagetypes.bbclass for the "simple" types and the conversions
- dedicated files for the more complex types:
- imagetypes_wic.bbclass
- imagetypes_vm.bbclass
- imagetypes_contaier.bbclass

Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
meta/classes/cpiogz-img.bbclass | 14 ---
meta/classes/ext4-img.bbclass | 15 ---
meta/classes/fit-img.bbclass | 22 -----
meta/classes/image.bbclass | 54 +++++------
meta/classes/imagetypes.bbclass | 93 +++++++++++++++++++
...g.bbclass => imagetypes_container.bbclass} | 0
.../{vm-img.bbclass => imagetypes_vm.bbclass} | 0
...wic-img.bbclass => imagetypes_wic.bbclass} | 0
meta/classes/targz-img.bbclass | 8 --
meta/classes/ubi-img.bbclass | 20 ----
meta/classes/ubifs-img.bbclass | 19 ----
11 files changed, 114 insertions(+), 131 deletions(-)
delete mode 100644 meta/classes/cpiogz-img.bbclass
delete mode 100644 meta/classes/ext4-img.bbclass
delete mode 100644 meta/classes/fit-img.bbclass
create mode 100644 meta/classes/imagetypes.bbclass
rename meta/classes/{container-img.bbclass => imagetypes_container.bbclass} (100%)
rename meta/classes/{vm-img.bbclass => imagetypes_vm.bbclass} (100%)
rename meta/classes/{wic-img.bbclass => imagetypes_wic.bbclass} (100%)
delete mode 100644 meta/classes/targz-img.bbclass
delete mode 100644 meta/classes/ubi-img.bbclass
delete mode 100644 meta/classes/ubifs-img.bbclass

diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
deleted file mode 100644
index f4c33bd9..00000000
--- a/meta/classes/cpiogz-img.bbclass
+++ /dev/null
@@ -1,14 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2020 Siemens AG
-#
-# SPDX-License-Identifier: MIT
-
-IMAGER_INSTALL_cpio += "cpio"
-CPIO_IMAGE_FORMAT ?= "newc"
-
-IMAGE_CMD_cpio() {
- ${SUDO_CHROOT} \
- sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
- /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
- ${IMAGE_FILE_CHROOT}"
-}
diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
deleted file mode 100644
index 73d1bb57..00000000
--- a/meta/classes/ext4-img.bbclass
+++ /dev/null
@@ -1,15 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
-
-IMAGER_INSTALL_ext4 += "e2fsprogs"
-
-MKE2FS_ARGS ?= "-t ext4"
-
-# Generate ext4 filesystem image
-IMAGE_CMD_ext4() {
- truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'
-
- ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
- -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
-}
-#IMAGE_CMD_ext4[vardepsexclude] = "ROOTFS_SIZE ROOTFS_EXTRA"
diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass
deleted file mode 100644
index ef65af88..00000000
--- a/meta/classes/fit-img.bbclass
+++ /dev/null
@@ -1,22 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-MKIMAGE_ARGS ??= ""
-
-FIT_IMAGE_SOURCE ??= "fitimage.its"
-
-IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"
-
-# Generate fit image
-IMAGE_CMD_fit() {
- if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
- die "FIT_IMAGE_SOURCE does not contain fitimage source file"
- fi
-
- # Create fit image using buildchroot tools
- ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
- -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
-}
-IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 91c7aaaa..26b1b0df 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -11,7 +11,6 @@ SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}

IMAGE_INSTALL ?= ""
IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4"}"
-IMAGE_CONVERSIONS = "gz xz"
IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"

KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
@@ -85,6 +84,26 @@ inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension

+# Extra space for rootfs in MB
+ROOTFS_EXTRA ?= "64"
+
+def get_rootfs_size(d):
+ import subprocess
+ rootfs_extra = int(d.getVar("ROOTFS_EXTRA", True))
+
+ output = subprocess.check_output(
+ ["sudo", "du", "-xs", "--block-size=1k", d.getVar("IMAGE_ROOTFS", True)]
+ )
+ base_size = int(output.split()[0])
+
+ return base_size + rootfs_extra * 1024
+
+python set_image_size () {
+ rootfs_size = get_rootfs_size(d)
+ d.setVar('ROOTFS_SIZE', str(rootfs_size))
+ d.setVarFlag('ROOTFS_SIZE', 'export', '1')
+}
+
def get_base_type(t, d):
bt = t
for c in d.getVar('IMAGE_CONVERSIONS').split():
@@ -120,22 +139,10 @@ IMAGE_BASETYPES = "${@get_image_basetypes(d)}"

# image types
IMAGE_CLASSES ??= ""
-IMGCLASSES = "container-img cpiogz-img ext4-img fit-img targz-img ubi-img ubifs-img vm-img wic-img"
+IMGCLASSES = "imagetypes imagetypes_wic imagetypes_vm imagetypes_container"
IMGCLASSES += "${IMAGE_CLASSES}"
inherit ${IMGCLASSES}

-# image conversions
-CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
-CONVERSION_DEPS_gz = "gzip"
-
-XZ_MEMLIMIT ?= "50%"
-XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
-XZ_THREADS[vardepvalue] = "1"
-XZ_OPTIONS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
-XZ_OPTIONS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
-CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'"
-CONVERSION_DEPS_xz = "xz-utils"
-
# hook up IMAGE_CMD_*
python() {
image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
@@ -276,19 +283,6 @@ python() {
d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
}

-# Extra space for rootfs in MB
-ROOTFS_EXTRA ?= "64"
-
-def get_rootfs_size(d):
- import subprocess
- rootfs_extra = int(d.getVar("ROOTFS_EXTRA", True))
-
- output = subprocess.check_output(
- ["sudo", "du", "-xs", "--block-size=1k", d.getVar("IMAGE_ROOTFS", True)]
- )
- base_size = int(output.split()[0])
-
- return base_size + rootfs_extra * 1024

# here we call a command that should describe your whole build system,
# this could be "git describe" or something similar.
@@ -307,12 +301,6 @@ get_build_id() {
fi
}

-python set_image_size () {
- rootfs_size = get_rootfs_size(d)
- d.setVar('ROOTFS_SIZE', str(rootfs_size))
- d.setVarFlag('ROOTFS_SIZE', 'export', '1')
-}
-
ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
image_configure_fstab[weight] = "2"
image_configure_fstab() {
diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass
new file mode 100644
index 00000000..07ad0cdb
--- /dev/null
+++ b/meta/classes/imagetypes.bbclass
@@ -0,0 +1,93 @@
+# This software is a part of ISAR.
+# Copyright (C) 2021 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+#image type: tar
+IMAGER_INSTALL_tar = "tar"
+TAR_OPTIIONS ?= ""
+
+IMAGE_CMD_tar() {
+ ${SUDO_CHROOT} tar ${TAR_OPTIONS} -cvzf \
+ ${IMAGE_FILE_CHROOT} --one-file-system -C ${PP_ROOTFS} .
+}
+
+# image type: ext4
+IMAGER_INSTALL_ext4 += "e2fsprogs"
+MKE2FS_ARGS ?= "-t ext4"
+
+IMAGE_CMD_ext4() {
+ truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'
+
+ ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
+ -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
+}
+
+# image type: cpio
+IMAGER_INSTALL_cpio += "cpio"
+CPIO_IMAGE_FORMAT ?= "newc"
+
+IMAGE_CMD_cpio() {
+ ${SUDO_CHROOT} \
+ sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
+ /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
+ ${IMAGE_FILE_CHROOT}"
+}
+
+# image type: fit
+MKIMAGE_ARGS ??= ""
+FIT_IMAGE_SOURCE ??= "fitimage.its"
+IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"
+
+IMAGE_CMD_fit() {
+ if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
+ die "FIT_IMAGE_SOURCE does not contain fitimage source file"
+ fi
+
+ ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
+ -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
+}
+IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
+
+# image type: ubifs
+IMAGER_INSTALL_ubifs += "mtd-utils"
+IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
+
+# glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
+# should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
+THIS_ISAR_CROSS_COMPILE := "${ISAR_CROSS_COMPILE}"
+ISAR_CROSS_COMPILE_armhf = "${@bb.utils.contains('IMAGE_BASETYPES', 'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"
+
+IMAGE_CMD_ubifs() {
+ ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
+ -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
+}
+
+# image type: ubi
+IMAGER_INSTALL_ubi += "mtd-utils"
+IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
+UBINIZE_CFG ??= "ubinize.cfg"
+
+IMAGE_CMD_ubi() {
+ if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
+ die "UBINIZE_CFG does not contain ubinize config file."
+ fi
+
+ ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
+ -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
+}
+IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
+
+# image conversions
+IMAGE_CONVERSIONS = "gz xz"
+
+CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
+CONVERSION_DEPS_gz = "gzip"
+
+XZ_MEMLIMIT ?= "50%"
+XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
+XZ_THREADS[vardepvalue] = "1"
+XZ_OPTIONS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
+XZ_OPTIONS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
+CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'"
+CONVERSION_DEPS_xz = "xz-utils"
diff --git a/meta/classes/container-img.bbclass b/meta/classes/imagetypes_container.bbclass
similarity index 100%
rename from meta/classes/container-img.bbclass
rename to meta/classes/imagetypes_container.bbclass
diff --git a/meta/classes/vm-img.bbclass b/meta/classes/imagetypes_vm.bbclass
similarity index 100%
rename from meta/classes/vm-img.bbclass
rename to meta/classes/imagetypes_vm.bbclass
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/imagetypes_wic.bbclass
similarity index 100%
rename from meta/classes/wic-img.bbclass
rename to meta/classes/imagetypes_wic.bbclass
diff --git a/meta/classes/targz-img.bbclass b/meta/classes/targz-img.bbclass
deleted file mode 100644
index 74d34e29..00000000
--- a/meta/classes/targz-img.bbclass
+++ /dev/null
@@ -1,8 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-IMAGE_CMD_tar() {
- sudo tar -cvzf ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz --one-file-system -C ${IMAGE_ROOTFS} .
-}
diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass
deleted file mode 100644
index 92acb6f8..00000000
--- a/meta/classes/ubi-img.bbclass
+++ /dev/null
@@ -1,20 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-UBINIZE_CFG ??= "ubinize.cfg"
-
-IMAGER_INSTALL_ubi += "mtd-utils"
-
-# Generate ubi filesystem image
-IMAGE_CMD_ubi() {
- if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
- die "UBINIZE_CFG does not contain ubinize config file."
- fi
-
- ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
- -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
-}
-IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
-IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
diff --git a/meta/classes/ubifs-img.bbclass b/meta/classes/ubifs-img.bbclass
deleted file mode 100644
index 5f1be89e..00000000
--- a/meta/classes/ubifs-img.bbclass
+++ /dev/null
@@ -1,19 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-IMAGER_INSTALL_ubifs += "mtd-utils"
-
-# glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
-# should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
-THIS_ISAR_CROSS_COMPILE := "${ISAR_CROSS_COMPILE}"
-ISAR_CROSS_COMPILE_armhf = "${@bb.utils.contains('IMAGE_BASETYPES', 'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"
-
-# Generate ubifs filesystem image
-IMAGE_CMD_ubifs() {
- # Create ubifs image using buildchroot tools
- ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
- -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
-}
-IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
--
2.30.2

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:42 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
- new image type names, basically replacing wic-img with wic, etc.
- adapt custom image type rpi-sdimg to the new scheme

Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
meta-isar/classes/rpi-sdimg.bbclass | 2 +-
meta-isar/classes/ubi-ubifs-img.bbclass | 11 ++++++++---
meta-isar/conf/machine/bananapi.conf | 2 +-
meta-isar/conf/machine/container-amd64.conf | 2 +-
meta-isar/conf/machine/de0-nano-soc.conf | 2 +-
meta-isar/conf/machine/hikey.conf | 2 +-
meta-isar/conf/machine/imx6-sabrelite.conf | 3 ++-
meta-isar/conf/machine/nanopi-neo.conf | 2 +-
meta-isar/conf/machine/phyboard-mira.conf | 3 ++-
meta-isar/conf/machine/qemuamd64.conf | 2 +-
meta-isar/conf/machine/qemuarm.conf | 2 +-
meta-isar/conf/machine/qemuarm64.conf | 2 +-
meta-isar/conf/machine/qemui386.conf | 2 +-
meta-isar/conf/machine/qemumipsel.conf | 2 +-
meta-isar/conf/machine/qemuriscv64.conf | 2 +-
meta-isar/conf/machine/rpi-common.conf | 2 +-
meta-isar/conf/machine/rpi.conf | 2 +-
meta-isar/conf/machine/sifive-fu540.conf | 2 +-
meta-isar/conf/machine/stm32mp15x.conf | 2 +-
meta-isar/conf/machine/virtualbox.conf | 2 +-
meta-isar/conf/machine/vmware.conf | 2 +-
.../conf/multiconfig/qemuamd64-buster-cpiogz.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 +-
meta-isar/recipes-core/images/isar-image-ubi.bb | 4 ++--
testsuite/cibase.py | 4 ++--
26 files changed, 37 insertions(+), 30 deletions(-)

diff --git a/meta-isar/classes/rpi-sdimg.bbclass b/meta-isar/classes/rpi-sdimg.bbclass
index 010c573c..508c4bc4 100644
--- a/meta-isar/classes/rpi-sdimg.bbclass
+++ b/meta-isar/classes/rpi-sdimg.bbclass
@@ -3,7 +3,7 @@
#
# Based on SD class from meta-raspberrypi

-inherit wic-img
+IMAGE_TYPEDEP_rpi_sdimg = "wic"

WKS_FILE ?= "rpi-sdimg"

diff --git a/meta-isar/classes/ubi-ubifs-img.bbclass b/meta-isar/classes/ubi-ubifs-img.bbclass
index 095719aa..7a917b47 100644
--- a/meta-isar/classes/ubi-ubifs-img.bbclass
+++ b/meta-isar/classes/ubi-ubifs-img.bbclass
@@ -5,6 +5,11 @@
#
# SPDX-License-Identifier: MIT

-inherit ubi-img ubifs-img fit-img
-addtask do_ubi_image after do_ubifs_image
-addtask do_ubi_image after do_fit_image
+IMAGE_TYPEDEP_ubi_ubifs = "ubi"
+IMAGE_TYPEDEP_ubi += "ubifs fit"
+
+IMAGE_CMD_ubi_ubifs() {
+ # we need to produce output (with extension .ubi-ubifs),
+ # so just create a symlink
+ ln -sf ${IMAGE_FULLNAME}.ubi ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.ubi-ubifs
+}
diff --git a/meta-isar/conf/machine/bananapi.conf b/meta-isar/conf/machine/bananapi.conf
index dd3a0667..36cda28a 100644
--- a/meta-isar/conf/machine/bananapi.conf
+++ b/meta-isar/conf/machine/bananapi.conf
@@ -9,6 +9,6 @@ KERNEL_NAME ?= "armmp"

IMAGE_INSTALL += "u-boot-script"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "bananapi"
IMAGER_INSTALL += "u-boot-sunxi"
diff --git a/meta-isar/conf/machine/container-amd64.conf b/meta-isar/conf/machine/container-amd64.conf
index 1f7669ad..e24729ff 100644
--- a/meta-isar/conf/machine/container-amd64.conf
+++ b/meta-isar/conf/machine/container-amd64.conf
@@ -4,4 +4,4 @@
# SPDX-License-Identifier: MIT

DISTRO_ARCH ?= "amd64"
-IMAGE_FSTYPES ?= "container-img"
+IMAGE_FSTYPES ?= "docker-archive.gz"
diff --git a/meta-isar/conf/machine/de0-nano-soc.conf b/meta-isar/conf/machine/de0-nano-soc.conf
index f822e24b..55343e9f 100644
--- a/meta-isar/conf/machine/de0-nano-soc.conf
+++ b/meta-isar/conf/machine/de0-nano-soc.conf
@@ -11,7 +11,7 @@ PREFERRED_PROVIDER_u-boot-de0-nano-soc = "u-boot-de0-nano-soc"
U_BOOT_CONFIG_de0-nano-soc = "socfpga_de0_nano_soc_defconfig"
U_BOOT_BIN_de0-nano-soc = "u-boot-with-spl.sfp"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "de0-nano-soc.wks.in"
IMAGER_INSTALL += "u-boot-de0-nano-soc"
IMAGER_BUILD_DEPS += "u-boot-de0-nano-soc"
diff --git a/meta-isar/conf/machine/hikey.conf b/meta-isar/conf/machine/hikey.conf
index 615d6749..7f9cccb2 100644
--- a/meta-isar/conf/machine/hikey.conf
+++ b/meta-isar/conf/machine/hikey.conf
@@ -7,7 +7,7 @@ DISTRO_ARCH = "arm64"

KERNEL_NAME ?= "mainline"

-IMAGE_FSTYPES = "wic-img"
+IMAGE_FSTYPES = "wic"
WKS_FILE ?= "hikey"
IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"

diff --git a/meta-isar/conf/machine/imx6-sabrelite.conf b/meta-isar/conf/machine/imx6-sabrelite.conf
index 5be337f9..d8cb93ad 100644
--- a/meta-isar/conf/machine/imx6-sabrelite.conf
+++ b/meta-isar/conf/machine/imx6-sabrelite.conf
@@ -6,8 +6,9 @@
MACHINE_SERIAL ?= "ttymxc0"
BAUDRATE_TTY ?= "115200"

+IMAGE_CLASSES += "ubi-ubifs-img"
MKUBIFS_ARGS := "-m 0x1000 -e 0x3e000 -c 1500"
UBINIZE_ARGS = "-vv -m 0x1000 -p 0x40000"
-IMAGE_FSTYPES ?= "ubi-ubifs-img"
+IMAGE_FSTYPES ?= "ubi-ubifs"

DTB_FILES = "imx6q-sabrelite.dtb"
diff --git a/meta-isar/conf/machine/nanopi-neo.conf b/meta-isar/conf/machine/nanopi-neo.conf
index 2475ce9d..5aa717e5 100644
--- a/meta-isar/conf/machine/nanopi-neo.conf
+++ b/meta-isar/conf/machine/nanopi-neo.conf
@@ -7,7 +7,7 @@ DISTRO_ARCH ?= "armhf"

KERNEL_NAME ?= "armmp"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "nanopi-neo.wks.in"

IMAGE_INSTALL += "u-boot-script"
diff --git a/meta-isar/conf/machine/phyboard-mira.conf b/meta-isar/conf/machine/phyboard-mira.conf
index 2d052d0c..0c6d29aa 100644
--- a/meta-isar/conf/machine/phyboard-mira.conf
+++ b/meta-isar/conf/machine/phyboard-mira.conf
@@ -6,8 +6,9 @@
MACHINE_SERIAL ?= "ttymxc0"
BAUDRATE_TTY ?= "115200"

+IMAGE_CLASSES += "ubi-ubifs-img"
MKUBIFS_ARGS := "-m 0x1000 -e 0x7e000 -c 8083"
UBINIZE_ARGS = "-vv -m 0x1000 -p 0x80000"
-IMAGE_FSTYPES ?= "ubi-ubifs-img"
+IMAGE_FSTYPES ?= "ubi-ubifs"

DTB_FILES = "imx6q-phytec-mira-rdk-nand.dtb"
diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index ccfde391..eca2628c 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -6,7 +6,7 @@ DISTRO_ARCH ?= "amd64"
KERNEL_NAME ?= "amd64"
KERNEL_NAME_ubuntu-focal ?= "generic"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "sdimage-efi"
IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"

diff --git a/meta-isar/conf/machine/qemuarm.conf b/meta-isar/conf/machine/qemuarm.conf
index 20a71ea4..06d4b34d 100644
--- a/meta-isar/conf/machine/qemuarm.conf
+++ b/meta-isar/conf/machine/qemuarm.conf
@@ -5,7 +5,7 @@ DISTRO_ARCH ?= "armhf"

KERNEL_NAME ?= "armmp"

-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"

QEMU_ROOTFS_DEV ?= "vda"

diff --git a/meta-isar/conf/machine/qemuarm64.conf b/meta-isar/conf/machine/qemuarm64.conf
index ca23cef5..b30fad47 100644
--- a/meta-isar/conf/machine/qemuarm64.conf
+++ b/meta-isar/conf/machine/qemuarm64.conf
@@ -6,7 +6,7 @@ DISTRO_ARCH ?= "arm64"
KERNEL_NAME ?= "arm64"
KERNEL_NAME_ubuntu-focal ?= "generic"

-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"

QEMU_ROOTFS_DEV ?= "vda"

diff --git a/meta-isar/conf/machine/qemui386.conf b/meta-isar/conf/machine/qemui386.conf
index 17cff1eb..ddfc077b 100644
--- a/meta-isar/conf/machine/qemui386.conf
+++ b/meta-isar/conf/machine/qemui386.conf
@@ -4,7 +4,7 @@ DISTRO_ARCH ?= "i386"

KERNEL_NAME ?= "686-pae"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "directdisk-isar"
IMAGER_INSTALL += "${SYSLINUX_BOOTLOADER_INSTALL}"

diff --git a/meta-isar/conf/machine/qemumipsel.conf b/meta-isar/conf/machine/qemumipsel.conf
index b34e46de..40f5e744 100644
--- a/meta-isar/conf/machine/qemumipsel.conf
+++ b/meta-isar/conf/machine/qemumipsel.conf
@@ -5,7 +5,7 @@ DISTRO_ARCH ?= "mipsel"

KERNEL_NAME ?= "4kc-malta"

-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"

QEMU_ROOTFS_DEV ?= "sda"

diff --git a/meta-isar/conf/machine/qemuriscv64.conf b/meta-isar/conf/machine/qemuriscv64.conf
index 5c687a03..cd04d873 100644
--- a/meta-isar/conf/machine/qemuriscv64.conf
+++ b/meta-isar/conf/machine/qemuriscv64.conf
@@ -8,7 +8,7 @@ DISTRO_ARCH ?= "riscv64"

KERNEL_NAME ?= "riscv64"

-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"

QEMU_ROOTFS_DEV ?= "vda"

diff --git a/meta-isar/conf/machine/rpi-common.conf b/meta-isar/conf/machine/rpi-common.conf
index 8d6dff9e..327ac91b 100644
--- a/meta-isar/conf/machine/rpi-common.conf
+++ b/meta-isar/conf/machine/rpi-common.conf
@@ -5,7 +5,7 @@ BOOT_SPACE ?= "81920"
MACHINE_SERIAL ?= "serial0"
BAUDRATE_TTY ?= "115200"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "rpi-sdimg"

KERNEL_TYPE = "raspios"
diff --git a/meta-isar/conf/machine/rpi.conf b/meta-isar/conf/machine/rpi.conf
index 0ab701c9..917527f0 100644
--- a/meta-isar/conf/machine/rpi.conf
+++ b/meta-isar/conf/machine/rpi.conf
@@ -4,5 +4,5 @@
MACHINE_SERIAL ?= "ttyAMA0"
BAUDRATE_TTY ?= "9600"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "rpi-sdimg"
diff --git a/meta-isar/conf/machine/sifive-fu540.conf b/meta-isar/conf/machine/sifive-fu540.conf
index d73f0475..1eb58db1 100644
--- a/meta-isar/conf/machine/sifive-fu540.conf
+++ b/meta-isar/conf/machine/sifive-fu540.conf
@@ -7,7 +7,7 @@ DISTRO_ARCH = "riscv64"

KERNEL_NAME ?= "mainline"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "sifive-fu540"
IMAGER_INSTALL += "opensbi-sifive-fu540"
IMAGER_BUILD_DEPS += "opensbi-sifive-fu540"
diff --git a/meta-isar/conf/machine/stm32mp15x.conf b/meta-isar/conf/machine/stm32mp15x.conf
index 22e88ea9..62de7dd1 100644
--- a/meta-isar/conf/machine/stm32mp15x.conf
+++ b/meta-isar/conf/machine/stm32mp15x.conf
@@ -11,7 +11,7 @@ PREFERRED_PROVIDER_u-boot-stm32mp15x = "u-boot-stm32mp15x"
U_BOOT_CONFIG_stm32mp15x = "stm32mp15_trusted_defconfig"
U_BOOT_BIN_stm32mp15x = "u-boot.stm32"

-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "stm32mp15x.wks.in"
IMAGER_INSTALL += "trusted-firmware-a-stm32mp15x optee-os-stm32mp15x u-boot-stm32mp15x"
IMAGER_BUILD_DEPS += "trusted-firmware-a-stm32mp15x optee-os-stm32mp15x u-boot-stm32mp15x"
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
index c0f65842..de3d04c3 100644
--- a/meta-isar/conf/machine/virtualbox.conf
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -12,4 +12,4 @@ WKS_FILE ?= "sdimage-efi"
IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"

VMDK_SUBFORMAT = "monolithicSparse"
-IMAGE_FSTYPES ?= "vm-img"
+IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
index 9bf1fd96..fba639b1 100644
--- a/meta-isar/conf/machine/vmware.conf
+++ b/meta-isar/conf/machine/vmware.conf
@@ -14,4 +14,4 @@ IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
OVF_TEMPLATE_FILE ?= "vm-img-vmware.ovf.tmpl"

VMDK_SUBFORMAT = "streamOptimized"
-IMAGE_FSTYPES ?= "vm-img"
+IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
index 37f89581..675d934a 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
@@ -6,4 +6,4 @@
MACHINE ?= "qemuamd64"
DISTRO ?= "debian-buster"

-IMAGE_FSTYPES = "cpiogz-img"
+IMAGE_FSTYPES = "cpio.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
index e3992e60..b48b0a6c 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
@@ -6,4 +6,4 @@
MACHINE ?= "qemuamd64"
DISTRO ?= "debian-buster"

-IMAGE_FSTYPES ?= "targz-img"
+IMAGE_FSTYPES ?= "tar.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
index 44fe8af9..79d0aa63 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
@@ -5,4 +5,4 @@ DISTRO ?= "debian-buster"
WKS_FILE ?= "efi-plus-pcbios.wks"
IMAGER_INSTALL += "${SYSLINUX_BOOTLOADER_INSTALL}"

-IMAGE_FSTYPES ?= "wic-img ext4-img"
+IMAGE_FSTYPES ?= "wic ext4"
diff --git a/meta-isar/recipes-core/images/isar-image-ubi.bb b/meta-isar/recipes-core/images/isar-image-ubi.bb
index ef2cf1c3..8c70c1f7 100644
--- a/meta-isar/recipes-core/images/isar-image-ubi.bb
+++ b/meta-isar/recipes-core/images/isar-image-ubi.bb
@@ -23,5 +23,5 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_IMAGE}"
# only one dtb file supported, pick the first
DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"

-UBIFS_IMG = "${PP_DEPLOY}/${UBIFS_IMAGE_FILE}"
-FIT_IMG = "${PP_DEPLOY}/${FIT_IMAGE_FILE}"
+UBIFS_IMG = "${PP_DEPLOY}/${IMAGE_FULLNAME}.ubifs"
+FIT_IMG = "${PP_DEPLOY}/${IMAGE_FULLNAME}.fit"
diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index 0281be41..1dd7c7a0 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -120,7 +120,7 @@ class CIBaseTest(CIBuilder):
['do_bootstrap_setscene', '!do_bootstrap']),
check_executed_tasks('buildchroot-target',
['do_rootfs_install_setscene', '!do_rootfs_install']),
- check_executed_tasks('isar-image-base-*-wic-img',
+ check_executed_tasks('isar-image-base-*',
['do_rootfs_install_setscene', '!do_rootfs_install'])
]):
self.fail("Failed rebuild image")
@@ -163,7 +163,7 @@ class CIBaseTest(CIBuilder):
# TODO: if we actually make a change to hello, then we could test
# that do_rootfs is executed. currently, hello is rebuilt,
# but its sstate sig/hash does not change.
- check_executed_tasks('isar-image-base-*-wic-img',
+ check_executed_tasks('isar-image-base-*',
['do_rootfs_install_setscene', '!do_rootfs_install'])
]):
self.fail("Failed rebuild package and image")
--
2.30.2

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:43 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
Now that we have support for building multiple image types using
IMAGE_FSTYPES, doing it via multiconfig is the wrong approach.
This removes the configs that are still using that pattern (buster-cpiogz
and buster-targz), and instead adds the building of cpio and tar
images to the qemuamd64-bullseye config.

Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
meta-isar/conf/local.conf.sample | 4 ----
.../conf/multiconfig/qemuamd64-bullseye-cpiogz.conf | 7 -------
meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf | 7 -------
meta-isar/conf/multiconfig/qemuamd64-bullseye.conf | 2 ++
meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf | 9 ---------
meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf | 9 ---------
testsuite/citest.py | 2 --
7 files changed, 2 insertions(+), 38 deletions(-)
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 71ee8579..58f3e1a2 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -52,11 +52,7 @@ BBMULTICONFIG = " \
qemui386-bookworm \
qemuamd64-stretch \
qemuamd64-buster \
- qemuamd64-buster-tgz \
- qemuamd64-buster-cpiogz \
qemuamd64-bullseye \
- qemuamd64-bullseye-tgz \
- qemuamd64-bullseye-cpiogz \
qemuamd64-bookworm \
container-amd64-stretch \
container-amd64-buster \
diff --git a/meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
deleted file mode 100644
index ff491b2c..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
+++ /dev/null
@@ -1,7 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2022 ilbers GmbH
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES = "cpiogz-img"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf b/meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
deleted file mode 100644
index 8eea112e..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
+++ /dev/null
@@ -1,7 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2022 ilbers GmbH
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES ?= "targz-img"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf b/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf
index 1badc889..13d870df 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf
@@ -2,3 +2,5 @@

MACHINE ?= "qemuamd64"
DISTRO ?= "debian-bullseye"
+
+IMAGE_FSTYPES_append = " cpio.gz tar.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
deleted file mode 100644
index 675d934a..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2020
-#
-# SPDX-License-Identifier: MIT
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES = "cpio.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
deleted file mode 100644
index b48b0a6c..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES ?= "tar.gz"
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 994c130a..69c66b1a 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -136,8 +136,6 @@ class NoCrossTest(CIBaseTest):
'mc:qemui386-bullseye:isar-image-base',
'mc:qemuamd64-buster:isar-image-base',
'mc:qemuamd64-bullseye:isar-image-base',
- 'mc:qemuamd64-bullseye-tgz:isar-image-base',
- 'mc:qemuamd64-bullseye-cpiogz:isar-image-base',
'mc:qemuamd64-bullseye:isar-initramfs',
'mc:qemumipsel-buster:isar-image-base',
'mc:qemumipsel-bullseye:isar-image-base',
--
2.30.2

Adriaan Schmidt

unread,
Apr 28, 2022, 2:06:52 AM4/28/22
to isar-...@googlegroups.com, Adriaan Schmidt
This makes a number of changes:
- Replace all do_*_image tasks with IMAGE_CMD_* definitions
- Remove all implicit conversions (gz, xz)
- Add IMAGE_CONVERSION_* commands
- Create tasks (do_image_*) on demand in image.bbclass for
all requested IMAGE_FSTYPES
- When creating the tasks:
- image_do_mounts is inserted automatically
- a final chown is inserted automatically
- variables IMAGE_FILE_HOST and IMAGE_FILE_CHROOT are
set to reference the image
- variable SUDO_CHROOT contains the chroot command
- Create conversions on demand based on IMAGE_FSTYPES.
A conversion is defined by CONVERSION_CMD_type, and its
dependencies given as CONVERSION_DEPS_type
- In conversion commands
- the input file is named ${IMAGE_FULLNAME}.${type}
- the conversions appends its own type, e.g. the output file
would be ${IMAGE_FULLNAME}.${type}.xz
- a final chown is appended automatically
- Image types now longer have a -img suffix, e.g., ext4 instead
of ext4-img, and conversions are appended like tar.gz instead
of targz-img
- Imager dependencies are set as IMAGER_INSTALL_type
- Dependencies between image types are modelled like
IMAGE_TYPEDEP_ova = "wic"
- Required arguments/variables are modelled by
IMAGE_CMD_REQUIRED_ARGUMENTS = "A B C"
- Container types (previously CONTAINER_IMAGE_FORMATS) are now
first class image types (oci, oci-archive, docker-archive,
docker-daemon, containers.storage)
- The logic of image-container-extension has moved to
container-img
- The VM image now has type ova (instead of vm-img)

Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
meta/classes/container-img.bbclass | 94 ++++++++-
meta/classes/cpiogz-img.bbclass | 20 +-
meta/classes/ext4-img.bbclass | 21 +-
meta/classes/fit-img.bbclass | 17 +-
.../classes/image-container-extension.bbclass | 83 --------
meta/classes/image.bbclass | 198 +++++++++++++++++-
meta/classes/rootfs.bbclass | 1 +
meta/classes/sdk.bbclass | 2 +-
meta/classes/targz-img.bbclass | 11 +-
meta/classes/ubi-img.bbclass | 24 +--
meta/classes/ubifs-img.bbclass | 27 +--
meta/classes/vm-img.bbclass | 24 +--
meta/classes/wic-img.bbclass | 45 ++--
13 files changed, 343 insertions(+), 224 deletions(-)
delete mode 100644 meta/classes/image-container-extension.bbclass

diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
index 322889a9..436a0051 100644
--- a/meta/classes/container-img.bbclass
+++ b/meta/classes/container-img.bbclass
@@ -3,15 +3,93 @@
#
# SPDX-License-Identifier: MIT
#
-# This class provides the task 'container_image'
+# This class provides the task 'containerize'
# to create container images containing the target rootfs.

-do_container_image[dirs] = "${DEPLOY_DIR_IMAGE}"
-do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-do_container_image[vardeps] += "CONTAINER_IMAGE_FORMATS"
-do_container_image(){
- bbdebug 1 "Generate container image in these formats: ${CONTAINER_IMAGE_FORMATS}"
- containerize_rootfs "${IMAGE_ROOTFS}" "${CONTAINER_IMAGE_FORMATS}"
+CONTAINER_TYPES = "oci oci-archive docker-archive docker-daemon containers-storage"
+USING_CONTAINER = "${@bb.utils.contains_any('IMAGE_BASETYPES', d.getVar('CONTAINER_TYPES').split(), '1', '0', d)}"
+
+CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
+CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
+
+python() {
+ if not d.getVar('USING_CONTAINER') == '1':
+ return
+ for t in d.getVar('CONTAINER_TYPES').split():
+ t_clean = t.replace('-', '_').replace('.', '_')
+ d.setVar('IMAGE_CMD_' + t_clean, 'convert_container %s "${CONTAINER_IMAGE_NAME}" "${IMAGE_FILE_HOST}"' % t)
+ d.setVar('IMAGE_FULLNAME_' + t_clean, '${PN}-${DISTRO}-${DISTRO_ARCH}')
+ bb.build.addtask('containerize', 'do_image_' + t_clean, 'do_image_tools', d)
+}
+
+do_containerize() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local tag="${CONTAINER_IMAGE_TAG}"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="${IMAGE_ROOTFS}"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ # clean-up temporary files
+ sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
+
+ # pack container image
+ bbdebug 1 "pack container image"
+ sudo umoci repack --image "${oci_img_dir}:${tag}" \
+ "${oci_img_dir}_unpacked"
+ sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
+ sudo rm -rf "${oci_img_dir}_unpacked"
+
+ # no root needed anymore
+ sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
}

-addtask container_image before do_image after do_image_tools
+convert_container() {
+ local tag="${CONTAINER_IMAGE_TAG}"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local container_type="$1"
+ local image_name="$2"
+ local image_archive="$3"
+
+ # convert the OCI container image to the desired format
+ bbdebug 1 "Creating container image type: ${container_type}"
+ case "${container_type}" in
+ "docker-archive" | "oci-archive")
+ if [ "${container_type}" = "oci-archive" ] ; then
+ target="${container_type}:${image_archive}:${tag}"
+ else
+ target="${container_type}:${image_archive}:${image_name}:${tag}"
+ fi
+ rm -f "${image_archive}"
+ bbdebug 2 "Converting OCI image to ${container_type}"
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${tag}" "${target}"
+ ;;
+ "oci")
+ tar --create --directory "${oci_img_dir}" \
+ --file "${image_archive}" .
+ ;;
+ "docker-daemon" | "containers-storage")
+ if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
+ die "Adding the container image to a container runtime (${container_type}) not supported if running from a container (e.g. 'kas-container')"
+ fi
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${tag}" \
+ "${container_type}:${image_name}:${tag}"
+ ;;
+ *)
+ die "Unsupported format for convert_container: ${container_type}"
+ ;;
+ esac
+}
diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
index 2a49456b..f4c33bd9 100644
--- a/meta/classes/cpiogz-img.bbclass
+++ b/meta/classes/cpiogz-img.bbclass
@@ -3,20 +3,12 @@
#
# SPDX-License-Identifier: MIT

-CPIOGZ_FNAME ?= "${IMAGE_FULLNAME}.cpio.gz"
-CPIOGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${CPIOGZ_FNAME}"
-IMAGER_INSTALL += "cpio"
+IMAGER_INSTALL_cpio += "cpio"
CPIO_IMAGE_FORMAT ?= "newc"

-do_cpiogz_image() {
- sudo rm -f ${CPIOGZ_IMAGE_FILE}
- image_do_mounts
- sudo chroot ${BUILDCHROOT_DIR} \
- sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
- /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o | /usr/bin/gzip -9 > \
- ${PP_DEPLOY}/${CPIOGZ_FNAME}"
- sudo chown $(id -u):$(id -g) ${CPIOGZ_IMAGE_FILE}
+IMAGE_CMD_cpio() {
+ ${SUDO_CHROOT} \
+ sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
+ /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
+ ${IMAGE_FILE_CHROOT}"
}
-
-addtask cpiogz_image before do_image after do_image_tools
-do_cpiogz_image[dirs] = "${DEPLOY_DIR_IMAGE}"
diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
index 5085afcc..73d1bb57 100644
--- a/meta/classes/ext4-img.bbclass
+++ b/meta/classes/ext4-img.bbclass
@@ -1,24 +1,15 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2017 ilbers GmbH

-EXT4_IMAGE_FILE = "${IMAGE_FULLNAME}.ext4.img"
-
-IMAGER_INSTALL += "e2fsprogs"
+IMAGER_INSTALL_ext4 += "e2fsprogs"

MKE2FS_ARGS ?= "-t ext4"

# Generate ext4 filesystem image
-do_ext4_image() {
- rm -f '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}'
-
- truncate -s ${ROOTFS_SIZE}K '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}'
+IMAGE_CMD_ext4() {
+ truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'

- image_do_mounts
-
- sudo chroot ${BUILDCHROOT_DIR} /sbin/mke2fs ${MKE2FS_ARGS} \
- -F -d '${PP_ROOTFS}' '${PP_DEPLOY}/${EXT4_IMAGE_FILE}'
+ ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
+ -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
}
-
-addtask ext4_image before do_image after do_image_tools
-do_ext4_image[prefuncs] = 'set_image_size'
-do_ext4_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+#IMAGE_CMD_ext4[vardepsexclude] = "ROOTFS_SIZE ROOTFS_EXTRA"
diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass
index 1ad0c5b8..ef65af88 100644
--- a/meta/classes/fit-img.bbclass
+++ b/meta/classes/fit-img.bbclass
@@ -6,24 +6,17 @@
MKIMAGE_ARGS ??= ""

FIT_IMAGE_SOURCE ??= "fitimage.its"
-FIT_IMAGE_FILE ?= "${IMAGE_FULLNAME}.fit.img"

-IMAGER_INSTALL += "u-boot-tools device-tree-compiler"
+IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"

# Generate fit image
-do_fit_image() {
+IMAGE_CMD_fit() {
if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
die "FIT_IMAGE_SOURCE does not contain fitimage source file"
fi

- rm -f '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}'
-
- image_do_mounts
-
# Create fit image using buildchroot tools
- sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \
- -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${PP_DEPLOY}/${FIT_IMAGE_FILE}'
- sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}'
+ ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
+ -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
}
-addtask fit_image before do_image after do_image_tools do_transform_template
-do_fit_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
deleted file mode 100644
index cdec4633..00000000
--- a/meta/classes/image-container-extension.bbclass
+++ /dev/null
@@ -1,83 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2021
-#
-# SPDX-License-Identifier: MIT
-#
-# This class extends the image.bbclass for containerizing the root filesystem.
-
-CONTAINER_IMAGE_FORMATS ?= "docker-archive"
-CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
-CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
-
-containerize_rootfs() {
- local cmd="/bin/dash"
- local empty_tag="empty"
- local tag="${CONTAINER_IMAGE_TAG}"
- local oci_img_dir="${WORKDIR}/oci-image"
- local rootfs="$1"
- local container_formats="$2"
- local container_name_prefix="$3"
-
- # prepare OCI container image skeleton
- bbdebug 1 "prepare OCI container image skeleton"
- sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
- sudo umoci init --layout "${oci_img_dir}"
- sudo umoci new --image "${oci_img_dir}:${empty_tag}"
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
- --config.cmd="${cmd}"
- sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
- "${oci_img_dir}_unpacked"
-
- # add root filesystem as the flesh of the skeleton
- sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
- # clean-up temporary files
- sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
-
- # pack container image
- bbdebug 1 "pack container image"
- sudo umoci repack --image "${oci_img_dir}:${tag}" \
- "${oci_img_dir}_unpacked"
- sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
- sudo rm -rf "${oci_img_dir}_unpacked"
-
- # no root needed anymore
- sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
-
- # convert the OCI container image to the desired format
- image_name="${container_name_prefix}${CONTAINER_IMAGE_NAME}"
- for image_type in ${CONTAINER_IMAGE_FORMATS} ; do
- image_archive="${DEPLOY_DIR_IMAGE}/${image_name}-${tag}-${image_type}.tar"
- bbdebug 1 "Creating container image type: ${image_type}"
- case "${image_type}" in
- "docker-archive" | "oci-archive")
- if [ "${image_type}" = "oci-archive" ] ; then
- target="${image_type}:${image_archive}:${tag}"
- else
- target="${image_type}:${image_archive}:${image_name}:${tag}"
- fi
- rm -f "${image_archive}" "${image_archive}.xz"
- bbdebug 2 "Converting OCI image to ${image_type}"
- skopeo --insecure-policy copy \
- "oci:${oci_img_dir}:${tag}" "${target}"
- bbdebug 2 "Compressing image"
- xz -T0 "${image_archive}"
- ;;
- "oci")
- tar --create --xz --directory "${oci_img_dir}" \
- --file "${image_archive}.xz" .
- ;;
- "docker-daemon" | "containers-storage")
- if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
- die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')"
- fi
- skopeo --insecure-policy copy \
- "oci:${oci_img_dir}:${tag}" \
- "${image_type}:${image_name}:${tag}"
- ;;
- *)
- die "Unsupported format for containerize_rootfs: ${image_type}"
- ;;
- esac
- done
-}
-
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index f87b76e7..91c7aaaa 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -10,7 +10,8 @@ STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"

IMAGE_INSTALL ?= ""
-IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4-img"}"
+IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4"}"
+IMAGE_CONVERSIONS = "gz xz"
IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"

KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
@@ -83,7 +84,197 @@ inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
-inherit image-container-extension
+
+def get_base_type(t, d):
+ bt = t
+ for c in d.getVar('IMAGE_CONVERSIONS').split():
+ if t.endswith('.' + c):
+ bt = t[:-len('.' + c)]
+ break
+ return bt if bt == t else get_base_type(bt, d)
+
+# Calculate IMAGE_BASETYPES as list of all image types that need to be built,
+# also due to dependencies, but withoug any conversions.
+# This is only for use in imagetype classes, e.g., for conditional expressions
+# in the form of "${@bb.utils.contains('IMAGE_BASETYPES', type, a, b, d)}"
+# All this dependency resolution (including conversions) is then done again
+# below when the actual image tasks are constructed.
+def get_image_basetypes(d):
+ def recurse(t):
+ bt = get_base_type(t, d)
+ if bt.endswith('-img'):
+ # be backwards-compatible
+ bt = bt[:-len('-img')]
+ bb.warn("IMAGE_TYPE '{0}-img' is deprecated. Please use '{0}' instead.".format(bt))
+ deps = (d.getVar('IMAGE_TYPEDEP_' + bt.replace('-', '_').replace('.', '_')) or '').split()
+ ret = set([bt])
+ for dep in deps:
+ ret |= recurse(dep)
+ return ret
+ basetypes = set()
+ for t in (d.getVar('IMAGE_FSTYPES') or '').split():
+ basetypes |= recurse(t)
+ return ' '.join(list(basetypes))
+
+IMAGE_BASETYPES = "${@get_image_basetypes(d)}"
+
+# image types
+IMAGE_CLASSES ??= ""
+IMGCLASSES = "container-img cpiogz-img ext4-img fit-img targz-img ubi-img ubifs-img vm-img wic-img"
+IMGCLASSES += "${IMAGE_CLASSES}"
+inherit ${IMGCLASSES}
+
+# image conversions
+CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
+CONVERSION_DEPS_gz = "gzip"
+
+XZ_MEMLIMIT ?= "50%"
+XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
+XZ_THREADS[vardepvalue] = "1"
+XZ_OPTIONS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
+XZ_OPTIONS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
+CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'"
+CONVERSION_DEPS_xz = "xz-utils"
+
+# hook up IMAGE_CMD_*
+python() {
+ image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
+ conversions = set(d.getVar('IMAGE_CONVERSIONS').split())
+
+ basetypes = {}
+ typedeps = {}
+ vardeps = set()
+
+ def collect_image_type(t):
+ bt = get_base_type(t, d)
+ if bt.endswith('-img'):
+ # be backwards-compatible
+ bt = bt[:-len('-img')]
+ bb.warn("IMAGE_TYPE '{0}-img' is deprecated. Please use '{0}' instead.".format(bt))
+
+ if bt not in basetypes:
+ basetypes[bt] = []
+ if t not in basetypes[bt]:
+ basetypes[bt].append(t)
+ t_clean = t.replace('-', '_').replace('.', '_')
+ deps = (d.getVar('IMAGE_TYPEDEP_' + t_clean) or '').split()
+ vardeps.add('IMAGE_TYPEDEP_' + t_clean)
+ if bt not in typedeps:
+ typedeps[bt] = set()
+ for dep in deps:
+ if dep not in image_types:
+ image_types.append(dep)
+ collect_image_type(dep)
+ typedeps[bt].add(get_base_type(dep, d))
+ if bt != t:
+ collect_image_type(bt)
+
+ for t in image_types[:]:
+ collect_image_type(t)
+
+ # TODO: OE uses do_image, but Isar is different...
+ d.appendVarFlag('do_image_tools', 'vardeps', ' '.join(vardeps))
+
+ imager_install = set()
+ imager_build_deps = set()
+ conversion_install = set()
+ for bt in basetypes:
+ vardeps = set()
+ cmds = []
+ bt_clean = bt.replace('-', '_').replace('.', '_')
+
+ # prepare local environment
+ localdata = bb.data.createCopy(d)
+ localdata.setVar('OVERRIDES', bt_clean + ':' + d.getVar('OVERRIDES', False))
+ localdata.setVar('PV', d.getVar('PV'))
+ localdata.delVar('DATETIME')
+ localdata.delVar('DATE')
+ localdata.delVar('TMPDIR')
+ vardepsexclude = (d.getVarFlag('IMAGE_CMD_' + bt_clean, 'vardepsexclude', True) or '').split()
+ for dep in vardepsexclude:
+ localdata.delVar(dep)
+
+ # check if required args are set
+ required_args = (localdata.getVar('IMAGE_CMD_REQUIRED_ARGS') or '').split()
+ if any([d.getVar(arg) is None for arg in required_args]):
+ bb.fatal("IMAGE_TYPE '%s' requires these arguments: %s" % (image_type, ', '.join(required_args)))
+
+ # convenience variables to be used by CMDs
+ localdata.setVar('IMAGE_FILE_HOST', '${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.${type}')
+ #bb.warn("FULLNAME is %s -> %s" % (localdata.getVar('IMAGE_FULLNAME', False), localdata.getVar('IMAGE_FULLNAME', True)))
+ localdata.setVar('IMAGE_FILE_CHROOT', '${PP_DEPLOY}/${IMAGE_FULLNAME}.${type}')
+ localdata.setVar('SUDO_CHROOT', localdata.expand('sudo chroot ${BUILDCHROOT_DIR}'))
+
+ # imager install
+ for dep in (d.getVar('IMAGER_INSTALL_' + bt_clean) or '').split():
+ imager_install.add(dep)
+ for dep in (d.getVar('IMAGER_BUILD_DEPS_' + bt_clean) or '').split():
+ imager_build_deps.add(dep)
+
+ # construct image command
+ cmds.append('\timage_do_mounts')
+ image_cmd = localdata.getVar('IMAGE_CMD_' + bt_clean)
+ if image_cmd:
+ localdata.setVar('type', bt)
+ cmds.append(localdata.expand(image_cmd))
+ #bb.warn("IMAGE_CMD\n*** %s\n*** %s" % (image_cmd, localdata.expand(image_cmd)))
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
+ else:
+ bb.fatal("No IMAGE_CMD for %s" % bt)
+ vardeps.add('IMAGE_CMD_' + bt_clean)
+ d.delVarFlag('IMAGE_CMD_' + bt_clean, 'func')
+ task_deps = d.getVarFlag('IMAGE_CMD_' + bt_clean, 'depends')
+
+ # add conversions
+ conversion_depends = set()
+ rm_images = set()
+ def create_conversions(t):
+ for c in sorted(conversions):
+ if t.endswith('.' + c):
+ t = t[:-len(c) - 1]
+ create_conversions(t)
+ localdata.setVar('type', t)
+ cmd = '\t' + localdata.getVar('CONVERSION_CMD_' + c)
+ if cmd not in cmds:
+ cmds.append(cmd)
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
+ vardeps.add('CONVERSION_CMD_' + c)
+ for dep in (localdata.getVar('CONVERSION_DEPS_' + c) or '').split():
+ conversion_install.add(dep)
+ # remove temporary image files
+ if t not in image_types:
+ rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
+
+ for t in basetypes[bt]:
+ create_conversions(t)
+
+ if bt not in image_types:
+ localdata.setVar('type', t)
+ rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
+
+ for image in rm_images:
+ cmds.append('\trm ' + image)
+
+ # image type dependencies
+ after = 'do_image_tools'
+ for dep in typedeps[bt]:
+ after += ' do_image_%s' % dep.replace('-', '_').replace('.', '_')
+
+ # create the task
+ task = 'do_image_%s' % bt_clean
+ d.setVar(task, '\n'.join(cmds))
+ d.setVarFlag(task, 'func', '1')
+ d.appendVarFlag(task, 'prefuncs', ' set_image_size')
+ d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps))
+ d.appendVarFlag(task, 'vardepsexclude', ' ' + ' '.join(vardepsexclude))
+ d.appendVarFlag(task, 'dirs', localdata.expand(' ${DEPLOY_DIR_IMAGE}'))
+ if task_deps:
+ d.appendVarFlag(task, 'depends', task_deps)
+ bb.build.addtask(task, 'do_image', after, d)
+
+ d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
+ d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
+}

# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
@@ -256,6 +447,3 @@ do_rootfs_quality_check() {
}

addtask rootfs_quality_check after do_rootfs_finalize before do_rootfs
-
-# Last so that the image type can overwrite tasks if needed
-inherit ${IMAGE_FSTYPES}
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index b021e728..7e9fb0f3 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -172,6 +172,7 @@ rootfs_install_pkgs_install() {

do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}"
+do_rootfs_install[vardepsexclude] += "IMAGE_ROOTFS"
do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build"
do_rootfs_install[recrdeptask] = "do_deploy_deb"
python do_rootfs_install() {
diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
index adf9a1fe..477dff70 100644
--- a/meta/classes/sdk.bbclass
+++ b/meta/classes/sdk.bbclass
@@ -31,7 +31,7 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${MACHINE}"

# SDK settings
SDK_INCLUDE_ISAR_APT ?= "0"
-SDK_FORMATS ?= "targz-img"
+SDK_FORMATS ?= "tar.xz"
SDK_INSTALL ?= ""
SDK_PREINSTALL += " \
debhelper \
diff --git a/meta/classes/targz-img.bbclass b/meta/classes/targz-img.bbclass
index bf94af02..74d34e29 100644
--- a/meta/classes/targz-img.bbclass
+++ b/meta/classes/targz-img.bbclass
@@ -3,13 +3,6 @@
#
# SPDX-License-Identifier: MIT

-TARGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz"
-
-do_targz_image() {
- rm -f ${TARGZ_IMAGE_FILE}
- sudo tar -cvzf ${TARGZ_IMAGE_FILE} --one-file-system -C ${IMAGE_ROOTFS} .
- sudo chown $(id -u):$(id -g) ${TARGZ_IMAGE_FILE}
+IMAGE_CMD_tar() {
+ sudo tar -cvzf ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz --one-file-system -C ${IMAGE_ROOTFS} .
}
-
-addtask targz_image before do_image after do_image_tools
-do_targz_image[dirs] = "${DEPLOY_DIR_IMAGE}"
diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass
index efaf058e..92acb6f8 100644
--- a/meta/classes/ubi-img.bbclass
+++ b/meta/classes/ubi-img.bbclass
@@ -3,30 +3,18 @@
#
# SPDX-License-Identifier: MIT

-python() {
- if not d.getVar("UBINIZE_ARGS"):
- raise bb.parse.SkipRecipe("UBINIZE_ARGS must be set")
-}
-
UBINIZE_CFG ??= "ubinize.cfg"
-UBI_IMAGE_FILE ?= "${IMAGE_FULLNAME}.ubi.img"

-IMAGER_INSTALL += "mtd-utils"
+IMAGER_INSTALL_ubi += "mtd-utils"

# Generate ubi filesystem image
-do_ubi_image() {
+IMAGE_CMD_ubi() {
if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
die "UBINIZE_CFG does not contain ubinize config file."
fi

- rm -f '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}'
-
- image_do_mounts
-
- # Create ubi image using buildchroot tools
- sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \
- -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}' '${PP_WORK}/${UBINIZE_CFG}'
- sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}'
+ ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
+ -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
}
-addtask ubi_image before do_image after do_image_tools do_transform_template
-do_ubi_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
+IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
diff --git a/meta/classes/ubifs-img.bbclass b/meta/classes/ubifs-img.bbclass
index 229eb3ef..5f1be89e 100644
--- a/meta/classes/ubifs-img.bbclass
+++ b/meta/classes/ubifs-img.bbclass
@@ -3,30 +3,17 @@
#
# SPDX-License-Identifier: MIT

-python() {
- if not d.getVar("MKUBIFS_ARGS"):
- raise bb.parse.SkipRecipe("mkubifs_args must be set")
-}
-
-UBIFS_IMAGE_FILE ?= "${IMAGE_FULLNAME}.ubifs.img"
-
-IMAGER_INSTALL += "mtd-utils"
+IMAGER_INSTALL_ubifs += "mtd-utils"
# should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
-ISAR_CROSS_COMPILE_armhf = "1"
+THIS_ISAR_CROSS_COMPILE := "${ISAR_CROSS_COMPILE}"
+ISAR_CROSS_COMPILE_armhf = "${@bb.utils.contains('IMAGE_BASETYPES', 'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"

# Generate ubifs filesystem image
-do_ubifs_image() {
- rm -f '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}'
-
- image_do_mounts
-
+IMAGE_CMD_ubifs() {
# Create ubifs image using buildchroot tools
- sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
- -r '${PP_ROOTFS}' '${PP_DEPLOY}/${UBIFS_IMAGE_FILE}'
- sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}'
+ ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
+ -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
}
-
-addtask ubifs_image before do_image after do_image_tools
-do_ubifs_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass
index 4bc977b9..8a676aca 100644
--- a/meta/classes/vm-img.bbclass
+++ b/meta/classes/vm-img.bbclass
@@ -5,16 +5,18 @@
#

inherit buildchroot
-inherit wic-img
+
+USING_OVA = "${@bb.utils.contains('IMAGE_BASETYPES', 'ova', '1', '0', d)}"

FILESEXTRAPATHS_prepend := "${LAYERDIR_core}/classes/vm-img:"
OVF_TEMPLATE_FILE ?= "vm-img-virtualbox.ovf.tmpl"
-SRC_URI += "file://${OVF_TEMPLATE_FILE}"
+SRC_URI += "${@'file://${OVF_TEMPLATE_FILE}' if d.getVar('USING_OVA') == '1' else ''}"

-IMAGER_INSTALL += "qemu-utils gawk uuid-runtime"
+IMAGE_TYPEDEP_ova = "wic"
+IMAGER_INSTALL_ova += "qemu-utils gawk uuid-runtime"

# virtual machine disk settings
-SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img"
+SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic"

# For VirtualBox, this needs to be "monolithicSparse" (default to it).
# VMware needs this to be "streamOptimized".
@@ -34,7 +36,7 @@ def set_convert_options(d):

CONVERSION_OPTIONS = "${@set_convert_options(d)}"

-do_convert_wic() {
+convert_wic() {
rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
image_do_mounts
bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${SOURCE_IMAGE_FILE}"
@@ -43,8 +45,6 @@ do_convert_wic() {
'${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${VIRTUAL_MACHINE_DISK}'
}

-addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template
-
# User settings for OVA
OVA_NAME ?= "${IMAGE_FULLNAME}"
OVA_MEMORY ?= "8192"
@@ -67,10 +67,11 @@ OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \
OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL \
OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE"

-TEMPLATE_FILES += "${OVF_TEMPLATE_FILE}"
+TEMPLATE_FILES += "${@'${OVF_TEMPLATE_FILE}' if d.getVar('USING_OVA') == '1' else ''}"
TEMPLATE_VARS += "${OVA_VARS}"

-do_create_ova() {
+do_image_ova[prefuncs] += "convert_wic"
+IMAGE_CMD_ova() {
if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then
exit 0
fi
@@ -81,10 +82,7 @@ do_create_ova() {
export PRIMARY_MAC=$(macgen)
export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
export OVA_FIRMWARE_UPPERCASE=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]')
-
export OVF_TEMPLATE_STAGE2=$(echo ${OVF_TEMPLATE_FILE} | sed 's/.tmpl$//' )
- image_do_mounts
-
sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO'
set -e
export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \
@@ -104,5 +102,3 @@ do_create_ova() {
tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE}
EOSUDO
}
-
-addtask do_create_ova after do_convert_wic before do_deploy
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 7537a27b..cfcc94c7 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -4,7 +4,8 @@
# this class is heavily inspired by OEs ./meta/classes/image_types_wic.bbclass
#

-WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}')}"
+USING_WIC = "${@bb.utils.contains('IMAGE_BASETYPES', 'wic', '1', '0', d)}"
+WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if d.getVar('USING_WIC') == '1' else ''}"

WKS_FILE ??= "sdimage-efi"

@@ -14,6 +15,9 @@ do_copy_wks_template () {
}

python () {
+ if not d.getVar('USING_WIC') == '1':
+ return
+
import itertools
import re

@@ -74,13 +78,13 @@ python () {
except (IOError, OSError) as exc:
pass
else:
- bb.build.addtask('do_copy_wks_template', 'do_transform_template do_wic_image', None, d)
- bb.build.addtask('do_transform_template', 'do_wic_image', None, d)
+ bb.build.addtask('do_copy_wks_template', 'do_transform_template do_image_wic', None, d)
+ bb.build.addtask('do_transform_template', 'do_image_wic', None, d)
}

inherit buildchroot

-IMAGER_INSTALL += "${WIC_IMAGER_INSTALL}"
+IMAGER_INSTALL_wic += "${WIC_IMAGER_INSTALL}"
# wic comes with reasonable defaults, and the proper interface is the wks file
ROOTFS_EXTRA ?= "0"

@@ -125,32 +129,23 @@ python do_rootfs_wicenv () {

}

-addtask do_rootfs_wicenv after do_rootfs before do_wic_image
+addtask do_rootfs_wicenv after do_rootfs before do_image_wic
do_rootfs_wicenv[vardeps] += "${WICVARS}"
do_rootfs_wicenv[prefuncs] = 'set_image_size'

-WIC_IMAGE_FILE ="${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.img"
-
-python check_for_wic_warnings() {
- with open("{}/log.do_wic_image".format(d.getVar("T"))) as f:
- for line in f.readlines():
- if line.startswith("WARNING"):
- bb.warn(line.strip())
+check_for_wic_warnings() {
+ WARN="$(grep -e '^WARNING' ${T}/log.do_image_wic || true)"
+ if [ -n "$WARN" ]; then
+ bbwarn "$WARN"
+ fi
}

-do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-do_wic_image[dirs] = "${DEPLOY_DIR_IMAGE}"
-python do_wic_image() {
- cmds = ['wic_do_mounts', 'generate_wic_image', 'check_for_wic_warnings']
- weights = [5, 90, 5]
- progress_reporter = bb.progress.MultiStageProgressReporter(d, weights)
-
- for cmd in cmds:
- progress_reporter.next_stage()
- bb.build.exec_func(cmd, d)
- progress_reporter.finish()
+do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
+IMAGE_CMD_wic() {
+ wic_do_mounts
+ generate_wic_image
+ check_for_wic_warnings
}
-addtask wic_image before do_image after do_image_tools

wic_do_mounts() {
buildchroot_do_mounts
@@ -209,7 +204,7 @@ generate_wic_image() {
sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
find ${BUILDCHROOT_DIR}/${WICTMP} -type f -name "*.direct*" | while read f; do
suffix=$(basename $f | sed 's/\(.*\)\(\.direct\)\(.*\)/\3/')
- mv -f ${f} ${WIC_IMAGE_FILE}${suffix}
+ mv -f ${f} "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic${suffix}"
done
rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
rm -rf ${IMAGE_ROOTFS}/../pseudo
--
2.30.2

Anton Mikanovich

unread,
May 4, 2022, 3:51:12 AM5/4/22
to Adriaan Schmidt, isar-...@googlegroups.com
Applied to next, thanks.

Bezdeka, Florian

unread,
May 4, 2022, 4:09:21 AM5/4/22
to ami...@ilbers.de, Schmidt, Adriaan, isar-...@googlegroups.com
[snip]

> >
> >
> Applied to next, thanks.
>

My understanding is that all downstream layers have to update
IMAGE_FSTYPES (mainly remove "-img"). Isn't that something that should
make it into a changelog? Maybe into the "recipe API changelog"?

Regards,
Florian

Jan Kiszka

unread,
May 4, 2022, 7:33:05 AM5/4/22
to Bezdeka, Florian, ami...@ilbers.de, Schmidt, Adriaan, isar-...@googlegroups.com
Yes, please, that will help downstream a lot.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

Schmidt, Adriaan

unread,
May 4, 2022, 8:19:19 AM5/4/22
to jan.k...@siemens.com, Bezdeka, Florian, ami...@ilbers.de, isar-...@googlegroups.com
Kiszka, Jan (T CED), 4. Mai 2022 13:33:
I just posted a proposal. It also has information on what has changed when
defining new image classes. Not sure if that is also part of the recipe API...

Adriaan

Reply all
Reply to author
Forward
0 new messages