[PATCH V2 0/2] Add support for containerized root filesystems
74 views
Skip to first unread message
Silvano Cirujano Cuesta
Feb 9, 2021, 9:10:09 AM2/9/21
to isar-...@googlegroups.com
This patch series provides support for containerized root filesystems,
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized rootfilesystems `IMAGE_TYPE` as to be
`container-img`, additionally the container image format can be selected
with the variable `CONTAINER_FORMAT`. The default format is
`docker-archive`.
More information about its usage is documented in the file
docs/user_manual.md.
A PoC/demo of this functionality (only the SDK part) has been created
based on the project https://github.com/siemens/meta-iot2050.
Jan Kiszka already tested and liked it! =>
https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845
In order to get a feeling about its usage (you need Docker or Podman),
follow these simple copy&paste instructions:
https://github.com/Silvanoc/meta-iot2050/blob/master/kas/BUILDING-SDK-CONTAINER.md#running-the-sdk
Build instructions are available in the upper part of that document.
Two new dependencies are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
https://github.com/siemens/kas (for `kas-container`, for example).
Silvano Cirujano Cuesta (2):
images: add support for container images
docs: document creation of container images
doc/user_manual.md | 127 +++++++++++++++++++++++++++++
meta/classes/container-img.bbclass | 99 ++++++++++++++++++++++
2 files changed, 226 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
--
2.30.0
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized rootfilesystems `IMAGE_TYPE` as to be
`container-img`, additionally the container image format can be selected
with the variable `CONTAINER_FORMAT`. The default format is
`docker-archive`.
More information about its usage is documented in the file
docs/user_manual.md.
A PoC/demo of this functionality (only the SDK part) has been created
based on the project https://github.com/siemens/meta-iot2050.
Jan Kiszka already tested and liked it! =>
https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845
In order to get a feeling about its usage (you need Docker or Podman),
follow these simple copy&paste instructions:
https://github.com/Silvanoc/meta-iot2050/blob/master/kas/BUILDING-SDK-CONTAINER.md#running-the-sdk
Build instructions are available in the upper part of that document.
Two new dependencies are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
https://github.com/siemens/kas (for `kas-container`, for example).
Silvano Cirujano Cuesta (2):
images: add support for container images
docs: document creation of container images
doc/user_manual.md | 127 +++++++++++++++++++++++++++++
meta/classes/container-img.bbclass | 99 ++++++++++++++++++++++
2 files changed, 226 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
--
2.30.0
Silvano Cirujano Cuesta
Feb 9, 2021, 9:10:10 AM2/9/21
to isar-...@googlegroups.com
Add support for creation of container images with the build root
filesystems.
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cir...@siemens.com>
---
meta/classes/container-img.bbclass | 99 ++++++++++++++++++++++++++++++
1 file changed, 99 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
index 0000000..85d9fb4
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,99 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk'
+# to create container images containing the target rootfs and the SDK
+# respectively.
+
+CONTAINER_FORMAT ?= "docker-archive"
+
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ rm -rf "${oci_img_dir}"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+
+ # pack container image
+ bbdebug 1 "pack container image"
+ sudo umoci repack --image "${oci_img_dir}:${full_tag}" \
+ "${oci_img_dir}_unpacked"
+ sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
+ sudo rm -rf "${oci_img_dir}_unpacked"
+
+ # no root needed anymore
+ sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
+
+ # convert the OCI container image to the desired format
+ image_name="isar-${rootfs_id}"
+ for image_type in ${CONTAINER_FORMAT} ; do
+ image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar"
+ bbdebug 1 "Creating container image type: ${image_type}"
+ case "${image_type}" in
+ "docker-archive" | "oci-archive")
+ if [ "${image_type}" = "oci-archive" ] ; then
+ target="${image_type}:${image_archive}:latest"
+ else
+ target="${image_type}:${image_archive}:${image_name}:latest"
+ fi
+ rm -f "${image_archive}" "${image_archive}.xz"
+ bbdebug 2 "Converting OCI image to ${image_type}"
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" "${target}"
+ bbdebug 2 "Compressing image"
+ xz -T0 "${image_archive}"
+ ;;
+ "oci")
+ tar --create --xz --directory "${oci_img_dir}" \
+ --file "${image_archive}.xz" .
+ ;;
+ "docker-daemon" | "containers-storage")
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" \
+ "${image_type}:${image_name}:latest"
+ ;;
+ *)
+ die "Unsupported format for containerize_rootfs: ${image_type}"
+ ;;
+ esac
+ done
+}
+
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMAT"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbnote "Generate container image in these formats: ${CONTAINER_FORMAT}"
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}"
+}
+
+addtask container_image before do_image after do_image_tools
+
+do_container_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_sdk[vardeps] += "CONTAINER_FORMAT"
+do_container_sdk(){
+ rootfs_id="sdk-${DISTRO}-${DISTRO_ARCH}"
+
+ bbnote "Generate containerized SDK in these formats: ${CONTAINER_FORMAT}"
+ containerize_rootfs "${SDKCHROOT_DIR}" "${rootfs_id}"
+}
+
+addtask container_sdk after do_populate_sdk
+
--
2.30.0
filesystems.
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cir...@siemens.com>
---
meta/classes/container-img.bbclass | 99 ++++++++++++++++++++++++++++++
1 file changed, 99 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
index 0000000..85d9fb4
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,99 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk'
+# to create container images containing the target rootfs and the SDK
+# respectively.
+
+CONTAINER_FORMAT ?= "docker-archive"
+
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ rm -rf "${oci_img_dir}"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+
+ # pack container image
+ bbdebug 1 "pack container image"
+ sudo umoci repack --image "${oci_img_dir}:${full_tag}" \
+ "${oci_img_dir}_unpacked"
+ sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
+ sudo rm -rf "${oci_img_dir}_unpacked"
+
+ # no root needed anymore
+ sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
+
+ # convert the OCI container image to the desired format
+ image_name="isar-${rootfs_id}"
+ for image_type in ${CONTAINER_FORMAT} ; do
+ image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar"
+ bbdebug 1 "Creating container image type: ${image_type}"
+ case "${image_type}" in
+ "docker-archive" | "oci-archive")
+ if [ "${image_type}" = "oci-archive" ] ; then
+ target="${image_type}:${image_archive}:latest"
+ else
+ target="${image_type}:${image_archive}:${image_name}:latest"
+ fi
+ rm -f "${image_archive}" "${image_archive}.xz"
+ bbdebug 2 "Converting OCI image to ${image_type}"
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" "${target}"
+ bbdebug 2 "Compressing image"
+ xz -T0 "${image_archive}"
+ ;;
+ "oci")
+ tar --create --xz --directory "${oci_img_dir}" \
+ --file "${image_archive}.xz" .
+ ;;
+ "docker-daemon" | "containers-storage")
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" \
+ "${image_type}:${image_name}:latest"
+ ;;
+ *)
+ die "Unsupported format for containerize_rootfs: ${image_type}"
+ ;;
+ esac
+ done
+}
+
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMAT"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbnote "Generate container image in these formats: ${CONTAINER_FORMAT}"
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}"
+}
+
+addtask container_image before do_image after do_image_tools
+
+do_container_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_sdk[vardeps] += "CONTAINER_FORMAT"
+do_container_sdk(){
+ rootfs_id="sdk-${DISTRO}-${DISTRO_ARCH}"
+
+ bbnote "Generate containerized SDK in these formats: ${CONTAINER_FORMAT}"
+ containerize_rootfs "${SDKCHROOT_DIR}" "${rootfs_id}"
+}
+
+addtask container_sdk after do_populate_sdk
+
--
2.30.0
Silvano Cirujano Cuesta
Feb 9, 2021, 9:10:13 AM2/9/21
to isar-...@googlegroups.com
doc/user_manual.md | 127 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 127 insertions(+)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index a4f3d1d..6b72a16 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -19,6 +19,7 @@ Copyright (C) 2016-2019, ilbers GmbH
- [Add a Custom Application](#add-a-custom-application)
- [Enabling Cross-compilation](#isar-cross-compilation)
- [Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)
+ - [Create a containerized ISAR SDK root filesystem](#create-a-containerized-isar-sdk-root-filesystem)
- [Creation of local apt repo caching upstream Debian packages](#creation-of-local-apt-repo-caching-upstream-debian-packages)
@@ -84,6 +85,9 @@ If your host is >= buster, also install the following package.
apt install python3-distutils
```
+If you want to generate containerized SDKs, also install the following packages: `umoci` and `skopeo`.
+Umoci is provided by Debian Buster and can be installed with `apt install umoci`, Skopeo is provided by Debian Bullseye/Unstable and has to be installed either manually downloading the DEB and installing it (no other packages required) or with `apt install -t bullseye skopeo` (if unstable/bullseye included in `/etc/apt/sources.list[.d]`).
+
Notes:
* BitBake requires Python 3.4+.
@@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
qemu-system-i386 -m 256M -nographic -hda tmp/deploy/images/qemui386/isar-image-base-debian-buster-qemui386.wic.img
```
+### Generate container image with root-filesystem
+
+A runnable container image is generated if you set IMAGE_TYPE to 'container-img'.
+Getting a container image can be the main purpose of an ISAR configuration, but not only.
+A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
+
+Container images can be generated in different formats, selected with the variable `CONTAINER_FORMAT`. One or more (whitespace separated) of following options can be given:
+ - `docker-archive`: (default) an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+Following formats don't work if running `bitbake ...` (to build the image) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting container image archives (only for `docker-archive` and `oci-archive`) are made available as `tmp/deploy/images/${MACHINE}/${DISTRO}-${DISTRO_ARCH}-${container_format}.tar.xz` (being `container_format` each one of the formats specified in `CONTAINER_FORMAT`).
+
+### Example
+
+ - Make the relevant environment variables available to the task
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_FORMAT"
+export IMAGE_TYPE="container-img"
+export CONTAINER_FORMAT="docker-archive"
+```
+
+ - Trigger creation of container image from root filesystem
+
+```
+bitbake mc:qemuarm-buster:isar-image-base
+```
+
+ - Load the container image into the Docker Daemon
+
+```
+xzcat build/tmp/deploy/images/qemuarm/debian-buster-armhf-docker-archive.tar.xz | docker load
+```
+
+ - Run a container using the container image (following commands starting with `#~:` are to be run in the container)
+
+```
+docker run --rm -ti --volume "$(pwd):/build" isar-buster-armhf:latest
+```
+
---
## Terms and Definitions
@@ -834,6 +886,81 @@ ii crossbuild-essential-armhf 12.3 all Inf
~#
```
+## Create a containerized ISAR SDK root filesystem
+
+### Motivation
+
+Distributing and using the SDK root filesystem created following the instructions in "[Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)" becomes easier using container images (at least for those using containers anyway)
+A "containerized" SDK adds to those advantages of a normal SDK root filesystem the comfort of container images.
+
+### Approach
+
+Create container image with SDK root filesystem with installed cross-toolchain for target architecture and ability to install already prebuilt target binary artifacts.
+Developer:
+ - runs a container based on the resulting container image mounting the source code to be built,
+ - develops applications for target platform on the container and
+ - leaves the container getting the results on the mounted directory.
+
+### Solution
+
+A container image containing the SDK is generated if you set IMAGE_TYPE to 'container-img'.
+
+Container images can be generated in different formats, selected with the variable `CONTAINER_FORMAT`. One or more (space separated) of following options can be given:
+ - `docker-archive`: (default) an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+User manually triggers creation of SDK formats for his target platform by launching the task `do_populate_sdk` for target image, f.e.
+`bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
+Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
+
+Following formats don't work if running `bitbake -c do_populate_sdk ...` (to generate the containerized SDK) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting SDK formats are archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_format}.tar.xz` (being `sdk_format` each one of the formats specified in `CONTAINER_FORMAT`).
+The SDK container directory `/isar-apt` contains a copy of isar-apt repo with locally prebuilt target debian packages (for <HOST_DISTRO>).
+One may get into an SDK container and install required target packages with the help of `apt-get install <package_name>:<DISTRO_ARCH>` command.
+The directory with the source code to develop on should be mounted on the container (with `--volume <host-directory>:<container-directory>`) to be able to edit files in the host with an IDE and build in the container.
+
+### Example
+
+ - Make the relevant environment variables available to the task
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_FORMAT"
+export IMAGE_TYPE="container-img"
+export CONTAINER_FORMAT="docker-archive"
+```
+
+ - Trigger creation of SDK root filesystem
+
+```
+bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base
+```
+
+ - Load the SDK container image into the Docker Daemon
+
+```
+xzcat build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf-docker-archive.tar.xz | docker load
+```
+
+ - Run a container using the SDK container image (following commands starting with `#~:` are to be run in the container)
+
+```
+docker run --rm -ti --volume "$(pwd):/build" isar-sdk-buster-armhf:latest
+```
+
+ - Check that cross toolchains are installed
+
+```
+:~# dpkg -l | grep crossbuild-essential-armhf
+ii crossbuild-essential-armhf 12.3 all Informational list of cross-build-essential packages
+```
+
## Creation of local apt repo caching upstream Debian packages
### Motivation
--
2.30.0
1 file changed, 127 insertions(+)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index a4f3d1d..6b72a16 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -19,6 +19,7 @@ Copyright (C) 2016-2019, ilbers GmbH
- [Add a Custom Application](#add-a-custom-application)
- [Enabling Cross-compilation](#isar-cross-compilation)
- [Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)
+ - [Create a containerized ISAR SDK root filesystem](#create-a-containerized-isar-sdk-root-filesystem)
- [Creation of local apt repo caching upstream Debian packages](#creation-of-local-apt-repo-caching-upstream-debian-packages)
@@ -84,6 +85,9 @@ If your host is >= buster, also install the following package.
apt install python3-distutils
```
+If you want to generate containerized SDKs, also install the following packages: `umoci` and `skopeo`.
+Umoci is provided by Debian Buster and can be installed with `apt install umoci`, Skopeo is provided by Debian Bullseye/Unstable and has to be installed either manually downloading the DEB and installing it (no other packages required) or with `apt install -t bullseye skopeo` (if unstable/bullseye included in `/etc/apt/sources.list[.d]`).
+
Notes:
* BitBake requires Python 3.4+.
@@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
qemu-system-i386 -m 256M -nographic -hda tmp/deploy/images/qemui386/isar-image-base-debian-buster-qemui386.wic.img
```
+### Generate container image with root-filesystem
+
+A runnable container image is generated if you set IMAGE_TYPE to 'container-img'.
+Getting a container image can be the main purpose of an ISAR configuration, but not only.
+A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
+
+Container images can be generated in different formats, selected with the variable `CONTAINER_FORMAT`. One or more (whitespace separated) of following options can be given:
+ - `docker-archive`: (default) an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+Following formats don't work if running `bitbake ...` (to build the image) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting container image archives (only for `docker-archive` and `oci-archive`) are made available as `tmp/deploy/images/${MACHINE}/${DISTRO}-${DISTRO_ARCH}-${container_format}.tar.xz` (being `container_format` each one of the formats specified in `CONTAINER_FORMAT`).
+
+### Example
+
+ - Make the relevant environment variables available to the task
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_FORMAT"
+export IMAGE_TYPE="container-img"
+export CONTAINER_FORMAT="docker-archive"
+```
+
+ - Trigger creation of container image from root filesystem
+
+```
+bitbake mc:qemuarm-buster:isar-image-base
+```
+
+ - Load the container image into the Docker Daemon
+
+```
+xzcat build/tmp/deploy/images/qemuarm/debian-buster-armhf-docker-archive.tar.xz | docker load
+```
+
+ - Run a container using the container image (following commands starting with `#~:` are to be run in the container)
+
+```
+docker run --rm -ti --volume "$(pwd):/build" isar-buster-armhf:latest
+```
+
---
## Terms and Definitions
@@ -834,6 +886,81 @@ ii crossbuild-essential-armhf 12.3 all Inf
~#
```
+## Create a containerized ISAR SDK root filesystem
+
+### Motivation
+
+Distributing and using the SDK root filesystem created following the instructions in "[Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)" becomes easier using container images (at least for those using containers anyway)
+A "containerized" SDK adds to those advantages of a normal SDK root filesystem the comfort of container images.
+
+### Approach
+
+Create container image with SDK root filesystem with installed cross-toolchain for target architecture and ability to install already prebuilt target binary artifacts.
+Developer:
+ - runs a container based on the resulting container image mounting the source code to be built,
+ - develops applications for target platform on the container and
+ - leaves the container getting the results on the mounted directory.
+
+### Solution
+
+A container image containing the SDK is generated if you set IMAGE_TYPE to 'container-img'.
+
+Container images can be generated in different formats, selected with the variable `CONTAINER_FORMAT`. One or more (space separated) of following options can be given:
+ - `docker-archive`: (default) an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+User manually triggers creation of SDK formats for his target platform by launching the task `do_populate_sdk` for target image, f.e.
+`bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
+Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
+
+Following formats don't work if running `bitbake -c do_populate_sdk ...` (to generate the containerized SDK) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting SDK formats are archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_format}.tar.xz` (being `sdk_format` each one of the formats specified in `CONTAINER_FORMAT`).
+The SDK container directory `/isar-apt` contains a copy of isar-apt repo with locally prebuilt target debian packages (for <HOST_DISTRO>).
+One may get into an SDK container and install required target packages with the help of `apt-get install <package_name>:<DISTRO_ARCH>` command.
+The directory with the source code to develop on should be mounted on the container (with `--volume <host-directory>:<container-directory>`) to be able to edit files in the host with an IDE and build in the container.
+
+### Example
+
+ - Make the relevant environment variables available to the task
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_FORMAT"
+export IMAGE_TYPE="container-img"
+export CONTAINER_FORMAT="docker-archive"
+```
+
+ - Trigger creation of SDK root filesystem
+
+```
+bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base
+```
+
+ - Load the SDK container image into the Docker Daemon
+
+```
+xzcat build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf-docker-archive.tar.xz | docker load
+```
+
+ - Run a container using the SDK container image (following commands starting with `#~:` are to be run in the container)
+
+```
+docker run --rm -ti --volume "$(pwd):/build" isar-sdk-buster-armhf:latest
+```
+
+ - Check that cross toolchains are installed
+
+```
+:~# dpkg -l | grep crossbuild-essential-armhf
+ii crossbuild-essential-armhf 12.3 all Informational list of cross-build-essential packages
+```
+
## Creation of local apt repo caching upstream Debian packages
### Motivation
--
2.30.0
Jan Kiszka
Feb 9, 2021, 11:50:09 AM2/9/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
On 09.02.21 15:10, [ext] Silvano Cirujano Cuesta wrote:
> This patch series provides support for containerized root filesystems,
> for both target images and SDKs.
>
> For containerized target images the new image type `container-img` has
> been added.
>
> For containerized SDKs the task `populate_sdk` has been extended.
>
> Containerized root filesystems are easy to distribute and run, enabling
> this way following scenarios:
> - Use ISAR to build container images meant to be run only in containers.
> - Use the same ISAR configuration to build images for containers, VMs
> and bare-metal.
> - Easy SDK distribution and "installation".
> - Quickly testing certain applications in the workstation using the
> target root filesystem.
>
> In order to build containerized rootfilesystems `IMAGE_TYPE` as to be
> `container-img`, additionally the container image format can be selected
> with the variable `CONTAINER_FORMAT`. The default format is
> `docker-archive`.
>
But you dropped control over building the SDK (and only the SDK) as
> This patch series provides support for containerized root filesystems,
> for both target images and SDKs.
>
> For containerized target images the new image type `container-img` has
> been added.
>
> For containerized SDKs the task `populate_sdk` has been extended.
>
> Containerized root filesystems are easy to distribute and run, enabling
> this way following scenarios:
> - Use ISAR to build container images meant to be run only in containers.
> - Use the same ISAR configuration to build images for containers, VMs
> and bare-metal.
> - Easy SDK distribution and "installation".
> - Quickly testing certain applications in the workstation using the
> target root filesystem.
>
> In order to build containerized rootfilesystems `IMAGE_TYPE` as to be
> `container-img`, additionally the container image format can be selected
> with the variable `CONTAINER_FORMAT`. The default format is
> `docker-archive`.
>
container, didn't you? Where did that SDK_FORMATS go?
Jan
Corporate Competence Center Embedded Linux
Silvano Cirujano Cuesta
Feb 9, 2021, 12:37:16 PM2/9/21
to Jan Kiszka, isar-...@googlegroups.com
On 09/02/2021 17:50, Jan Kiszka wrote:
> On 09.02.21 15:10, [ext] Silvano Cirujano Cuesta wrote:
>> This patch series provides support for containerized root filesystems,
>> for both target images and SDKs.
>>
>> For containerized target images the new image type `container-img` has
>> been added.
>>
>> For containerized SDKs the task `populate_sdk` has been extended.
>>
>> Containerized root filesystems are easy to distribute and run, enabling
>> this way following scenarios:
>> - Use ISAR to build container images meant to be run only in containers.
>> - Use the same ISAR configuration to build images for containers, VMs
>> and bare-metal.
>> - Easy SDK distribution and "installation".
>> - Quickly testing certain applications in the workstation using the
>> target root filesystem.
>>
>> In order to build containerized rootfilesystems `IMAGE_TYPE` as to be
>> `container-img`, additionally the container image format can be selected
>> with the variable `CONTAINER_FORMAT`. The default format is
>> `docker-archive`.
>>
> But you dropped control over building the SDK (and only the SDK) as
> container, didn't you? Where did that SDK_FORMATS go?
>
> Jan
Two of them to select the container format(s) to generate. For these I'd propose CONTAINER_FORMAT and SDK_FORMAT.
Another one to select the build of a container image for target, IMAGE_TYPE="container-img" would be my proposal.
The selection of containerized SDK would be implicit with the presence of a valid SDK_FORMAT. If that selection should be explicit, then a 4th variable would be needed.
I don't have a strong opinion on this, so simply make a wish.
Silvano
>
>> More information about its usage is documented in the file
>> docs/user_manual.md.
>>
>> A PoC/demo of this functionality (only the SDK part) has been created
>> based on the project https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cb07a56105809427e9b7308d8cd1ac275%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637484862105605050%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=odtZPJwWKcqoXH5VrvsRtfqDzHS8wwYSS0M6WsNtzmU%3D&reserved=0.
>> More information about its usage is documented in the file
>> docs/user_manual.md.
>>
>> A PoC/demo of this functionality (only the SDK part) has been created
>> Jan Kiszka already tested and liked it! =>
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050%2Fissues%2F86%23issuecomment-768907845&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cb07a56105809427e9b7308d8cd1ac275%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637484862105605050%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bXFPjdM8f5VwsFozf5vGHoMRwfo6kv2D%2FxvL76mtZtw%3D&reserved=0
>>
>> In order to get a feeling about its usage (you need Docker or Podman),
>> follow these simple copy&paste instructions:
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Fblob%2Fmaster%2Fkas%2FBUILDING-SDK-CONTAINER.md%23running-the-sdk&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cb07a56105809427e9b7308d8cd1ac275%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637484862105605050%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mONGFofFottYo4fVJ%2BMADZ%2Fx%2FZ6NmOj8w6POjw1FlAk%3D&reserved=0
>> In order to get a feeling about its usage (you need Docker or Podman),
>> follow these simple copy&paste instructions:
>> Build instructions are available in the upper part of that document.
>>
>> Two new dependencies are required to create containerized root
>> filesystems (as specified in the documentation).
>>
>> Typical container image management actions (e.g. push an image to a
>> container image regitry) are out of scope. Available tools (Docker,
>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>
>> A patch will follow this one to get the dependencies into the container
>> images being provided by the project
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fkas&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cb07a56105809427e9b7308d8cd1ac275%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637484862105605050%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X2OZUddd2z%2BWIRHIZuSdBXTI%2BJvr9Jh%2FXw%2BgrqO1Ni4%3D&reserved=0 (for `kas-container`, for example).
>>
>> Two new dependencies are required to create containerized root
>> filesystems (as specified in the documentation).
>>
>> Typical container image management actions (e.g. push an image to a
>> container image regitry) are out of scope. Available tools (Docker,
>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>
>> A patch will follow this one to get the dependencies into the container
>> images being provided by the project
>>
>> Silvano Cirujano Cuesta (2):
>> images: add support for container images
>> docs: document creation of container images
>>
>> doc/user_manual.md | 127 +++++++++++++++++++++++++++++
>> meta/classes/container-img.bbclass | 99 ++++++++++++++++++++++
>> 2 files changed, 226 insertions(+)
>> create mode 100644 meta/classes/container-img.bbclass
>>
--
Siemens AG, T RDA IOT SES-DE
>> Silvano Cirujano Cuesta (2):
>> images: add support for container images
>> docs: document creation of container images
>>
>> doc/user_manual.md | 127 +++++++++++++++++++++++++++++
>> meta/classes/container-img.bbclass | 99 ++++++++++++++++++++++
>> 2 files changed, 226 insertions(+)
>> create mode 100644 meta/classes/container-img.bbclass
>>
--
Jan Kiszka
Feb 9, 2021, 1:39:45 PM2/9/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
container), and those should be independently configurable.
I was fine with your previous way of configuring the SDK_FORMATS. Here,
we should add container-img as another IMAGE_TYPE, further configurable
regarding its output formats (CONTAINER_FORMATS). And both use cases
should share container-img.bbclass for their common bits.
Jan
--
Siemens AG, T RDA IOT
Silvano Cirujano Cuesta
Feb 12, 2021, 4:01:16 AM2/12/21
to isar-...@googlegroups.com
This patch series provides support for containerized root filesystems,
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized target root filesystems `IMAGE_TYPE` has
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
to be `container-img`, additionally the container image format can be
selected with the variable `CONTAINER_FORMATS`. The default format is
`docker-archive`.
In order to build containerized SDKs the variable `SDK_FORMAT` has to
provide any of the supported container formats (e.g. `docker-archive`).
The default format is the legacy non-containerized: `tar_xz`.
More information about its usage is documented in the file
docs/user_manual.md.
A PoC/demo of this functionality (only the SDK part) has been created
Jan Kiszka already tested and liked it! =>
https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845
Successful builds of both containerized target and SDK are available on
the same PoC project:
- https://github.com/Silvanoc/meta-iot2050/actions/runs/558311580
- https://github.com/Silvanoc/meta-iot2050/actions/runs/558311581
and also the resulting images:
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-arm64
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-sdk-arm64
In order to get a feeling about its usage (you need Docker or Podman),
follow these simple copy&paste instructions:
Build instructions are available in the upper part of that document.
Two new dependencies are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
https://github.com/siemens/kas (for `kas-container`, for example).
Two new dependencies are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
Silvano Cirujano Cuesta (2):
images: add support for container images
docs: document creation of container images
doc/user_manual.md | 127 +++++++++++++++++++++++
meta/classes/image-sdk-extension.bbclass | 51 +++++++--
meta/classes/image.bbclass | 1 +
4 files changed, 260 insertions(+), 7 deletions(-)
create mode 100644 meta/classes/container-img.bbclass
--
2.30.0
--
Silvano Cirujano Cuesta
Feb 12, 2021, 4:06:16 AM2/12/21
to isar-...@googlegroups.com
Add support for creation of container images with the build root
filesystems.
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
meta/classes/container-img.bbclass | 88 ++++++++++++++++++++++++
filesystems.
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
meta/classes/image-sdk-extension.bbclass | 51 ++++++++++++--
meta/classes/image.bbclass | 1 +
3 files changed, 133 insertions(+), 7 deletions(-)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,88 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk'
+# to create container images containing the target rootfs and the SDK
+# respectively.
+
+CONTAINER_FORMATS ?= "docker-archive"
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk'
+# to create container images containing the target rootfs and the SDK
+# respectively.
+
+
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+ local container_formats="$3"
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbnote "Generate container image in these formats: ${CONTAINER_FORMATS}"
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}"
+}
+
+addtask container_image before do_image after do_image_tools
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
+
+addtask container_image before do_image after do_image_tools
index a8c708a..63138da 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -6,11 +6,25 @@
# This class extends the image.bbclass to supply the creation of a sdk
SDK_INCLUDE_ISAR_APT ?= "0"
+SDK_FORMATS ?= "tar-xz"
+
+sdk_tar_xz() {
+ # Copy mount_chroot.sh for convenience
+ sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+
+ # Create SDK archive
+ cd -P ${SDKCHROOT_DIR}/..
+ sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
+ -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ bbnote "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz"
+}
do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
do_populate_sdk[depends] = "sdkchroot:do_build"
-do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT"
+do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS"
do_populate_sdk() {
+ local sdk_container_formats=""
+
if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then
# Copy isar-apt with deployed Isar packages
sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt
@@ -48,12 +62,35 @@ do_populate_sdk() {
done
done
- # Copy mount_chroot.sh for convenience
- sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+ # separate SDK formats: TAR and container formats
+ for sdk_format in ${SDK_FORMATS} ; do
+ case ${sdk_format} in
+ "tar-xz")
+ sdk_tar_xz
+ ;;
+ "docker-archive" | "oci" | "oci-archive")
+ if [ -z "${sdk_container_formats}" ] ; then
+ sdk_container_formats="${sdk_format}"
+ else
+ sdk_container_formats="${sdk_container_formats} ${sdk_format}"
+ fi
+ ;;
+ "docker-daemon" | "containers-storage")
+ if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
+ "docker-daemon" | "containers-storage")
+ die "Adding the SDK container image to a container runtime (${sdk_format}) not supported if running from a container (e.g. 'kas-container')"
+ fi
+ ;;
+ *)
+ die "unsupported SDK format specified: ${sdk_format}"
+ ;;
+ esac
+ done
- # Create SDK archive
+ esac
+ done
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ # generate the SDK in all the desired container formats
+ if [ -n "${sdk_container_formats}" ] ; then
+ bbnote "Generating SDK container in ${sdk_container_formats} format"
+ containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+ fi
}
+
addtask populate_sdk after do_rootfs
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index eddc444..7fb7b7e 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -76,6 +76,7 @@ inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
+inherit container-img
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
--
2.30.0
Silvano Cirujano Cuesta
Feb 12, 2021, 4:06:16 AM2/12/21
to isar-...@googlegroups.com
doc/user_manual.md | 127 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 127 insertions(+)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index a4f3d1d..f6f49bc 100644
1 file changed, 127 insertions(+)
diff --git a/doc/user_manual.md b/doc/user_manual.md
+
+Supported formats are:
+ - `tar-xz`: (default) is the non-containerized format that results from following the instructions in "[Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)"
+ - `docker-archive`: an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+User manually triggers creation of SDK formats for his target platform by launching the task `do_populate_sdk` for target image, f.e.
+`bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
+Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
+
+Following formats don't work if running `bitbake -c do_populate_sdk ...` (to generate the containerized SDK) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting SDK formats are archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_format}.tar.xz` (being `sdk_format` each one of the formats specified in `SDK_FORMATS`).
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+User manually triggers creation of SDK formats for his target platform by launching the task `do_populate_sdk` for target image, f.e.
+`bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
+Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
+
+Following formats don't work if running `bitbake -c do_populate_sdk ...` (to generate the containerized SDK) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The SDK container directory `/isar-apt` contains a copy of isar-apt repo with locally prebuilt target debian packages (for <HOST_DISTRO>).
+One may get into an SDK container and install required target packages with the help of `apt-get install <package_name>:<DISTRO_ARCH>` command.
+The directory with the source code to develop on should be mounted on the container (with `--volume <host-directory>:<container-directory>`) to be able to edit files in the host with an IDE and build in the container.
+
+### Example
+
+ - Make the SDK formats to generate available to the task
+One may get into an SDK container and install required target packages with the help of `apt-get install <package_name>:<DISTRO_ARCH>` command.
+The directory with the source code to develop on should be mounted on the container (with `--volume <host-directory>:<container-directory>`) to be able to edit files in the host with an IDE and build in the container.
+
+### Example
+
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMATS"
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export SDK_FORMATS="docker-archive"
Jan Kiszka
Feb 12, 2021, 12:15:11 PM2/12/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
On 12.02.21 09:51, [ext] Silvano Cirujano Cuesta wrote:
> Add support for creation of container images with the build root
> filesystems.
>
> Extend also task "populate_sdk" to support the creation of a container image
> containing the SDK.
Should be done in to steps: container-img.bbclass frirst, and then a
> Add support for creation of container images with the build root
> filesystems.
>
> Extend also task "populate_sdk" to support the creation of a container image
> containing the SDK.
patch to use it for the SDK as well.
>
> Signed-off-by: Silvano Cirujano Cuesta <silvano.cir...@siemens.com>
> ---
> meta/classes/container-img.bbclass | 88 ++++++++++++++++++++++++
> meta/classes/image-sdk-extension.bbclass | 51 ++++++++++++--
> meta/classes/image.bbclass | 1 +
> 3 files changed, 133 insertions(+), 7 deletions(-)
> create mode 100644 meta/classes/container-img.bbclass
>
> diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
> new file mode 100644
> index 0000000..35c7bbc
> --- /dev/null
> +++ b/meta/classes/container-img.bbclass
> @@ -0,0 +1,88 @@
> +# This software is a part of ISAR.
> +# Copyright (C) Siemens AG, 2021
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk'
that test here and share.
> + *)
> + die "Unsupported format for containerize_rootfs: ${image_type}"
> + ;;
> + esac
> + done
> +}
> +
> +do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> +do_container_image[vardeps] += "CONTAINER_FORMATS"
> +do_container_image(){
> + rootfs_id="${DISTRO}-${DISTRO_ARCH}"
> +
> + bbnote "Generate container image in these formats: ${CONTAINER_FORMATS}"
core so far. Nor bbdebug, though.
> + sdk_container_formats="${sdk_format}"
> + else
> + sdk_container_formats="${sdk_container_formats} ${sdk_format}"
> + fi
> + ;;
> + "docker-daemon" | "containers-storage")
> + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
> + die "Adding the SDK container image to a container runtime (${sdk_format}) not supported if running from a container (e.g. 'kas-container')"
> + fi
Silvano Cirujano Cuesta
Feb 12, 2021, 12:46:19 PM2/12/21
to Jan Kiszka, isar-...@googlegroups.com
On 12/02/2021 18:10, Jan Kiszka wrote:
> On 12.02.21 09:51, [ext] Silvano Cirujano Cuesta wrote:
>> Add support for creation of container images with the build root
>> filesystems.
>>
>> Extend also task "populate_sdk" to support the creation of a container image
>> containing the SDK.
> Should be done in to steps: container-img.bbclass frirst, and then a
> patch to use it for the SDK as well.
>
>> Signed-off-by: Silvano Cirujano Cuesta <silvano.cir...@siemens.com>
>> ---
>> meta/classes/container-img.bbclass | 88 ++++++++++++++++++++++++
>> meta/classes/image-sdk-extension.bbclass | 51 ++++++++++++--
>> meta/classes/image.bbclass | 1 +
>> 3 files changed, 133 insertions(+), 7 deletions(-)
>> create mode 100644 meta/classes/container-img.bbclass
>>
>> diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
>> new file mode 100644
>> index 0000000..35c7bbc
>> --- /dev/null
>> +++ b/meta/classes/container-img.bbclass
>> @@ -0,0 +1,88 @@
>> +# This software is a part of ISAR.
>> +# Copyright (C) Siemens AG, 2021
>> +#
>> +# SPDX-License-Identifier: MIT
>> +#
>> +# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk'
> Nope, it now only provides the former.
In the case of the SDK the same task is provides the non-containerized format tar-xz.
>
>> + *)
>> + die "Unsupported format for containerize_rootfs: ${image_type}"
>> + ;;
>> + esac
>> + done
>> +}
>> +
>> +do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
>> +do_container_image[vardeps] += "CONTAINER_FORMATS"
>> +do_container_image(){
>> + rootfs_id="${DISTRO}-${DISTRO_ARCH}"
>> +
>> + bbnote "Generate container image in these formats: ${CONTAINER_FORMATS}"
> Probabably more "bbdebug"? Unsure. But we aren't using bbnote in the
> core so far. Nor bbdebug, though.
BTW I'm using bbdebug a lot in the containerize_rootfs section because I've missed those kind of traces much too often when trying to debug some issues on ISAR recipes.
Perhaps we should have more debug verbosity in the logs to ease debugging...
Not really useless, but not important (in fact that was my 1st version). I can change it in the next patch series version that I need anyway.
>
>> + sdk_container_formats="${sdk_format}"
>> + else
>> + sdk_container_formats="${sdk_container_formats} ${sdk_format}"
>> + fi
>> + ;;
>> + "docker-daemon" | "containers-storage")
>> + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
>> + die "Adding the SDK container image to a container runtime (${sdk_format}) not supported if running from a container (e.g. 'kas-container')"
>> + fi
> See above, should likely go into containerize_rootfs().
In fact this case section is really messed up, I have to clean it up completely.
--
Siemens AG, T RDA IOT SES-DE
Silvano Cirujano Cuesta
Feb 12, 2021, 1:04:27 PM2/12/21
to isar-...@googlegroups.com
The current structure that I have come up to seems weird to me, it isn't obvious that "containerize_rootfs" is meant to be reused.
(-) an additional class for a single function
(+) better structured code
Jan Kiszka
Feb 12, 2021, 1:11:23 PM2/12/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
On 12.02.21 18:46, Silvano Cirujano Cuesta wrote:
>
> On 12/02/2021 18:10, Jan Kiszka wrote:
>> On 12.02.21 09:51, [ext] Silvano Cirujano Cuesta wrote:
>>> Add support for creation of container images with the build root
>>> filesystems.
>>>
>>> Extend also task "populate_sdk" to support the creation of a container image
>>> containing the SDK.
>> Should be done in to steps: container-img.bbclass frirst, and then a
>> patch to use it for the SDK as well.
>
> Ok. There are some many different tastes WRT to big vs small commits :-)
Rather /wrt logically separatable steps.
>
> On 12/02/2021 18:10, Jan Kiszka wrote:
>> On 12.02.21 09:51, [ext] Silvano Cirujano Cuesta wrote:
>>> Add support for creation of container images with the build root
>>> filesystems.
>>>
>>> Extend also task "populate_sdk" to support the creation of a container image
>>> containing the SDK.
>> Should be done in to steps: container-img.bbclass frirst, and then a
>> patch to use it for the SDK as well.
>
> Ok. There are some many different tastes WRT to big vs small commits :-)
CONTAINER_FORMATS="docker-daemon" and SDK_FORMATS="docker-daemon" when
running inside a kas build container? Both do not work, do they?
But if you dislike the leading whitespaces in the debug logs, make it
trailing (prepend rather than append).
>>
>>> + sdk_container_formats="${sdk_format}"
>>> + else
>>> + sdk_container_formats="${sdk_container_formats} ${sdk_format}"
>>> + fi
>>> + ;;
>>> + "docker-daemon" | "containers-storage")
>>> + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
>>> + die "Adding the SDK container image to a container runtime (${sdk_format}) not supported if running from a container (e.g. 'kas-container')"
>>> + fi
>> See above, should likely go into containerize_rootfs().
>
> Right, will fix it.
>
> In fact this case section is really messed up, I have to clean it up completely.
>
--
Siemens AG, T RDA IOT
Silvano Cirujano Cuesta
Feb 12, 2021, 1:23:31 PM2/12/21
to Jan Kiszka, isar-...@googlegroups.com
But I got it now, and that's what I meant with the messed up case section.
In the next version the "Am I a container?" is in the function, no need to do it twice.
--
Siemens AG, T RDA IOT SES-DE
Silvano Cirujano Cuesta
Feb 15, 2021, 4:46:53 AM2/15/21
to isar-...@googlegroups.com
Wouldn't it make sense to put the "containerize_rootfs" function in a separate class and let "image.bbclass" inherit from it?
The current structure that I have come up to seems weird to me, it isn't obvious that "containerize_rootfs" is meant to be reused.
(-) an additional class for a single function
(+) better structured code
Possibilities that seem to fit somehow:
- specific class "container.bbclass",
- specific class "image-container-extension.bbclass"
- existing class already being inherited by "image.bbclass" ("rootfs.bbclass" -is it a rootfs feature?-, "image-postproc-extension.bbclass", "image-tools-extension.bbclass")
and I cannot tell which one fits best.
Silvano
The current structure that I have come up to seems weird to me, it isn't obvious that "containerize_rootfs" is meant to be reused.
(-) an additional class for a single function
(+) better structured code
- specific class "container.bbclass",
- specific class "image-container-extension.bbclass"
- existing class already being inherited by "image.bbclass" ("rootfs.bbclass" -is it a rootfs feature?-, "image-postproc-extension.bbclass", "image-tools-extension.bbclass")
and I cannot tell which one fits best.
Silvano
Jan Kiszka
Feb 15, 2021, 5:31:53 AM2/15/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
On 15.02.21 10:46, [ext] Silvano Cirujano Cuesta wrote:
> Wouldn't it make sense to put the "containerize_rootfs" function in a separate class and let "image.bbclass" inherit from it?
>
Sounds reasonable to me, even if we end up with only one function in
> Wouldn't it make sense to put the "containerize_rootfs" function in a separate class and let "image.bbclass" inherit from it?
>
that class, at least so far.
Jan
--
Siemens AG, T RDA IOT
Siemens AG, T RDA IOT
Silvano Cirujano Cuesta
Feb 15, 2021, 8:39:20 AM2/15/21
to isar-...@googlegroups.com
classes: add root filesystem containerizing class
classes: add new image type 'container-img'
sdk: add support for containerized sdk
docs: document creation of container images
doc/user_manual.md | 127 ++++++++++++++++++
meta/classes/container-img.bbclass | 18 +++
doc/user_manual.md | 127 ++++++++++++++++++
.../classes/image-container-extension.bbclass | 79 +++++++++++
meta/classes/image-sdk-extension.bbclass | 42 +++++-
meta/classes/image.bbclass | 1 +
5 files changed, 260 insertions(+), 7 deletions(-)
create mode 100644 meta/classes/container-img.bbclass
create mode 100644 meta/classes/image-container-extension.bbclass
--
2.30.0
Silvano Cirujano Cuesta
Feb 15, 2021, 8:44:21 AM2/15/21
to isar-...@googlegroups.com
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
meta/classes/image-sdk-extension.bbclass | 42 ++++++++++++++++++++----
containing the SDK.
1 file changed, 35 insertions(+), 7 deletions(-)
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index a8c708a..426b925 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -6,11 +6,25 @@
# This class extends the image.bbclass to supply the creation of a sdk
SDK_INCLUDE_ISAR_APT ?= "0"
+SDK_FORMATS ?= "tar-xz"
+
+sdk_tar_xz() {
+ # Copy mount_chroot.sh for convenience
+ sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+
+ # Create SDK archive
+ cd -P ${SDKCHROOT_DIR}/..
+ sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
+ -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ bbdebug 1 "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz"
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -6,11 +6,25 @@
# This class extends the image.bbclass to supply the creation of a sdk
SDK_INCLUDE_ISAR_APT ?= "0"
+SDK_FORMATS ?= "tar-xz"
+
+sdk_tar_xz() {
+ # Copy mount_chroot.sh for convenience
+ sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+
+ # Create SDK archive
+ cd -P ${SDKCHROOT_DIR}/..
+ sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
+ -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+}
do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
do_populate_sdk[depends] = "sdkchroot:do_build"
-do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT"
+do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS"
do_populate_sdk() {
+ local sdk_container_formats=""
+
if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then
# Copy isar-apt with deployed Isar packages
sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt
@@ -48,12 +62,26 @@ do_populate_sdk() {
do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
do_populate_sdk[depends] = "sdkchroot:do_build"
-do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT"
+do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS"
do_populate_sdk() {
+ local sdk_container_formats=""
+
if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then
# Copy isar-apt with deployed Isar packages
sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt
done
done
- # Copy mount_chroot.sh for convenience
- sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+ # separate SDK formats: TAR and container formats
+ for sdk_format in ${SDK_FORMATS} ; do
+ case ${sdk_format} in
+ "tar-xz")
+ sdk_tar_xz
+ ;;
+ "docker-archive" | "oci" | "oci-archive" | "docker-daemon" | "containers-storage")
done
- # Copy mount_chroot.sh for convenience
- sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+ # separate SDK formats: TAR and container formats
+ for sdk_format in ${SDK_FORMATS} ; do
+ case ${sdk_format} in
+ "tar-xz")
+ sdk_tar_xz
+ ;;
+ sdk_container_formats="${sdk_container_formats} ${sdk_format}"
+ ;;
+ *)
+ die "unsupported SDK format specified: ${sdk_format}"
+ ;;
+ esac
+ done
- # Create SDK archive
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ # generate the SDK in all the desired container formats
+ if [ -n "${sdk_container_formats}" ] ; then
+ bbnote "Generating SDK container in ${sdk_container_formats} format"
+ containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+ fi
}
+
addtask populate_sdk after do_rootfs
--
+ *)
+ die "unsupported SDK format specified: ${sdk_format}"
+ ;;
+ esac
+ done
- # Create SDK archive
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ # generate the SDK in all the desired container formats
+ if [ -n "${sdk_container_formats}" ] ; then
+ bbnote "Generating SDK container in ${sdk_container_formats} format"
+ containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+ fi
}
+
addtask populate_sdk after do_rootfs
2.30.0
Silvano Cirujano Cuesta
Feb 15, 2021, 8:44:31 AM2/15/21
to isar-...@googlegroups.com
Silvano Cirujano Cuesta
Feb 15, 2021, 8:44:38 AM2/15/21
to isar-...@googlegroups.com
Add a new "image" class for generating a container image containing the
target root filesystem.
meta/classes/container-img.bbclass | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,18 @@
+# to create container images containing the target rootfs.
2.30.0
target root filesystem.
meta/classes/container-img.bbclass | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
index 0000000..79ef3e8
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,18 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the task 'containerize_rootfs'
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# to create container images containing the target rootfs.
+
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMATS"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbdebug 1 "Generate container image in these formats: ${CONTAINER_FORMATS}"
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMATS"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}"
+}
+
+addtask container_image before do_image after do_image_tools
--
+}
+
+addtask container_image before do_image after do_image_tools
2.30.0
Silvano Cirujano Cuesta
Feb 15, 2021, 8:49:20 AM2/15/21
to isar-...@googlegroups.com
This class can be used to create container images which root filesystem
is that generated by the do_rootfs task.
Containerized root filesystems have following possible use-cases:
- Using ISAR as a container image builder.
- Simplify distribution of runtime rootfs (binaries, libraries,
configurations, ...) for application development or testing.
- Distributing SDKs.
.../classes/image-container-extension.bbclass | 79 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
2 files changed, 80 insertions(+)
create mode 100644 meta/classes/image-container-extension.bbclass
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
new file mode 100644
index 0000000..a49a435
--- /dev/null
+++ b/meta/classes/image-container-extension.bbclass
@@ -0,0 +1,79 @@
+ fi
+ *)
+
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index eddc444..ec93cab 100644
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
--
2.30.0
is that generated by the do_rootfs task.
Containerized root filesystems have following possible use-cases:
- Using ISAR as a container image builder.
- Simplify distribution of runtime rootfs (binaries, libraries,
configurations, ...) for application development or testing.
- Distributing SDKs.
.../classes/image-container-extension.bbclass | 79 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
2 files changed, 80 insertions(+)
create mode 100644 meta/classes/image-container-extension.bbclass
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
new file mode 100644
index 0000000..a49a435
--- /dev/null
+++ b/meta/classes/image-container-extension.bbclass
@@ -0,0 +1,79 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass for containerizing the root filesystem.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+ if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
+ die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')"
+ fi
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" \
+ "${image_type}:${image_name}:latest"
+ ;;
+ "oci:${oci_img_dir}:${full_tag}" \
+ "${image_type}:${image_name}:latest"
+ *)
+ die "Unsupported format for containerize_rootfs: ${image_type}"
+ ;;
+ esac
+ done
+}
+ esac
+ done
+
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index eddc444..ec93cab 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -76,6 +76,7 @@ inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
+inherit image-container-extension
+++ b/meta/classes/image.bbclass
@@ -76,6 +76,7 @@ inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
2.30.0
Silvano Cirujano Cuesta
Feb 22, 2021, 4:16:24 AM2/22/21
to isar-...@googlegroups.com, Kiszka, Jan (CT RDA IOT SES-DE)
Friendly reminder
On 15/02/2021 14:39, [ext] Silvano Cirujano Cuesta wrote:
> This patch series provides support for containerized root filesystems,
> for both target images and SDKs.
>
> For containerized target images the new image type `container-img` has
> been added.
>
> For containerized SDKs the task `populate_sdk` has been extended.
>
> Containerized root filesystems are easy to distribute and run, enabling
> this way following scenarios:
> - Use ISAR to build container images meant to be run only in containers.
> - Use the same ISAR configuration to build images for containers, VMs
> and bare-metal.
> - Easy SDK distribution and "installation".
> - Quickly testing certain applications in the workstation using the
> target root filesystem.
>
> In order to build containerized target root filesystems `IMAGE_TYPE` has
> to be `container-img`, additionally the container image format can be
> selected with the variable `CONTAINER_FORMATS`. The default format is
> `docker-archive`.
>
> In order to build containerized SDKs the variable `SDK_FORMAT` has to
> provide any of the supported container formats (e.g. `docker-archive`).
> The default format is the legacy non-containerized: `tar_xz`.
>
> More information about its usage is documented in the file
> docs/user_manual.md.
>
> A PoC/demo of this functionality (only the SDK part) has been created
> based on the project https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WkM3UYp9Bfm7izayxUFAnLH%2FWZMpCPQxZOH5vXz1SM0%3D&reserved=0.
> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311581&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=u4jV%2BcWk5cu5m5SflXNtV%2FeejeO5Lks1VqTfKR8riXI%3D&reserved=0
> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-sdk-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CPd2YFNnth%2Fokhh3F9CL2T0tOvKKBoX22R1kHPHsI6M%3D&reserved=0
On 15/02/2021 14:39, [ext] Silvano Cirujano Cuesta wrote:
> This patch series provides support for containerized root filesystems,
> for both target images and SDKs.
>
> For containerized target images the new image type `container-img` has
> been added.
>
> For containerized SDKs the task `populate_sdk` has been extended.
>
> Containerized root filesystems are easy to distribute and run, enabling
> this way following scenarios:
> - Use ISAR to build container images meant to be run only in containers.
> - Use the same ISAR configuration to build images for containers, VMs
> and bare-metal.
> - Easy SDK distribution and "installation".
> - Quickly testing certain applications in the workstation using the
> target root filesystem.
>
> In order to build containerized target root filesystems `IMAGE_TYPE` has
> to be `container-img`, additionally the container image format can be
> selected with the variable `CONTAINER_FORMATS`. The default format is
> `docker-archive`.
>
> In order to build containerized SDKs the variable `SDK_FORMAT` has to
> provide any of the supported container formats (e.g. `docker-archive`).
> The default format is the legacy non-containerized: `tar_xz`.
>
> More information about its usage is documented in the file
> docs/user_manual.md.
>
> A PoC/demo of this functionality (only the SDK part) has been created
> Jan Kiszka already tested and liked it! =>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050%2Fissues%2F86%23issuecomment-768907845&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GjNKEX9eFMYjTDF3f17k9KUammBDStvCdi9yonqgV1o%3D&reserved=0
>
> Successful builds of both containerized target and SDK are available on
> the same PoC project:
> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311580&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Omk6MunatUrP06oYwsWLKeyeUnifRInL9uW1rde4pig%3D&reserved=0
> Successful builds of both containerized target and SDK are available on
> the same PoC project:
> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311581&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=u4jV%2BcWk5cu5m5SflXNtV%2FeejeO5Lks1VqTfKR8riXI%3D&reserved=0
> and also the resulting images:
> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7ThrUDCV94UIEGLDJwnuakDATL43r3KV9YJX1Gi%2F84%3D&reserved=0
> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-sdk-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CPd2YFNnth%2Fokhh3F9CL2T0tOvKKBoX22R1kHPHsI6M%3D&reserved=0
>
> In order to get a feeling about its usage (you need Docker or Podman),
> follow these simple copy&paste instructions:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Fblob%2Fmaster%2Fkas%2FBUILDING-SDK-CONTAINER.md%23running-the-sdk&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z3%2FQeloU%2FAfPTeAWT7QLB9ORtEJ2V4Y9v0%2B3zX8D4L4%3D&reserved=0
> In order to get a feeling about its usage (you need Docker or Podman),
> follow these simple copy&paste instructions:
> Build instructions are available in the upper part of that document.
>
> Two new dependencies are required to create containerized root
> filesystems (as specified in the documentation).
>
> Typical container image management actions (e.g. push an image to a
> container image regitry) are out of scope. Available tools (Docker,
> Skopeo, Buildah, Podman,...) should be used for these actions.
>
> A patch will follow this one to get the dependencies into the container
> images being provided by the project
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fkas&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C2cd3aa178ab34860762f08d8d1b71e87%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637489931711752194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=lwdYFdlfKLiKEYMhyx3VW15%2B5t70noxUgJhwOCXQ5Fo%3D&reserved=0 (for `kas-container`, for example).
>
> Two new dependencies are required to create containerized root
> filesystems (as specified in the documentation).
>
> Typical container image management actions (e.g. push an image to a
> container image regitry) are out of scope. Available tools (Docker,
> Skopeo, Buildah, Podman,...) should be used for these actions.
>
> A patch will follow this one to get the dependencies into the container
> images being provided by the project
>
> Silvano Cirujano Cuesta (4):
> classes: add root filesystem containerizing class
> classes: add new image type 'container-img'
> sdk: add support for containerized sdk
> docs: document creation of container images
>
> doc/user_manual.md | 127 ++++++++++++++++++
> meta/classes/container-img.bbclass | 18 +++
> .../classes/image-container-extension.bbclass | 79 +++++++++++
> meta/classes/image-sdk-extension.bbclass | 42 +++++-
> meta/classes/image.bbclass | 1 +
> 5 files changed, 260 insertions(+), 7 deletions(-)
> create mode 100644 meta/classes/container-img.bbclass
> create mode 100644 meta/classes/image-container-extension.bbclass
>
--
Siemens AG, T RDA IOT SES-DE
> Silvano Cirujano Cuesta (4):
> classes: add root filesystem containerizing class
> classes: add new image type 'container-img'
> sdk: add support for containerized sdk
> docs: document creation of container images
>
> doc/user_manual.md | 127 ++++++++++++++++++
> meta/classes/container-img.bbclass | 18 +++
> .../classes/image-container-extension.bbclass | 79 +++++++++++
> meta/classes/image-sdk-extension.bbclass | 42 +++++-
> meta/classes/image.bbclass | 1 +
> 5 files changed, 260 insertions(+), 7 deletions(-)
> create mode 100644 meta/classes/container-img.bbclass
> create mode 100644 meta/classes/image-container-extension.bbclass
>
--
Jan Kiszka
Feb 23, 2021, 3:08:45 AM2/23/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
Jan
--
Siemens AG, T RDA IOT
Anton Mikanovich
Mar 2, 2021, 4:05:26 AM3/2/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
(in docker-archive format probably, not to ruin kas-docker building)?
--
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov
Silvano Cirujano Cuesta
Mar 2, 2021, 4:30:52 AM3/2/21
to Anton Mikanovich, isar-...@googlegroups.com
>> Jan Kiszka already tested and liked it! =>
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050%2Fissues%2F86%23issuecomment-768907845&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EbMCfrcIp7Ox6g3qu76ZAEbfD0Hi2uFdQg8k3vRRjfQ%3D&reserved=0
>>
>> Successful builds of both containerized target and SDK are available on
>> the same PoC project:
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311580&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GrSzLMcn5FCpBxbfiN6MR1JmDM2br4SiIZ8Sr3T5VZ0%3D&reserved=0
>> Successful builds of both containerized target and SDK are available on
>> the same PoC project:
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311581&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KOK%2FrnO2koZGMMvmVTvY2btxk8pQtSk94erdp6kSgjw%3D&reserved=0
>> and also the resulting images:
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=a7PR38IxA8bv3w3AEytnAI3AmTxWEMNsgsdMh%2FB09z4%3D&reserved=0
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-sdk-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cUwlR32HYINq2uZFbsAgDpc5RVh4OeECEb8z1tFy%2Fpc%3D&reserved=0
>>
>> In order to get a feeling about its usage (you need Docker or Podman),
>> follow these simple copy&paste instructions:
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Fblob%2Fmaster%2Fkas%2FBUILDING-SDK-CONTAINER.md%23running-the-sdk&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=johK8ZhCGDLqu1ylY0zhdsTNs4H9xi8fTXR5PZiG0Co%3D&reserved=0
>> In order to get a feeling about its usage (you need Docker or Podman),
>> follow these simple copy&paste instructions:
>> Build instructions are available in the upper part of that document.
>>
>> Two new dependencies are required to create containerized root
>> filesystems (as specified in the documentation).
>>
>> Typical container image management actions (e.g. push an image to a
>> container image regitry) are out of scope. Available tools (Docker,
>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>
>> A patch will follow this one to get the dependencies into the container
>> images being provided by the project
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fkas&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7Cad4bf8937210493b054b08d8dd5a524a%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637502727284417779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XTlGc05JGsY%2FidOi7OTUU1znz4cAOrU5zO5WLo251kA%3D&reserved=0 (for `kas-container`, for example).
>>
>> Two new dependencies are required to create containerized root
>> filesystems (as specified in the documentation).
>>
>> Typical container image management actions (e.g. push an image to a
>> container image regitry) are out of scope. Available tools (Docker,
>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>
>> A patch will follow this one to get the dependencies into the container
>> images being provided by the project
>>
>> Silvano Cirujano Cuesta (4):
>> classes: add root filesystem containerizing class
>> classes: add new image type 'container-img'
>> sdk: add support for containerized sdk
>> docs: document creation of container images
>>
>> doc/user_manual.md | 127 ++++++++++++++++++
>> meta/classes/container-img.bbclass | 18 +++
>> .../classes/image-container-extension.bbclass | 79 +++++++++++
>> meta/classes/image-sdk-extension.bbclass | 42 +++++-
>> meta/classes/image.bbclass | 1 +
>> 5 files changed, 260 insertions(+), 7 deletions(-)
>> create mode 100644 meta/classes/container-img.bbclass
>> create mode 100644 meta/classes/image-container-extension.bbclass
>>
> Do you plan to put both sdk and rootfs container creation under CI case (in docker-archive format probably, not to ruin kas-docker building)?
Sorry, I don't get the question. Could you please rephrase the question?
>> Silvano Cirujano Cuesta (4):
>> classes: add root filesystem containerizing class
>> classes: add new image type 'container-img'
>> sdk: add support for containerized sdk
>> docs: document creation of container images
>>
>> doc/user_manual.md | 127 ++++++++++++++++++
>> meta/classes/container-img.bbclass | 18 +++
>> .../classes/image-container-extension.bbclass | 79 +++++++++++
>> meta/classes/image-sdk-extension.bbclass | 42 +++++-
>> meta/classes/image.bbclass | 1 +
>> 5 files changed, 260 insertions(+), 7 deletions(-)
>> create mode 100644 meta/classes/container-img.bbclass
>> create mode 100644 meta/classes/image-container-extension.bbclass
>>
> Do you plan to put both sdk and rootfs container creation under CI case (in docker-archive format probably, not to ruin kas-docker building)?
In any case both of them can be run locally or on CI. In fact I've carefully selected the tools so that they can run on unprivileged containers being widely extended tools (umoci in fact is part of the OCI toolset).
BR,
Silvano
Anton Mikanovich
Mar 2, 2021, 8:30:08 AM3/2/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
not used by any target in Isar, so it makes those parts of code to be
not checked during ci_build.sh execution and not maintained in future.
We should avoid merging unused code.
Hope I'm clarified it now.
Silvano Cirujano Cuesta
Mar 9, 2021, 4:02:41 PM3/9/21
to isar-...@googlegroups.com
This patch series provides support for containerized root filesystems,
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized target root filesystems `IMAGE_TYPE` has
to be `container-img`, additionally the container image format can be
selected with the variable `CONTAINER_FORMATS`. The default format is
`docker-archive`.
In order to build containerized SDKs the variable `SDK_FORMAT` has to
provide any of the supported container formats (e.g. `docker-archive`).
The default format is the legacy non-containerized: `tar_xz`.
It also provides a sample machine, multiconfigs and ci-testing.
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized target root filesystems `IMAGE_TYPE` has
to be `container-img`, additionally the container image format can be
selected with the variable `CONTAINER_FORMATS`. The default format is
`docker-archive`.
In order to build containerized SDKs the variable `SDK_FORMAT` has to
provide any of the supported container formats (e.g. `docker-archive`).
The default format is the legacy non-containerized: `tar_xz`.
More information about its usage is documented in the file
docs/user_manual.md.
A PoC/demo of this functionality (only the SDK part) has been created
Jan Kiszka already tested and liked it! =>
https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845
Successful builds of both containerized target and SDK are available on
the same PoC project:
- https://github.com/Silvanoc/meta-iot2050/actions/runs/558311581
and also the resulting images:
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-arm64
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-sdk-arm64
In order to get a feeling about its usage (you need Docker or Podman),
follow these simple copy&paste instructions:
Build instructions are available in the upper part of that document.
Two new dependencies (umoci and skopeo -backporting from bullseye to
buster works easily) are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
https://github.com/siemens/kas (for `kas-container`, for example).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
Silvano Cirujano Cuesta (5):
classes: add root filesystem containerizing class
classes: add new image type 'container-img'
sdk: add support for containerized sdk
docs: document creation of container images
ci: add container image testing configurations
classes: add new image type 'container-img'
sdk: add support for containerized sdk
docs: document creation of container images
doc/user_manual.md | 127 ++++++++++++++++++
meta-isar/conf/machine/container.conf | 5 +
.../conf/multiconfig/container-bullseye.conf | 4 +
.../conf/multiconfig/container-buster.conf | 4 +
.../conf/multiconfig/container-focal.conf | 4 +
.../conf/multiconfig/container-stretch.conf | 4 +
meta/classes/container-img.bbclass | 18 +++
.../classes/image-container-extension.bbclass | 81 +++++++++++
meta/classes/image-sdk-extension.bbclass | 42 +++++-
meta/classes/image.bbclass | 1 +
scripts/ci_build.sh | 11 +-
meta/classes/image.bbclass | 1 +
11 files changed, 293 insertions(+), 8 deletions(-)
create mode 100644 meta-isar/conf/machine/container.conf
create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
create mode 100644 meta-isar/conf/multiconfig/container-focal.conf
create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
create mode 100644 meta/classes/container-img.bbclass
create mode 100644 meta/classes/image-container-extension.bbclass
--
create mode 100644 meta/classes/image-container-extension.bbclass
2.30.1
Silvano Cirujano Cuesta
Mar 9, 2021, 4:02:42 PM3/9/21
to isar-...@googlegroups.com
Add a new "image" class for generating a container image containing the
target root filesystem.
target root filesystem.
meta/classes/container-img.bbclass | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
1 file changed, 18 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
index 0000000..79ef3e8
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,18 @@
--- /dev/null
+++ b/meta/classes/container-img.bbclass
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the task 'containerize_rootfs'
+# to create container images containing the target rootfs.
+
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMATS"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbdebug 1 "Generate container image in these formats: ${CONTAINER_FORMATS}"
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}"
+}
+
+addtask container_image before do_image after do_image_tools
--
2.30.1
+# to create container images containing the target rootfs.
+
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMATS"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbdebug 1 "Generate container image in these formats: ${CONTAINER_FORMATS}"
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}"
+}
+
+addtask container_image before do_image after do_image_tools
--
Silvano Cirujano Cuesta
Mar 9, 2021, 4:02:42 PM3/9/21
to isar-...@googlegroups.com
Anton Mikanovich
Mar 10, 2021, 11:18:10 AM3/10/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
14:17:17 + SDK_FORMAT=docker-archive
14:17:17 + BB_ENV_WHITELIST=' SDK_FORMAT'
14:17:17 + bitbake -c do_populate_sdk mc:qemuarm-stretch:isar-image-base
...
14:17:18 ERROR: Unable to parse
/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb
14:17:18 Traceback (most recent call last):
14:17:18 File
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py",
line 382, in
multi_finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>):
14:17:18 try:
14:17:18 > finalize(fn, d)
14:17:18 except bb.parse.SkipRecipe as e:
14:17:18 File
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py",
line 347, in
finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>, variant=None):
14:17:18
14:17:18 > bb.parse.siggen.finalise(fn, d, variant)
14:17:18
14:17:18 File
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py", line
182, in
SignatureGeneratorBasicHash.finalise(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>, variant=None):
14:17:18 try:
14:17:18 > taskdeps = self._build_data(fn, d)
14:17:18 except bb.parse.SkipRecipe:
14:17:18 File
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py", line
150, in
SignatureGeneratorBasicHash._build_data(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>):
14:17:18 ignore_mismatch =
((d.getVar("BB_HASH_IGNORE_MISMATCH") or '') == '1')
14:17:18 > tasklist, gendeps, lookupcache =
bb.data.generate_dependencies(d, self.basewhitelist)
14:17:18
14:17:18 File
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/data.py", line
371, in generate_dependencies(d=<bb.data_smart.DataSmart object at
0x7f12917e1c50>, whitelist={'SHELL', 'DEPLOY_DIR', 'PRSERV_DUMPFILE',
'SDKPKGSUFFIX', 'FILE_DIRNAME', 'TERM', 'COREBASE', 'LICENSE_PATH',
'STAGING_DIR_HOST', 'SSTATE_PKGARCH', 'PKGDATA_DIR', 'THISDIR', 'USER',
'EXTERNAL_TOOLCHAIN', 'PWD', 'DL_DIR', 'LOGNAME', 'PATH',
'BB_LIMITEDDEPS', 'CCACHE', 'BB_TASKHASH', 'CCACHE_DIR',
'PARALLEL_MAKE', 'BBSERVER', 'PRSERV_LOCKDOWN', 'BBPATH',
'BB_WORKERCONTEXT', 'PRSERV_HOST', 'WORKDIR', 'FILE',
'CCACHE_NOHASHDIR', 'PRSERV_DUMPDIR', 'FILESEXTRAPATHS', 'HOME',
'BUILD_ARCH', 'STAMPCLEAN', 'SSTATE_DIR', 'TMPDIR', 'FILESPATH',
'STAGING_DIR_TARGET'}):
14:17:18 keys = set(key for key in d if not key.startswith("__"))
14:17:18 > shelldeps = set(key for key in
d.getVar("__exportlist", False) if d.getVarFlag(key, "export", False)
and not d.getVarFlag(key, "unexport", False))
14:17:18 varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS')
14:17:18 TypeError: 'NoneType' object is not iterable
Full log:
http://ci.isar-build.org:8080/job/isar_am_devel_fast/92/consoleFull
Henning Schild
Mar 10, 2021, 11:28:15 AM3/10/21
to [ext] Silvano Cirujano Cuesta, isar-...@googlegroups.com
Silvano you might want to push this to
https://code.siemens.com/ebsy/debian/isar to make use of our CI.
Henning
Am Tue, 9 Mar 2021 21:52:34 +0100
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cir...@siemens.com>:
https://code.siemens.com/ebsy/debian/isar to make use of our CI.
Henning
Am Tue, 9 Mar 2021 21:52:34 +0100
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cir...@siemens.com>:
Silvano Cirujano Cuesta
Mar 10, 2021, 12:18:26 PM3/10/21
to Anton Mikanovich, isar-...@googlegroups.com
>> Jan Kiszka already tested and liked it! =>
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050%2Fissues%2F86%23issuecomment-768907845&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Os3cY15D2myT5sdPINARBh63iwb3GpE5IlupRGnA0v0%3D&reserved=0
>>
>> Successful builds of both containerized target and SDK are available on
>> the same PoC project:
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311580&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=N3DUdGxhNOWlvbatGxopSWPWsMXsXdvsKmIsOh1KtyU%3D&reserved=0
>> Successful builds of both containerized target and SDK are available on
>> the same PoC project:
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311581&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cOkongJ%2BtbrEvaquB241NbaK1nQHHiY%2FrgbYHPFAjeo%3D&reserved=0
>> and also the resulting images:
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UegaXdy58CUxX8%2FsDnG8cy3V9uy9IUyh8yA00U%2FSaR4%3D&reserved=0
>> - https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-sdk-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wss3OIVsS2ulaQ4Jr7ria0wMna6Y0U8Ahp9M3XQU5PM%3D&reserved=0
>>
>> In order to get a feeling about its usage (you need Docker or Podman),
>> follow these simple copy&paste instructions:
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Fblob%2Fmaster%2Fkas%2FBUILDING-SDK-CONTAINER.md%23running-the-sdk&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FiIv247NEqKV%2F2osF6hL5S4QiA6T8yqd5TQpi3EbuGQ%3D&reserved=0
>> In order to get a feeling about its usage (you need Docker or Podman),
>> follow these simple copy&paste instructions:
>> Build instructions are available in the upper part of that document.
>>
>> Two new dependencies (umoci and skopeo -backporting from bullseye to
>> buster works easily) are required to create containerized root
>> filesystems (as specified in the documentation).
>>
>> Typical container image management actions (e.g. push an image to a
>> container image regitry) are out of scope. Available tools (Docker,
>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>
>> A patch will follow this one to get the dependencies into the container
>> images being provided by the project
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fkas&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980099579%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eozG3LQR9pRGIcNiz9a0DckXhkBu8pcas9c%2F%2BWon9mU%3D&reserved=0 (for `kas-container`, for example).
>>
>> Two new dependencies (umoci and skopeo -backporting from bullseye to
>> buster works easily) are required to create containerized root
>> filesystems (as specified in the documentation).
>>
>> Typical container image management actions (e.g. push an image to a
>> container image regitry) are out of scope. Available tools (Docker,
>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>
>> A patch will follow this one to get the dependencies into the container
>> images being provided by the project
>
Henning Schild
Mar 10, 2021, 12:25:17 PM3/10/21
to [ext] Silvano Cirujano Cuesta, Anton Mikanovich, isar-...@googlegroups.com
Am Wed, 10 Mar 2021 18:18:23 +0100
AFAIK the jenkins runner is debian buster running "shell" jobs. New
deps will not work here.
I did not review the changes, but hope those new deps have been added
to the docs.
Henning
deps will not work here.
I did not review the changes, but hope those new deps have been added
to the docs.
Henning
Silvano Cirujano Cuesta
Mar 10, 2021, 12:30:19 PM3/10/21
to Henning Schild, Anton Mikanovich, isar-...@googlegroups.com
>>>> Jan Kiszka already tested and liked it! =>
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050%2Fissues%2F86%23issuecomment-768907845&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186866083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KobF91tR3qTPXkVlm1siBENSZ8GI2yVgV%2FUR4wA%2Fa1g%3D&reserved=0
>>>>
>>>> Successful builds of both containerized target and SDK are
>>>> available on the same PoC project:
>>>> -
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311580&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186866083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SfmgjQlfFHt5GTWS1Bkh9fgYzdVus%2BxAQWL1HXU%2FEiM%3D&reserved=0
>>>> Successful builds of both containerized target and SDK are
>>>> available on the same PoC project:
>>>> -
>>>> -
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311581&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186866083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tXRuiVa7uhPLnRG7Ll2oLrz%2BGP85bR6HEOOzTp7J%2Bxo%3D&reserved=0
>>>> and also the resulting images:
>>>> -
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186876071%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BKHSEDqOAHeum%2FGYdREIysEJ1JCTm68npNBZhgYkOr4%3D&reserved=0
>>>> -
>>>> -
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-sdk-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186876071%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EhFGqF5PnTNkeZub1%2FzDzFCClAT0cN4v2A95D85NEjo%3D&reserved=0
>>>>
>>>> In order to get a feeling about its usage (you need Docker or
>>>> Podman), follow these simple copy&paste instructions:
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Fblob%2Fmaster%2Fkas%2FBUILDING-SDK-CONTAINER.md%23running-the-sdk&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186876071%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GMOqu5mVVazU%2FnvpjiWLDoatp2Yi4XmYMkpZNLCMZts%3D&reserved=0
>>>> In order to get a feeling about its usage (you need Docker or
>>>> Podman), follow these simple copy&paste instructions:
>>>> Build instructions are available in the upper part of that
>>>> document.
>>>>
>>>> Two new dependencies (umoci and skopeo -backporting from bullseye
>>>> to buster works easily) are required to create containerized root
>>>> filesystems (as specified in the documentation).
>>>>
>>>> Typical container image management actions (e.g. push an image to a
>>>> container image regitry) are out of scope. Available tools (Docker,
>>>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>>>
>>>> A patch will follow this one to get the dependencies into the
>>>> container images being provided by the project
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fkas&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C9aeeebc3bbcd43639e3408d8e3e97906%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509939186876071%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0XCjNwZVf%2By0WL2C%2BRTRiVmgZT34dL0auBRFila0Uqs%3D&reserved=0
>>>> document.
>>>>
>>>> Two new dependencies (umoci and skopeo -backporting from bullseye
>>>> to buster works easily) are required to create containerized root
>>>> filesystems (as specified in the documentation).
>>>>
>>>> Typical container image management actions (e.g. push an image to a
>>>> container image regitry) are out of scope. Available tools (Docker,
>>>> Skopeo, Buildah, Podman,...) should be used for these actions.
>>>>
>>>> A patch will follow this one to get the dependencies into the
>>>> container images being provided by the project
Silvano
--
Siemens AG, T RDA IOT SES-DE
Silvano Cirujano Cuesta
Mar 12, 2021, 2:58:46 PM3/12/21
to isar-...@googlegroups.com
NEW: issues in sample configurations and CI script fixed and tested on
two different CI environments.
This patch series provides support for containerized root filesystems,
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized target root filesystems `IMAGE_TYPE` has
to be `container-img`, additionally the container image format can be
selected with the variable `CONTAINER_FORMATS`. The default format is
`docker-archive`.
In order to build containerized SDKs the variable `SDK_FORMAT` has to
provide any of the supported container formats (e.g. `docker-archive`).
The default format is the legacy non-containerized: `tar_xz`.
It also provides a sample machine, multiconfigs and ci-testing.
More information about its usage is documented in the file
docs/user_manual.md.
A PoC/demo of this functionality (only the SDK part) has been created
based on the project https://github.com/siemens/meta-iot2050.
Successful builds of both containerized target and SDK are available on
the same PoC project:
- https://github.com/Silvanoc/meta-iot2050/actions/runs/558311580
- https://github.com/Silvanoc/meta-iot2050/actions/runs/558311581
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-sdk-arm64
In order to get a feeling about its usage (you need Docker or Podman),
follow these simple copy&paste instructions:
https://github.com/Silvanoc/meta-iot2050/blob/master/kas/BUILDING-SDK-CONTAINER.md#running-the-sdk
Silvano Cirujano Cuesta (5):
classes: add root filesystem containerizing class
classes: add new image type 'container-img'
sdk: add support for containerized sdk
docs: document creation of container images
ci: add container image sample configurations
doc/user_manual.md | 127 ++++++++++++++++++
meta-isar/conf/local.conf.sample | 3 +
11 files changed, 311 insertions(+), 8 deletions(-)
2.30.1
two different CI environments.
This patch series provides support for containerized root filesystems,
for both target images and SDKs.
For containerized target images the new image type `container-img` has
been added.
For containerized SDKs the task `populate_sdk` has been extended.
Containerized root filesystems are easy to distribute and run, enabling
this way following scenarios:
- Use ISAR to build container images meant to be run only in containers.
- Use the same ISAR configuration to build images for containers, VMs
and bare-metal.
- Easy SDK distribution and "installation".
- Quickly testing certain applications in the workstation using the
target root filesystem.
In order to build containerized target root filesystems `IMAGE_TYPE` has
to be `container-img`, additionally the container image format can be
selected with the variable `CONTAINER_FORMATS`. The default format is
`docker-archive`.
In order to build containerized SDKs the variable `SDK_FORMAT` has to
provide any of the supported container formats (e.g. `docker-archive`).
The default format is the legacy non-containerized: `tar_xz`.
It also provides a sample machine, multiconfigs and ci-testing.
More information about its usage is documented in the file
docs/user_manual.md.
A PoC/demo of this functionality (only the SDK part) has been created
Jan Kiszka already tested and liked it! =>
https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845
Successful builds of both containerized target and SDK are available on
the same PoC project:
- https://github.com/Silvanoc/meta-iot2050/actions/runs/558311581
and also the resulting images:
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-arm64
- https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-sdk-arm64
In order to get a feeling about its usage (you need Docker or Podman),
follow these simple copy&paste instructions:
Build instructions are available in the upper part of that document.
Two new dependencies (umoci and skopeo -backporting from bullseye to
buster works easily) are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
https://github.com/siemens/kas (for `kas-container`, for example).
Two new dependencies (umoci and skopeo -backporting from bullseye to
buster works easily) are required to create containerized root
filesystems (as specified in the documentation).
Typical container image management actions (e.g. push an image to a
container image regitry) are out of scope. Available tools (Docker,
Skopeo, Buildah, Podman,...) should be used for these actions.
A patch will follow this one to get the dependencies into the container
images being provided by the project
Silvano Cirujano Cuesta (5):
classes: add root filesystem containerizing class
classes: add new image type 'container-img'
sdk: add support for containerized sdk
docs: document creation of container images
doc/user_manual.md | 127 ++++++++++++++++++
meta-isar/conf/local.conf.sample | 3 +
meta-isar/conf/machine/container.conf | 5 +
.../conf/multiconfig/container-bullseye.conf | 4 +
.../conf/multiconfig/container-buster.conf | 4 +
.../conf/multiconfig/container-bullseye.conf | 4 +
.../conf/multiconfig/container-buster.conf | 4 +
.../conf/multiconfig/container-stretch.conf | 4 +
meta/classes/container-img.bbclass | 18 +++
.../classes/image-container-extension.bbclass | 82 +++++++++++
meta/classes/container-img.bbclass | 18 +++
meta/classes/image-sdk-extension.bbclass | 42 +++++-
meta/classes/image.bbclass | 1 +
scripts/ci_build.sh | 29 +++-
meta/classes/image.bbclass | 1 +
11 files changed, 311 insertions(+), 8 deletions(-)
create mode 100644 meta-isar/conf/machine/container.conf
create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
create mode 100644 meta/classes/container-img.bbclass
create mode 100644 meta/classes/image-container-extension.bbclass
--
create mode 100644 meta/classes/container-img.bbclass
create mode 100644 meta/classes/image-container-extension.bbclass
2.30.1
Silvano Cirujano Cuesta
Mar 12, 2021, 2:58:47 PM3/12/21
to isar-...@googlegroups.com
Add a new "image" class for generating a container image containing the
target root filesystem.
target root filesystem.
Silvano Cirujano Cuesta
Mar 12, 2021, 3:03:48 PM3/12/21
to isar-...@googlegroups.com
Add samples for the creation of container images and containerized SDKs
as well as testing configurations to the CI script.
meta-isar/conf/local.conf.sample | 3 ++
meta-isar/conf/machine/container.conf | 5 ++++
.../conf/multiconfig/container-bullseye.conf | 4 +++
.../conf/multiconfig/container-buster.conf | 4 +++
.../conf/multiconfig/container-stretch.conf | 4 +++
scripts/ci_build.sh | 29 ++++++++++++++++++-
6 files changed, 48 insertions(+), 1 deletion(-)
index 107496c..87ba26f 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -51,6 +51,9 @@ BBMULTICONFIG = " \
qemuamd64-buster \
qemuamd64-buster-tgz \
qemuamd64-bullseye \
+ container-stretch \
+ container-buster \
+ container-bullseye \
qemumipsel-stretch \
qemumipsel-buster \
qemumipsel-bullseye \
diff --git a/meta-isar/conf/machine/container.conf b/meta-isar/conf/machine/container.conf
new file mode 100644
index 0000000..367d790
--- /dev/null
+++ b/meta-isar/conf/machine/container.conf
@@ -0,0 +1,5 @@
+
+IMAGE_TYPE = "container-img"
+
diff --git a/meta-isar/conf/multiconfig/container-bullseye.conf b/meta-isar/conf/multiconfig/container-bullseye.conf
new file mode 100644
index 0000000..200b241
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-bullseye.conf
@@ -0,0 +1,4 @@
+DISTRO ?= "debian-bullseye"
diff --git a/meta-isar/conf/multiconfig/container-buster.conf b/meta-isar/conf/multiconfig/container-buster.conf
new file mode 100644
index 0000000..78b0324
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-buster.conf
@@ -0,0 +1,4 @@
+DISTRO ?= "debian-buster"
diff --git a/meta-isar/conf/multiconfig/container-stretch.conf b/meta-isar/conf/multiconfig/container-stretch.conf
new file mode 100644
index 0000000..3ff8bcb
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-stretch.conf
@@ -0,0 +1,4 @@
+DISTRO ?= "debian-stretch"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 3868fb6..a8d861d 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -17,6 +17,15 @@ cd "$(dirname "$0")/.."
# Start build in Isar tree by default
BUILD_DIR=./build
+# Check dependencies
+DEPENDENCIES="umoci skopeo"
+for prog in ${DEPENDENCIES} ; do
+ if [ ! -x "$(which $prog)" ] ; then
+ echo "missing $prog in PATH, exiting" >&2
+ exit 1
+ fi
+done
+
BB_ARGS="-v"
TARGETS_SET="\
@@ -33,7 +42,8 @@ TARGETS_SET="\
mc:qemumipsel-buster:isar-image-base \
mc:nand-ubi-demo-buster:isar-image-ubi \
mc:rpi-stretch:isar-image-base \
- mc:qemuamd64-focal:isar-image-base"
+ mc:qemuamd64-focal:isar-image-base \
+ "
# qemu-user-static of <= buster too old to build that
# mc:qemuarm64-buster:isar-image-base
# mc:qemuarm64-bullseye:isar-image-base
@@ -45,6 +55,12 @@ TARGETS_SET_BULLSEYE="\
mc:qemumipsel-bullseye:isar-image-base \
"
+TARGETS_CONTAINERS="\
+ mc:container-stretch:isar-image-base \
+ mc:container-buster:isar-image-base \
+ mc:container-bullseye:isar-image-base \
+"
+
CROSS_TARGETS_SET="\
mc:qemuarm-stretch:isar-image-base \
mc:qemuarm-buster:isar-image-base \
@@ -237,3 +253,14 @@ bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img
+
+# Finalize with containerized images, since they remove some not-needed packages from the local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf
+bitbake $BB_ARGS $TARGETS_CONTAINERS
+while [ -e bitbake.sock ]; do sleep 1; done
+# and SDK container image creation
+SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
+ bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
+while [ -e bitbake.sock ]; do sleep 1; done
+
--
2.30.1
as well as testing configurations to the CI script.
meta-isar/conf/local.conf.sample | 3 ++
meta-isar/conf/machine/container.conf | 5 ++++
.../conf/multiconfig/container-bullseye.conf | 4 +++
.../conf/multiconfig/container-buster.conf | 4 +++
.../conf/multiconfig/container-stretch.conf | 4 +++
scripts/ci_build.sh | 29 ++++++++++++++++++-
6 files changed, 48 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/conf/machine/container.conf
create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
index 107496c..87ba26f 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -51,6 +51,9 @@ BBMULTICONFIG = " \
qemuamd64-buster \
qemuamd64-buster-tgz \
qemuamd64-bullseye \
+ container-stretch \
+ container-buster \
+ container-bullseye \
qemumipsel-stretch \
qemumipsel-buster \
qemumipsel-bullseye \
diff --git a/meta-isar/conf/machine/container.conf b/meta-isar/conf/machine/container.conf
new file mode 100644
index 0000000..367d790
--- /dev/null
+++ b/meta-isar/conf/machine/container.conf
@@ -0,0 +1,5 @@
+# This software is a part of ISAR.
+# Copyright (C) 2015-2017 ilbers GmbH
+
+IMAGE_TYPE = "container-img"
+
diff --git a/meta-isar/conf/multiconfig/container-bullseye.conf b/meta-isar/conf/multiconfig/container-bullseye.conf
new file mode 100644
index 0000000..200b241
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-bullseye.conf
@@ -0,0 +1,4 @@
+# This software is a part of ISAR.
+
+MACHINE ?= "container"
+
+DISTRO ?= "debian-bullseye"
diff --git a/meta-isar/conf/multiconfig/container-buster.conf b/meta-isar/conf/multiconfig/container-buster.conf
new file mode 100644
index 0000000..78b0324
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-buster.conf
@@ -0,0 +1,4 @@
+# This software is a part of ISAR.
+
+MACHINE ?= "container"
+
+DISTRO ?= "debian-buster"
diff --git a/meta-isar/conf/multiconfig/container-stretch.conf b/meta-isar/conf/multiconfig/container-stretch.conf
new file mode 100644
index 0000000..3ff8bcb
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-stretch.conf
@@ -0,0 +1,4 @@
+# This software is a part of ISAR.
+
+MACHINE ?= "container"
+
+DISTRO ?= "debian-stretch"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 3868fb6..a8d861d 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -17,6 +17,15 @@ cd "$(dirname "$0")/.."
# Start build in Isar tree by default
BUILD_DIR=./build
+# Check dependencies
+DEPENDENCIES="umoci skopeo"
+for prog in ${DEPENDENCIES} ; do
+ if [ ! -x "$(which $prog)" ] ; then
+ echo "missing $prog in PATH, exiting" >&2
+ exit 1
+ fi
+done
+
BB_ARGS="-v"
TARGETS_SET="\
@@ -33,7 +42,8 @@ TARGETS_SET="\
mc:qemumipsel-buster:isar-image-base \
mc:nand-ubi-demo-buster:isar-image-ubi \
mc:rpi-stretch:isar-image-base \
- mc:qemuamd64-focal:isar-image-base"
+ mc:qemuamd64-focal:isar-image-base \
+ "
# qemu-user-static of <= buster too old to build that
# mc:qemuarm64-buster:isar-image-base
# mc:qemuarm64-bullseye:isar-image-base
@@ -45,6 +55,12 @@ TARGETS_SET_BULLSEYE="\
mc:qemumipsel-bullseye:isar-image-base \
"
+TARGETS_CONTAINERS="\
+ mc:container-stretch:isar-image-base \
+ mc:container-buster:isar-image-base \
+ mc:container-bullseye:isar-image-base \
+"
+
CROSS_TARGETS_SET="\
mc:qemuarm-stretch:isar-image-base \
mc:qemuarm-buster:isar-image-base \
@@ -237,3 +253,14 @@ bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img
+
+# Finalize with containerized images, since they remove some not-needed packages from the local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf
+bitbake $BB_ARGS $TARGETS_CONTAINERS
+while [ -e bitbake.sock ]; do sleep 1; done
+# and SDK container image creation
+SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
+ bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
+while [ -e bitbake.sock ]; do sleep 1; done
+
--
2.30.1
Silvano Cirujano Cuesta
Mar 12, 2021, 3:08:46 PM3/12/21
to isar-...@googlegroups.com
This class can be used to create container images which root filesystem
is that generated by the do_rootfs task.
Containerized root filesystems have following possible use-cases:
- Using ISAR as a container image builder.
- Simplify distribution of runtime rootfs (binaries, libraries,
configurations, ...) for application development or testing.
- Distributing SDKs.
.../classes/image-container-extension.bbclass | 82 +++++++++++++++++++
is that generated by the do_rootfs task.
Containerized root filesystems have following possible use-cases:
- Using ISAR as a container image builder.
- Simplify distribution of runtime rootfs (binaries, libraries,
configurations, ...) for application development or testing.
- Distributing SDKs.
meta/classes/image.bbclass | 1 +
2 files changed, 83 insertions(+)
create mode 100644 meta/classes/image-container-extension.bbclass
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
new file mode 100644
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
index 0000000..f693627
--- /dev/null
+++ b/meta/classes/image-container-extension.bbclass
@@ -0,0 +1,82 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass for containerizing the root filesystem.
+
+CONTAINER_FORMATS ?= "docker-archive"
+IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache"
+
+CONTAINER_FORMATS ?= "docker-archive"
+
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+ local container_formats="$3"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ rm -rf "${oci_img_dir}"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ # clean-up temporary files
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+ local container_formats="$3"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ rm -rf "${oci_img_dir}"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
Silvano Cirujano Cuesta
Mar 12, 2021, 3:08:46 PM3/12/21
to isar-...@googlegroups.com
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
+ ;;
+ *)
containing the SDK.
+ *)
+ die "unsupported SDK format specified: ${sdk_format}"
+ ;;
+ esac
+ done
+ esac
+ done
- # Create SDK archive
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ # generate the SDK in all the desired container formats
+ if [ -n "${sdk_container_formats}" ] ; then
+ bbnote "Generating SDK container in ${sdk_container_formats} format"
+ containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+ fi
}
+
addtask populate_sdk after do_rootfs
--
2.30.1
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ # generate the SDK in all the desired container formats
+ if [ -n "${sdk_container_formats}" ] ; then
+ bbnote "Generating SDK container in ${sdk_container_formats} format"
+ containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+ fi
}
+
addtask populate_sdk after do_rootfs
--
Silvano Cirujano Cuesta
Mar 12, 2021, 3:08:47 PM3/12/21
to isar-...@googlegroups.com
Baurzhan Ismagulov
Mar 19, 2021, 12:11:26 PM3/19/21
to isar-...@googlegroups.com
On Fri, Mar 12, 2021 at 08:58:42PM +0100, Silvano Cirujano Cuesta wrote:
> --- a/doc/user_manual.md
...
me.
I'd suggest a couple of cosmetic changes:
* "root-filesystem" -> "root filesystem"
* "ISAR" -> "Isar"
* "container. What can endanger" -> "container, which can endanger"
* Wrap at column 79.
What do you think? Would it be ok to do that when we apply it, or should I send
an updated patch?
We'll play a bit with the images, but I'm quite positive we'll be able to merge
this soon.
With kind regards,
Baurzhan.
> --- a/doc/user_manual.md
...
> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
...
> +### Generate container image with root-filesystem
...
> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
...
> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
> +What can endanger the stability and security of the host.
Thanks for implementing this long-standing feature, the series looks good to
> +What can endanger the stability and security of the host.
me.
I'd suggest a couple of cosmetic changes:
* "root-filesystem" -> "root filesystem"
* "ISAR" -> "Isar"
* "container. What can endanger" -> "container, which can endanger"
* Wrap at column 79.
What do you think? Would it be ok to do that when we apply it, or should I send
an updated patch?
We'll play a bit with the images, but I'm quite positive we'll be able to merge
this soon.
With kind regards,
Baurzhan.
Silvano Cirujano Cuesta
Mar 22, 2021, 8:48:25 AM3/22/21
to isar-...@googlegroups.com
On 19/03/2021 17:11, Baurzhan Ismagulov wrote:
> On Fri, Mar 12, 2021 at 08:58:42PM +0100, Silvano Cirujano Cuesta wrote:
>> --- a/doc/user_manual.md
> ...
>> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
> ...
>> +### Generate container image with root-filesystem
> ...
>> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
> ...
>> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
>> +What can endanger the stability and security of the host.
> Thanks for implementing this long-standing feature, the series looks good to
> me.
>
>
> I'd suggest a couple of cosmetic changes:
>
> * "root-filesystem" -> "root filesystem"
> * "ISAR" -> "Isar"
> * "container. What can endanger" -> "container, which can endanger"
> * Wrap at column 79.
>
> What do you think? Would it be ok to do that when we apply it, or should I send
> an updated patch?
>
>
> We'll play a bit with the images, but I'm quite positive we'll be able to merge
> this soon.
I'd be happy to see it happening :-D
>
> We'll play a bit with the images, but I'm quite positive we'll be able to merge
> this soon.
BR,
Silvano
>
>
> With kind regards,
> Baurzhan.
>
Anton Mikanovich
Mar 22, 2021, 8:59:13 AM3/22/21
to Silvano Cirujano Cuesta, isar-...@googlegroups.com
12.03.2021 22:58, Silvano Cirujano Cuesta wrote:
> +# and SDK container image creation
> +SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
> + bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
> +while [ -e bitbake.sock ]; do sleep 1; done
> +
Should be SDK_FORMATS here (or SDK_FORMAT in image-sdk-extension.bbclass).
> +# and SDK container image creation
> +SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
> + bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
> +while [ -e bitbake.sock ]; do sleep 1; done
> +
And it's better to add this to local.conf without BB_ENV_EXTRAWHITE.
Silvano Cirujano Cuesta
Mar 22, 2021, 9:11:40 AM3/22/21
to Anton Mikanovich, isar-...@googlegroups.com
On 22/03/2021 13:59, Anton Mikanovich wrote:
> 12.03.2021 22:58, Silvano Cirujano Cuesta wrote:
>> +# and SDK container image creation
>> +SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
>> + bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
>> +while [ -e bitbake.sock ]; do sleep 1; done
>> +
>
> Should be SDK_FORMATS here (or SDK_FORMAT in image-sdk-extension.bbclass).
> And it's better to add this to local.conf without BB_ENV_EXTRAWHITE.
In general yes, but being it the CI script I was just trying to make it somehow more explicit than changing the local.conf.
But since I don't have a strong opinion on it, I don't mind changing it as you propose.
Silvano
Silvano Cirujano Cuesta
Mar 26, 2021, 8:17:51 AM3/26/21
to isar-...@googlegroups.com
On 19/03/2021 17:11, Baurzhan Ismagulov wrote:
> On Fri, Mar 12, 2021 at 08:58:42PM +0100, Silvano Cirujano Cuesta wrote:
>> --- a/doc/user_manual.md
> ...
>> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
> ...
>> +### Generate container image with root-filesystem
> ...
>> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
> ...
>> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
>> +What can endanger the stability and security of the host.
> Thanks for implementing this long-standing feature, the series looks good to
> me.
>
>
> I'd suggest a couple of cosmetic changes:
>
> * "root-filesystem" -> "root filesystem"
> * "ISAR" -> "Isar"
It's not being consistently used in the document. I'll use it as you propose, but the inconsistency will remain.
>> --- a/doc/user_manual.md
> ...
>> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
> ...
>> +### Generate container image with root-filesystem
> ...
>> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
> ...
>> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
>> +What can endanger the stability and security of the host.
> Thanks for implementing this long-standing feature, the series looks good to
> me.
>
>
> I'd suggest a couple of cosmetic changes:
>
> * "root-filesystem" -> "root filesystem"
> * "ISAR" -> "Isar"
> * "container. What can endanger" -> "container, which can endanger"
> * Wrap at column 79.
FYI there are enough other areas in the document beyond column 79. I'll apply it nevertheless on my patch.
> * Wrap at column 79.
Silvano
Reply all
Reply to author
Forward
0 new messages