[PATCH 0/4] add support for OE's ROOTFS_*_COMMAND

12 views
Skip to first unread message

Cedric Hombourger

unread,
Oct 29, 2018, 12:13:42 PM10/29/18
to isar-...@googlegroups.com, Cedric Hombourger
OpenEmbedded allows custom functions to be called at various stages of
the root file-system generation process. Add similar capabilities to
Isar.

Cedric Hombourger (4):
isar-image-base: introduce and use isar-image class
isar-image: refactor do_rootfs()
base: add 'lib' folder of each layer to python's module search path
isar-image: add support for OE's ROOTFS_*_COMMAND

doc/user_manual.md | 43 +++++++++++
meta-isar/classes/isar-image.bbclass | 89 ++++++++++++++++++++++
.../files => conf/distro}/debian-configscript.sh | 0
.../files => conf/distro}/raspbian-configscript.sh | 0
meta-isar/recipes-core/images/isar-image-base.bb | 57 +-------------
meta/classes/base.bbclass | 4 +
meta/lib/oe/utils.py | 11 +++
7 files changed, 150 insertions(+), 54 deletions(-)
create mode 100644 meta-isar/classes/isar-image.bbclass
rename meta-isar/{recipes-core/images/files => conf/distro}/debian-configscript.sh (100%)
rename meta-isar/{recipes-core/images/files => conf/distro}/raspbian-configscript.sh (100%)
create mode 100644 meta/lib/oe/utils.py

--
2.11.0

Cedric Hombourger

unread,
Oct 29, 2018, 12:13:53 PM10/29/18
to isar-...@googlegroups.com, Cedric Hombourger
Provide an isar-image class to ease the creation of custom image recipes.
A second objective behind this change is for recipes (.bb files) to only
contain meta-data: no code/scripts.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
doc/user_manual.md | 33 ++++++++++++
meta-isar/classes/isar-image.bbclass | 59 ++++++++++++++++++++++
.../files => conf/distro}/debian-configscript.sh | 0
.../files => conf/distro}/raspbian-configscript.sh | 0
meta-isar/recipes-core/images/isar-image-base.bb | 57 ++-------------------
5 files changed, 95 insertions(+), 54 deletions(-)
create mode 100644 meta-isar/classes/isar-image.bbclass
rename meta-isar/{recipes-core/images/files => conf/distro}/debian-configscript.sh (100%)
rename meta-isar/{recipes-core/images/files => conf/distro}/raspbian-configscript.sh (100%)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 42bcd6b..b702c28 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -496,6 +496,39 @@ Packages in the `IMAGE_TRANSIENT_PACKAGES` variable are installed to the image a

---

+## Create a Custom Image Recipe
+
+A custom image recipe may be created to assemble packages of your choice into a root file-system image. The `isar-image` class
+implements a `do_rootfs` function to compile and configure the file-system for you. Prebuilt packages may be selected for
+installation by appending them to the `IMAGE_PREINSTALL` variable while packages created by ISAR should be appended to
+`IMAGE_INSTALL`. A sample image recipe follows.
+
+### Example
+```
+DESCRIPTION = "Sample image recipe for ISAR"
+
+LICENSE = "gpl-2.0"
+LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
+
+PV = "1.0"
+
+IMAGE_PREINSTALL = " \
+ openssh-server \
+"
+
+inherit isar-image
+
+```
+
+### Additional Notes
+
+The distribution selected via the `DISTRO` variable may need to run a post-configuration script after the root file-system
+was assembled. Isar provides scripts for Debian and Raspbian. In the event where a different Debian-based distribution is
+used, your custom image recipe may need to set `DISTRO_CONFIG_SCRIPT` and use `SRC_URI` and `FILESPATH` for the script to
+be copied into the work directory (`WORKDIR`).
+
+---
+
## Add a Custom Application

Before creating a new recipe it's highly recommended to take a look into the BitBake user manual mentioned in Terms and Definitions section.
diff --git a/meta-isar/classes/isar-image.bbclass b/meta-isar/classes/isar-image.bbclass
new file mode 100644
index 0000000..ec2b9e3
--- /dev/null
+++ b/meta-isar/classes/isar-image.bbclass
@@ -0,0 +1,59 @@
+# Root filesystem for target installation
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2018 ilbers GmbH
+
+inherit image
+inherit isar-bootstrap-helper
+
+FILESPATH =. "${LAYERDIR_isar}/conf/distro:"
+SRC_URI += "${@ 'file://${DISTRO_CONFIG_SCRIPT}' if '${DISTRO_CONFIG_SCRIPT}' else '' }"
+
+DEPENDS += "${IMAGE_INSTALL} ${IMAGE_TRANSIENT_PACKAGES}"
+
+IMAGE_TRANSIENT_PACKAGES += "isar-cfg-localepurge"
+
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
+
+ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_isar} describe --tags --dirty --match 'v[0-9].[0-9]*'"
+ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
+
+do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
+ ${IMAGE_ROOTFS}/isar-apt"
+
+do_rootfs() {
+ cat > ${WORKDIR}/fstab << EOF
+# Begin /etc/fstab
+/dev/root / auto defaults 0 0
+proc /proc proc nosuid,noexec,nodev 0 0
+sysfs /sys sysfs nosuid,noexec,nodev 0 0
+devpts /dev/pts devpts gid=5,mode=620 0 0
+tmpfs /run tmpfs defaults 0 0
+devtmpfs /dev devtmpfs mode=0755,nosuid 0 0
+
+# End /etc/fstab
+EOF
+
+ setup_root_file_system --clean --keep-apt-cache \
+ --fstab "${WORKDIR}/fstab" \
+ "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+
+ # Configure root filesystem
+ if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
+ sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
+ sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} "${MACHINE_SERIAL}" \
+ "${BAUDRATE_TTY}"
+ sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
+ fi
+
+ # Cleanup
+ sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list"
+ test ! -e "${IMAGE_ROOTFS}/usr/share/doc/qemu-user-static" && \
+ sudo find "${IMAGE_ROOTFS}/usr/bin" \
+ -maxdepth 1 -name 'qemu-*-static' -type f -delete
+ sudo umount -l ${IMAGE_ROOTFS}/isar-apt
+ sudo rmdir ${IMAGE_ROOTFS}/isar-apt
+ sudo umount -l ${IMAGE_ROOTFS}/dev
+ sudo umount -l ${IMAGE_ROOTFS}/proc
+ sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
+}
diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/conf/distro/debian-configscript.sh
similarity index 100%
rename from meta-isar/recipes-core/images/files/debian-configscript.sh
rename to meta-isar/conf/distro/debian-configscript.sh
diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/conf/distro/raspbian-configscript.sh
similarity index 100%
rename from meta-isar/recipes-core/images/files/raspbian-configscript.sh
rename to meta-isar/conf/distro/raspbian-configscript.sh
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index bf606cc..074e566 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -1,64 +1,13 @@
-# Root filesystem for target installation
+# Base image recipe for ISAR
#
# This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
+# Copyright (C) 2015-2018 ilbers GmbH

DESCRIPTION = "Isar target filesystem"

LICENSE = "gpl-2.0"
LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"

-FILESPATH =. "${LAYERDIR_isar}/recipes-core/images/files:"
-SRC_URI = "file://${DISTRO_CONFIG_SCRIPT}"
-
PV = "1.0"

-inherit image
-inherit isar-bootstrap-helper
-
-DEPENDS += "${IMAGE_INSTALL} ${IMAGE_TRANSIENT_PACKAGES}"
-
-IMAGE_TRANSIENT_PACKAGES += "isar-cfg-localepurge"
-
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
-
-ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_isar} describe --tags --dirty --match 'v[0-9].[0-9]*'"
-ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
-
-do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
- ${IMAGE_ROOTFS}/isar-apt"
-
-do_rootfs() {
- cat > ${WORKDIR}/fstab << EOF
-# Begin /etc/fstab
-/dev/root / auto defaults 0 0
-proc /proc proc nosuid,noexec,nodev 0 0
-sysfs /sys sysfs nosuid,noexec,nodev 0 0
-devpts /dev/pts devpts gid=5,mode=620 0 0
-tmpfs /run tmpfs defaults 0 0
-devtmpfs /dev devtmpfs mode=0755,nosuid 0 0
-
-# End /etc/fstab
-EOF
-
- setup_root_file_system --clean --keep-apt-cache \
- --fstab "${WORKDIR}/fstab" \
- "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
-
- # Configure root filesystem
- sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
- sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} ${MACHINE_SERIAL} \
- ${BAUDRATE_TTY}
-
- # Cleanup
- sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
- sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list"
- test ! -e "${IMAGE_ROOTFS}/usr/share/doc/qemu-user-static" && \
- sudo find "${IMAGE_ROOTFS}/usr/bin" \
- -maxdepth 1 -name 'qemu-*-static' -type f -delete
- sudo umount -l ${IMAGE_ROOTFS}/isar-apt
- sudo rmdir ${IMAGE_ROOTFS}/isar-apt
- sudo umount -l ${IMAGE_ROOTFS}/dev
- sudo umount -l ${IMAGE_ROOTFS}/proc
- sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
-}
+inherit isar-image
--
2.11.0

Cedric Hombourger

unread,
Oct 29, 2018, 12:13:53 PM10/29/18
to isar-...@googlegroups.com, Cedric Hombourger
Split the do_rootfs() shell function into four (smaller) shell functions to
generate the fstab file (isar_image_gen_fstab), populate the root file-system
(isar_image_gen_rootfs), configure it (isar_image_conf_rootfs) and perform
some house-keeping (isar_image_cleanup). do_rootfs is now a python function
and sequentially calls the helper functions listed above.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
meta-isar/classes/isar-image.bbclass | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/meta-isar/classes/isar-image.bbclass b/meta-isar/classes/isar-image.bbclass
index ec2b9e3..e0508be 100644
--- a/meta-isar/classes/isar-image.bbclass
+++ b/meta-isar/classes/isar-image.bbclass
@@ -21,7 +21,7 @@ ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
${IMAGE_ROOTFS}/isar-apt"

-do_rootfs() {
+isar_image_gen_fstab() {
cat > ${WORKDIR}/fstab << EOF
# Begin /etc/fstab
/dev/root / auto defaults 0 0
@@ -33,11 +33,15 @@ devtmpfs /dev devtmpfs mode=0755,nosuid 0 0

# End /etc/fstab
EOF
+}

+isar_image_gen_rootfs() {
setup_root_file_system --clean --keep-apt-cache \
--fstab "${WORKDIR}/fstab" \
"${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+}

+isar_image_conf_rootfs() {
# Configure root filesystem
if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
@@ -45,7 +49,9 @@ EOF
"${BAUDRATE_TTY}"
sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
fi
+}

+isar_image_cleanup() {
# Cleanup
sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list"
test ! -e "${IMAGE_ROOTFS}/usr/share/doc/qemu-user-static" && \
@@ -57,3 +63,10 @@ EOF
sudo umount -l ${IMAGE_ROOTFS}/proc
sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
}
+
+python do_rootfs() {
+ bb.build.exec_func("isar_image_gen_fstab", d)
+ bb.build.exec_func("isar_image_gen_rootfs", d)
+ bb.build.exec_func("isar_image_conf_rootfs", d)
+ bb.build.exec_func("isar_image_cleanup", d)
+}
--
2.11.0

Cedric Hombourger

unread,
Oct 29, 2018, 12:13:55 PM10/29/18
to isar-...@googlegroups.com, Cedric Hombourger
This is something that OpenEmbedded does and desirable for Isar to
further extend its helper classes.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
meta/classes/base.bbclass | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index fce1084..af6a379 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -69,6 +69,10 @@ root_cleandirs() {
}

python() {
+ import sys
+ bbpath = d.getVar("BBPATH").split(":")
+ sys.path[0:0] = [os.path.join(dir, "lib") for dir in bbpath]
+
import re
for e in d.keys():
flags = d.getVarFlags(e)
--
2.11.0

Cedric Hombourger

unread,
Oct 29, 2018, 12:13:58 PM10/29/18
to isar-...@googlegroups.com, Cedric Hombourger
OpenEmbedded allows custom functions to be called at various stages of
the root file-system generation process. Add similar capabilities to
Isar.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
doc/user_manual.md | 10 ++++++++++
meta-isar/classes/isar-image.bbclass | 17 +++++++++++++++++
meta/lib/oe/utils.py | 11 +++++++++++
3 files changed, 38 insertions(+)
create mode 100644 meta/lib/oe/utils.py

diff --git a/doc/user_manual.md b/doc/user_manual.md
index b702c28..0c4ed4a 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -527,6 +527,16 @@ was assembled. Isar provides scripts for Debian and Raspbian. In the event where
used, your custom image recipe may need to set `DISTRO_CONFIG_SCRIPT` and use `SRC_URI` and `FILESPATH` for the script to
be copied into the work directory (`WORKDIR`).

+Custom (python) functions may also be called at various steps during the root file-system generation process. They should
+be listed in either `ROOTFS_PREPROCESS_COMMAND` for any functions to be called before the root file-system gets populated,
+`ROOTFS_POSTINSTALL_COMMAND` for functions to be called right after selected packages were installed and finally
+`ROOTFS_POSTPROCESS_COMMAND` after the final configuration script and clean-up tasks were called. Functions listed in these
+variables should be separated by a semi-colon sign (;).
+
+It should also be noted that Isar may no longer execute commands from the created root file-system after it was cleaned-up
+(binaries required to support emulation of foreign architectures would have been removed). Please consider this when
+designing functions to be called from `ROOTFS_POSTPROCESS_COMMAND`.
+
---

## Add a Custom Application
diff --git a/meta-isar/classes/isar-image.bbclass b/meta-isar/classes/isar-image.bbclass
index e0508be..d5358e0 100644
--- a/meta-isar/classes/isar-image.bbclass
+++ b/meta-isar/classes/isar-image.bbclass
@@ -64,9 +64,26 @@ isar_image_cleanup() {
sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
}

+ROOTFS_PREPROCESS_COMMAND ?= ""
+ROOTFS_POSTINSTALL_COMMAND ?= ""
+ROOTFS_POSTPROCESS_COMMAND ?= ""
+
python do_rootfs() {
+ from oe.utils import execute_pre_post_process
+
bb.build.exec_func("isar_image_gen_fstab", d)
+
+ pre_process_cmds = d.getVar("ROOTFS_PREPROCESS_COMMAND")
+ execute_pre_post_process(d, pre_process_cmds)
+
bb.build.exec_func("isar_image_gen_rootfs", d)
+
+ post_install_cmds = d.getVar("ROOTFS_POSTINSTALL_COMMAND")
+ execute_pre_post_process(d, post_install_cmds)
+
bb.build.exec_func("isar_image_conf_rootfs", d)
bb.build.exec_func("isar_image_cleanup", d)
+
+ post_process_cmds = d.getVar("ROOTFS_POSTPROCESS_COMMAND")
+ execute_pre_post_process(d, post_process_cmds)
}
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
new file mode 100644
index 0000000..dc3a872
--- /dev/null
+++ b/meta/lib/oe/utils.py
@@ -0,0 +1,11 @@
+# heavily stripped-down copy of OE's meta/lib/oe/utils.py
+
+def execute_pre_post_process(d, cmds):
+ if cmds is None:
+ return
+
+ for cmd in cmds.strip().split(';'):
+ cmd = cmd.strip()
+ if cmd != '':
+ bb.note("Executing %s ..." % cmd)
+ bb.build.exec_func(cmd, d)
--
2.11.0

Henning Schild

unread,
Oct 29, 2018, 12:45:09 PM10/29/18
to Cedric Hombourger, isar-...@googlegroups.com
Hey,

could you please elaborate on why this is needed and what you want to
do with it? I suspect there are already other ways of doing what you
might want to do.

One golden rule is that _everything_ inside the rootfs comes from
a .deb package. If you need to execute scripts, go for post/pre inst
etc.

Henning

Am Mon, 29 Oct 2018 17:12:59 +0100
schrieb Cedric Hombourger <Cedric_H...@mentor.com>:

Hombourger, Cedric

unread,
Oct 29, 2018, 12:55:15 PM10/29/18
to Henning Schild, isar-...@googlegroups.com
Sure thing. This will allow us to generate license/package manifests like Yocto does amongst other things (eg rootfs optimization functions). In other words system wide tweaks. The former is the subject of an upcoming patch series.

Sent from a mobile

Henning Schild

unread,
Oct 30, 2018, 5:25:28 AM10/30/18
to Hombourger, Cedric, isar-...@googlegroups.com
Am Mon, 29 Oct 2018 16:55:10 +0000
schrieb "Hombourger, Cedric" <Cedric_H...@mentor.com>:

> Sure thing. This will allow us to generate license/package manifests
> like Yocto does amongst other things (eg rootfs optimization
> functions). In other words system wide tweaks. The former is the
> subject of an upcoming patch series.

rootfs optimization functions sounds very much like hacking your way
around the golden rule i mentioned earlier. And since people will abuse
such a mechanism it is best to not introduce it.

For the license manifests ... that sounds interesting indeed but will
likely not require such a generic mechanism.

I guess concrete examples would be useful. But looking just at the
mechanism without such examples, i am strongly against merging that
code.

Henning

Hombourger, Cedric

unread,
Oct 30, 2018, 7:02:54 AM10/30/18
to Henning Schild, isar-...@googlegroups.com
prelink or ima would not be considered hacks or kludges but definitely things you apply on the complete file-system. I am failing to understand your resistance... Yocto is no different, have you looked at their uses of the pre/post rootfs commands? They also have packages and apply package level optimizations in their recipes (and we also should) but the aforementioned examples do not fit this category at all

Looking forward to your response

Sent from a mobile

Henning Schild

unread,
Oct 30, 2018, 8:41:39 AM10/30/18
to Hombourger, Cedric, isar-...@googlegroups.com
Am Tue, 30 Oct 2018 11:02:49 +0000
schrieb "Hombourger, Cedric" <Cedric_H...@mentor.com>:

> prelink or ima would not be considered hacks or kludges but
> definitely things you apply on the complete file-system. I am failing
> to understand your resistance... Yocto is no different, have you
> looked at their uses of the pre/post rootfs commands? They also have
> packages and apply package level optimizations in their recipes (and
> we also should) but the aforementioned examples do not fit this
> category at all

The resistance comes from a mechanism without a user, a mechanism that
can (and will) be abused. First hit on google ...
https://stackoverflow.com/questions/36863682/yoctos-rootfs-postprocess-command-not-working
Once we have the use-case we can find the right mechanism or add a new
one.

Have a look at IMAGE_TRANSIENT_PACKAGES. I think prelink could be done
with an apt hook as well, so you have a raw pkg depending on prelink and
shipping the apt config.

Henning

Hombourger, Cedric

unread,
Oct 30, 2018, 8:46:03 AM10/30/18
to Henning Schild, isar-...@googlegroups.com
Ok i will submit a new patch series with one or several users. The Yocto flow is IMO easier to control than postinst scripts (e.g. another postinst script could alter some of the files you have just hashed for IMA).

Sent from a mobile

Cedric Hombourger

unread,
Oct 31, 2018, 2:11:07 AM10/31/18
to isar-...@googlegroups.com, Cedric Hombourger
Reduce the previous patch series to only include the first two patches.
The changes will allow users to easily create their own image recipes.
Documentation was updated with instructions.

The second patch refactors do_rootfs() into smaller functions to reduce
its complexity (one function == one purpose/goal). It was also changed
to become a Python function so it may be extended by other classes with
Python code.

Patches introducing ROOTFS_*_COMMAND are on hold. I will investigate use
of e.g. IMAGE_TRANSIENT_PACKAGES to implement some of the features we are
currently carrying in our own project layers.

Cedric Hombourger (2):
isar-image-base: introduce and use isar-image class
isar-image: refactor do_rootfs()

doc/user_manual.md | 33 ++++++++++
meta-isar/classes/isar-image.bbclass | 72 ++++++++++++++++++++++
.../files => conf/distro}/debian-configscript.sh | 0
.../files => conf/distro}/raspbian-configscript.sh | 0
meta-isar/recipes-core/images/isar-image-base.bb | 57 +----------------
5 files changed, 108 insertions(+), 54 deletions(-)
create mode 100644 meta-isar/classes/isar-image.bbclass
rename meta-isar/{recipes-core/images/files => conf/distro}/debian-configscript.sh (100%)
rename meta-isar/{recipes-core/images/files => conf/distro}/raspbian-configscript.sh (100%)

--
2.11.0

Cedric Hombourger

unread,
Oct 31, 2018, 2:11:10 AM10/31/18
to isar-...@googlegroups.com, Cedric Hombourger
Provide an isar-image class to ease the creation of custom image recipes.
A second objective behind this change is for recipes (.bb files) to only
contain meta-data: no code/scripts.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
doc/user_manual.md | 33 ++++++++++++
meta-isar/classes/isar-image.bbclass | 59 ++++++++++++++++++++++
.../files => conf/distro}/debian-configscript.sh | 0
.../files => conf/distro}/raspbian-configscript.sh | 0
meta-isar/recipes-core/images/isar-image-base.bb | 57 ++-------------------
5 files changed, 95 insertions(+), 54 deletions(-)
create mode 100644 meta-isar/classes/isar-image.bbclass
rename meta-isar/{recipes-core/images/files => conf/distro}/debian-configscript.sh (100%)
rename meta-isar/{recipes-core/images/files => conf/distro}/raspbian-configscript.sh (100%)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 42bcd6b..b702c28 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
+used, your custom image recipe may need to set `DISTRO_CONFIG_SCRIPT` and use `SRC_URI` and `FILESPATH` for the script to
+be copied into the work directory (`WORKDIR`).
+
+---
+
## Add a Custom Application

Before creating a new recipe it's highly recommended to take a look into the BitBake user manual mentioned in Terms and Definitions section.
diff --git a/meta-isar/classes/isar-image.bbclass b/meta-isar/classes/isar-image.bbclass
new file mode 100644
+ "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+

Cedric Hombourger

unread,
Oct 31, 2018, 2:11:12 AM10/31/18
to isar-...@googlegroups.com, Cedric Hombourger
Split the do_rootfs() shell function into four (smaller) shell functions to
generate the fstab file (isar_image_gen_fstab), populate the root file-system
(isar_image_gen_rootfs), configure it (isar_image_conf_rootfs) and perform
some house-keeping (isar_image_cleanup). do_rootfs is now a python function
and sequentially calls the helper functions listed above.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
meta-isar/classes/isar-image.bbclass | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/meta-isar/classes/isar-image.bbclass b/meta-isar/classes/isar-image.bbclass
index ec2b9e3..e0508be 100644
--- a/meta-isar/classes/isar-image.bbclass
+++ b/meta-isar/classes/isar-image.bbclass
@@ -21,7 +21,7 @@ ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
${IMAGE_ROOTFS}/isar-apt"

-do_rootfs() {
+isar_image_gen_fstab() {
cat > ${WORKDIR}/fstab << EOF
# Begin /etc/fstab
/dev/root / auto defaults 0 0
@@ -33,11 +33,15 @@ devtmpfs /dev devtmpfs mode=0755,nosuid 0 0

# End /etc/fstab
EOF
+}

+isar_image_gen_rootfs() {
setup_root_file_system --clean --keep-apt-cache \
--fstab "${WORKDIR}/fstab" \
"${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+}

+isar_image_conf_rootfs() {
# Configure root filesystem
if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
@@ -45,7 +49,9 @@ EOF
"${BAUDRATE_TTY}"
sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
fi
+}

+isar_image_cleanup() {
# Cleanup
sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list"
test ! -e "${IMAGE_ROOTFS}/usr/share/doc/qemu-user-static" && \
@@ -57,3 +63,10 @@ EOF
sudo umount -l ${IMAGE_ROOTFS}/proc
sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
}
+
+python do_rootfs() {
+ bb.build.exec_func("isar_image_gen_fstab", d)
+ bb.build.exec_func("isar_image_gen_rootfs", d)
+ bb.build.exec_func("isar_image_conf_rootfs", d)
+ bb.build.exec_func("isar_image_cleanup", d)
+}
--
2.11.0

Jan Kiszka

unread,
Oct 31, 2018, 2:39:14 AM10/31/18
to Cedric Hombourger, isar-...@googlegroups.com, Baurzhan Ismagulov
Before we move things in meta-isar around: I'd like to first of all get rid of
the dependency on meta-isar for building typical images. Everything that is not
of demonstration purpose (specific boards, example recipes etc.) should go to
the core layer (meta). Therefore, we should not introduce further generic
classes to meta-isar.

One background: I already ran into conflict with machine conf files in meta-isar
that I wanted to replace with own configs in a downstream layer. While config
overloading is possible, it's not as obvious as bbappend, nor does it work well
in the presence of kas. So the better approach is avoiding it in the first place
- e.g. by not pulling meta-isar.

Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

chomb...@gmail.com

unread,
Oct 31, 2018, 2:51:57 AM10/31/18
to isar-users


On Wednesday, October 31, 2018 at 7:39:14 AM UTC+1, Jan Kiszka wrote:
Before we move things in meta-isar around: I'd like to first of all get rid of
the dependency on meta-isar for building typical images. Everything that is not
of demonstration purpose (specific boards, example recipes etc.) should go to
the core layer (meta). Therefore, we should not introduce further generic
classes to meta-isar. 

Hi Jan,

That's great to know, I was wondering if we had some guidelines on what should go to meta vs meta-isar
We will probably need to document this somewhere and introduce some mechanisms to have our CI jobs check if isar/meta is buildable without isar/meta-isar

In the short term, I can amend the patch series to move the class to isar/meta
I believe we could also place isar-image-base there but leave isar-image-debug in isar/meta-isar since the definition of a debug image we have there appears to be just an example
What do you think?

Thanks
Cedric 

Cedric Hombourger

unread,
Nov 1, 2018, 6:13:46 AM11/1/18
to isar-...@googlegroups.com, Cedric Hombourger
Provide an isar-image class to ease the creation of custom image recipes.
A second objective behind this change is for recipes (.bb files) to only
contain meta-data: no code/scripts.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
doc/user_manual.md | 33 ++++++++++++
.../files => conf/distro}/debian-configscript.sh | 0
.../files => conf/distro}/raspbian-configscript.sh | 0
meta-isar/recipes-core/images/isar-image-base.bb | 57 ++-------------------
meta/classes/isar-image.bbclass | 59 ++++++++++++++++++++++
5 files changed, 95 insertions(+), 54 deletions(-)
rename meta-isar/{recipes-core/images/files => conf/distro}/debian-configscript.sh (100%)
rename meta-isar/{recipes-core/images/files => conf/distro}/raspbian-configscript.sh (100%)
create mode 100644 meta/classes/isar-image.bbclass
- sudo umount -l ${IMAGE_ROOTFS}/dev
- sudo umount -l ${IMAGE_ROOTFS}/proc
- sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
-}
+inherit isar-image
diff --git a/meta/classes/isar-image.bbclass b/meta/classes/isar-image.bbclass
new file mode 100644
index 0000000..ec2b9e3
--- /dev/null
+++ b/meta/classes/isar-image.bbclass
+ "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+
+ # Configure root filesystem
+ if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
+ sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
+ sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} "${MACHINE_SERIAL}" \
+ "${BAUDRATE_TTY}"
+ sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
+ fi
+
+ # Cleanup
+ sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list"
+ test ! -e "${IMAGE_ROOTFS}/usr/share/doc/qemu-user-static" && \
+ sudo find "${IMAGE_ROOTFS}/usr/bin" \
+ -maxdepth 1 -name 'qemu-*-static' -type f -delete
+ sudo umount -l ${IMAGE_ROOTFS}/isar-apt
+ sudo rmdir ${IMAGE_ROOTFS}/isar-apt
+ sudo umount -l ${IMAGE_ROOTFS}/dev
+ sudo umount -l ${IMAGE_ROOTFS}/proc
+ sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
+}
--
2.11.0

Cedric Hombourger

unread,
Nov 1, 2018, 6:13:50 AM11/1/18
to isar-...@googlegroups.com, Cedric Hombourger
Split the do_rootfs() shell function into four (smaller) shell functions to
generate the fstab file (isar_image_gen_fstab), populate the root file-system
(isar_image_gen_rootfs), configure it (isar_image_conf_rootfs) and perform
some house-keeping (isar_image_cleanup). do_rootfs is now a python function
and sequentially calls the helper functions listed above.

Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
---
meta/classes/isar-image.bbclass | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/meta/classes/isar-image.bbclass b/meta/classes/isar-image.bbclass
index ec2b9e3..e0508be 100644
--- a/meta/classes/isar-image.bbclass
+++ b/meta/classes/isar-image.bbclass
@@ -21,7 +21,7 @@ ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
${IMAGE_ROOTFS}/isar-apt"

-do_rootfs() {
+isar_image_gen_fstab() {
cat > ${WORKDIR}/fstab << EOF
# Begin /etc/fstab
/dev/root / auto defaults 0 0
@@ -33,11 +33,15 @@ devtmpfs /dev devtmpfs mode=0755,nosuid 0 0

# End /etc/fstab
EOF
+}

+isar_image_gen_rootfs() {
setup_root_file_system --clean --keep-apt-cache \
--fstab "${WORKDIR}/fstab" \
"${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+}

+isar_image_conf_rootfs() {
# Configure root filesystem
if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
@@ -45,7 +49,9 @@ EOF
"${BAUDRATE_TTY}"
sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
fi
+}

+isar_image_cleanup() {
# Cleanup
sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list"
test ! -e "${IMAGE_ROOTFS}/usr/share/doc/qemu-user-static" && \
@@ -57,3 +63,10 @@ EOF
sudo umount -l ${IMAGE_ROOTFS}/proc
sudo rm -f "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"
}
+

Maxim Yu. Osipov

unread,
Nov 1, 2018, 7:43:24 AM11/1/18
to chomb...@gmail.com, isar-users, Jan Kiszka
On 10/31/18 9:51 AM, chomb...@gmail.com wrote:
>
>
> On Wednesday, October 31, 2018 at 7:39:14 AM UTC+1, Jan Kiszka wrote:
>
> Before we move things in meta-isar around: I'd like to first of all
> get rid of
> the dependency on meta-isar for building typical images. Everything
> that is not
> of demonstration purpose (specific boards, example recipes etc.)
> should go to
> the core layer (meta). Therefore, we should not introduce further
> generic
> classes to meta-isar.
>
>
> Hi Jan,
>
> That's great to know, I was wondering if we had some guidelines on what
> should go to meta vs meta-isar
> We will probably need to document this somewhere and introduce some
> mechanisms to have our CI jobs check if isar/meta is buildable without
> isar/meta-isar

Definitely we need to discuss such meta-isar refactoring:

1) Should we keep under meta-isar only support for several reference
boards with some board specific recipes (bsp related, demo apps etc)?
So support for other third-party boards shipped as separate layer
will not include meta-isar?

2) What can be incorporated into isar from one of the isar users -
jailhouse-images?

> In the short term, I can amend the patch series to move the class to
> isar/meta
> I believe we could also place isar-image-base there but leave
> isar-image-debug in isar/meta-isar since the definition of a debug image
> we have there appears to be just an example
> What do you think?

In my opinion, debug image is generic enough so it has to be kept
together with isar-image-base.

Thanks,
Maxim.


> Thanks
> Cedric
>
> --
> You received this message because you are subscribed to the Google
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to isar-users+...@googlegroups.com
> <mailto:isar-users+...@googlegroups.com>.
> To post to this group, send email to isar-...@googlegroups.com
> <mailto:isar-...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/f1c6cbd0-fd05-4e6e-a77d-b5867f501ec3%40googlegroups.com
> <https://groups.google.com/d/msgid/isar-users/f1c6cbd0-fd05-4e6e-a77d-b5867f501ec3%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mos...@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Maxim Yu. Osipov

unread,
Nov 7, 2018, 8:07:51 AM11/7/18
to Cedric Hombourger, isar-...@googlegroups.com
On 11/1/18 1:13 PM, Cedric Hombourger wrote:
> Provide an isar-image class to ease the creation of custom image recipes.
> A second objective behind this change is for recipes (.bb files) to only
> contain meta-data: no code/scripts.

Applied V3 series to the 'next',

Thanks,
Maxim.

Jan Kiszka

unread,
Nov 7, 2018, 8:20:12 AM11/7/18
to Cedric Hombourger, isar-...@googlegroups.com, Maksim Osipov
Why making this a python function?

What is the reasoning for the chosen split-up? These are all recipe APIs, so we
must be more careful with such declarations.

Henning Schild

unread,
Nov 7, 2018, 1:45:33 PM11/7/18
to Cedric Hombourger, isar-...@googlegroups.com
Am Thu, 1 Nov 2018 11:13:02 +0100
schrieb Cedric Hombourger <Cedric_H...@mentor.com>:
I am loosing my proxies somewhere in current next ... i suspect this
guy but did not confirm yet.

Henning

Cedric Hombourger

unread,
Nov 7, 2018, 2:46:50 PM11/7/18
to Henning Schild, Cedric Hombourger, isar-...@googlegroups.com
Hi Henning,

I apologize if that's the case. On my side, the proxy appears to be used just fine. I have added a trace in isar_image_gen_rootfs() and I can see:

DEBUG: Executing python function do_rootfs
DEBUG: Executing shell function root_cleandirs
DEBUG: Shell function root_cleandirs finished
DEBUG: Executing shell function isar_image_gen_fstab
DEBUG: Shell function isar_image_gen_fstab finished
DEBUG: Executing shell function isar_image_gen_rootfs
######## http_proxy=http://127.0.0.1:3142 ##########

I also checked my apt-cacher-ng and I saw a bunch of requests coming in while do_rootfs was executing.
Will double-check on a machine that does not have direct access to the Internet to make sure.

Cedric

--
You received this message because you are subscribed to a topic in the Google Groups "isar-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/isar-users/Pzt7vsbc8VE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to isar-users+...@googlegroups.com.
To post to this group, send email to isar-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20181107194531.17356c59%40md1za8fc.ad001.siemens.net.

Cedric Hombourger

unread,
Nov 7, 2018, 3:16:23 PM11/7/18
to Henning Schild, Cedric Hombourger, isar-...@googlegroups.com
Hi again,

My test build on a machine _without_ a direct Internet connection has completed.
Any idea how I can reproduce your issue? or help troubleshoot?

Note: tested with HEAD = 60db83cdcae60bdca6d3e3ed2cc7c44712d4dd91

Cedric

Jan Kiszka

unread,
Nov 8, 2018, 1:52:17 AM11/8/18
to Cedric Hombourger, Henning Schild, Cedric Hombourger, isar-...@googlegroups.com
On 07.11.18 21:16, Cedric Hombourger wrote:
> Hi again,
>
> My test build on a machine _without_ a direct Internet connection has completed.
> Any idea how I can reproduce your issue? or help troubleshoot?
>
> Note: tested with HEAD = 60db83cdcae60bdca6d3e3ed2cc7c44712d4dd91
>

Works for me as well, and the code changes do not touch anything proxy-related.

I'm more concerned about the structure of this change, as commented in the other
mail. Cedric, could you follow up on that?

Thanks,
Jan

> Cedric
>
> On Wed, 7 Nov 2018 at 20:46, Cedric Hombourger <chomb...@gmail.com
> <mailto:chomb...@gmail.com>> wrote:
>
> Hi Henning,
>
> I apologize if that's the case. On my side, the proxy appears to be used
> just fine. I have added a trace in isar_image_gen_rootfs() and I can see:
>
> DEBUG: Executing python function do_rootfs
> DEBUG: Executing shell function root_cleandirs
> DEBUG: Shell function root_cleandirs finished
> DEBUG: Executing shell function isar_image_gen_fstab
> DEBUG: Shell function isar_image_gen_fstab finished
> DEBUG: Executing shell function isar_image_gen_rootfs
> ######## http_proxy=http://127.0.0.1:3142 ##########
>
> I also checked my apt-cacher-ng and I saw a bunch of requests coming in
> while do_rootfs was executing.
> Will double-check on a machine that does not have direct access to the
> Internet to make sure.
>
> Cedric
>
> On Wed, 7 Nov 2018 at 19:45, Henning Schild <henning...@siemens.com
> <mailto:henning...@siemens.com>> wrote:
>
> Am Thu, 1 Nov 2018 11:13:02 +0100
> schrieb Cedric Hombourger <Cedric_H...@mentor.com
> <mailto:Cedric_H...@mentor.com>>:
>
> > Split the do_rootfs() shell function into four (smaller) shell
> > functions to generate the fstab file (isar_image_gen_fstab), populate
> > the root file-system (isar_image_gen_rootfs), configure it
> > (isar_image_conf_rootfs) and perform some house-keeping
> > (isar_image_cleanup). do_rootfs is now a python function and
> > sequentially calls the helper functions listed above.
> >
> > Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com
> <mailto:Cedric_H...@mentor.com>>
> <mailto:isar-users%2Bunsu...@googlegroups.com>.
> To post to this group, send email to isar-...@googlegroups.com
> <mailto:isar-...@googlegroups.com>.
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+...@googlegroups.com
> <mailto:isar-users+...@googlegroups.com>.
> To post to this group, send email to isar-...@googlegroups.com
> <mailto:isar-...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/CAKbb94fWOV4NN2WMuXCvRF%2B5%2BV73w8N0ZcS43vZbPmS_kckQ_A%40mail.gmail.com
> <https://groups.google.com/d/msgid/isar-users/CAKbb94fWOV4NN2WMuXCvRF%2B5%2BV73w8N0ZcS43vZbPmS_kckQ_A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


Hombourger, Cedric

unread,
Nov 8, 2018, 2:59:33 AM11/8/18
to Jan Kiszka, Maksim Osipov, isar-...@googlegroups.com
Hi Jan,

> Why making this a python function?

To align with Yocto and ease extension of do_rootfs should we need to (we probably want to have some guidelines for public APIs and one of them should be: make them Python functions because it allows children classes to use the full Python paradigm instead of the somewhat limited shell environment)

> What is the reasoning for the chosen split-up?

If you are referring to the split of do_rootfs() into smaller functions, the primary objective was the reduction of the function complexity. Another candidate for such refactoring is IMO setup_root_file_system(): it's a function doing dozens of little things. It also allows classes inheriting from isar-image to hook themselves at specific points in the do_rootfs flow instead of being limited to pre-processing and post-processing extension points (do_rootfs_prepend / do_rootfs_append).

> These are all recipe APIs, so we must be more careful with such declarations.

You are right. If isar-image-base was viewed as an API, we probably want to highlight the change in the ChangeLog. I can do that.

Cedric

Henning Schild

unread,
Nov 8, 2018, 3:13:17 AM11/8/18
to Cedric Hombourger, Cedric Hombourger, isar-...@googlegroups.com
Am Wed, 7 Nov 2018 21:16:08 +0100
schrieb Cedric Hombourger <chomb...@gmail.com>:

> Hi again,
>
> My test build on a machine _without_ a direct Internet connection has
> completed.
> Any idea how I can reproduce your issue? or help troubleshoot?
>
> Note: tested with HEAD = 60db83cdcae60bdca6d3e3ed2cc7c44712d4dd91

False alarm. Last night several machines had issues fetching files in
debootstrap (with and without proxies), must have been a temporary
server-side issue.

Henning

Jan Kiszka

unread,
Nov 8, 2018, 3:28:53 AM11/8/18
to Hombourger, Cedric, Maksim Osipov, isar-...@googlegroups.com
On 08.11.18 08:59, Hombourger, Cedric wrote:
> Hi Jan,
>
>> Why making this a python function?
>
> To align with Yocto and ease extension of do_rootfs should we need to (we probably want to have some guidelines for public APIs and one of them should be: make them Python functions because it allows children classes to use the full Python paradigm instead of the somewhat limited shell environment)

That is not the actual reason why Yocto did this. The reason, which would have
helped me to understand the design, is that it allows downstream to define the
language of the subfunctions freely. bb.build.exec_func will take care of
calling the function as shell or as python, whatever the user chose.

But, as the subfunctions are shell code, appending/prepending still requires
shell. If you do that for the rootfs function, that must now be python, even if
shell would be more appropriate. Here, it is impossible with bitbake to provide
freedom. If you want flexibility, you better model this with tasks. That is
something to consider seriously if there is a hooking use case.

>
>> What is the reasoning for the chosen split-up?
>
> If you are referring to the split of do_rootfs() into smaller functions, the primary objective was the reduction of the function complexity. Another candidate for such refactoring is IMO setup_root_file_system(): it's a function doing dozens of little things. It also allows classes inheriting from isar-image to hook themselves at specific points in the do_rootfs flow instead of being limited to pre-processing and post-processing extension points (do_rootfs_prepend / do_rootfs_append).

I'm not against breaking this task up, but the question is where, and also how.
And that should be driven by foreseeable use cases.

Eg., isar_image_conf_rootfs seems pointless at this stage because there is
already DISTRO_CONFIG_SCRIPT to do customizations. If we want that thing to be
extensible (rather than just replaceable), we need something else.

isar_image_gen_fstab is possibly something we do not want users to play with
because that could easily conflict with what wic does.

>
>> These are all recipe APIs, so we must be more careful with such declarations.
>
> You are right. If isar-image-base was viewed as an API, we probably want to highlight the change in the ChangeLog. I can do that.

Once it is clear why and how a user should overload / append to those APIs.

Baurzhan Ismagulov

unread,
Nov 9, 2018, 8:26:25 AM11/9/18
to isar-users
On Tue, Oct 30, 2018 at 11:51:57PM -0700, chomb...@gmail.com wrote:
> That's great to know, I was wondering if we had some guidelines on what
> should go to meta vs meta-isar

meta is the generic one ("Isar itself") and meta-isar is a demo one that may be
used as a template for your product.


> We will probably need to document this somewhere and introduce some
> mechanisms to have our CI jobs check if isar/meta is buildable without
> isar/meta-isar

That would be a good litmus test for meta vs. meta-isar decisions. Currently,
we don't have an image in meta.


> In the short term, I can amend the patch series to move the class to
> isar/meta

If isar-image.bbclass would help creating product images, it's indeed worth
having in meta.


> I believe we could also place isar-image-base there but leave
> isar-image-debug in isar/meta-isar since the definition of a debug image we
> have there appears to be just an example

Originally, isar-image-base was also just an example reusable by copy-pasting.
If isar-image.bbclass goes to meta and isar-image-base.bb becomes a two-liner,
it would make sense to include the latter in meta as well. That would allow CI
testing of meta alone. I'd keep isar-image-debug.bb alongside
isar-image-base.bb, separating those would be unhandy.

Regarding machine support, one approach could be supporting all Debian QEMU
targets in meta, leaving the rest in meta-isar. I'm in favor of adding new
boards to meta-isar, since people end up implementing them in their own repos.
In the long term, if meta-isar grows, we could consider moving it into a
separate repo and renaming e.g. meta -> meta-isar, meta-isar -> meta-isar-demo
if that is more intuitive.


With kind regards,
Baurzhan.

Baurzhan Ismagulov

unread,
Nov 9, 2018, 8:42:39 AM11/9/18
to isar-...@googlegroups.com
On Mon, Oct 29, 2018 at 05:13:00PM +0100, Cedric Hombourger wrote:
> A second objective behind this change is for recipes (.bb files) to only
> contain meta-data: no code/scripts.
...
> .../files => conf/distro}/debian-configscript.sh | 0
> .../files => conf/distro}/raspbian-configscript.sh | 0

The scripts have been deliberately placed alongside the recipe till we have
enough evidence that they are 1. universal enough (serve 80% of users as they
are) and 2. readily overridable (so that users are not forced to copy-paste the
whole thing to remove an undesired customization). If they contain generic
pluggable parts selectively reusable downstream, that could be useful. Besides,
I find having scripts under conf/ not very intuitive. What is the problem with
having the scripts in the recipe?

With kind regards,
Baurzhan.

Jan Kiszka

unread,
Nov 29, 2018, 10:06:56 AM11/29/18
to Cedric Hombourger, isar-...@googlegroups.com
On 29.10.18 17:13, Cedric Hombourger wrote:
> Split the do_rootfs() shell function into four (smaller) shell functions to
> generate the fstab file (isar_image_gen_fstab), populate the root file-system
> (isar_image_gen_rootfs), configure it (isar_image_conf_rootfs) and perform
> some house-keeping (isar_image_cleanup). do_rootfs is now a python function
> and sequentially calls the helper functions listed above.
>
> Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
> ---
> meta-isar/classes/isar-image.bbclass | 15 ++++++++++++++-
> 1 file changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/meta-isar/classes/isar-image.bbclass b/meta-isar/classes/isar-image.bbclass
> index ec2b9e3..e0508be 100644
> --- a/meta-isar/classes/isar-image.bbclass
> +++ b/meta-isar/classes/isar-image.bbclass
Just realized a downside of this pattern: run.do_rootfs logs do not contain the
subcommand traces, only this rather boring top-level function. Not critical, but
also not nice (because I'm hunting a bug down there).

If you have an idea how to restoring logging, that would be great!
Reply all
Reply to author
Forward
0 new messages