[PATCH 0/4] Add test-cases for dracut-based initrds

[Cedric Hombourger]

[1] is adding support for dracut as the initrd generator. That
patch series missed to add tests. Add several smoke tests.
Note that a small change (add "set -e" to the sudo script used
to generate the initrd) is needed for all initrd tests to pass.

Build results with pristine "Add support for dracut" v5:

builder@ed1e0b7e8d4e:/work/testsuite$ trun --max-parallel-tasks=1 -p sstate=1 -t initrd citest.py
JOB ID : f1601d733e2a811ad04738eef7a1bdba28dbcd52
JOB LOG : /tmp/tmp72bozbdi/avocado/job-results/job-2025-11-01T10.19-f1601d7/job.log
(1/5) citest.py:InitRdTest.test_dracut_in_image: STARTED
(1/5) citest.py:InitRdTest.test_dracut_in_image: PASS (206.52 s)
(2/5) citest.py:InitRdTest.test_dracut_build_initrd: STARTED
(2/5) citest.py:InitRdTest.test_dracut_build_initrd: PASS (57.86 s)
(3/5) citest.py:InitRdTest.test_dracut_build_failure: STARTED
(3/5) citest.py:InitRdTest.test_dracut_build_failure: FAIL: Bitbake suceeded but was expected to fail! (159.91 s)
(4/5) citest.py:InitRdCrossTests.test_dracut_in_image: STARTED
(4/5) citest.py:InitRdCrossTests.test_dracut_in_image: PASS (1000.05 s)
(5/5) citest.py:InitRdCrossTests.test_dracut_build_initrd: STARTED
(5/5) citest.py:InitRdCrossTests.test_dracut_build_initrd: PASS (233.79 s)
RESULTS : PASS 4 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 1663.01 s

With the suggested change:

builder@ed1e0b7e8d4e:/work/testsuite$ trun --max-parallel-tasks=1 -p sstate=1 -t initrd citest.py
JOB ID : dc55f13f05f5997669aaccce9760bca4bed7a400
JOB LOG : /tmp/tmp72bozbdi/avocado/job-results/job-2025-11-01T11.54-dc55f13/job.log
(1/5) citest.py:InitRdTest.test_dracut_in_image: STARTED
(1/5) citest.py:InitRdTest.test_dracut_in_image: PASS (117.49 s)
(2/5) citest.py:InitRdTest.test_dracut_build_initrd: STARTED
(2/5) citest.py:InitRdTest.test_dracut_build_initrd: PASS (30.33 s)
(3/5) citest.py:InitRdTest.test_dracut_build_failure: STARTED
(3/5) citest.py:InitRdTest.test_dracut_build_failure: PASS (73.31 s)
(4/5) citest.py:InitRdCrossTests.test_dracut_in_image: STARTED

(4/5) citest.py:InitRdCrossTests.test_dracut_in_image: PASS (430.85 s)
(5/5) citest.py:InitRdCrossTests.test_dracut_build_initrd: STARTED
(5/5) citest.py:InitRdCrossTests.test_dracut_build_initrd: PASS (55.30 s)
RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 711.89 s

[1] https://lists.isar-build.org/isar-users/20251030094451.13038...@siemens.com/T/#t

Cedric Hombourger (4):
CI: support negative build tests
CI: add smoke-tests for switch to dracut in isar-image-ci
CI: add build tests for isar-dracut initrd images
CI: check if the build fails when dracut fails

testsuite/cibase.py | 4 +--
testsuite/cibuilder.py | 12 ++++---
testsuite/citest.py | 81 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 91 insertions(+), 6 deletions(-)

--
2.47.3

[Cedric Hombourger]

Use IMAGE_PREINSTALL to install dracut in the isar-image-ci
image and verify that it builds and boots. We also check that
a dracut-shutdown service is active (as an indication that
the system was booted using a dracut initrd). A single test
is provided as well as cross-{machine,distro} tests.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---
testsuite/citest.py | 58 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 58 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index a1214e9c..23900a6c 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -278,6 +278,64 @@ class KernelTests(CIBaseTest):
self.perform_build_test(targets, image_install=' '.join(modules), lines=lines)


+class InitRdBaseTest(CIBaseTest):
+ """
+ Helper class for InitRd build tests
+ """
+
+ DRACUT_CONF = [
+ "IMAGE_PREINSTALL:append = ' dracut'",
+ "IMAGE_INSTALL:remove = 'enable-fsck'",
+ ]
+
+ def dracut_in_image(self, targets):
+ machine = 'qemuamd64'
+ distro = 'bookworm'
+ image = 'isar-image-ci'
+ self.init()
+ self.perform_build_test(targets, lines=InitRdBaseTest.DRACUT_CONF)
+
+ for target in targets:
+ config = target.split(':')
+ machine = config[1].split('-')[0]
+ distro = config[1].split('-')[1]
+ image = config[2]
+
+ # systems booted via a dracut initrd have a dracut-shutdown service
+ # check its presence
+ self.vm_start(
+ machine.removeprefix('qemu'), distro, image=image,
+ cmd="systemctl is-active dracut-shutdown"
+ )
+
+
+class InitRdTest(InitRdBaseTest):
+ """
+ Tests for initrd generation
+ :avocado: tags=initrd,single,full
+ """
+
+ def test_dracut_in_image(self):
+ """Test switch to dracut in an image recipe."""
+ self.dracut_in_image(['mc:qemuamd64-bookworm:isar-image-ci'])
+
+
+class InitRdCrossTests(InitRdBaseTest):
+ """
+ Tests for initrd generation
+ :avocado: tags=initrd,full
+ """
+
+ def test_dracut_in_image(self):
+ """Test switch to dracut in an image recipe."""
+ self.dracut_in_image([
+ 'mc:qemuamd64-bookworm:isar-image-ci',
+ 'mc:qemuamd64-trixie:isar-image-ci',
+ 'mc:qemuarm64-bookworm:isar-image-ci',
+ 'mc:qemuarm64-trixie:isar-image-ci'
+ ])
+
+
class WicTest(CIBaseTest):

"""
--
2.47.3

[Cedric Hombourger]

It appears that our build tests are only check for successful
builds. We also want to make sure builds fail when they are
expected to. This adds a should_fail boolean to bitbake() and
perform_build_test()

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

testsuite/cibase.py | 4 ++--
testsuite/cibuilder.py | 12 ++++++++----
2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index 6ffedcea..0a006a21 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -14,12 +14,12 @@ from avocado.utils import process


class CIBaseTest(CIBuilder):
- def perform_build_test(self, targets, **kwargs):
+ def perform_build_test(self, targets, should_fail=False, **kwargs):
self.configure(**kwargs)

self.log.info("Starting build...")

- self.bitbake(targets, **kwargs)
+ self.bitbake(targets, should_fail=should_fail, **kwargs)

def perform_wic_partition_test(self, targets, wic_deploy_parts, **kwargs):
self.configure(wic_deploy_parts=wic_deploy_parts, **kwargs)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index cd5e70ba..366f6a1f 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -281,7 +281,8 @@ class CIBuilder(Test):
if os.path.exists(self.build_dir + '/' + src):
shutil.move(self.build_dir + '/' + src, self.build_dir + '/' + dst)

- def bitbake(self, target, bitbake_cmd=None, sig_handler=None, **kwargs):
+ def bitbake(self, target, bitbake_cmd=None, should_fail=False,
+ sig_handler=None, **kwargs):
self.check_init()
self.log.info("===================================================")
self.log.info(f"Building {str(target)}")
@@ -318,13 +319,16 @@ class CIBuilder(Test):
continue
if fd == p1.stdout.fileno():
self.log.info(p1.stdout.readline().rstrip())
- if fd == p1.stderr.fileno():
+ if fd == p1.stderr.fileno() and should_fail is False:
app_log.error(p1.stderr.readline().rstrip())
if p1.poll() is not None:
break
p1.wait()
- if p1.returncode:
- self.fail("Bitbake failed")
+ if should_fail is False:
+ if p1.returncode:
+ self.fail("Bitbake failed")
+ elif p1.returncode == 0:
+ self.fail("Bitbake suceeded but was expected to fail!")

def backupfile(self, path):
self.check_init()
--
2.47.3

[Cedric Hombourger]

The build should not succeed if dracut fails to generate an
initrd. Amend the dracut command line to add an invalid option
(--unknown-option) to force a build failure and check that
bitbake does fail (it did not in the "Add support for dracut"
patch series v5).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

testsuite/citest.py | 8 ++++++++
1 file changed, 8 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 7f046a7f..e53dfa9b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -324,6 +324,14 @@ class InitRdTest(InitRdBaseTest):
self.init()
self.perform_build_test(['mc:qemuamd64-bookworm:isar-dracut'])

+ def test_dracut_build_failure(self):
+ """ Check if the build fails if dracut fails to generate an initrd."""
+ lines = InitRdBaseTest.DRACUT_CONF
+ lines.append("ROOTFS_INITRAMFS_GENERATOR_CMDLINE:append = ' --unknown-option'")
+ self.init()
+ self.perform_build_test('mc:qemuamd64-bookworm:isar-image-ci',
+ should_fail=True, lines=lines)
+

class InitRdCrossTests(InitRdBaseTest):
"""
--
2.47.3

[Zhihang Wei]

I would suggest to move the definition of class InitRdBaseTest(CIBaseTest)
to cibase.py. In citest.py there should only be definition of tests that we
can directly run.

Thanks for adding new test cases. I have tested the v7 dracut patch set
against these test cases and others in current CI, didn't show any error.
We'll continue with this patch set after dracut can be applied.

Best regards,
Zhihang

[cedric.h...@siemens.com]

Are we sure we want to do this? While that's an easy change, I fear
that it would negatively impact readability of the tests that are
defined below. cibase is really at the framework level. The above class
is getting into the specifics.

We will anyhow need to revisit the structure of our test. If we are
successful in getting more test-case contributions (as I am really
hoping and now pushing for), then we would end-up with an enormous
citest.py file. We will need to break it down.

I would therefore like to give us some time to re-think the overall
structure of our testsuite/.

Is that agreeable?

--
Cedric Hombourger
Siemens AG
www.siemens.com