[PATCH] test: provide ci_shell as a convienence wrapper

[Cedric Hombourger]

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script
to invoke `kas-container` for you. The script uses a simple
kas fragment (`kas/ci-shell.yml`) to use your existing copy
of the Isar code checkout: this makes it easier to test your
work-in-progress changes. Lastly, `ci_setup.sh` will produce
less output (e.g. when installing dependencies) and will
print a few hints to remind users with useful commands.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---
CONTRIBUTING.md | 52 +++++++++++----------------------------------
kas/ci-shell.yml | 19 +++++++++++++++++
scripts/ci_setup.sh | 32 +++++++++++++++++++++++-----
scripts/ci_shell | 12 +++++++++++
4 files changed, 70 insertions(+), 45 deletions(-)
create mode 100644 kas/ci-shell.yml
create mode 100755 scripts/ci_shell

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 07621232..6970477b 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -81,54 +81,26 @@ Plan merges to `master` so that both fit the two-week window; short extensions s
by launching CI tests scripts. The procedure is described below:

```
- git clone https://github.com/siemens/kas
- cat > kas.yml <<EOF
- header:
- version: 14
- build_system: isar
- defaults:
- repos:
- patches:
- repo: isar
- repos:
- isar:
- url: "http://github.com:/ilbers/isar"
- branch: next
- layers:
- meta:
- meta-isar:
- EOF
- kas/kas-container shell --command /work/isar/scripts/ci_setup.sh kas.yml
+ ./scripts/ci_shell
```

- In kas shell:
+ This will spawn a shell where you may just run:

```
- cd /work/isar/testsuite
- avocado run citest.py -t dev --max-parallel-tasks=1
+ trun citest.py -t dev --max-parallel-tasks=1
```

- Your git-formatpatches may be listed in the `kas.yml` file as illustrated below:
+ Note: (`trun` is an alias for `avocado run`).

- ```
- ...
- repos:
- isar:
- url: "http://github.com:/ilbers/isar"
- branch: next
- patches:
- 0001:
- path: /work/0001-my-contribution-to-isar.patch
- layers:
- meta:
- meta-isar:
- ```
-
- Perform the above steps from a clean directory for your CI run to be as close as
- possible to the environment that our project maintainers will be using. That
- directory would contain: *.patch isar/ kas/ kas.yml
+ The container has `/work` bind-mounted to your local copy of the Isar code
+ base (and therefore with your changes). Make sure to track upstream changes
+ and regularly rebase your tree: your patches shall be sent against the latest
+ HEAD available when submitting. Keep an eye on changes while your series are
+ under review: you may need to send a refreshed version if contributions from
+ others were accepted and touch the same areas (make sure to re-run the test
+ suite after rebasing).

- Be also mindful of community-provided resources such as deb.debian.org or
+ Be mindful of community-provided resources such as deb.debian.org or
snapshot.debian.org and consider using a caching proxy in your setup to
reduce traffic as much as possible.

diff --git a/kas/ci-shell.yml b/kas/ci-shell.yml
new file mode 100644
index 00000000..16ce8b42
--- /dev/null
+++ b/kas/ci-shell.yml
@@ -0,0 +1,19 @@
+# This software is a part of Isar.
+# Copyright (C) 2023-2024 ilbers GmbH
+
+header:
+ version: 14
+
+build_system: isar
+
+repos:
+ isar:
+ layers:
+ meta:
+ meta-isar:
+ meta-test:
+
+bblayers_conf_header:
+ standard: |
+ BBPATH = "${TOPDIR}"
+ BBFILES ?= ""
diff --git a/scripts/ci_setup.sh b/scripts/ci_setup.sh
index f373d1c0..03cf3452 100755
--- a/scripts/ci_setup.sh
+++ b/scripts/ci_setup.sh
@@ -9,24 +9,46 @@ gpg_key=/etc/apt/trusted.gpg.d/debian-isar.gpg
[ -f "${gpg_key}" ] || {
wget -q http://deb.isar-build.org/debian-isar.key -O- \
| gpg --dearmor \
- | sudo dd of="${gpg_key}"
+ | sudo dd of="${gpg_key}" status=none
}

list=/etc/apt/sources.list.d/10-isar_build.list
[ -f "${list}" ] || {
echo "deb [signed-by=/etc/apt/trusted.gpg.d/debian-isar.gpg] \
http://deb.isar-build.org/debian-isar bookworm-isar main" \
- | sudo tee /etc/apt/sources.list.d/10-isar_build.list
+ | sudo tee /etc/apt/sources.list.d/10-isar_build.list >/dev/null
}

+echo "🔍 Checking dependencies..."
tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64"
need_install=0
for tool in ${tools}; do
which "${tool}" || need_install=1
done
[ "${need_install}" = "0" ] || {
- sudo apt-get update
- sudo apt-get install -y avocado qemu-system-arm qemu-system-x86
+ echo "⏳ Installing missing dependencies..."
+ (
+ sudo apt-get update
+ sudo apt-get install -y avocado qemu-system-arm qemu-system-x86
+ ) >/dev/null
}

-exec /container-entrypoint ${*}
+(
+ cat <<' EOF'
+ alias tlist="avocado list"
+ alias trun="avocado run"
+ cd /work/testsuite
+ EOF
+) | sudo tee -a /etc/profile.d/ci.sh >/dev/null
+
+cat <<'EOF'
+
+💡 Use "tlist citest.py" to list existing test cases
+💡 Use "trun citest.py:TestClass.test_method" to run a specific test
+💡 Use "trun -p sstate=1 citest.py" to enable sstate
+
+✅ Test environment ready for use!
+
+EOF
+
+exec /container-entrypoint bash -l
diff --git a/scripts/ci_shell b/scripts/ci_shell
new file mode 100755
index 00000000..308bb9fd
--- /dev/null
+++ b/scripts/ci_shell
@@ -0,0 +1,12 @@
+#!/bin/bash
+# A simple wrapper to run a container to run CI tests
+#
+# Cedric Hombourger <cedric.h...@siemens.com>
+# Copyright (c) Siemens AG, 2025
+# SPDX-License-Identifier: MIT
+
+self=${0}
+topdir=$(readlink -f $(dirname ${self})/..)
+
+cd ${topdir}
+kas/kas-container shell --command /work/scripts/ci_setup.sh kas/ci-shell.yml
--
2.47.3

[Cedric Hombourger]

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script
to invoke `kas-container` for you. The script uses a simple
kas fragment (`kas/ci-shell.yml`) to use your existing copy
of the Isar code checkout: this makes it easier to test your
work-in-progress changes. Lastly, `ci_setup.sh` will produce
less output (e.g. when installing dependencies) and will
print a few hints to remind users with useful commands.

Changes since v1:
- use a temporary KAS_BUILD_DIR for the build container since we are
not using kas for building test cases but only as a runtime
environment. By that, the build folder created by kas does not
conflict with the folder created by avocado builds.

Cedric Hombourger (1):
test: provide ci_shell as a convenience wrapper

CONTRIBUTING.md | 52 +++++++++++----------------------------------
kas/ci-shell.yml | 19 +++++++++++++++++
scripts/ci_setup.sh | 32 +++++++++++++++++++++++-----

scripts/ci_shell | 18 ++++++++++++++++
4 files changed, 76 insertions(+), 45 deletions(-)

create mode 100644 kas/ci-shell.yml
create mode 100755 scripts/ci_shell

--
2.47.3

[Cedric Hombourger]

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script
to invoke `kas-container` for you. The script uses a simple
kas fragment (`kas/ci-shell.yml`) to use your existing copy
of the Isar code checkout: this makes it easier to test your
work-in-progress changes. Lastly, `ci_setup.sh` will produce
less output (e.g. when installing dependencies) and will
print a few hints to remind users with useful commands.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

CONTRIBUTING.md | 52 +++++++++++----------------------------------
kas/ci-shell.yml | 19 +++++++++++++++++
scripts/ci_setup.sh | 32 +++++++++++++++++++++++-----
scripts/ci_shell | 18 ++++++++++++++++
4 files changed, 76 insertions(+), 45 deletions(-)
create mode 100644 kas/ci-shell.yml
create mode 100755 scripts/ci_shell

index 00000000..bcb7ffeb
--- /dev/null
+++ b/scripts/ci_shell
@@ -0,0 +1,18 @@

+#!/bin/bash
+# A simple wrapper to run a container to run CI tests
+#
+# Cedric Hombourger <cedric.h...@siemens.com>
+# Copyright (c) Siemens AG, 2025
+# SPDX-License-Identifier: MIT
+
+self=${0}
+topdir=$(readlink -f $(dirname ${self})/..)
+

+export KAS_BUILD_DIR=$(mktemp -d)
+cleanup() {
+ rm -rf --one-file-system ${KAS_BUILD_DIR}
+}
+trap "cleanup" EXIT

[Jan Kiszka]

This is a redundant copy of kas/isar.yaml. Just use the latter.

Having an even easier start wrapper for the testsuite is definitely welcome.

To make the startup times even shorter, we could furthermore host a
prebuilt kas-derived test container image on ghcr.io.

Jan

--
Siemens AG, Foundational Technologies
Linux Expert Center

[Cedric Hombourger]

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script

to invoke `kas-container` for you. The script uses the existing
kas/isar.yml kas file to use your existing checkout of the Isar
code: this makes it easier to test your work-in-progress changes.

Lastly, `ci_setup.sh` will produce less output (e.g. when
installing dependencies) and will print a few hints to remind
users with useful commands.

Changes since v2:
- drop redundant kas/ci-shell.yml since identical to the
existing kas/isar.yml file

Changes since v1:
- use a temporary KAS_BUILD_DIR for the build container since we are
not using kas for building test cases but only as a runtime
environment. By that, the build folder created by kas does not
conflict with the folder created by avocado builds.

Future work:
- Provide a Containerfile to build a container image with package
dependencies pre-installed (as suggested by Jan Kiszka).

Cedric Hombourger (1):
test: provide ci_shell as a convenience wrapper

CONTRIBUTING.md | 52 +++++++++++----------------------------------

scripts/ci_setup.sh | 32 +++++++++++++++++++++++-----
scripts/ci_shell | 18 ++++++++++++++++

3 files changed, 57 insertions(+), 45 deletions(-)
create mode 100755 scripts/ci_shell

--
2.47.3

[Cedric Hombourger]

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script
to invoke `kas-container` for you. The script uses the existing
kas/isar.yml kas file to use your existing checkout of the Isar
code: this makes it easier to test your work-in-progress changes.

Lastly, `ci_setup.sh` will produce less output (e.g. when
installing dependencies) and will print a few hints to remind
users with useful commands.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

CONTRIBUTING.md | 52 +++++++++++----------------------------------
scripts/ci_setup.sh | 32 +++++++++++++++++++++++-----
scripts/ci_shell | 18 ++++++++++++++++
3 files changed, 57 insertions(+), 45 deletions(-)
create mode 100755 scripts/ci_shell

index 00000000..9481a8ef

--- /dev/null
+++ b/scripts/ci_shell
@@ -0,0 +1,18 @@
+#!/bin/bash
+# A simple wrapper to run a container to run CI tests
+#
+# Cedric Hombourger <cedric.h...@siemens.com>
+# Copyright (c) Siemens AG, 2025
+# SPDX-License-Identifier: MIT
+
+self=${0}
+topdir=$(readlink -f $(dirname ${self})/..)
+
+export KAS_BUILD_DIR=$(mktemp -d)
+cleanup() {
+ rm -rf --one-file-system ${KAS_BUILD_DIR}
+}
+trap "cleanup" EXIT
+
+cd ${topdir}

+kas/kas-container shell --command /work/scripts/ci_setup.sh kas/isar.yml
--
2.47.3

[Jan Kiszka]

It's isar.yaml by now - do you plan to update this patch?

I'm still convinced we need this as workaround while we do not have
pre-packaged test framework or test container. I would would even go
further and think about forwarding parameters into the shell so that one
could directly kick off a specific test.

Also way more handy would be if the testsuite could be temporarily
parameterized to reduce larger test cases to a setup which still
triggers the issue, just faster. This would allow to share the
reproduction setup with patch contributor via a single line:

call_avocado testname --test-parameters

Testing must be thought more from the contributor perspective, less the
(existing) infrastructure.

[MOESSBAUER, Felix]

I'm also looking into this and already have a couple of improvements to
the script. Will send them out today.

This also handles the stupid case of the generated bitbake
pyshtables.py that break the test if bitbake ever was locally executed.

>
> I'm still convinced we need this as workaround while we do not have
> pre-packaged test framework or test container. I would would even go
> further and think about forwarding parameters into the shell so that one
> could directly kick off a specific test.

For that, we need an up-to-date test container. Installing the
dependencies takes too long. But I would also like to see this.

>
> Also way more handy would be if the testsuite could be temporarily
> parameterized to reduce larger test cases to a setup which still
> triggers the issue, just faster. This would allow to share the
> reproduction setup with patch contributor via a single line:
>
> call_avocado testname --test-parameters
>
> Testing must be thought more from the contributor perspective, less the
> (existing) infrastructure.

Agree.

PS: Can we please make caching the default? All tests should be
executed with ccache, sstate cache and download cache per default. Only
tests that explicitly check these features should diverge.

Felix

>
> Jan
>
> --
> Siemens AG, Foundational Technologies
> Linux Expert Center
>

> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/01d937a1-5778-4470-9d6a-2706cf8fd47c%40siemens.com.

--
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

[Felix Moessbauer]

From: Cedric Hombourger' via isar-users <isar-...@googlegroups.com>

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script
to invoke `kas-container` for you. The script uses the existing
kas/isar.yml kas file to use your existing checkout of the Isar
code: this makes it easier to test your work-in-progress changes.

Lastly, `ci_setup.sh` will produce less output (e.g. when
installing dependencies) and will print a few hints to remind
users with useful commands.

[Felix]
- fix pyhashtables cleanup
- speedup installation by skipping recommends
- add more helper commands
- always run with sstate cache
- run everything in tmux to access logs while test is running
- provide pycache path to speedup python startup
- add builtin (memory only) squid cache
- use host network instead of container network (allows use of federated proxies)

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
@Cedric: This is RFC is based on your patchset and tries to further
simplify and speedup the testsuite invocation. Feel free to integrate
whatever you think makes sense. With your patch and my additions,
I finally found a way to efficiently execute single tests. However,
there is still much room for improvement in the testsuite itself.

Best regards,
Felix

CONTRIBUTING.md | 52 ++++++++++----------------------------------
scripts/ci_setup.sh | 53 ++++++++++++++++++++++++++++++++++++++++-----
scripts/ci_shell | 36 ++++++++++++++++++++++++++++++
3 files changed, 95 insertions(+), 46 deletions(-)

index f373d1c0..bbca4ab8 100755
--- a/scripts/ci_setup.sh
+++ b/scripts/ci_setup.sh
@@ -5,28 +5,69 @@

# Copyright (c) Siemens AG, 2025

# SPDX-License-Identifier: MIT

+# start a squid http proxy in the container
+with_squid="1"
+

gpg_key=/etc/apt/trusted.gpg.d/debian-isar.gpg
[ -f "${gpg_key}" ] || {
wget -q http://deb.isar-build.org/debian-isar.key -O- \
| gpg --dearmor \
- | sudo dd of="${gpg_key}"
+ | sudo dd of="${gpg_key}" status=none
}

list=/etc/apt/sources.list.d/10-isar_build.list
[ -f "${list}" ] || {
echo "deb [signed-by=/etc/apt/trusted.gpg.d/debian-isar.gpg] \
http://deb.isar-build.org/debian-isar bookworm-isar main" \
- | sudo tee /etc/apt/sources.list.d/10-isar_build.list
+ | sudo tee /etc/apt/sources.list.d/10-isar_build.list >/dev/null
}

-tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64"
+echo "🔍 Checking dependencies..."
+tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64 htop"
+if [ "${with_squid}" = "1" ]; then
+ tools="${tools} squid"
+ squid_pkg="squid"
+ start_squid="sudo /usr/sbin/squid;"
+ echo "export http_proxy=http://localhost:3128" \
+ | sudo tee -a /etc/profile.d/squid.sh >/dev/null
+fi
+

need_install=0
for tool in ${tools}; do
which "${tool}" || need_install=1
done
[ "${need_install}" = "0" ] || {
- sudo apt-get update
- sudo apt-get install -y avocado qemu-system-arm qemu-system-x86
+ echo "⏳ Installing missing dependencies..."
+ (
+ sudo apt-get update

+ sudo apt-get install -y --no-install-recommends \
+ avocado qemu-system-arm qemu-system-x86 ovmf htop $squid_pkg
+ )

}

-exec /container-entrypoint ${*}
+(
+ cat <<' EOF'
+ alias tlist="avocado list"

+ alias trun="avocado run -p sstate=1 --max-parallel-tasks=1 --failfast"
+ alias trun-fg="avocado --show avocado.test run -p sstate=1 --max-parallel-tasks=1 --failfast"
+ alias tquit="tmux kill-session -t isar_ci"
+ alias thelp="cat ~/.trun-help"
+ export PYTHONPYCACHEPREFIX=/work/build/pycache
+ mkdir -p ${PYTHONPYCACHEPREFIX}

+ cd /work/testsuite
+ EOF
+) | sudo tee -a /etc/profile.d/ci.sh >/dev/null
+

+cat <<'EOF' > ~/.trun-help

+
+💡 Use "tlist citest.py" to list existing test cases
+💡 Use "trun citest.py:TestClass.test_method" to run a specific test

+💡 Use "trun-fg <test> to run a specific test and emit the logs
+💡 Use "tquit" to leave the testing environment
+💡 Use "thelp" to show this help message

+
+✅ Test environment ready for use!
+
+EOF
+

+exec /container-entrypoint tmux new-session -A -s isar_ci "$start_squid cat ~/.trun-help && bash -l"

diff --git a/scripts/ci_shell b/scripts/ci_shell
new file mode 100755

index 00000000..917d1272
--- /dev/null
+++ b/scripts/ci_shell
@@ -0,0 +1,36 @@

+#!/bin/bash
+# A simple wrapper to run a container to run CI tests
+#
+# Cedric Hombourger <cedric.h...@siemens.com>
+# Copyright (c) Siemens AG, 2025
+# SPDX-License-Identifier: MIT

+#
+# Usage: ci_shell [kas-container-version]

+
+self=${0}
+topdir=$(readlink -f $(dirname ${self})/..)
+
+export KAS_BUILD_DIR=$(mktemp -d)
+

+if [[ -n "$1" ]]; then
+ export KAS_IMAGE_VERSION="$1"
+fi
+
+drop_pyhashtables() {
+ # Remove possibly outdated pyhashtables.py file that is generated on-the-fly.
+ # The file is owned by the (bitbake) calling user, hence it's lifetime needs
+ # to be bound to the container's lifetime.
+ rm -f bitbake/lib/bb/pysh/pyshtables.py
+}

+cleanup() {
+ rm -rf --one-file-system ${KAS_BUILD_DIR}

+ drop_pyhashtables

+}
+trap "cleanup" EXIT
+
+cd ${topdir}

+drop_pyhashtables
+kas/kas-container \
+ --runtime-args "--network=host" \
+ shell --command "rm -rf /work/build/conf && /work/scripts/ci_setup.sh" \
+ kas/isar.yaml
--
2.51.0

[Zhihang Wei]

I would suggest to add some instructions on using avocado, like the ones
mentioned in [1], but that can be a separate patch.
[1]
https://lists.isar-build.org/isar-users/c3db73d0-5753-41cd...@ilbers.de

Zhihang

[Jan Kiszka]

On 12.12.25 16:01, Felix Moessbauer wrote:
> From: Cedric Hombourger' via isar-users <isar-...@googlegroups.com>

^^^^^^^^^^^^^^^^^^^^^^^^^^
(found by https://github.com/jan-kiszka/copypatch because signed-off
differs)

[Jan Kiszka]

On 12.12.25 16:01, Felix Moessbauer wrote:

> From: Cedric Hombourger' via isar-users <isar-...@googlegroups.com>
>
> Ease use of the container environment that may be used to run
> tests from this project by adding a `scripts/ci_shell` script
> to invoke `kas-container` for you. The script uses the existing
> kas/isar.yml kas file to use your existing checkout of the Isar
> code: this makes it easier to test your work-in-progress changes.
>
> Lastly, `ci_setup.sh` will produce less output (e.g. when
> installing dependencies) and will print a few hints to remind
> users with useful commands.
>
> [Felix]
> - fix pyhashtables cleanup
> - speedup installation by skipping recommends
> - add more helper commands
> - always run with sstate cache
> - run everything in tmux to access logs while test is running

This is a nice feature - if you like it. I hate it and would strongly
recommend to make this opt-in.

Jan

[MOESSBAUER, Felix]

On Fri, 2025-12-12 at 17:23 +0100, Jan Kiszka wrote:
> On 12.12.25 16:01, Felix Moessbauer wrote:
> > From: Cedric Hombourger' via isar-users <isar-...@googlegroups.com>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
> (found by https://github.com/jan-kiszka/copypatch because signed-off
> differs)

Good catch. I'm wondering why the author association got lost. But this
patch is anyways not for merging but merely a way to express feedback
regarding what the ci wrapper should offer.

Felix

[Jan Kiszka]

On 12.12.25 16:01, Felix Moessbauer wrote:

> +cat <<'EOF' > ~/.trun-help
> +
> +💡 Use "tlist citest.py" to list existing test cases
> +💡 Use "trun citest.py:TestClass.test_method" to run a specific test

In our context, it is almost always "citest.py", no? Then please enhance
the aliases, making it a default so that we do not have to type this all
the time, e.g. "tlist [citest.py]" and "trun
[citest.py:]TestClass.test_method".

> +💡 Use "trun-fg <test> to run a specific test and emit the logs
> +💡 Use "tquit" to leave the testing environment
> +💡 Use "thelp" to show this help message
> +

Jan

[MOESSBAUER, Felix]

On Fri, 2025-12-12 at 17:34 +0100, Jan Kiszka wrote:
> On 12.12.25 16:01, Felix Moessbauer wrote:
> > +cat <<'EOF' > ~/.trun-help
> > +
> > +💡 Use "tlist citest.py" to list existing test cases
> > +💡 Use "trun citest.py:TestClass.test_method" to run a specific test
>
> In our context, it is almost always "citest.py", no? Then please enhance

We also have repro-build-test.py and hopefully more testcase specific
files in the future. However, I'm wondering if there is any value in
letting the user select the test file. Otherwise we could also add a
python module that includes all downstream tests - which is then used
by the trun alias.