[RFC v2 0/4] kunit: tool: add support for QEMU
25 views
Skip to first unread message
Brendan Higgins
Apr 29, 2021, 4:51:14 PM4/29/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
TL;DR: Add support to kunit_tool to dispatch tests via QEMU. Also add
support to immediately shutdown a kernel after running KUnit tests.
Background
----------
KUnit has supported running on all architectures for quite some time;
however, kunit_tool - the script commonly used to invoke KUnit tests -
has only fully supported KUnit run on UML. Its functionality has been
broken up for some time to separate the configure, build, run, and parse
phases making it possible to be used in part on other architectures to a
small extent. Nevertheless, kunit_tool has not supported running tests
on other architectures.
What this patchset does
-----------------------
This patchset introduces first class support to kunit_tool for KUnit to
be run on many popular architectures via QEMU. It does this by adding
two new flags: `--arch` and `--cross_compile`.
`--arch` allows an architecture to be specified by the name the
architecture is given in `arch/`. It uses the specified architecture to
select a minimal amount of Kconfigs and QEMU configs needed for the
architecture to run in QEMU and provide a console from which KTAP
results can be scraped.
`--cross_compile` allows a toolchain prefix to be specified to make
similar to how `CROSS_COMPILE` is used.
Additionally, this patchset revives the previously considered "kunit:
tool: add support for QEMU"[1] patchs. The motivation for this new
kernel command line flags, `kunit_shutdown`, is to better support
running KUnit tests inside of QEMU. For most popular architectures, QEMU
can be made to terminate when the Linux kernel that is being run is
reboted, halted, or powered off. As Kees pointed out in a previous
discussion[2], it is possible to make a kernel initrd that can reboot
the kernel immediately, doing this for every architecture would likely
be infeasible. Instead, just having an option for the kernel to shutdown
when it is done with testing seems a lot simpler, especially since it is
an option which would only available in testing configurations of the
kernel anyway.
What discussion remains for this patchset?
------------------------------------------
The first most obvious thing is settling the debate about
`kunit_shutdown`. If I recall correctly, Kees suggested that it might be
better to just add a new initrd; however, as I mentioned above, now to
support many new architectures, it may be substantially easier to
support this option. So I am hoping with this new usecase, the argument
for `kunit_shutdown` will be more compelling.
The second and likely harder issue is figuring out the best way to
configure and provide configs for running KUnit tests via QEMU. I
provide a pretty primitive way in this patchset which is not super
flexible; for example, for our PPC support we have it set to build big
endian, and POWER8 - we currently don't support a way to change that.
Nevertheless, having sensible defaults is handy too, so we will probably
want to have some support for overriding defaults, while still being
able to have defaults.
[1] http://patches.linaro.org/patch/208336/
[2] https://lkml.org/lkml/2020/6/26/988
Brendan Higgins (3):
Documentation: Add kunit_shutdown to kernel-parameters.txt
kunit: tool: add support for QEMU
Documentation: kunit: document support for QEMU in kunit_tool
David Gow (1):
kunit: Add 'kunit_shutdown' option
.../admin-guide/kernel-parameters.txt | 8 +
Documentation/dev-tools/kunit/usage.rst | 37 +++-
lib/kunit/executor.c | 20 ++
tools/testing/kunit/kunit.py | 33 ++-
tools/testing/kunit/kunit_config.py | 2 +-
tools/testing/kunit/kunit_kernel.py | 209 +++++++++++++++---
tools/testing/kunit/kunit_parser.py | 2 +-
tools/testing/kunit/kunit_tool_test.py | 15 +-
8 files changed, 278 insertions(+), 48 deletions(-)
base-commit: 7af08140979a6e7e12b78c93b8625c8d25b084e2
--
2.31.1.498.g6c1eba8ee3d-goog
support to immediately shutdown a kernel after running KUnit tests.
Background
----------
KUnit has supported running on all architectures for quite some time;
however, kunit_tool - the script commonly used to invoke KUnit tests -
has only fully supported KUnit run on UML. Its functionality has been
broken up for some time to separate the configure, build, run, and parse
phases making it possible to be used in part on other architectures to a
small extent. Nevertheless, kunit_tool has not supported running tests
on other architectures.
What this patchset does
-----------------------
This patchset introduces first class support to kunit_tool for KUnit to
be run on many popular architectures via QEMU. It does this by adding
two new flags: `--arch` and `--cross_compile`.
`--arch` allows an architecture to be specified by the name the
architecture is given in `arch/`. It uses the specified architecture to
select a minimal amount of Kconfigs and QEMU configs needed for the
architecture to run in QEMU and provide a console from which KTAP
results can be scraped.
`--cross_compile` allows a toolchain prefix to be specified to make
similar to how `CROSS_COMPILE` is used.
Additionally, this patchset revives the previously considered "kunit:
tool: add support for QEMU"[1] patchs. The motivation for this new
kernel command line flags, `kunit_shutdown`, is to better support
running KUnit tests inside of QEMU. For most popular architectures, QEMU
can be made to terminate when the Linux kernel that is being run is
reboted, halted, or powered off. As Kees pointed out in a previous
discussion[2], it is possible to make a kernel initrd that can reboot
the kernel immediately, doing this for every architecture would likely
be infeasible. Instead, just having an option for the kernel to shutdown
when it is done with testing seems a lot simpler, especially since it is
an option which would only available in testing configurations of the
kernel anyway.
What discussion remains for this patchset?
------------------------------------------
The first most obvious thing is settling the debate about
`kunit_shutdown`. If I recall correctly, Kees suggested that it might be
better to just add a new initrd; however, as I mentioned above, now to
support many new architectures, it may be substantially easier to
support this option. So I am hoping with this new usecase, the argument
for `kunit_shutdown` will be more compelling.
The second and likely harder issue is figuring out the best way to
configure and provide configs for running KUnit tests via QEMU. I
provide a pretty primitive way in this patchset which is not super
flexible; for example, for our PPC support we have it set to build big
endian, and POWER8 - we currently don't support a way to change that.
Nevertheless, having sensible defaults is handy too, so we will probably
want to have some support for overriding defaults, while still being
able to have defaults.
[1] http://patches.linaro.org/patch/208336/
[2] https://lkml.org/lkml/2020/6/26/988
Brendan Higgins (3):
Documentation: Add kunit_shutdown to kernel-parameters.txt
kunit: tool: add support for QEMU
Documentation: kunit: document support for QEMU in kunit_tool
David Gow (1):
kunit: Add 'kunit_shutdown' option
.../admin-guide/kernel-parameters.txt | 8 +
Documentation/dev-tools/kunit/usage.rst | 37 +++-
lib/kunit/executor.c | 20 ++
tools/testing/kunit/kunit.py | 33 ++-
tools/testing/kunit/kunit_config.py | 2 +-
tools/testing/kunit/kunit_kernel.py | 209 +++++++++++++++---
tools/testing/kunit/kunit_parser.py | 2 +-
tools/testing/kunit/kunit_tool_test.py | 15 +-
8 files changed, 278 insertions(+), 48 deletions(-)
base-commit: 7af08140979a6e7e12b78c93b8625c8d25b084e2
--
2.31.1.498.g6c1eba8ee3d-goog
Brendan Higgins
Apr 29, 2021, 4:51:16 PM4/29/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
From: David Gow <davi...@google.com>
Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.
Signed-off-by: David Gow <davi...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
---
lib/kunit/executor.c | 20 ++++++++++++++++++++
tools/testing/kunit/kunit_kernel.py | 2 +-
tools/testing/kunit/kunit_parser.py | 2 +-
3 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c
index 15832ed446685..7db619624437c 100644
--- a/lib/kunit/executor.c
+++ b/lib/kunit/executor.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
+#include <linux/reboot.h>
#include <kunit/test.h>
#include <linux/glob.h>
#include <linux/moduleparam.h>
@@ -18,6 +19,9 @@ module_param(filter_glob, charp, 0);
MODULE_PARM_DESC(filter_glob,
"Filter which KUnit test suites run at boot-time, e.g. list*");
+static char *kunit_shutdown;
+core_param(kunit_shutdown, kunit_shutdown, charp, 0644);
+
static struct kunit_suite * const *
kunit_filter_subsuite(struct kunit_suite * const * const subsuite)
{
@@ -82,6 +86,20 @@ static struct suite_set kunit_filter_suites(void)
return filtered;
}
+static void kunit_handle_shutdown(void)
+{
+ if (!kunit_shutdown)
+ return;
+
+ if (!strcmp(kunit_shutdown, "poweroff"))
+ kernel_power_off();
+ else if (!strcmp(kunit_shutdown, "halt"))
+ kernel_halt();
+ else if (!strcmp(kunit_shutdown, "reboot"))
+ kernel_restart(NULL);
+
+}
+
static void kunit_print_tap_header(struct suite_set *suite_set)
{
struct kunit_suite * const * const *suites, * const *subsuite;
@@ -112,6 +130,8 @@ int kunit_run_all_tests(void)
kfree(suite_set.start);
}
+ kunit_handle_shutdown();
+
return 0;
}
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index f309a33256cd3..7d5b77967d48f 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -206,7 +206,7 @@ class LinuxSourceTree(object):
def run_kernel(self, args=None, build_dir='', filter_glob='', timeout=None) -> Iterator[str]:
if not args:
args = []
- args.extend(['mem=1G', 'console=tty'])
+ args.extend(['mem=1G', 'console=tty','kunit_shutdown=halt'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
self._ops.linux_bin(args, timeout, build_dir)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index e8bcc139702e2..8d8d4d70b39dd 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -49,7 +49,7 @@ class TestStatus(Enum):
kunit_start_re = re.compile(r'TAP version [0-9]+$')
kunit_end_re = re.compile('(List of all partitions:|'
- 'Kernel panic - not syncing: VFS:)')
+ 'Kernel panic - not syncing: VFS:|reboot: System halted)')
def isolate_kunit_output(kernel_output) -> Iterator[str]:
started = False
--
2.31.1.498.g6c1eba8ee3d-goog
Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.
Signed-off-by: David Gow <davi...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
---
lib/kunit/executor.c | 20 ++++++++++++++++++++
tools/testing/kunit/kunit_kernel.py | 2 +-
tools/testing/kunit/kunit_parser.py | 2 +-
3 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c
index 15832ed446685..7db619624437c 100644
--- a/lib/kunit/executor.c
+++ b/lib/kunit/executor.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
+#include <linux/reboot.h>
#include <kunit/test.h>
#include <linux/glob.h>
#include <linux/moduleparam.h>
@@ -18,6 +19,9 @@ module_param(filter_glob, charp, 0);
MODULE_PARM_DESC(filter_glob,
"Filter which KUnit test suites run at boot-time, e.g. list*");
+static char *kunit_shutdown;
+core_param(kunit_shutdown, kunit_shutdown, charp, 0644);
+
static struct kunit_suite * const *
kunit_filter_subsuite(struct kunit_suite * const * const subsuite)
{
@@ -82,6 +86,20 @@ static struct suite_set kunit_filter_suites(void)
return filtered;
}
+static void kunit_handle_shutdown(void)
+{
+ if (!kunit_shutdown)
+ return;
+
+ if (!strcmp(kunit_shutdown, "poweroff"))
+ kernel_power_off();
+ else if (!strcmp(kunit_shutdown, "halt"))
+ kernel_halt();
+ else if (!strcmp(kunit_shutdown, "reboot"))
+ kernel_restart(NULL);
+
+}
+
static void kunit_print_tap_header(struct suite_set *suite_set)
{
struct kunit_suite * const * const *suites, * const *subsuite;
@@ -112,6 +130,8 @@ int kunit_run_all_tests(void)
kfree(suite_set.start);
}
+ kunit_handle_shutdown();
+
return 0;
}
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index f309a33256cd3..7d5b77967d48f 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -206,7 +206,7 @@ class LinuxSourceTree(object):
def run_kernel(self, args=None, build_dir='', filter_glob='', timeout=None) -> Iterator[str]:
if not args:
args = []
- args.extend(['mem=1G', 'console=tty'])
+ args.extend(['mem=1G', 'console=tty','kunit_shutdown=halt'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
self._ops.linux_bin(args, timeout, build_dir)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index e8bcc139702e2..8d8d4d70b39dd 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -49,7 +49,7 @@ class TestStatus(Enum):
kunit_start_re = re.compile(r'TAP version [0-9]+$')
kunit_end_re = re.compile('(List of all partitions:|'
- 'Kernel panic - not syncing: VFS:)')
+ 'Kernel panic - not syncing: VFS:|reboot: System halted)')
def isolate_kunit_output(kernel_output) -> Iterator[str]:
started = False
--
2.31.1.498.g6c1eba8ee3d-goog
Brendan Higgins
Apr 29, 2021, 4:51:18 PM4/29/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Add kunit_shutdown, an option to specify that the kernel shutsdown after
running KUnit tests, to the kernel-parameters.txt documentation.
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 04545725f187f..fb2aacb29a4be 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2233,6 +2233,14 @@
0: force disabled
1: force enabled
+ kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after
+ running built-in tests. Tests configured as modules will
+ not be run.
+ Default: (flag not present) don't shutdown
+ poweroff: poweroff the kernel after running tests
+ halt: halt the kernel after running tests
+ reboot: reboot the kernel after running tests
+
kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.
Default is 0 (don't ignore, but inject #GP)
--
2.31.1.498.g6c1eba8ee3d-goog
running KUnit tests, to the kernel-parameters.txt documentation.
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
---
1 file changed, 8 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 04545725f187f..fb2aacb29a4be 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2233,6 +2233,14 @@
0: force disabled
1: force enabled
+ kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after
+ running built-in tests. Tests configured as modules will
+ not be run.
+ Default: (flag not present) don't shutdown
+ poweroff: poweroff the kernel after running tests
+ halt: halt the kernel after running tests
+ reboot: reboot the kernel after running tests
+
kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.
Default is 0 (don't ignore, but inject #GP)
--
2.31.1.498.g6c1eba8ee3d-goog
Brendan Higgins
Apr 29, 2021, 4:51:20 PM4/29/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Add basic support to run QEMU via kunit_tool. Add support for i386,
x86_64, arm, arm64, and a bunch more.
Signed-off-by: Brendan Higgins <brendan...@google.com>
---
tools/testing/kunit/kunit.py | 33 +++-
tools/testing/kunit/kunit_config.py | 2 +-
tools/testing/kunit/kunit_kernel.py | 207 +++++++++++++++++++++----
tools/testing/kunit/kunit_tool_test.py | 15 +-
4 files changed, 217 insertions(+), 40 deletions(-)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index d5144fcb03acd..07ef80062873b 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -70,10 +70,10 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree,
kunit_parser.print_with_timestamp('Building KUnit Kernel ...')
build_start = time.time()
- success = linux.build_um_kernel(request.alltests,
- request.jobs,
- request.build_dir,
- request.make_options)
+ success = linux.build_kernel(request.alltests,
+ request.jobs,
+ request.build_dir,
+ request.make_options)
build_end = time.time()
if not success:
return KunitResult(KunitStatus.BUILD_FAILURE,
@@ -187,6 +187,14 @@ def add_common_opts(parser) -> None:
help='Path to Kconfig fragment that enables KUnit tests',
metavar='kunitconfig')
+ parser.add_argument('--arch',
+ help='Specifies the architecture to run tests under.',
+ type=str, default='um', metavar='arch')
+
+ parser.add_argument('--cross_compile',
+ help='Sets make\'s CROSS_COMPILE variable.',
+ metavar='cross_compile')
+
def add_build_opts(parser) -> None:
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
@@ -268,7 +276,10 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
+ cross_compile=cli_args.cross_compile)
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
@@ -287,7 +298,9 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch)
request = KunitConfigRequest(cli_args.build_dir,
cli_args.make_options)
@@ -299,7 +312,9 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'build':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch)
request = KunitBuildRequest(cli_args.jobs,
cli_args.build_dir,
@@ -313,7 +328,9 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'exec':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch)
exec_request = KunitExecRequest(cli_args.timeout,
cli_args.build_dir,
diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py
index 1e2683dcc0e7a..07d76be392544 100644
--- a/tools/testing/kunit/kunit_config.py
+++ b/tools/testing/kunit/kunit_config.py
@@ -53,7 +53,7 @@ class Kconfig(object):
return True
def write_to_file(self, path: str) -> None:
- with open(path, 'w') as f:
+ with open(path, 'a+') as f:
for entry in self.entries():
f.write(str(entry) + '\n')
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 7d5b77967d48f..b8b3b76aaa17e 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -15,6 +15,8 @@ from typing import Iterator
from contextlib import ExitStack
+from collections import namedtuple
+
import kunit_config
import kunit_parser
@@ -40,6 +42,10 @@ class BuildError(Exception):
class LinuxSourceTreeOperations(object):
"""An abstraction over command line operations performed on a source tree."""
+ def __init__(self, linux_arch, cross_compile):
+ self._linux_arch = linux_arch
+ self._cross_compile = cross_compile
+
def make_mrproper(self) -> None:
try:
subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT)
@@ -48,12 +54,18 @@ class LinuxSourceTreeOperations(object):
except subprocess.CalledProcessError as e:
raise ConfigError(e.output.decode())
+ def make_arch_qemuconfig(self, build_dir):
+ pass
+
def make_olddefconfig(self, build_dir, make_options) -> None:
- command = ['make', 'ARCH=um', 'olddefconfig']
+ command = ['make', 'ARCH=' + self._linux_arch, 'olddefconfig']
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
if make_options:
command.extend(make_options)
if build_dir:
command += ['O=' + build_dir]
+ print(' '.join(command))
try:
subprocess.check_output(command, stderr=subprocess.STDOUT)
except OSError as e:
@@ -61,6 +73,154 @@ class LinuxSourceTreeOperations(object):
except subprocess.CalledProcessError as e:
raise ConfigError(e.output.decode())
+ def make(self, jobs, build_dir, make_options) -> None:
+ command = ['make', 'ARCH=' + self._linux_arch, '--jobs=' + str(jobs)]
+ if make_options:
+ command.extend(make_options)
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
+ if build_dir:
+ command += ['O=' + build_dir]
+ print(' '.join(command))
+ try:
+ proc = subprocess.Popen(command,
+ stderr=subprocess.PIPE,
+ stdout=subprocess.DEVNULL)
+ except OSError as e:
+ raise BuildError('Could not call execute make: ' + e)
+ except subprocess.CalledProcessError as e:
+ raise BuildError(e.output)
+ _, stderr = proc.communicate()
+ if proc.returncode != 0:
+ raise BuildError(stderr.decode())
+ if stderr: # likely only due to build warnings
+ print(stderr.decode())
+
+ def run(self, params, timeout, build_dir, outfile) -> None:
+ pass
+
+
+QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
+ 'qemuconfig',
+ 'qemu_arch',
+ 'kernel_path',
+ 'kernel_command_line',
+ 'extra_qemu_params'])
+
+
+QEMU_ARCHS = {
+ 'i386' : QemuArchParams(linux_arch='i386',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['']),
+ 'x86_64' : QemuArchParams(linux_arch='x86_64',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['']),
+ 'arm' : QemuArchParams(linux_arch='arm',
+ qemuconfig='''CONFIG_ARCH_VIRT=y
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='arm',
+ kernel_path='arch/arm/boot/zImage',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt']),
+ 'arm64' : QemuArchParams(linux_arch='arm64',
+ qemuconfig='''CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='aarch64',
+ kernel_path='arch/arm64/boot/Image.gz',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt', '-cpu cortex-a57']),
+ 'alpha' : QemuArchParams(linux_arch='alpha',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='alpha',
+ kernel_path='arch/alpha/boot/vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['']),
+ 'powerpc' : QemuArchParams(linux_arch='powerpc',
+ qemuconfig='CONFIG_PPC64=y\nCONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y\nCONFIG_HVC_CONSOLE=y',
+ qemu_arch='ppc64',
+ kernel_path='vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-M pseries', '-cpu power8']),
+ 'riscv' : QemuArchParams(linux_arch='riscv',
+ qemuconfig='CONFIG_SOC_VIRT=y\nCONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y\nCONFIG_SERIAL_OF_PLATFORM=y\nCONFIG_SERIAL_EARLYCON_RISCV_SBI=y',
+ qemu_arch='riscv64',
+ kernel_path='arch/riscv/boot/Image',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-machine virt', '-cpu rv64', '-bios opensbi-riscv64-generic-fw_dynamic.bin']),
+ 's390' : QemuArchParams(linux_arch='s390',
+ qemuconfig='CONFIG_EXPERT=y\nCONFIG_TUNE_ZEC12=y\nCONFIG_NUMA=y\nCONFIG_MODULES=y',
+ qemu_arch='s390x',
+ kernel_path='arch/s390/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine s390-ccw-virtio',
+ '-cpu qemu',]),
+ 'sparc' : QemuArchParams(linux_arch='sparc',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='sparc',
+ kernel_path='arch/sparc/boot/zImage',
+ kernel_command_line='console=ttyS0 mem=256M',
+ extra_qemu_params=['-m 256']),
+}
+
+class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
+
+ def __init__(self, qemu_arch_params, cross_compile):
+ super().__init__(linux_arch=qemu_arch_params.linux_arch,
+ cross_compile=cross_compile)
+ self._qemuconfig = qemu_arch_params.qemuconfig
+ self._qemu_arch = qemu_arch_params.qemu_arch
+ self._kernel_path = qemu_arch_params.kernel_path
+ print(self._kernel_path)
+ self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
+ self._extra_qemu_params = qemu_arch_params.extra_qemu_params
+
+ def make_arch_qemuconfig(self, build_dir):
+ qemuconfig = kunit_config.Kconfig()
+ qemuconfig.parse_from_string(self._qemuconfig)
+ qemuconfig.write_to_file(get_kconfig_path(build_dir))
+
+ def run(self, params, timeout, build_dir, outfile):
+ kernel_path = os.path.join(build_dir, self._kernel_path)
+ qemu_command = ['qemu-system-' + self._qemu_arch,
+ '-nodefaults',
+ '-m', '1024',
+ '-kernel', kernel_path,
+ '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
+ '-no-reboot',
+ '-nographic',
+ '-serial stdio'] + self._extra_qemu_params
+ print(' '.join(qemu_command))
+ with open(outfile, 'w') as output:
+ process = subprocess.Popen(' '.join(qemu_command),
+ stdin=subprocess.PIPE,
+ stdout=output,
+ stderr=subprocess.STDOUT,
+ text=True, shell=True)
+ try:
+ process.wait(timeout=timeout)
+ except Exception as e:
+ print(e)
+ process.terminate()
+ return process
+
+class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
+ """An abstraction over command line operations performed on a source tree."""
+
+ def __init__(self):
+ super().__init__(linux_arch='um', cross_compile=None)
+
def make_allyesconfig(self, build_dir, make_options) -> None:
kunit_parser.print_with_timestamp(
'Enabling all CONFIGs for UML...')
@@ -83,32 +243,16 @@ class LinuxSourceTreeOperations(object):
kunit_parser.print_with_timestamp(
'Starting Kernel with all configs takes a few minutes...')
- def make(self, jobs, build_dir, make_options) -> None:
- command = ['make', 'ARCH=um', '--jobs=' + str(jobs)]
- if make_options:
- command.extend(make_options)
- if build_dir:
- command += ['O=' + build_dir]
- try:
- proc = subprocess.Popen(command,
- stderr=subprocess.PIPE,
- stdout=subprocess.DEVNULL)
- except OSError as e:
- raise BuildError('Could not call make command: ' + str(e))
- _, stderr = proc.communicate()
- if proc.returncode != 0:
- raise BuildError(stderr.decode())
- if stderr: # likely only due to build warnings
- print(stderr.decode())
-
- def linux_bin(self, params, timeout, build_dir) -> None:
+ def run(self, params, timeout, build_dir, outfile):
"""Runs the Linux UML binary. Must be named 'linux'."""
linux_bin = get_file_path(build_dir, 'linux')
outfile = get_outfile_path(build_dir)
with open(outfile, 'w') as output:
process = subprocess.Popen([linux_bin] + params,
+ stdin=subprocess.PIPE,
stdout=output,
- stderr=subprocess.STDOUT)
+ stderr=subprocess.STDOUT,
+ text=True)
process.wait(timeout)
def get_kconfig_path(build_dir) -> str:
@@ -123,10 +267,17 @@ def get_outfile_path(build_dir) -> str:
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
+ def __init__(self, build_dir: str, load_config=True, kunitconfig_path='', arch=None, cross_compile=None) -> None:
signal.signal(signal.SIGINT, self.signal_handler)
-
- self._ops = LinuxSourceTreeOperations()
+ self._ops = None
+ if arch is None or arch == 'um':
+ self._arch = 'um'
+ self._ops = LinuxSourceTreeOperationsUml()
+ elif arch in QEMU_ARCHS:
+ self._arch = arch
+ self._ops = LinuxSourceTreeOperationsQemu(QEMU_ARCHS[arch], cross_compile=cross_compile)
+ else:
+ raise ConfigError(arch + ' is not a valid arch')
if not load_config:
return
@@ -170,6 +321,7 @@ class LinuxSourceTree(object):
os.mkdir(build_dir)
self._kconfig.write_to_file(kconfig_path)
try:
+ self._ops.make_arch_qemuconfig(build_dir)
self._ops.make_olddefconfig(build_dir, make_options)
except ConfigError as e:
logging.error(e)
@@ -192,10 +344,13 @@ class LinuxSourceTree(object):
print('Generating .config ...')
return self.build_config(build_dir, make_options)
- def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
+ def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
+ if self._arch != 'um':
+ raise ConfigError('Only the "um" arch is supported for alltests')
self._ops.make_allyesconfig(build_dir, make_options)
+ self._ops.make_arch_qemuconfig(build_dir)
self._ops.make_olddefconfig(build_dir, make_options)
self._ops.make(jobs, build_dir, make_options)
except (ConfigError, BuildError) as e:
@@ -209,8 +364,8 @@ class LinuxSourceTree(object):
outfile = get_outfile_path(build_dir)
+ self._ops.run(args, timeout, build_dir, outfile)
subprocess.call(['stty', 'sane'])
with open(outfile, 'r') as file:
for line in file:
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 1ad3049e90698..25e8be95a575d 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -297,7 +297,7 @@ class KUnitMainTest(unittest.TestCase):
self.linux_source_mock = mock.Mock()
self.linux_source_mock.build_reconfig = mock.Mock(return_value=True)
- self.linux_source_mock.build_um_kernel = mock.Mock(return_value=True)
+ self.linux_source_mock.build_kernel = mock.Mock(return_value=True)
self.linux_source_mock.run_kernel = mock.Mock(return_value=all_passed_log)
def test_config_passes_args_pass(self):
@@ -308,7 +308,7 @@ class KUnitMainTest(unittest.TestCase):
def test_build_passes_args_pass(self):
kunit.main(['build'], self.linux_source_mock)
self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 0)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, '.kunit', None)
self.assertEqual(self.linux_source_mock.run_kernel.call_count, 0)
def test_exec_passes_args_pass(self):
@@ -390,7 +390,7 @@ class KUnitMainTest(unittest.TestCase):
def test_build_builddir(self):
build_dir = '.kunit'
kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, build_dir, None)
def test_exec_builddir(self):
build_dir = '.kunit'
@@ -404,14 +404,19 @@ class KUnitMainTest(unittest.TestCase):
mock_linux_init.return_value = self.linux_source_mock
kunit.main(['run', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ arch='um',
+ cross_compile=None)
@mock.patch.object(kunit_kernel, 'LinuxSourceTree')
def test_config_kunitconfig(self, mock_linux_init):
mock_linux_init.return_value = self.linux_source_mock
kunit.main(['config', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ arch='um')
if __name__ == '__main__':
unittest.main()
--
2.31.1.498.g6c1eba8ee3d-goog
x86_64, arm, arm64, and a bunch more.
Signed-off-by: Brendan Higgins <brendan...@google.com>
---
tools/testing/kunit/kunit.py | 33 +++-
tools/testing/kunit/kunit_config.py | 2 +-
tools/testing/kunit/kunit_kernel.py | 207 +++++++++++++++++++++----
tools/testing/kunit/kunit_tool_test.py | 15 +-
4 files changed, 217 insertions(+), 40 deletions(-)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index d5144fcb03acd..07ef80062873b 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -70,10 +70,10 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree,
kunit_parser.print_with_timestamp('Building KUnit Kernel ...')
build_start = time.time()
- success = linux.build_um_kernel(request.alltests,
- request.jobs,
- request.build_dir,
- request.make_options)
+ success = linux.build_kernel(request.alltests,
+ request.jobs,
+ request.build_dir,
+ request.make_options)
build_end = time.time()
if not success:
return KunitResult(KunitStatus.BUILD_FAILURE,
@@ -187,6 +187,14 @@ def add_common_opts(parser) -> None:
help='Path to Kconfig fragment that enables KUnit tests',
metavar='kunitconfig')
+ parser.add_argument('--arch',
+ help='Specifies the architecture to run tests under.',
+ type=str, default='um', metavar='arch')
+
+ parser.add_argument('--cross_compile',
+ help='Sets make\'s CROSS_COMPILE variable.',
+ metavar='cross_compile')
+
def add_build_opts(parser) -> None:
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
@@ -268,7 +276,10 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
+ cross_compile=cli_args.cross_compile)
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
@@ -287,7 +298,9 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch)
request = KunitConfigRequest(cli_args.build_dir,
cli_args.make_options)
@@ -299,7 +312,9 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'build':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch)
request = KunitBuildRequest(cli_args.jobs,
cli_args.build_dir,
@@ -313,7 +328,9 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'exec':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch)
exec_request = KunitExecRequest(cli_args.timeout,
cli_args.build_dir,
diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py
index 1e2683dcc0e7a..07d76be392544 100644
--- a/tools/testing/kunit/kunit_config.py
+++ b/tools/testing/kunit/kunit_config.py
@@ -53,7 +53,7 @@ class Kconfig(object):
return True
def write_to_file(self, path: str) -> None:
- with open(path, 'w') as f:
+ with open(path, 'a+') as f:
for entry in self.entries():
f.write(str(entry) + '\n')
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 7d5b77967d48f..b8b3b76aaa17e 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -15,6 +15,8 @@ from typing import Iterator
from contextlib import ExitStack
+from collections import namedtuple
+
import kunit_config
import kunit_parser
@@ -40,6 +42,10 @@ class BuildError(Exception):
class LinuxSourceTreeOperations(object):
"""An abstraction over command line operations performed on a source tree."""
+ def __init__(self, linux_arch, cross_compile):
+ self._linux_arch = linux_arch
+ self._cross_compile = cross_compile
+
def make_mrproper(self) -> None:
try:
subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT)
@@ -48,12 +54,18 @@ class LinuxSourceTreeOperations(object):
except subprocess.CalledProcessError as e:
raise ConfigError(e.output.decode())
+ def make_arch_qemuconfig(self, build_dir):
+ pass
+
def make_olddefconfig(self, build_dir, make_options) -> None:
- command = ['make', 'ARCH=um', 'olddefconfig']
+ command = ['make', 'ARCH=' + self._linux_arch, 'olddefconfig']
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
if make_options:
command.extend(make_options)
if build_dir:
command += ['O=' + build_dir]
+ print(' '.join(command))
try:
subprocess.check_output(command, stderr=subprocess.STDOUT)
except OSError as e:
@@ -61,6 +73,154 @@ class LinuxSourceTreeOperations(object):
except subprocess.CalledProcessError as e:
raise ConfigError(e.output.decode())
+ def make(self, jobs, build_dir, make_options) -> None:
+ command = ['make', 'ARCH=' + self._linux_arch, '--jobs=' + str(jobs)]
+ if make_options:
+ command.extend(make_options)
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
+ if build_dir:
+ command += ['O=' + build_dir]
+ print(' '.join(command))
+ try:
+ proc = subprocess.Popen(command,
+ stderr=subprocess.PIPE,
+ stdout=subprocess.DEVNULL)
+ except OSError as e:
+ raise BuildError('Could not call execute make: ' + e)
+ except subprocess.CalledProcessError as e:
+ raise BuildError(e.output)
+ _, stderr = proc.communicate()
+ if proc.returncode != 0:
+ raise BuildError(stderr.decode())
+ if stderr: # likely only due to build warnings
+ print(stderr.decode())
+
+ def run(self, params, timeout, build_dir, outfile) -> None:
+ pass
+
+
+QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
+ 'qemuconfig',
+ 'qemu_arch',
+ 'kernel_path',
+ 'kernel_command_line',
+ 'extra_qemu_params'])
+
+
+QEMU_ARCHS = {
+ 'i386' : QemuArchParams(linux_arch='i386',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['']),
+ 'x86_64' : QemuArchParams(linux_arch='x86_64',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['']),
+ 'arm' : QemuArchParams(linux_arch='arm',
+ qemuconfig='''CONFIG_ARCH_VIRT=y
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='arm',
+ kernel_path='arch/arm/boot/zImage',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt']),
+ 'arm64' : QemuArchParams(linux_arch='arm64',
+ qemuconfig='''CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='aarch64',
+ kernel_path='arch/arm64/boot/Image.gz',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt', '-cpu cortex-a57']),
+ 'alpha' : QemuArchParams(linux_arch='alpha',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='alpha',
+ kernel_path='arch/alpha/boot/vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['']),
+ 'powerpc' : QemuArchParams(linux_arch='powerpc',
+ qemuconfig='CONFIG_PPC64=y\nCONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y\nCONFIG_HVC_CONSOLE=y',
+ qemu_arch='ppc64',
+ kernel_path='vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-M pseries', '-cpu power8']),
+ 'riscv' : QemuArchParams(linux_arch='riscv',
+ qemuconfig='CONFIG_SOC_VIRT=y\nCONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y\nCONFIG_SERIAL_OF_PLATFORM=y\nCONFIG_SERIAL_EARLYCON_RISCV_SBI=y',
+ qemu_arch='riscv64',
+ kernel_path='arch/riscv/boot/Image',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-machine virt', '-cpu rv64', '-bios opensbi-riscv64-generic-fw_dynamic.bin']),
+ 's390' : QemuArchParams(linux_arch='s390',
+ qemuconfig='CONFIG_EXPERT=y\nCONFIG_TUNE_ZEC12=y\nCONFIG_NUMA=y\nCONFIG_MODULES=y',
+ qemu_arch='s390x',
+ kernel_path='arch/s390/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine s390-ccw-virtio',
+ '-cpu qemu',]),
+ 'sparc' : QemuArchParams(linux_arch='sparc',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='sparc',
+ kernel_path='arch/sparc/boot/zImage',
+ kernel_command_line='console=ttyS0 mem=256M',
+ extra_qemu_params=['-m 256']),
+}
+
+class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
+
+ def __init__(self, qemu_arch_params, cross_compile):
+ super().__init__(linux_arch=qemu_arch_params.linux_arch,
+ cross_compile=cross_compile)
+ self._qemuconfig = qemu_arch_params.qemuconfig
+ self._qemu_arch = qemu_arch_params.qemu_arch
+ self._kernel_path = qemu_arch_params.kernel_path
+ print(self._kernel_path)
+ self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
+ self._extra_qemu_params = qemu_arch_params.extra_qemu_params
+
+ def make_arch_qemuconfig(self, build_dir):
+ qemuconfig = kunit_config.Kconfig()
+ qemuconfig.parse_from_string(self._qemuconfig)
+ qemuconfig.write_to_file(get_kconfig_path(build_dir))
+
+ def run(self, params, timeout, build_dir, outfile):
+ kernel_path = os.path.join(build_dir, self._kernel_path)
+ qemu_command = ['qemu-system-' + self._qemu_arch,
+ '-nodefaults',
+ '-m', '1024',
+ '-kernel', kernel_path,
+ '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
+ '-no-reboot',
+ '-nographic',
+ '-serial stdio'] + self._extra_qemu_params
+ print(' '.join(qemu_command))
+ with open(outfile, 'w') as output:
+ process = subprocess.Popen(' '.join(qemu_command),
+ stdin=subprocess.PIPE,
+ stdout=output,
+ stderr=subprocess.STDOUT,
+ text=True, shell=True)
+ try:
+ process.wait(timeout=timeout)
+ except Exception as e:
+ print(e)
+ process.terminate()
+ return process
+
+class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
+ """An abstraction over command line operations performed on a source tree."""
+
+ def __init__(self):
+ super().__init__(linux_arch='um', cross_compile=None)
+
def make_allyesconfig(self, build_dir, make_options) -> None:
kunit_parser.print_with_timestamp(
'Enabling all CONFIGs for UML...')
@@ -83,32 +243,16 @@ class LinuxSourceTreeOperations(object):
kunit_parser.print_with_timestamp(
'Starting Kernel with all configs takes a few minutes...')
- def make(self, jobs, build_dir, make_options) -> None:
- command = ['make', 'ARCH=um', '--jobs=' + str(jobs)]
- if make_options:
- command.extend(make_options)
- if build_dir:
- command += ['O=' + build_dir]
- try:
- proc = subprocess.Popen(command,
- stderr=subprocess.PIPE,
- stdout=subprocess.DEVNULL)
- except OSError as e:
- raise BuildError('Could not call make command: ' + str(e))
- _, stderr = proc.communicate()
- if proc.returncode != 0:
- raise BuildError(stderr.decode())
- if stderr: # likely only due to build warnings
- print(stderr.decode())
-
- def linux_bin(self, params, timeout, build_dir) -> None:
+ def run(self, params, timeout, build_dir, outfile):
"""Runs the Linux UML binary. Must be named 'linux'."""
linux_bin = get_file_path(build_dir, 'linux')
outfile = get_outfile_path(build_dir)
with open(outfile, 'w') as output:
process = subprocess.Popen([linux_bin] + params,
+ stdin=subprocess.PIPE,
stdout=output,
- stderr=subprocess.STDOUT)
+ stderr=subprocess.STDOUT,
+ text=True)
process.wait(timeout)
def get_kconfig_path(build_dir) -> str:
@@ -123,10 +267,17 @@ def get_outfile_path(build_dir) -> str:
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
+ def __init__(self, build_dir: str, load_config=True, kunitconfig_path='', arch=None, cross_compile=None) -> None:
signal.signal(signal.SIGINT, self.signal_handler)
-
- self._ops = LinuxSourceTreeOperations()
+ self._ops = None
+ if arch is None or arch == 'um':
+ self._arch = 'um'
+ self._ops = LinuxSourceTreeOperationsUml()
+ elif arch in QEMU_ARCHS:
+ self._arch = arch
+ self._ops = LinuxSourceTreeOperationsQemu(QEMU_ARCHS[arch], cross_compile=cross_compile)
+ else:
+ raise ConfigError(arch + ' is not a valid arch')
if not load_config:
return
@@ -170,6 +321,7 @@ class LinuxSourceTree(object):
os.mkdir(build_dir)
self._kconfig.write_to_file(kconfig_path)
try:
+ self._ops.make_arch_qemuconfig(build_dir)
self._ops.make_olddefconfig(build_dir, make_options)
except ConfigError as e:
logging.error(e)
@@ -192,10 +344,13 @@ class LinuxSourceTree(object):
print('Generating .config ...')
return self.build_config(build_dir, make_options)
- def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
+ def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
+ if self._arch != 'um':
+ raise ConfigError('Only the "um" arch is supported for alltests')
self._ops.make_allyesconfig(build_dir, make_options)
+ self._ops.make_arch_qemuconfig(build_dir)
self._ops.make_olddefconfig(build_dir, make_options)
self._ops.make(jobs, build_dir, make_options)
except (ConfigError, BuildError) as e:
@@ -209,8 +364,8 @@ class LinuxSourceTree(object):
args.extend(['mem=1G', 'console=tty','kunit_shutdown=halt'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
- self._ops.linux_bin(args, timeout, build_dir)
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
outfile = get_outfile_path(build_dir)
+ self._ops.run(args, timeout, build_dir, outfile)
subprocess.call(['stty', 'sane'])
with open(outfile, 'r') as file:
for line in file:
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 1ad3049e90698..25e8be95a575d 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -297,7 +297,7 @@ class KUnitMainTest(unittest.TestCase):
self.linux_source_mock = mock.Mock()
self.linux_source_mock.build_reconfig = mock.Mock(return_value=True)
- self.linux_source_mock.build_um_kernel = mock.Mock(return_value=True)
+ self.linux_source_mock.build_kernel = mock.Mock(return_value=True)
self.linux_source_mock.run_kernel = mock.Mock(return_value=all_passed_log)
def test_config_passes_args_pass(self):
@@ -308,7 +308,7 @@ class KUnitMainTest(unittest.TestCase):
def test_build_passes_args_pass(self):
kunit.main(['build'], self.linux_source_mock)
self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 0)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, '.kunit', None)
self.assertEqual(self.linux_source_mock.run_kernel.call_count, 0)
def test_exec_passes_args_pass(self):
@@ -390,7 +390,7 @@ class KUnitMainTest(unittest.TestCase):
def test_build_builddir(self):
build_dir = '.kunit'
kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, build_dir, None)
def test_exec_builddir(self):
build_dir = '.kunit'
@@ -404,14 +404,19 @@ class KUnitMainTest(unittest.TestCase):
mock_linux_init.return_value = self.linux_source_mock
kunit.main(['run', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ arch='um',
+ cross_compile=None)
@mock.patch.object(kunit_kernel, 'LinuxSourceTree')
def test_config_kunitconfig(self, mock_linux_init):
mock_linux_init.return_value = self.linux_source_mock
kunit.main(['config', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ arch='um')
if __name__ == '__main__':
unittest.main()
--
2.31.1.498.g6c1eba8ee3d-goog
Brendan Higgins
Apr 29, 2021, 4:51:21 PM4/29/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Document QEMU support, what it does, and how to use it in kunit_tool.
Documentation/dev-tools/kunit/usage.rst | 37 +++++++++++++++++++++----
1 file changed, 31 insertions(+), 6 deletions(-)
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 650f99590df57..b74bd7c87cc20 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -612,14 +612,39 @@ only things to be aware of when doing so.
The biggest impediment will likely be that certain KUnit features and
infrastructure may not support your target environment. For example, at this
time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside
-of UML. Unfortunately, there is no way around this. Using UML (or even just a
-particular architecture) allows us to make a lot of assumptions that make it
-possible to do things which might otherwise be impossible.
+of UML and QEMU. Unfortunately, there is no way around this. Using UML and QEMU
+(or even just a particular architecture) allows us to make a lot of assumptions
+that make it possible to do things which might otherwise be impossible.
Nevertheless, all core KUnit framework features are fully supported on all
-architectures, and using them is straightforward: all you need to do is to take
-your kunitconfig, your Kconfig options for the tests you would like to run, and
-merge them into whatever config your are using for your platform. That's it!
+architectures, and using them is straightforward: Most popular architectures
+are supported directly in the KUnit Wrapper via QEMU. Currently, supported
+architectures on QEMU include:
+
+* i386
+* x86_64
+* arm
+* arm64
+* alpha
+* powerpc
+* riscv
+* s390
+* sparc
+
+In order to run KUnit tests on one of these architectures via QEMU with the
+KUnit wrapper, all you need to do is specify the flags ``--arch`` and
+``--cross_compile`` when invoking the KUnit Wrapper. For example, we could run
+the default KUnit tests on ARM in the following manner (assuming we have an ARM
+toolchain installed):
+
+.. code-block:: bash
+
+ tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --arch=arm --cross_compile=arm-linux-gnueabihf-
+
+Alternatively, if you want to run your tests on real hardware or in some other
+emulation environment, all you need to do is to take your kunitconfig, your
+Kconfig options for the tests you would like to run, and merge them into
+whatever config your are using for your platform. That's it!
For example, let's say you have the following kunitconfig:
--
2.31.1.498.g6c1eba8ee3d-goog
Documentation/dev-tools/kunit/usage.rst | 37 +++++++++++++++++++++----
1 file changed, 31 insertions(+), 6 deletions(-)
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 650f99590df57..b74bd7c87cc20 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -612,14 +612,39 @@ only things to be aware of when doing so.
The biggest impediment will likely be that certain KUnit features and
infrastructure may not support your target environment. For example, at this
time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside
-of UML. Unfortunately, there is no way around this. Using UML (or even just a
-particular architecture) allows us to make a lot of assumptions that make it
-possible to do things which might otherwise be impossible.
+of UML and QEMU. Unfortunately, there is no way around this. Using UML and QEMU
+(or even just a particular architecture) allows us to make a lot of assumptions
+that make it possible to do things which might otherwise be impossible.
Nevertheless, all core KUnit framework features are fully supported on all
-architectures, and using them is straightforward: all you need to do is to take
-your kunitconfig, your Kconfig options for the tests you would like to run, and
-merge them into whatever config your are using for your platform. That's it!
+architectures, and using them is straightforward: Most popular architectures
+are supported directly in the KUnit Wrapper via QEMU. Currently, supported
+architectures on QEMU include:
+
+* i386
+* x86_64
+* arm
+* arm64
+* alpha
+* powerpc
+* riscv
+* s390
+* sparc
+
+In order to run KUnit tests on one of these architectures via QEMU with the
+KUnit wrapper, all you need to do is specify the flags ``--arch`` and
+``--cross_compile`` when invoking the KUnit Wrapper. For example, we could run
+the default KUnit tests on ARM in the following manner (assuming we have an ARM
+toolchain installed):
+
+.. code-block:: bash
+
+ tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --arch=arm --cross_compile=arm-linux-gnueabihf-
+
+Alternatively, if you want to run your tests on real hardware or in some other
+emulation environment, all you need to do is to take your kunitconfig, your
+Kconfig options for the tests you would like to run, and merge them into
+whatever config your are using for your platform. That's it!
For example, let's say you have the following kunitconfig:
--
2.31.1.498.g6c1eba8ee3d-goog
Daniel Latypov
Apr 29, 2021, 5:53:00 PM4/29/21
to Brendan Higgins, sh...@kernel.org, David Gow, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, linu...@vger.kernel.org, Stephen Boyd, Kees Cook, frowan...@gmail.com
On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
<brendan...@google.com> wrote:
>
> From: David Gow <davi...@google.com>
>
> Add a new kernel command-line option, 'kunit_shutdown', which allows the
> user to specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running KUnit tests
> on UML or a VM so that the UML/VM process exits cleanly immediately
> after running all tests without needing a special initramfs.
>
> Signed-off-by: David Gow <davi...@google.com>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
Tested-By: Daniel Latypov <dlat...@google.com>
<brendan...@google.com> wrote:
>
> From: David Gow <davi...@google.com>
>
> Add a new kernel command-line option, 'kunit_shutdown', which allows the
> user to specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running KUnit tests
> on UML or a VM so that the UML/VM process exits cleanly immediately
> after running all tests without needing a special initramfs.
>
> Signed-off-by: David Gow <davi...@google.com>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
Testing this out, I think this also helps fix coverage on UML.
I used to have to hack in something like this:
diff --git a/arch/um/os-Linux/util.c b/arch/um/os-Linux/util.c
index 07327425d06e..e13763faedd9 100644
--- a/arch/um/os-Linux/util.c
+++ b/arch/um/os-Linux/util.c
@@ -146,7 +146,7 @@ void os_dump_core(void)
while ((pid = waitpid(-1, NULL, WNOHANG | __WALL)) > 0)
os_kill_ptraced_process(pid, 0);
- uml_abort();
+ exit(127);
}
void um_early_printk(const char *s, unsigned int n)
And afaict, this patch removes the need to do so.
(But I made a few mistakes when initially trying to test that out, so
I'm not 100% certain I didn't make another).
Daniel Latypov
Apr 29, 2021, 7:40:04 PM4/29/21
to Brendan Higgins, sh...@kernel.org, David Gow, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, linu...@vger.kernel.org, Stephen Boyd, Kees Cook, frowan...@gmail.com
> Add basic support to run QEMU via kunit_tool. Add support for i386,
> x86_64, arm, arm64, and a bunch more.
Hmmm, I'm wondering if I'm seeing unrelated breakages.
> x86_64, arm, arm64, and a bunch more.
Applied these patches on top of 55ba0fe059a5 ("Merge tag
'for-5.13-tag' of
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
$ make mrproper
$ rm -rf .kunit/* # just in case
$ ./tools/testing/kunit/kunit.py run --arch=arm64
...
ERROR:root:arch/arm64/Makefile:25: ld does not support
--fix-cortex-a53-843419; kernel may be susceptible to erratum
arch/arm64/Makefile:44: Detected assembler with broken .inst;
disassembly will be unreliable
gcc: error: unrecognized command-line option ‘-mlittle-endian’
So it seems like my version of ld might be out of date, but my gcc is 10.2.1
Is there a minimum version of the toolchain this would expect that we
can call out in the commit message (and in the Documentation)?
--arch=x86_64 worked just fine for me, however, which is mainly what I
was interested in.
I've mainly just left some nits and comments regarding typechecking.
Most people using this would be able to infer that, but I prefer
strings that are basically enums to document examples of useful
values.
(And x86 is probably the one most people want to use this flag for).
w => a means we wouldn't truncate the file if it exists. I can imagine
I'm missing something that makes it necessary.
+ => opens for read/write: we don't do any reads with f afaict.
> for entry in self.entries():
> f.write(str(entry) + '\n')
>
> diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
> index 7d5b77967d48f..b8b3b76aaa17e 100644
> --- a/tools/testing/kunit/kunit_kernel.py
> +++ b/tools/testing/kunit/kunit_kernel.py
> @@ -15,6 +15,8 @@ from typing import Iterator
>
> from contextlib import ExitStack
>
> +from collections import namedtuple
> +
> import kunit_config
> import kunit_parser
>
> @@ -40,6 +42,10 @@ class BuildError(Exception):
> class LinuxSourceTreeOperations(object):
> """An abstraction over command line operations performed on a source tree."""
>
> + def __init__(self, linux_arch, cross_compile):
is it Optional[str] here?)
I can see a reader thinking arch might be an enum and that
cross_compile might be a bool.
You'd want `+ str(e)`
I don't know enough to say if there's a better way of doing this.
But I think we should probably split this out into a separate python
file, if this mapping remains necessary.
E.g. in a qemu_configs.py file or the like.
> +
> +class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
I've called out the two errors pytype gives already, but mypy is even
worse about this new class :(
$ mypy tools/testing/kunit/*.py
tools/testing/kunit/kunit_kernel.py:90: error: Unsupported operand
types for + ("str" and "OSError")
tools/testing/kunit/kunit_kernel.py:278: error: Incompatible types in
assignment (expression has type "LinuxSourceTreeOperationsQemu",
variable has type "Optional[LinuxSourceTreeOperationsUml]")
tools/testing/kunit/kunit_kernel.py:298: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute
"make_mrproper"
tools/testing/kunit/kunit_kernel.py:324: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute
"make_arch_qemuconfig"
tools/testing/kunit/kunit_kernel.py:325: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute
"make_olddefconfig"
tools/testing/kunit/kunit_kernel.py:352: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute
"make_allyesconfig"
tools/testing/kunit/kunit_kernel.py:353: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute
"make_arch_qemuconfig"
tools/testing/kunit/kunit_kernel.py:354: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute
"make_olddefconfig"
tools/testing/kunit/kunit_kernel.py:355: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute "make"
tools/testing/kunit/kunit_kernel.py:368: error: Item "None" of
"Optional[LinuxSourceTreeOperationsUml]" has no attribute "run"
So to make up for mypy being less smart, we can do this and get it down 2 errors
diff --git a/tools/testing/kunit/kunit_kernel.py
b/tools/testing/kunit/kunit_kernel.py
index b8b3b76aaa17..a0aaa28db4c1 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -11,7 +11,7 @@ import subprocess
import os
import shutil
import signal
-from typing import Iterator
+from typing import Iterator, Union
from contextlib import ExitStack
@@ -269,15 +269,16 @@ class LinuxSourceTree(object):
def __init__(self, build_dir: str, load_config=True,
kunitconfig_path='', arch=None, cross_compile=None) -> None:
signal.signal(signal.SIGINT, self.signal_handler)
+ ops = None # type: Union[None,
LinuxSourceTreeOperationsUml, LinuxSourceTreeOperationsQemu]
if arch is None or arch == 'um':
self._arch = 'um'
- self._ops = LinuxSourceTreeOperationsUml()
+ ops = LinuxSourceTreeOperationsUml()
elif arch in QEMU_ARCHS:
self._arch = arch
- self._ops =
LinuxSourceTreeOperationsQemu(QEMU_ARCHS[arch],
cross_compile=cross_compile)
+ ops =
LinuxSourceTreeOperationsQemu(QEMU_ARCHS[arch],
cross_compile=cross_compile)
else:
raise ConfigError(arch + ' is not a valid arch')
+ self._ops = ops
if not load_config:
return
> +
> + def __init__(self, qemu_arch_params, cross_compile):
> + super().__init__(linux_arch=qemu_arch_params.linux_arch,
> + cross_compile=cross_compile)
> + self._qemuconfig = qemu_arch_params.qemuconfig
> + self._qemu_arch = qemu_arch_params.qemu_arch
> + self._kernel_path = qemu_arch_params.kernel_path
> + print(self._kernel_path)
> + self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
> + self._extra_qemu_params = qemu_arch_params.extra_qemu_params
> +
> + def make_arch_qemuconfig(self, build_dir):
> + qemuconfig = kunit_config.Kconfig()
> + qemuconfig.parse_from_string(self._qemuconfig)
> + qemuconfig.write_to_file(get_kconfig_path(build_dir))
> +
> + def run(self, params, timeout, build_dir, outfile):
> + kernel_path = os.path.join(build_dir, self._kernel_path)
> + qemu_command = ['qemu-system-' + self._qemu_arch,
> + '-nodefaults',
> + '-m', '1024',
> + '-kernel', kernel_path,
> + '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
> + '-no-reboot',
> + '-nographic',
> + '-serial stdio'] + self._extra_qemu_params
> + print(' '.join(qemu_command))
Though this one could be useful to actually print out for the user if
we include some more context in the message.
The code is correct, but pytype can't figure out that ops can't be the
QEMU variant.
Pytype recognizes comments like " # pytype: disable=attribute-error"
but mypy doesn't.
So I don't know that there's a way we can make both of them happy :/
Brendan Higgins
Apr 30, 2021, 4:01:51 PM4/30/21
to Daniel Latypov, shuah, David Gow, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand
On Thu, Apr 29, 2021 at 4:40 PM Daniel Latypov <dlat...@google.com> wrote:
>
> On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
> <brendan...@google.com> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
>
> Hmmm, I'm wondering if I'm seeing unrelated breakages.
> Applied these patches on top of 55ba0fe059a5 ("Merge tag
> 'for-5.13-tag' of
> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
>
> $ make mrproper
> $ rm -rf .kunit/* # just in case
> $ ./tools/testing/kunit/kunit.py run --arch=arm64
> ...
Huh, did you use a arm64 cross compiler? Maybe that's your issue.
>
> On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
> <brendan...@google.com> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
>
> Hmmm, I'm wondering if I'm seeing unrelated breakages.
> Applied these patches on top of 55ba0fe059a5 ("Merge tag
> 'for-5.13-tag' of
> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
>
> $ make mrproper
> $ rm -rf .kunit/* # just in case
> $ ./tools/testing/kunit/kunit.py run --arch=arm64
> ...
the argument passed into the ARCH make variable.
> (And x86 is probably the one most people want to use this flag for).
adding the arch configs. Idea is that we want to append those onto
whatever a base kunitconfig.
Nevertheless, your concern is valid, might be a better way to do this. Thoughts?
> > for entry in self.entries():
> > f.write(str(entry) + '\n')
> >
> > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
> > index 7d5b77967d48f..b8b3b76aaa17e 100644
> > --- a/tools/testing/kunit/kunit_kernel.py
> > +++ b/tools/testing/kunit/kunit_kernel.py
> > @@ -15,6 +15,8 @@ from typing import Iterator
> >
> > from contextlib import ExitStack
> >
> > +from collections import namedtuple
> > +
> > import kunit_config
> > import kunit_parser
> >
> > @@ -40,6 +42,10 @@ class BuildError(Exception):
> > class LinuxSourceTreeOperations(object):
> > """An abstraction over command line operations performed on a source tree."""
> >
> > + def __init__(self, linux_arch, cross_compile):
>
> nit: let's annotate these as `linux_arch: str, cross_compile: str` (or
> is it Optional[str] here?)
> I can see a reader thinking arch might be an enum and that
> cross_compile might be a bool.
into until I got some feedback on it.
> But I think we should probably split this out into a separate python
> file, if this mapping remains necessary.
> E.g. in a qemu_configs.py file or the like.
less gross?
> > + self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
> > + self._extra_qemu_params = qemu_arch_params.extra_qemu_params
> > +
> > + def make_arch_qemuconfig(self, build_dir):
> > + qemuconfig = kunit_config.Kconfig()
> > + qemuconfig.parse_from_string(self._qemuconfig)
> > + qemuconfig.write_to_file(get_kconfig_path(build_dir))
> > +
> > + def run(self, params, timeout, build_dir, outfile):
> > + kernel_path = os.path.join(build_dir, self._kernel_path)
> > + qemu_command = ['qemu-system-' + self._qemu_arch,
> > + '-nodefaults',
> > + '-m', '1024',
> > + '-kernel', kernel_path,
> > + '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
> > + '-no-reboot',
> > + '-nographic',
> > + '-serial stdio'] + self._extra_qemu_params
> > + print(' '.join(qemu_command))
>
> ditto, a debug statement?
> Though this one could be useful to actually print out for the user if
> we include some more context in the message.
pretty useful since it spits out a full qemu command that you can just
run outside of the tool, so I left it in.
it raise. That might be cleaner too.
Daniel Latypov
Apr 30, 2021, 4:14:43 PM4/30/21
to Brendan Higgins, shuah, David Gow, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand
On Fri, Apr 30, 2021 at 1:01 PM 'Brendan Higgins' via KUnit
Development <kuni...@googlegroups.com> wrote:
>
> On Thu, Apr 29, 2021 at 4:40 PM Daniel Latypov <dlat...@google.com> wrote:
> >
> > On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
> > <brendan...@google.com> wrote:
> > >
> > > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > > x86_64, arm, arm64, and a bunch more.
> >
> > Hmmm, I'm wondering if I'm seeing unrelated breakages.
> > Applied these patches on top of 55ba0fe059a5 ("Merge tag
> > 'for-5.13-tag' of
> > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
> >
> > $ make mrproper
> > $ rm -rf .kunit/* # just in case
> > $ ./tools/testing/kunit/kunit.py run --arch=arm64
> > ...
>
> Huh, did you use a arm64 cross compiler? Maybe that's your issue.
I didn't (and realized that like 2 minutes after I sent the email).
Development <kuni...@googlegroups.com> wrote:
>
> On Thu, Apr 29, 2021 at 4:40 PM Daniel Latypov <dlat...@google.com> wrote:
> >
> > On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
> > <brendan...@google.com> wrote:
> > >
> > > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > > x86_64, arm, arm64, and a bunch more.
> >
> > Hmmm, I'm wondering if I'm seeing unrelated breakages.
> > Applied these patches on top of 55ba0fe059a5 ("Merge tag
> > 'for-5.13-tag' of
> > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
> >
> > $ make mrproper
> > $ rm -rf .kunit/* # just in case
> > $ ./tools/testing/kunit/kunit.py run --arch=arm64
> > ...
>
> Huh, did you use a arm64 cross compiler? Maybe that's your issue.
Please disregard.
having the write_to_file() function do appends?
I'm worried this might lead to unintentional statefulness in other cases.
better ideas how we can maybe push this config out of python.
But if we don't find anything else, I think having the hard-coding
isolated into its own package is fine, tbh.
Maybe something like
print('Running tests with:\n$', ' '.join(qemu_command))
Side note: I think we might want to keep the existing check as well
just so it's more obvious to a reader that only uml is supported, even
at the cost of some duplication. But that's just personal preference.
> You received this message because you are subscribed to the Google Groups "KUnit Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kunit-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kunit-dev/CAFd5g44OL0BQ74ZGEnYzsLYVXEbOB-hBiNAf1%2BVLO%3DDoZwKOvA%40mail.gmail.com.
Brendan Higgins
May 3, 2021, 5:19:52 PM5/3/21
to Daniel Latypov, shuah, David Gow, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand
On Fri, Apr 30, 2021 at 01:14:29PM -0700, Daniel Latypov wrote:
> On Fri, Apr 30, 2021 at 1:01 PM 'Brendan Higgins' via KUnit
> Development <kuni...@googlegroups.com> wrote:
> >
> > On Thu, Apr 29, 2021 at 4:40 PM Daniel Latypov <dlat...@google.com> wrote:
> > >
> > > On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
> > > <brendan...@google.com> wrote:
> > > >
> > > > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > > > x86_64, arm, arm64, and a bunch more.
> > >
> > > Hmmm, I'm wondering if I'm seeing unrelated breakages.
> > > Applied these patches on top of 55ba0fe059a5 ("Merge tag
> > > 'for-5.13-tag' of
> > > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
> > >
> > > $ make mrproper
> > > $ rm -rf .kunit/* # just in case
> > > $ ./tools/testing/kunit/kunit.py run --arch=arm64
> > > ...
> >
> > Huh, did you use a arm64 cross compiler? Maybe that's your issue.
>
> I didn't (and realized that like 2 minutes after I sent the email).
> Please disregard.
As I mention below, I had a conversation offline with David regarding
> On Fri, Apr 30, 2021 at 1:01 PM 'Brendan Higgins' via KUnit
> Development <kuni...@googlegroups.com> wrote:
> >
> > On Thu, Apr 29, 2021 at 4:40 PM Daniel Latypov <dlat...@google.com> wrote:
> > >
> > > On Thu, Apr 29, 2021 at 1:51 PM Brendan Higgins
> > > <brendan...@google.com> wrote:
> > > >
> > > > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > > > x86_64, arm, arm64, and a bunch more.
> > >
> > > Hmmm, I'm wondering if I'm seeing unrelated breakages.
> > > Applied these patches on top of 55ba0fe059a5 ("Merge tag
> > > 'for-5.13-tag' of
> > > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux")
> > >
> > > $ make mrproper
> > > $ rm -rf .kunit/* # just in case
> > > $ ./tools/testing/kunit/kunit.py run --arch=arm64
> > > ...
> >
> > Huh, did you use a arm64 cross compiler? Maybe that's your issue.
>
> I didn't (and realized that like 2 minutes after I sent the email).
> Please disregard.
the problem of figuring out the best way to specify qemu_configs, and I
wanted to post the fruits of that discussion to get some early feedback
on it. I am still working on addressing your other comments, but I think
most of those are pretty straightforward and probably won't require much
discussion.
[...]
the configs in Python, but make the configs dynamically loaded somehow
with some kind of predefined format so that they are simple to add,
*and* more importantly that adding new ones that don't go upstream won't
be a burden to maintain. Basically a developer could have a QEMU config
that is customized for her usecase, and is no way dependent on any kinds
of changes to any upstream kunit_tool files.
So here is what I came up with: We have these qemu_config files which
can be specified anywhere within (subdirectories included) the
kunit_tool root directory. The qemu_config is a Python file that imports
the `QemuArchParams` object (above) defined by kunit_tool. It specifies
one variable: `QEMU_ARCH` which is of type `QemuArchParams` which is
then used as the arch listed above.
Below I have a proof-of-concept diff, you can see view the proposed
change on Gerrit here:
https://kunit-review.googlesource.com/c/linux/+/4489
diff --git a/tools/testing/kunit/__init__.py b/tools/testing/kunit/__init__.py
new file mode 100644
index 0000000000000..e69de29bb2d1d
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index f89def9e14dcd..eb9daf6896194 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -198,6 +198,11 @@ def add_common_opts(parser) -> None:
help='Sets make\'s CROSS_COMPILE variable.',
metavar='cross_compile')
+ parser.add_argument('--qemu_config',
+ help=('Takes a path to a path to a file containing '
+ 'a QemuArchParams object.'),
+ type=str, metavar='qemu_config')
+
def add_build_opts(parser) -> None:
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
@@ -282,7 +287,8 @@ def main(argv, linux=None):
def add_build_opts(parser) -> None:
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
kunitconfig_path=cli_args.kunitconfig,
arch=cli_args.arch,
- cross_compile=cli_args.cross_compile)
+ cross_compile=cli_args.cross_compile,
+ qemu_config_path=cli_args.qemu_config)
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 64d0fffc5b86e..bc9b847bda658 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -6,6 +6,7 @@
# Author: Felix Guo <felixgu...@gmail.com>
# Author: Brendan Higgins <brendan...@google.com>
+import importlib.util
import logging
import subprocess
import os
@@ -20,7 +21,7 @@ from collections import namedtuple
import kunit_config
import kunit_parser
-import qemu_configs
+import qemu_config
KCONFIG_PATH = '.config'
KUNITCONFIG_PATH = '.kunitconfig'
@@ -107,7 +108,7 @@ class LinuxSourceTreeOperations(object):
class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
- def __init__(self, qemu_arch_params: qemu_configs.QemuArchParams, cross_compile: Optional[str]):
+ def __init__(self, qemu_arch_params: qemu_config.QemuArchParams, cross_compile: Optional[str]):
super().__init__(linux_arch=qemu_arch_params.linux_arch,
cross_compile=cross_compile)
self._qemuconfig = qemu_arch_params.qemuconfig
@@ -197,19 +198,34 @@ def get_outfile_path(build_dir) -> str:
def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations:
if arch == 'um':
return LinuxSourceTreeOperationsUml()
- elif arch in qemu_configs.QEMU_ARCHS:
- return LinuxSourceTreeOperationsQemu(qemu_configs.QEMU_ARCHS[arch],
+ elif arch in qemu_config.QEMU_ARCHS:
+ return LinuxSourceTreeOperationsQemu(qemu_config.QEMU_ARCHS[arch],
cross_compile=cross_compile)
else:
raise ConfigError(arch + ' is not a valid arch')
+def get_source_tree_ops_from_qemu_config(config_path: str, cross_compile: Optional[str]) -> qemu_config.QemuArchParams:
else:
raise ConfigError(arch + ' is not a valid arch')
+ abs_tool_path = os.path.abspath(os.path.dirname(__file__))
+ abs_config_path = os.path.abspath(config_path)
+ if os.path.commonpath([abs_tool_path, abs_config_path]) != abs_tool_path:
+ raise ConfigError('Given QEMU config file is not in a child directory of KUnit tool.')
+ relative_config_path = os.path.relpath(abs_config_path, abs_tool_path)
+ spec = importlib.util.spec_from_file_location('.' + relative_config_path, config_path)
+ config = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(config)
+ return config.QEMU_ARCH.linux_arch, LinuxSourceTreeOperationsQemu(
+ config.QEMU_ARCH, cross_compile=cross_compile)
+
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, kunitconfig_path='', arch=None, cross_compile=None) -> None:
"""Represents a Linux kernel source tree with KUnit tests."""
+ def __init__(self, build_dir: str, load_config=True, kunitconfig_path='', arch=None, cross_compile=None, qemu_config_path=None) -> None:
signal.signal(signal.SIGINT, self.signal_handler)
- self._arch = 'um' if arch is None else arch
- self._ops = get_source_tree_ops(self._arch, cross_compile)
+ if qemu_config_path:
+ self._arch, self._ops = get_source_tree_ops_from_qemu_config(qemu_config_path, cross_compile)
+ else:
+ self._arch = 'um' if arch is None else arch
+ self._ops = get_source_tree_ops(self._arch, cross_compile)
if not load_config:
return
diff --git a/tools/testing/kunit/qemu_configs.py b/tools/testing/kunit/qemu_config.py
similarity index 100%
rename from tools/testing/kunit/qemu_configs.py
rename to tools/testing/kunit/qemu_config.py
diff --git a/tools/testing/kunit/qemu_configs/__init__.py b/tools/testing/kunit/qemu_configs/__init__.py
new file mode 100644
index 0000000000000..e69de29bb2d1d
diff --git a/tools/testing/kunit/qemu_configs/i386.py b/tools/testing/kunit/qemu_configs/i386.py
new file mode 100644
index 0000000000000..7f079db044cc7
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/i386.py
@@ -0,0 +1,8 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='i386',
+ qemuconfig='CONFIG_SERIAL_8250=y\nCONFIG_SERIAL_8250_CONSOLE=y',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
An example of how kunit_tool with the above change could be invoked is
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
as follows:
tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --qemu_config=./tools/testing/kunit/qemu_configs/i386.py
Let me know wha'cha think!
[...]
David Gow
May 4, 2021, 2:08:55 AM5/4/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Fri, Apr 30, 2021 at 4:51 AM Brendan Higgins
<brendan...@google.com> wrote:
>
> Add basic support to run QEMU via kunit_tool. Add support for i386,
> x86_64, arm, arm64, and a bunch more.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
I've tested this out on a few architectures (x86-32, ppc64, etc), and
<brendan...@google.com> wrote:
>
> Add basic support to run QEMU via kunit_tool. Add support for i386,
> x86_64, arm, arm64, and a bunch more.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
it all seems to work pretty well. I've got quite a few comments below.
The main one -- the location of the qemu configs -- is basically
echoing what Daniel said and what you've done on the Gerrit version,
but I felt was worth having on the record.
Otherwise, a bunch of nitpicks around missing places cross_compile
should be supported.
Still, great to see this happening: it's nice to be able to test on
non-UML architectures more easily.
Tested-by: David Gow <davi...@google.com>
-- David
> +
> + parser.add_argument('--cross_compile',
> + help='Sets make\'s CROSS_COMPILE variable.',
> + metavar='cross_compile')
> +
its the compiler name's prefix or something).
important for the 'build' step.
As mentioned below, moving the QemuArchParams to a separate file
helps, but even then it'd be nice to at least have each entry on a
separate line like with 'arm' below.
The other option would be to have this as a path to a separate
Kconfig/.kunitconfig file with these configs, rather than having them
inline here. I fear that could lead to every architecture needing
several support files, though some (e.g. x86/x86_64/alpha/sparc) could
share a file.
doesn't belong here. Ideally, I think these configs should be
specified somewhere else, either as a (alas, very long) series of
command-line arguments, or in a separate file. Moving them to a single
'qemu_configs.py' as Daniel mentioned would be an improvement, but
IMHO still falls well short of ideal. I think each config should be in
its own file, as there are several reasons someone might want to have
multiple different configs for a given architecture and we'll want
people to be able to add/remove these easily in their own
environments.
The proposed solution of having each config in a separate file --
which we discussed offline -- is a big improvement:
https://kunit-review.googlesource.com/c/linux/+/4489
I still think that the python format here is a bit verbose and ugly,
but it's probably worth it compared to adding a parser for a whole new
type of config file. Similarly, I'm not sure about keeping a separate
qemu_configs.py around with the builtin defaults rather than just
having them all as separate files all the time. (Even appreciating
that that requires some filename magic to find the correct file.)
These are all more nitpicky though.
compile UML kernels? (This should, I think, be possible for i386 on an
x86_64 host, for instance.)
Brendan Higgins
May 4, 2021, 4:07:56 PM5/4/21
to David Gow, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Tue, May 04, 2021 at 02:08:41PM +0800, David Gow wrote:
> On Fri, Apr 30, 2021 at 4:51 AM Brendan Higgins
> <brendan...@google.com> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
> >
> > Signed-off-by: Brendan Higgins <brendan...@google.com>
>
> I've tested this out on a few architectures (x86-32, ppc64, etc), and
> it all seems to work pretty well. I've got quite a few comments below.
> The main one -- the location of the qemu configs -- is basically
> echoing what Daniel said and what you've done on the Gerrit version,
> but I felt was worth having on the record.
>
> Otherwise, a bunch of nitpicks around missing places cross_compile
> should be supported.
>
> Still, great to see this happening: it's nice to be able to test on
> non-UML architectures more easily.
>
> Tested-by: David Gow <davi...@google.com>
Sweet, thanks!
> On Fri, Apr 30, 2021 at 4:51 AM Brendan Higgins
> <brendan...@google.com> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
> >
> > Signed-off-by: Brendan Higgins <brendan...@google.com>
>
> I've tested this out on a few architectures (x86-32, ppc64, etc), and
> it all seems to work pretty well. I've got quite a few comments below.
> The main one -- the location of the qemu configs -- is basically
> echoing what Daniel said and what you've done on the Gerrit version,
> but I felt was worth having on the record.
>
> Otherwise, a bunch of nitpicks around missing places cross_compile
> should be supported.
>
> Still, great to see this happening: it's nice to be able to test on
> non-UML architectures more easily.
>
> Tested-by: David Gow <davi...@google.com>
> > +
> > + parser.add_argument('--cross_compile',
> > + help='Sets make\'s CROSS_COMPILE variable.',
> > + metavar='cross_compile')
> > +
>
> It'd be nice to have a slightly more detailed description here (that
> its the compiler name's prefix or something).
Sets make's CROSS_COMPILE variable; it should be set to a toolchain path
prefix (the prefix of gcc and other tools in your toolchain, for example
`sparc64-linux-gnu-` if you have the sparc toolchain installed on your
system, or `$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-`
if you have downloaded the microblaze toolchain from the 0-day website
to a directory in your home directory called `toolchains`).
thought they were an improvement. Part of me thought they were horribly
ugly.
> The other option would be to have this as a path to a separate
> Kconfig/.kunitconfig file with these configs, rather than having them
> inline here. I fear that could lead to every architecture needing
> several support files, though some (e.g. x86/x86_64/alpha/sparc) could
> share a file.
all your configs in one place. The other part of me agrees with what you
said here. I think I will stick with the way I am doing it now for no
other reason than that's the way I already have it. If someone feels
strongly though, I would be happy to change it.
> I still think that the python format here is a bit verbose and ugly,
> but it's probably worth it compared to adding a parser for a whole new
> type of config file.
parsers to the kernel tree. :-)
> Similarly, I'm not sure about keeping a separate
> qemu_configs.py around with the builtin defaults rather than just
> having them all as separate files all the time. (Even appreciating
> that that requires some filename magic to find the correct file.)
> These are all more nitpicky though.
if we add the --qemu_config flag?
The only reason I just added --qemu_config without touching the existing
configs was just because I was being lazy; nevertheless, my original
plan was just to drop --arch entirely and just expect the user to know
that to run KUnit on i386 you need:
tools/testing/kunit/kunit.py run --qemu_config=./tools/testing/kunit/qemu_configs/i386.py
and to run KUnit on arm you need:
tools/testing/kunit/kunit.py run --qemu_config=./tools/testing/kunit/qemu_configs/arm.py --cross_compile=arm-linux-gnueabihf-
(or something like that).
Is that OK, or as you hint, should we do some file system magic so that
you have the option of
tools/testing/kunit/kunit.py run --qemu_config=./tools/testing/kunit/qemu_configs/i386.py
OR
tools/testing/kunit/kunit.py run --arch=i386
?
My preference is for the former since it is more explicit about what it
is doing.
[...]
Brendan Higgins
May 7, 2021, 5:31:31 PM5/7/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
---------------------------
Mostly fixed lots of minor issues suggested/poited out by David and
Daniel. Also reworked how qemu_configs are loaded: Now each config is in
its own Python file and is loaded dynamically. Given the number of
improvements that address the biggest concerns I had in the last RFC, I
decided to promote this to a normal patch set.
What discussion remains for this patchset?
------------------------------------------
patch: I want to make sure with the added context of QEMU running under
kunit_tool that this is now a reasonable approach. Nevertheless, I am
pretty happy with this patchset as is, and I did not get any negative
feedback on the previous revision, so I think we can probably just move
forward as is.
Brendan Higgins (3):
Documentation: Add kunit_shutdown to kernel-parameters.txt
kunit: tool: add support for QEMU
Documentation: kunit: document support for QEMU in kunit_tool
David Gow (1):
kunit: Add 'kunit_shutdown' option
.../admin-guide/kernel-parameters.txt | 8 +
Documentation/dev-tools/kunit/usage.rst | 37 +++-
lib/kunit/executor.c | 20 ++
tools/testing/kunit/kunit_config.py | 7 +-
tools/testing/kunit/kunit_kernel.py | 172 +++++++++++++++---
tools/testing/kunit/kunit_parser.py | 2 +-
tools/testing/kunit/kunit_tool_test.py | 18 +-
tools/testing/kunit/qemu_config.py | 17 ++
tools/testing/kunit/qemu_configs/alpha.py | 10 +
tools/testing/kunit/qemu_configs/arm.py | 13 ++
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/i386.py | 10 +
tools/testing/kunit/qemu_configs/powerpc.py | 12 ++
tools/testing/kunit/qemu_configs/riscv.py | 31 ++++
tools/testing/kunit/qemu_configs/s390.py | 14 ++
tools/testing/kunit/qemu_configs/sparc.py | 10 +
tools/testing/kunit/qemu_configs/x86_64.py | 10 +
18 files changed, 411 insertions(+), 49 deletions(-)
create mode 100644 tools/testing/kunit/qemu_config.py
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
base-commit: 38182162b50aa4e970e5997df0a0c4288147a153
--
2.31.1.607.g51e8a6a459-goog
Brendan Higgins
May 7, 2021, 5:31:33 PM5/7/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
From: David Gow <davi...@google.com>
Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.
Signed-off-by: David Gow <davi...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
Tested-By: Daniel Latypov <dlat...@google.com>
Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.
Signed-off-by: David Gow <davi...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
Tested-By: Daniel Latypov <dlat...@google.com>
index 89a7d4024e878..e22ade9d91ad5 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -208,7 +208,7 @@ class LinuxSourceTree(object):
def run_kernel(self, args=None, build_dir='', filter_glob='', timeout=None) -> Iterator[str]:
if not args:
args = []
- args.extend(['mem=1G', 'console=tty'])
+ args.extend(['mem=1G', 'console=tty','kunit_shutdown=halt'])
if not args:
args = []
- args.extend(['mem=1G', 'console=tty'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
args.append('kunit.filter_glob='+filter_glob)
self._ops.linux_bin(args, timeout, build_dir)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index e8bcc139702e2..8d8d4d70b39dd 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -49,7 +49,7 @@ class TestStatus(Enum):
kunit_start_re = re.compile(r'TAP version [0-9]+$')
kunit_end_re = re.compile('(List of all partitions:|'
- 'Kernel panic - not syncing: VFS:)')
+ 'Kernel panic - not syncing: VFS:|reboot: System halted)')
def isolate_kunit_output(kernel_output) -> Iterator[str]:
started = False
--
2.31.1.607.g51e8a6a459-goog
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index e8bcc139702e2..8d8d4d70b39dd 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -49,7 +49,7 @@ class TestStatus(Enum):
kunit_start_re = re.compile(r'TAP version [0-9]+$')
kunit_end_re = re.compile('(List of all partitions:|'
- 'Kernel panic - not syncing: VFS:)')
+ 'Kernel panic - not syncing: VFS:|reboot: System halted)')
def isolate_kunit_output(kernel_output) -> Iterator[str]:
started = False
--
Brendan Higgins
May 7, 2021, 5:31:34 PM5/7/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Add kunit_shutdown, an option to specify that the kernel shutsdown after
running KUnit tests, to the kernel-parameters.txt documentation.
running KUnit tests, to the kernel-parameters.txt documentation.
---
Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 5bcc229d31e24..bfd64c01698ba 100644
Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2244,6 +2244,14 @@
0: force disabled
1: force enabled
+ kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after
+ running built-in tests. Tests configured as modules will
+ not be run.
+ Default: (flag not present) don't shutdown
+ poweroff: poweroff the kernel after running tests
+ halt: halt the kernel after running tests
+ reboot: reboot the kernel after running tests
+
kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.
Default is 0 (don't ignore, but inject #GP)
--
2.31.1.607.g51e8a6a459-goog
1: force enabled
+ kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after
+ running built-in tests. Tests configured as modules will
+ not be run.
+ Default: (flag not present) don't shutdown
+ poweroff: poweroff the kernel after running tests
+ halt: halt the kernel after running tests
+ reboot: reboot the kernel after running tests
+
kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.
Default is 0 (don't ignore, but inject #GP)
--
Brendan Higgins
May 7, 2021, 5:31:37 PM5/7/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Add basic support to run QEMU via kunit_tool. Add support for i386,
x86_64, arm, arm64, and a bunch more.
Signed-off-by: Brendan Higgins <brendan...@google.com>
Tested-by: David Gow <davi...@google.com>
x86_64, arm, arm64, and a bunch more.
Signed-off-by: Brendan Higgins <brendan...@google.com>
---
Changes since last revision:
- A number of minor obvious issues pointed out by David and Daniel.
- Added facility for merging Kconfigs at Daniel's suggestion.
- Broke out qemu_configs each into their own config file which is loaded
dynamically - mostly at David's suggestion.
---
tools/testing/kunit/kunit.py | 57 ++++++-
tools/testing/kunit/kunit_config.py | 7 +-
tools/testing/kunit/kunit_kernel.py | 170 ++++++++++++++++----
tools/testing/kunit/kunit_tool_test.py | 18 ++-
tools/testing/kunit/qemu_config.py | 17 ++
tools/testing/kunit/qemu_configs/alpha.py | 10 ++
tools/testing/kunit/qemu_configs/arm.py | 13 ++
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/i386.py | 10 ++
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/powerpc.py | 12 ++
tools/testing/kunit/qemu_configs/riscv.py | 31 ++++
tools/testing/kunit/qemu_configs/s390.py | 14 ++
tools/testing/kunit/qemu_configs/sparc.py | 10 ++
tools/testing/kunit/qemu_configs/riscv.py | 31 ++++
tools/testing/kunit/qemu_configs/s390.py | 14 ++
tools/testing/kunit/qemu_configs/x86_64.py | 10 ++
14 files changed, 350 insertions(+), 41 deletions(-)
create mode 100644 tools/testing/kunit/qemu_config.py
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
index 5da8fb3762f98..be8d8d4a4e08f 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -70,10 +70,10 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree,
kunit_parser.print_with_timestamp('Building KUnit Kernel ...')
build_start = time.time()
- success = linux.build_um_kernel(request.alltests,
- request.jobs,
- request.build_dir,
- request.make_options)
+ success = linux.build_kernel(request.alltests,
+ request.jobs,
+ request.build_dir,
+ request.make_options)
build_end = time.time()
if not success:
return KunitResult(KunitStatus.BUILD_FAILURE,
@@ -189,6 +189,31 @@ def add_common_opts(parser) -> None:
+++ b/tools/testing/kunit/kunit.py
@@ -70,10 +70,10 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree,
kunit_parser.print_with_timestamp('Building KUnit Kernel ...')
build_start = time.time()
- success = linux.build_um_kernel(request.alltests,
- request.jobs,
- request.build_dir,
- request.make_options)
+ success = linux.build_kernel(request.alltests,
+ request.jobs,
+ request.build_dir,
+ request.make_options)
build_end = time.time()
if not success:
return KunitResult(KunitStatus.BUILD_FAILURE,
'will get automatically appended.',
metavar='kunitconfig')
+ parser.add_argument('--arch',
+ help=('Specifies the architecture to run tests under. '
+ parser.add_argument('--arch',
+ 'The architecture specified here must match the '
+ 'string passed to the ARCH make param, '
+ 'e.g. i386, x86_64, arm, um, etc. Non-UML '
+ 'architectures run on QEMU.'),
+ type=str, default='um', metavar='arch')
+
+ parser.add_argument('--cross_compile',
+ help=('Sets make\'s CROSS_COMPILE variable; it should '
+ parser.add_argument('--cross_compile',
+ 'be set to a toolchain path prefix (the prefix '
+ 'of gcc and other tools in your toolchain, for '
+ 'example `sparc64-linux-gnu-` if you have the '
+ 'sparc toolchain installed on your system, or '
+ '`$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-` '
+ 'if you have downloaded the microblaze toolchain '
+ 'from the 0-day website to a directory in your '
+ 'home directory called `toolchains`).'),
+ metavar='cross_compile')
+
+
+ parser.add_argument('--qemu_config',
+ help=('Takes a path to a path to a file containing '
+ 'a QemuArchParams object.'),
+ type=str, metavar='qemu_config')
+
+ help=('Takes a path to a path to a file containing '
+ 'a QemuArchParams object.'),
+ type=str, metavar='qemu_config')
+
def add_build_opts(parser) -> None:
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
@@ -270,7 +295,11 @@ def main(argv, linux=None):
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
+ cross_compile=cli_args.cross_compile,
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
+ qemu_config_path=cli_args.qemu_config)
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
@@ -289,7 +318,11 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
+ cross_compile=cli_args.cross_compile,
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
+ qemu_config_path=cli_args.qemu_config)
request = KunitConfigRequest(cli_args.build_dir,
cli_args.make_options)
@@ -301,7 +334,11 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'build':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
elif cli_args.subcommand == 'build':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ cross_compile=cli_args.cross_compile,
+ qemu_config_path=cli_args.qemu_config)
request = KunitBuildRequest(cli_args.jobs,
cli_args.build_dir,
@@ -315,7 +352,11 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'exec':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ arch=cli_args.arch,
elif cli_args.subcommand == 'exec':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
+ kunitconfig_path=cli_args.kunitconfig,
+ cross_compile=cli_args.cross_compile,
+ qemu_config_path=cli_args.qemu_config)
exec_request = KunitExecRequest(cli_args.timeout,
cli_args.build_dir,
diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py
--- a/tools/testing/kunit/kunit_config.py
+++ b/tools/testing/kunit/kunit_config.py
@@ -52,8 +52,13 @@ class Kconfig(object):
return False
return True
+ def merge_in_entries(self, other: 'Kconfig') -> None:
+ if other.is_subset_of(self):
+ return
+ self._entries = list(self.entries().union(other.entries()))
+
def write_to_file(self, path: str) -> None:
- with open(path, 'w') as f:
+ with open(path, 'a+') as f:
for entry in self.entries():
f.write(str(entry) + '\n')
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
- with open(path, 'w') as f:
+ with open(path, 'a+') as f:
for entry in self.entries():
f.write(str(entry) + '\n')
index e22ade9d91ad5..2bd196fd69e5c 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -6,23 +6,31 @@
+from __future__ import annotations
+import importlib.util
import logging
import subprocess
import os
import shutil
import logging
import subprocess
import os
import signal
from typing import Iterator
+from typing import Optional
from contextlib import ExitStack
+from collections import namedtuple
+
import kunit_config
import kunit_parser
+import qemu_config
KCONFIG_PATH = '.config'
KUNITCONFIG_PATH = '.kunitconfig'
DEFAULT_KUNITCONFIG_PATH = 'arch/um/configs/kunit_defconfig'
KCONFIG_PATH = '.config'
KUNITCONFIG_PATH = '.kunitconfig'
BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
OUTFILE_PATH = 'test.log'
+ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__))
+QEMU_CONFIGS_DIR = os.path.join(ABS_TOOL_PATH, 'qemu_configs')
def get_file_path(build_dir, default):
if build_dir:
@@ -40,6 +48,10 @@ class BuildError(Exception):
class LinuxSourceTreeOperations(object):
"""An abstraction over command line operations performed on a source tree."""
+ def __init__(self, linux_arch: str, cross_compile: Optional[str]):
"""An abstraction over command line operations performed on a source tree."""
+ self._linux_arch = linux_arch
+ self._cross_compile = cross_compile
+
def make_mrproper(self) -> None:
try:
subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT)
@@ -48,12 +60,21 @@ class LinuxSourceTreeOperations(object):
+ self._cross_compile = cross_compile
+
def make_mrproper(self) -> None:
try:
subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
raise ConfigError(e.output.decode())
+ def make_arch_qemuconfig(self, kconfig: kunit_config.Kconfig) -> None:
raise ConfigError(e.output.decode())
+ pass
+
+ def make_allyesconfig(self, build_dir, make_options) -> None:
+ raise ConfigError('Only the "um" arch is supported for alltests')
+
def make_olddefconfig(self, build_dir, make_options) -> None:
- command = ['make', 'ARCH=um', 'olddefconfig']
+ command = ['make', 'ARCH=' + self._linux_arch, 'olddefconfig']
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
if make_options:
command.extend(make_options)
if build_dir:
command += ['O=' + build_dir]
+ print('Populating config with:\n$', ' '.join(command))
def make_olddefconfig(self, build_dir, make_options) -> None:
- command = ['make', 'ARCH=um', 'olddefconfig']
+ command = ['make', 'ARCH=' + self._linux_arch, 'olddefconfig']
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
if make_options:
command.extend(make_options)
if build_dir:
command += ['O=' + build_dir]
try:
subprocess.check_output(command, stderr=subprocess.STDOUT)
except OSError as e:
@@ -61,6 +82,79 @@ class LinuxSourceTreeOperations(object):
subprocess.check_output(command, stderr=subprocess.STDOUT)
except OSError as e:
except subprocess.CalledProcessError as e:
raise ConfigError(e.output.decode())
+ def make(self, jobs, build_dir, make_options) -> None:
+ command = ['make', 'ARCH=' + self._linux_arch, '--jobs=' + str(jobs)]
+ if make_options:
+ command.extend(make_options)
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
+ if build_dir:
+ command += ['O=' + build_dir]
+ print('Building with:\n$', ' '.join(command))
raise ConfigError(e.output.decode())
+ def make(self, jobs, build_dir, make_options) -> None:
+ command = ['make', 'ARCH=' + self._linux_arch, '--jobs=' + str(jobs)]
+ if make_options:
+ command.extend(make_options)
+ if self._cross_compile:
+ command += ['CROSS_COMPILE=' + self._cross_compile]
+ if build_dir:
+ command += ['O=' + build_dir]
+ try:
+ proc = subprocess.Popen(command,
+ stderr=subprocess.PIPE,
+ stdout=subprocess.DEVNULL)
+ except OSError as e:
+ raise BuildError('Could not call execute make: ' + str(e))
+ proc = subprocess.Popen(command,
+ stderr=subprocess.PIPE,
+ stdout=subprocess.DEVNULL)
+ except OSError as e:
+ except subprocess.CalledProcessError as e:
+ raise BuildError(e.output)
+ _, stderr = proc.communicate()
+ if proc.returncode != 0:
+ raise BuildError(stderr.decode())
+ if stderr: # likely only due to build warnings
+ print(stderr.decode())
+
+ def run(self, params, timeout, build_dir, outfile) -> None:
+ pass
+
+
+class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
+ raise BuildError(e.output)
+ _, stderr = proc.communicate()
+ if proc.returncode != 0:
+ raise BuildError(stderr.decode())
+ if stderr: # likely only due to build warnings
+ print(stderr.decode())
+
+ def run(self, params, timeout, build_dir, outfile) -> None:
+ pass
+
+
+
+ def __init__(self, qemu_arch_params: qemu_config.QemuArchParams, cross_compile: Optional[str]):
+ super().__init__(linux_arch=qemu_arch_params.linux_arch,
+ cross_compile=cross_compile)
+ self._qemuconfig = qemu_arch_params.qemuconfig
+ self._qemu_arch = qemu_arch_params.qemu_arch
+ self._kernel_path = qemu_arch_params.kernel_path
+ cross_compile=cross_compile)
+ self._qemuconfig = qemu_arch_params.qemuconfig
+ self._qemu_arch = qemu_arch_params.qemu_arch
+ self._kernel_path = qemu_arch_params.kernel_path
+ self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
+ self._extra_qemu_params = qemu_arch_params.extra_qemu_params
+
+ def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None:
+ self._extra_qemu_params = qemu_arch_params.extra_qemu_params
+
+ qemuconfig = kunit_config.Kconfig()
+ qemuconfig.parse_from_string(self._qemuconfig)
+ base_kunitconfig.merge_in_entries(qemuconfig)
+ qemuconfig.parse_from_string(self._qemuconfig)
+
+ def run(self, params, timeout, build_dir, outfile):
+ kernel_path = os.path.join(build_dir, self._kernel_path)
+ qemu_command = ['qemu-system-' + self._qemu_arch,
+ '-nodefaults',
+ '-m', '1024',
+ '-kernel', kernel_path,
+ '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
+ '-no-reboot',
+ '-nographic',
+ '-serial stdio'] + self._extra_qemu_params
+ print('Running tests with:\n$', ' '.join(qemu_command))
+ def run(self, params, timeout, build_dir, outfile):
+ kernel_path = os.path.join(build_dir, self._kernel_path)
+ qemu_command = ['qemu-system-' + self._qemu_arch,
+ '-nodefaults',
+ '-m', '1024',
+ '-kernel', kernel_path,
+ '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
+ '-no-reboot',
+ '-nographic',
+ '-serial stdio'] + self._extra_qemu_params
+ with open(outfile, 'w') as output:
+ process = subprocess.Popen(' '.join(qemu_command),
+ stdin=subprocess.PIPE,
+ stdout=output,
+ stderr=subprocess.STDOUT,
+ text=True, shell=True)
+ try:
+ process.wait(timeout=timeout)
+ except Exception as e:
+ print(e)
+ process.terminate()
+ return process
+
+class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
+ """An abstraction over command line operations performed on a source tree."""
+
+ def __init__(self, cross_compile=None):
+ process = subprocess.Popen(' '.join(qemu_command),
+ stdin=subprocess.PIPE,
+ stdout=output,
+ stderr=subprocess.STDOUT,
+ text=True, shell=True)
+ try:
+ process.wait(timeout=timeout)
+ except Exception as e:
+ print(e)
+ process.terminate()
+ return process
+
+class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
+ """An abstraction over command line operations performed on a source tree."""
+
+ super().__init__(linux_arch='um', cross_compile=cross_compile)
+
def make_allyesconfig(self, build_dir, make_options) -> None:
kunit_parser.print_with_timestamp(
'Enabling all CONFIGs for UML...')
@@ -83,32 +177,16 @@ class LinuxSourceTreeOperations(object):
kunit_parser.print_with_timestamp(
'Enabling all CONFIGs for UML...')
def get_outfile_path(build_dir) -> str:
return get_file_path(build_dir, OUTFILE_PATH)
+def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations:
+ config_path = os.path.join(QEMU_CONFIGS_DIR, arch + '.py')
+ if arch == 'um':
+ return LinuxSourceTreeOperationsUml(cross_compile=cross_compile)
+ elif os.path.isfile(config_path):
+ return get_source_tree_ops_from_qemu_config(config_path, cross_compile)[1]
+ else:
+ raise ConfigError(arch + ' is not a valid arch')
+
+ raise ConfigError(arch + ' is not a valid arch')
+def get_source_tree_ops_from_qemu_config(config_path: str,
+ cross_compile: Optional[str]) -> tuple[
+ str, LinuxSourceTreeOperations]:
+ abs_config_path = os.path.abspath(config_path)
+ if os.path.commonpath([ABS_TOOL_PATH, abs_config_path]) != ABS_TOOL_PATH:
+ raise ConfigError('Given QEMU config file is not in a child directory of KUnit tool.')
+ relative_config_path = os.path.relpath(abs_config_path, ABS_TOOL_PATH)
+ spec = importlib.util.spec_from_file_location('.' + relative_config_path, config_path)
+ config = importlib.util.module_from_spec(spec)
+ # TODO(brendan...@google.com): I looked this up and apparently other
+ config = importlib.util.module_from_spec(spec)
+ # Python projects have noted that pytype complains that "No attribute
+ # 'exec_module' on _importlib_modulespec._Loader". Disabling for now.
+ spec.loader.exec_module(config) # pytype: disable=attribute-error
+ return config.QEMU_ARCH.linux_arch, LinuxSourceTreeOperationsQemu(
+ config.QEMU_ARCH, cross_compile=cross_compile)
+
+ config.QEMU_ARCH, cross_compile=cross_compile)
+
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
+ def __init__(
+ self,
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
+ def __init__(
+ build_dir: str,
+ load_config=True,
+ kunitconfig_path='',
+ arch=None,
+ cross_compile=None,
+ qemu_config_path=None) -> None:
signal.signal(signal.SIGINT, self.signal_handler)
-
- self._ops = LinuxSourceTreeOperations()
-
- self._ops = LinuxSourceTreeOperations()
+ if qemu_config_path:
+ self._arch, self._ops = get_source_tree_ops_from_qemu_config(
+ qemu_config_path, cross_compile)
+ self._arch, self._ops = get_source_tree_ops_from_qemu_config(
+ else:
+ self._arch = 'um' if arch is None else arch
+ self._ops = get_source_tree_ops(self._arch, cross_compile)
if not load_config:
+ self._arch = 'um' if arch is None else arch
+ self._ops = get_source_tree_ops(self._arch, cross_compile)
return
@@ -170,8 +284,9 @@ class LinuxSourceTree(object):
kconfig_path = get_kconfig_path(build_dir)
if build_dir and not os.path.exists(build_dir):
os.mkdir(build_dir)
- self._kconfig.write_to_file(kconfig_path)
try:
+ self._ops.make_arch_qemuconfig(self._kconfig)
+ self._kconfig.write_to_file(kconfig_path)
self._ops.make_olddefconfig(build_dir, make_options)
except ConfigError as e:
logging.error(e)
@@ -184,6 +299,7 @@ class LinuxSourceTree(object):
except ConfigError as e:
logging.error(e)
if os.path.exists(kconfig_path):
existing_kconfig = kunit_config.Kconfig()
existing_kconfig.read_from_file(kconfig_path)
+ self._ops.make_arch_qemuconfig(self._kconfig)
if not self._kconfig.is_subset_of(existing_kconfig):
print('Regenerating .config ...')
os.remove(kconfig_path)
@@ -194,7 +310,7 @@ class LinuxSourceTree(object):
print('Generating .config ...')
return self.build_config(build_dir, make_options)
- def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
+ def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
self._ops.make_allyesconfig(build_dir, make_options)
return self.build_config(build_dir, make_options)
- def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
+ def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
@@ -211,8 +327,8 @@ class LinuxSourceTree(object):
args.extend(['mem=1G', 'console=tty','kunit_shutdown=halt'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
- self._ops.linux_bin(args, timeout, build_dir)
outfile = get_outfile_path(build_dir)
+ self._ops.run(args, timeout, build_dir, outfile)
subprocess.call(['stty', 'sane'])
with open(outfile, 'r') as file:
for line in file:
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 2e809dd956a77..a3b7f68e1cf9f 100755
outfile = get_outfile_path(build_dir)
+ self._ops.run(args, timeout, build_dir, outfile)
subprocess.call(['stty', 'sane'])
with open(outfile, 'r') as file:
for line in file:
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -303,7 +303,7 @@ class KUnitMainTest(unittest.TestCase):
self.linux_source_mock = mock.Mock()
self.linux_source_mock.build_reconfig = mock.Mock(return_value=True)
- self.linux_source_mock.build_um_kernel = mock.Mock(return_value=True)
+ self.linux_source_mock.build_kernel = mock.Mock(return_value=True)
self.linux_source_mock.run_kernel = mock.Mock(return_value=all_passed_log)
def test_config_passes_args_pass(self):
def test_build_passes_args_pass(self):
kunit.main(['build'], self.linux_source_mock)
self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 0)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, '.kunit', None)
self.assertEqual(self.linux_source_mock.run_kernel.call_count, 0)
def test_exec_passes_args_pass(self):
@@ -396,7 +396,7 @@ class KUnitMainTest(unittest.TestCase):
kunit.main(['build'], self.linux_source_mock)
self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 0)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, '.kunit', None)
self.assertEqual(self.linux_source_mock.run_kernel.call_count, 0)
def test_exec_passes_args_pass(self):
def test_build_builddir(self):
build_dir = '.kunit'
kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, build_dir, None)
def test_exec_builddir(self):
build_dir = '.kunit'
@@ -410,14 +410,22 @@ class KUnitMainTest(unittest.TestCase):
build_dir = '.kunit'
kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
- self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, build_dir, None)
def test_exec_builddir(self):
build_dir = '.kunit'
mock_linux_init.return_value = self.linux_source_mock
kunit.main(['run', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ arch='um',
+ cross_compile=None,
kunit.main(['run', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ arch='um',
+ qemu_config_path=None)
@mock.patch.object(kunit_kernel, 'LinuxSourceTree')
def test_config_kunitconfig(self, mock_linux_init):
mock_linux_init.return_value = self.linux_source_mock
kunit.main(['config', '--kunitconfig=mykunitconfig'])
# Just verify that we parsed and initialized it correctly here.
- mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path='mykunitconfig',
+ cross_compile=None,
+ qemu_config_path=None)
if __name__ == '__main__':
unittest.main()
new file mode 100644
index 0000000000000..aff1fe0442dbc
--- /dev/null
+++ b/tools/testing/kunit/qemu_config.py
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Collection of configs for building non-UML kernels and running them on QEMU.
+#
+# Copyright (C) 2021, Google LLC.
+# Author: Brendan Higgins <brendan...@google.com>
+
+from collections import namedtuple
+
+
+
+QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
+ 'qemuconfig',
+ 'qemu_arch',
+ 'kernel_path',
+ 'kernel_command_line',
+ 'extra_qemu_params'])
+
diff --git a/tools/testing/kunit/qemu_configs/alpha.py b/tools/testing/kunit/qemu_configs/alpha.py
+QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
+ 'qemuconfig',
+ 'qemu_arch',
+ 'kernel_path',
+ 'kernel_command_line',
+ 'extra_qemu_params'])
+
new file mode 100644
index 0000000000000..2cc64f848ca2c
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/alpha.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='alpha',
+
+ qemuconfig='''
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='alpha',
+ kernel_path='arch/alpha/boot/vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
diff --git a/tools/testing/kunit/qemu_configs/arm.py b/tools/testing/kunit/qemu_configs/arm.py
+ kernel_path='arch/alpha/boot/vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
new file mode 100644
index 0000000000000..29a043b0531a0
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/arm.py
@@ -0,0 +1,13 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='arm',
+
+ qemuconfig='''
+CONFIG_ARCH_VIRT=y
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='arm',
+ kernel_path='arch/arm/boot/zImage',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt'])
diff --git a/tools/testing/kunit/qemu_configs/arm64.py b/tools/testing/kunit/qemu_configs/arm64.py
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='arm',
+ kernel_path='arch/arm/boot/zImage',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt'])
new file mode 100644
index 0000000000000..1ba200bc99f0f
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/arm64.py
@@ -0,0 +1,12 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='arm64',
+
+ qemuconfig='''
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='aarch64',
+ kernel_path='arch/arm64/boot/Image.gz',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt', '-cpu cortex-a57'])
+ kernel_path='arch/arm64/boot/Image.gz',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt', '-cpu cortex-a57'])
diff --git a/tools/testing/kunit/qemu_configs/i386.py b/tools/testing/kunit/qemu_configs/i386.py
new file mode 100644
index 0000000000000..3998af306468e
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/i386.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='i386',
+
+ qemuconfig='''
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
diff --git a/tools/testing/kunit/qemu_configs/powerpc.py b/tools/testing/kunit/qemu_configs/powerpc.py
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
new file mode 100644
index 0000000000000..46292ce9e368e
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/powerpc.py
@@ -0,0 +1,12 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='powerpc',
+
+ qemuconfig='''
+CONFIG_PPC64=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_HVC_CONSOLE=y''',
+ qemu_arch='ppc64',
+ kernel_path='vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-M pseries', '-cpu power8'])
diff --git a/tools/testing/kunit/qemu_configs/riscv.py b/tools/testing/kunit/qemu_configs/riscv.py
+ kernel_path='vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-M pseries', '-cpu power8'])
new file mode 100644
index 0000000000000..de8c62d465723
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/riscv.py
@@ -0,0 +1,31 @@
+from ..qemu_config import QemuArchParams
+import os
+import os.path
+import sys
+
+GITHUB_OPENSBI_URL = 'https://github.com/qemu/qemu/raw/master/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin'
+OPENSBI_FILE = os.path.basename(GITHUB_OPENSBI_URL)
+
+if not os.path.isfile(OPENSBI_FILE):
+ print('\n\nOpenSBI file is not in the current working directory.\n'
+ 'Would you like me to download it for you from:\n' + GITHUB_OPENSBI_URL + ' ?\n')
+ response = input('yes/[no]: ')
+ if response.strip() == 'yes':
+ os.system('wget ' + GITHUB_OPENSBI_URL)
+ else:
+ sys.exit()
+
+QEMU_ARCH = QemuArchParams(linux_arch='riscv',
+ qemuconfig='''
+CONFIG_SOC_VIRT=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_OF_PLATFORM=y
+CONFIG_SERIAL_EARLYCON_RISCV_SBI=y''',
+ qemu_arch='riscv64',
+ kernel_path='arch/riscv/boot/Image',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine virt',
+ kernel_path='arch/riscv/boot/Image',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-cpu rv64',
+ '-bios opensbi-riscv64-generic-fw_dynamic.bin'])
diff --git a/tools/testing/kunit/qemu_configs/s390.py b/tools/testing/kunit/qemu_configs/s390.py
new file mode 100644
index 0000000000000..04c90332f1098
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/s390.py
@@ -0,0 +1,14 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='s390',
+
+ qemuconfig='''
+CONFIG_EXPERT=y
+CONFIG_TUNE_ZEC12=y
+CONFIG_NUMA=y
+CONFIG_MODULES=y''',
+ qemu_arch='s390x',
+ kernel_path='arch/s390/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine s390-ccw-virtio',
+ '-cpu qemu',])
diff --git a/tools/testing/kunit/qemu_configs/sparc.py b/tools/testing/kunit/qemu_configs/sparc.py
+ kernel_path='arch/s390/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine s390-ccw-virtio',
+ '-cpu qemu',])
new file mode 100644
index 0000000000000..f26b5f27cc5a1
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/sparc.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='sparc',
+
+ qemuconfig='''
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='sparc',
+ kernel_path='arch/sparc/boot/zImage',
+ kernel_command_line='console=ttyS0 mem=256M',
+ extra_qemu_params=['-m 256'])
diff --git a/tools/testing/kunit/qemu_configs/x86_64.py b/tools/testing/kunit/qemu_configs/x86_64.py
+ kernel_path='arch/sparc/boot/zImage',
+ kernel_command_line='console=ttyS0 mem=256M',
+ extra_qemu_params=['-m 256'])
new file mode 100644
index 0000000000000..bd5ab733b92ac
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/x86_64.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='x86_64',
+
+ qemuconfig='''
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
--
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
2.31.1.607.g51e8a6a459-goog
Brendan Higgins
May 7, 2021, 5:31:38 PM5/7/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Document QEMU support, what it does, and how to use it in kunit_tool.
David Gow
May 15, 2021, 3:58:20 AM5/15/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
<brendan...@google.com> wrote:
>
> From: David Gow <davi...@google.com>
>
> Add a new kernel command-line option, 'kunit_shutdown', which allows the
> user to specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running KUnit tests
> on UML or a VM so that the UML/VM process exits cleanly immediately
> after running all tests without needing a special initramfs.
>
> Signed-off-by: David Gow <davi...@google.com>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
> Tested-By: Daniel Latypov <dlat...@google.com>
> ---
Obviously I'm okay with this change, but I did find a minor whitespace
<brendan...@google.com> wrote:
>
> From: David Gow <davi...@google.com>
>
> Add a new kernel command-line option, 'kunit_shutdown', which allows the
> user to specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running KUnit tests
> on UML or a VM so that the UML/VM process exits cleanly immediately
> after running all tests without needing a special initramfs.
>
> Signed-off-by: David Gow <davi...@google.com>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
> Tested-By: Daniel Latypov <dlat...@google.com>
> ---
nitpick below.
David Gow
May 15, 2021, 3:58:42 AM5/15/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
> Add kunit_shutdown, an option to specify that the kernel shutsdown after
> running KUnit tests, to the kernel-parameters.txt documentation.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
> ---
Looks good to me.
> running KUnit tests, to the kernel-parameters.txt documentation.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
> ---
Reviewed-by: David Gow <davi...@google.com>
David Gow
May 15, 2021, 3:59:44 AM5/15/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
> Add basic support to run QEMU via kunit_tool. Add support for i386,
> x86_64, arm, arm64, and a bunch more.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Tested-by: David Gow <davi...@google.com>
> ---
>
> Changes since last revision:
>
> - A number of minor obvious issues pointed out by David and Daniel.
> - Added facility for merging Kconfigs at Daniel's suggestion.
> - Broke out qemu_configs each into their own config file which is loaded
> dynamically - mostly at David's suggestion.
>
> ---
This seems pretty good to me. I only have one real complaint --
> x86_64, arm, arm64, and a bunch more.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Tested-by: David Gow <davi...@google.com>
> ---
>
> Changes since last revision:
>
> - A number of minor obvious issues pointed out by David and Daniel.
> - Added facility for merging Kconfigs at Daniel's suggestion.
> - Broke out qemu_configs each into their own config file which is loaded
> dynamically - mostly at David's suggestion.
>
> ---
qemu_configs needing to be in a subdirectory of ./tools/testing/kunit
-- but am able to tolerate that (even if I'd prefer not to have it) if
it's documented properly.
Otherwise, save for a couple of minor nitpicks, this seems good to go.
Reviewed-by: David Gow <davi...@google.com>
much longer than all the surrounding lines?
> + 'if you have downloaded the microblaze toolchain '
> + 'from the 0-day website to a directory in your '
> + 'home directory called `toolchains`).'),
> + metavar='cross_compile')
> +
> + parser.add_argument('--qemu_config',
> + help=('Takes a path to a path to a file containing '
> + 'a QemuArchParams object.'),
> + type=str, metavar='qemu_config')
> +
kunit_tool directory, we should document this, or maybe make this path
relative to the kunit_tool directory.
undercuts the value of the --qemu_config option a bit: we almost might
as well keep everything in the QEMU_CONFIG_DIR.
I assume it's here because of the need to import the QemuArchParams
definition: I take it there's no convenient way to do that?
At the very least, let's document this restriction properly, as it was
a bit of a weird one I wasn't expecting. Then people can either put
their custom qemu configs in the qemu_configs/ directory, or in
something like a custom_qemu_configs/ one. And if we later can work
out a more convenient way of removing this restriction entirely, we
can do so.
David Gow
May 15, 2021, 4:01:32 AM5/15/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
> Document QEMU support, what it does, and how to use it in kunit_tool.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> ---
This is a good start, and probably meets the minimum requirements, but
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> ---
I do have a number of comments and suggestions below.
Cheers,
-- David
> Documentation/dev-tools/kunit/usage.rst | 37 +++++++++++++++++++++----
> 1 file changed, 31 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
> index 650f99590df57..b74bd7c87cc20 100644
> --- a/Documentation/dev-tools/kunit/usage.rst
> +++ b/Documentation/dev-tools/kunit/usage.rst
> @@ -612,14 +612,39 @@ only things to be aware of when doing so.
> The biggest impediment will likely be that certain KUnit features and
> infrastructure may not support your target environment. For example, at this
> time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside
> -of UML. Unfortunately, there is no way around this. Using UML (or even just a
> -particular architecture) allows us to make a lot of assumptions that make it
> -possible to do things which might otherwise be impossible.
> +of UML and QEMU. Unfortunately, there is no way around this. Using UML and QEMU
> +(or even just a particular architecture) allows us to make a lot of assumptions
> +that make it possible to do things which might otherwise be impossible.
right impression. Particularly the "Unfortunately, there is no way
around this." bit: there's no fundamental reason that someone couldn't
implement support for some other emulator (or even a setup which
pushed to real hardware and read results over serial), it'd just take
a bit of work to implement (like this patch series has done for qemu).
Personally, I think it'd be easiest to simplify this section and say
that kunit_tool currently only fully supports some architectures, via
UML and QEMU.
(Particularly given the restriction on its path?) Or is that something
best added to the kunit_tool page?
That being said, changes to the kunit_tool page are probably worth
adding as a section in the updated page:
https://patchwork.kernel.org/project/linux-kselftest/patch/20210417034553.1...@google.com/
At the very least, it'd be nice to have the new QEMU-related options
documented there.
David Gow
May 17, 2021, 11:01:56 PM5/17/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Sat, May 15, 2021 at 3:59 PM David Gow <davi...@google.com> wrote:
>
> On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
> <brendan...@google.com> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
> >
> > Signed-off-by: Brendan Higgins <brendan...@google.com>
> > Tested-by: David Gow <davi...@google.com>
> > ---
> >
> > Changes since last revision:
> >
> > - A number of minor obvious issues pointed out by David and Daniel.
> > - Added facility for merging Kconfigs at Daniel's suggestion.
> > - Broke out qemu_configs each into their own config file which is loaded
> > dynamically - mostly at David's suggestion.
> >
> > ---
>
> This seems pretty good to me. I only have one real complaint --
> qemu_configs needing to be in a subdirectory of ./tools/testing/kunit
> -- but am able to tolerate that (even if I'd prefer not to have it) if
> it's documented properly.
>
> Otherwise, save for a couple of minor nitpicks, this seems good to go.
>
> Reviewed-by: David Gow <davi...@google.com>
>
>
One thing I forgot to mention is that I'm not 100% sure about the
>
> On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
> <brendan...@google.com> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
> >
> > Signed-off-by: Brendan Higgins <brendan...@google.com>
> > Tested-by: David Gow <davi...@google.com>
> > ---
> >
> > Changes since last revision:
> >
> > - A number of minor obvious issues pointed out by David and Daniel.
> > - Added facility for merging Kconfigs at Daniel's suggestion.
> > - Broke out qemu_configs each into their own config file which is loaded
> > dynamically - mostly at David's suggestion.
> >
> > ---
>
> This seems pretty good to me. I only have one real complaint --
> qemu_configs needing to be in a subdirectory of ./tools/testing/kunit
> -- but am able to tolerate that (even if I'd prefer not to have it) if
> it's documented properly.
>
> Otherwise, save for a couple of minor nitpicks, this seems good to go.
>
> Reviewed-by: David Gow <davi...@google.com>
>
>
Kconfig fragments being embedded in the qemu_configs. I still kind-of
prefer the idea of them being in separate config files. While I don't
think this is necessarily a blocker, I did just realise that, by
default, kunit.py run --arch=<non-UM-arch> will pull its default
.kunitconfig from arch/um/configs/kunit_defconfig, which definitely
feels awkward when UML is not otherwise involved.
Some further thoughts below (which range a bit from "practical
suggestion" to "overcomplicated ponderings", so don't feel the
pressure to take all of them).
(...snip...)
builds. Is it worth either:
(a) moving this somewhere else (e.g., tools/testing/kunit/configs as
with the BROKEN_ALLCONFIG_PATH beflow), or
(b) giving each architecture its own kunit_defconfig, possibly in
place of the qemuconfig member of QemuArchParams
I'm leaning towards (b), which solves two different sources of
ugliness in one go, though it would appear to have the downside that
the default .kunitconfig could end up being architecture specific,
which isn't great.
> > BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
> > OUTFILE_PATH = 'test.log'
> > +ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__))
> > +QEMU_CONFIGS_DIR = os.path.join(ABS_TOOL_PATH, 'qemu_configs')
> >
> > diff --git a/tools/testing/kunit/qemu_config.py b/tools/testing/kunit/qemu_config.py
> > new file mode 100644
> > index 0000000000000..aff1fe0442dbc
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_config.py
> > @@ -0,0 +1,17 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# Collection of configs for building non-UML kernels and running them on QEMU.
> > +#
> > +# Copyright (C) 2021, Google LLC.
> > +# Author: Brendan Higgins <brendan...@google.com>
> > +
> > +from collections import namedtuple
> > +
> > +
> > +QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
> > + 'qemuconfig',
and would kind-of prefer it to be in another file. I could live with
it if I have to, though. Regardless, 'qemuconfig' is not a
super-descriptive name, particularly as it's not clear if this is
configuring QEMU (no, that's extra_qemu_params'), or configuring the
kernel for QEMU compatibility.
> > + 'qemu_arch',
> > + 'kernel_path',
> > + 'kernel_command_line',
> > + 'extra_qemu_params'])
> > +
>
> Nit: newline at end of file.
>
>
>
> > diff --git a/tools/testing/kunit/qemu_configs/alpha.py b/tools/testing/kunit/qemu_configs/alpha.py
> > new file mode 100644
> > index 0000000000000..2cc64f848ca2c
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/alpha.py
> > @@ -0,0 +1,10 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='alpha',
> > + qemuconfig='''
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y''',
i386, x86_64, etc. Of course, that wouldn't gel well with putting them
in arch/.../config. If there were some way of listing multiple files,
it could form part of the config for several more architectures,
though that's probably overcomplicating things.
> > + qemu_arch='alpha',
> > + kernel_path='arch/alpha/boot/vmlinux',
> > + kernel_command_line='console=ttyS0',
> > + extra_qemu_params=[''])
> > diff --git a/tools/testing/kunit/qemu_configs/arm.py b/tools/testing/kunit/qemu_configs/arm.py
> > new file mode 100644
> > index 0000000000000..29a043b0531a0
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/arm.py
> > @@ -0,0 +1,13 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='arm',
> > + qemuconfig='''
> > +CONFIG_ARCH_VIRT=y
> > +CONFIG_SERIAL_AMBA_PL010=y
> > +CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
> > +CONFIG_SERIAL_AMBA_PL011=y
> > +CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
mechanism, these could mostly be shared between arm & arm64 (ARCH_VIRT
being the only problem). Again, probably overcomplicating it at this
point though.
Brendan Higgins
May 18, 2021, 4:28:04 PM5/18/21
to David Gow, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
toolchains and they don't get much shorter than this (I guess I could
drop $HOME/toolchains/microblaze, but it would still be very long).
Think maybe I should just drop this secondary example entirely? Anyone
trying to download and use a toolchain probably has better resources
to follow anyway.
> > + 'if you have downloaded the microblaze toolchain '
> > + 'from the 0-day website to a directory in your '
> > + 'home directory called `toolchains`).'),
> > + metavar='cross_compile')
> > +
> > + parser.add_argument('--qemu_config',
> > + help=('Takes a path to a path to a file containing '
> > + 'a QemuArchParams object.'),
> > + type=str, metavar='qemu_config')
> > +
>
> If (as noted below) this file needs to be in a subdirectory of the
> kunit_tool directory, we should document this, or maybe make this path
> relative to the kunit_tool directory.
a config from anywhere.
In the next revision, no such requirement will exist.
> At the very least, let's document this restriction properly, as it was
> a bit of a weird one I wasn't expecting. Then people can either put
> their custom qemu configs in the qemu_configs/ directory, or in
> something like a custom_qemu_configs/ one. And if we later can work
> out a more convenient way of removing this restriction entirely, we
> can do so.
location in future revisions.
Brendan Higgins
May 18, 2021, 4:43:29 PM5/18/21
to David Gow, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Mon, May 17, 2021 at 8:01 PM David Gow <davi...@google.com> wrote:
>
> On Sat, May 15, 2021 at 3:59 PM David Gow <davi...@google.com> wrote:
> >
> > On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
> > <brendan...@google.com> wrote:
> > >
> > > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > > x86_64, arm, arm64, and a bunch more.
> > >
> > > Signed-off-by: Brendan Higgins <brendan...@google.com>
> > > Tested-by: David Gow <davi...@google.com>
> > > ---
> > >
> > > Changes since last revision:
> > >
> > > - A number of minor obvious issues pointed out by David and Daniel.
> > > - Added facility for merging Kconfigs at Daniel's suggestion.
> > > - Broke out qemu_configs each into their own config file which is loaded
> > > dynamically - mostly at David's suggestion.
> > >
> > > ---
> >
> > This seems pretty good to me. I only have one real complaint --
> > qemu_configs needing to be in a subdirectory of ./tools/testing/kunit
> > -- but am able to tolerate that (even if I'd prefer not to have it) if
> > it's documented properly.
> >
> > Otherwise, save for a couple of minor nitpicks, this seems good to go.
> >
> > Reviewed-by: David Gow <davi...@google.com>
> >
> >
>
> One thing I forgot to mention is that I'm not 100% sure about the
> Kconfig fragments being embedded in the qemu_configs. I still kind-of
> prefer the idea of them being in separate config files. While I don't
I don't feel strongly either way, but I don't have a good idea on how
>
> On Sat, May 15, 2021 at 3:59 PM David Gow <davi...@google.com> wrote:
> >
> > On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
> > <brendan...@google.com> wrote:
> > >
> > > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > > x86_64, arm, arm64, and a bunch more.
> > >
> > > Signed-off-by: Brendan Higgins <brendan...@google.com>
> > > Tested-by: David Gow <davi...@google.com>
> > > ---
> > >
> > > Changes since last revision:
> > >
> > > - A number of minor obvious issues pointed out by David and Daniel.
> > > - Added facility for merging Kconfigs at Daniel's suggestion.
> > > - Broke out qemu_configs each into their own config file which is loaded
> > > dynamically - mostly at David's suggestion.
> > >
> > > ---
> >
> > This seems pretty good to me. I only have one real complaint --
> > qemu_configs needing to be in a subdirectory of ./tools/testing/kunit
> > -- but am able to tolerate that (even if I'd prefer not to have it) if
> > it's documented properly.
> >
> > Otherwise, save for a couple of minor nitpicks, this seems good to go.
> >
> > Reviewed-by: David Gow <davi...@google.com>
> >
> >
>
> One thing I forgot to mention is that I'm not 100% sure about the
> Kconfig fragments being embedded in the qemu_configs. I still kind-of
> prefer the idea of them being in separate config files. While I don't
to implement your idea well. How about we leave it for now, and if you
decide you really want to do something about it, you can do it?
> think this is necessarily a blocker, I did just realise that, by
> default, kunit.py run --arch=<non-UM-arch> will pull its default
> .kunitconfig from arch/um/configs/kunit_defconfig, which definitely
> feels awkward when UML is not otherwise involved.
tools/testing/kunit/configs/all_tests.config. Maybe we should just use
that instead?
file is identical.
> (b) giving each architecture its own kunit_defconfig, possibly in
> place of the qemuconfig member of QemuArchParams
>
> I'm leaning towards (b), which solves two different sources of
> ugliness in one go, though it would appear to have the downside that
> the default .kunitconfig could end up being architecture specific,
> which isn't great.
This is sounding more and more like what should be follow-on work to
me.
> super-descriptive name, particularly as it's not clear if this is
> configuring QEMU (no, that's extra_qemu_params'), or configuring the
> kernel for QEMU compatibility.
configs contain configs for configuring, building, and running kernels
on QEMU.
> > > + 'qemu_arch',
> > > + 'kernel_path',
> > > + 'kernel_command_line',
> > > + 'extra_qemu_params'])
> > > +
> >
> > Nit: newline at end of file.
> >
> >
> >
> > > diff --git a/tools/testing/kunit/qemu_configs/alpha.py b/tools/testing/kunit/qemu_configs/alpha.py
> > > new file mode 100644
> > > index 0000000000000..2cc64f848ca2c
> > > --- /dev/null
> > > +++ b/tools/testing/kunit/qemu_configs/alpha.py
> > > @@ -0,0 +1,10 @@
> > > +from ..qemu_config import QemuArchParams
> > > +
> > > +QEMU_ARCH = QemuArchParams(linux_arch='alpha',
> > > + qemuconfig='''
> > > +CONFIG_SERIAL_8250=y
> > > +CONFIG_SERIAL_8250_CONSOLE=y''',
>
> If these were in a separate file, they could be shared across alpha,
> i386, x86_64, etc. Of course, that wouldn't gel well with putting them
> in arch/.../config. If there were some way of listing multiple files,
> it could form part of the config for several more architectures,
> though that's probably overcomplicating things.
it really feels like something that can and should be addressed in
follow on patches. We could totally address this issue later by
expanding this field to take either a string containing a Kconfig, or
a path to an external Kconfig; if we do so, it won't cause any
migration issues in the future.
> > > + qemu_arch='alpha',
> > > + kernel_path='arch/alpha/boot/vmlinux',
> > > + kernel_command_line='console=ttyS0',
> > > + extra_qemu_params=[''])
> > > diff --git a/tools/testing/kunit/qemu_configs/arm.py b/tools/testing/kunit/qemu_configs/arm.py
> > > new file mode 100644
> > > index 0000000000000..29a043b0531a0
> > > --- /dev/null
> > > +++ b/tools/testing/kunit/qemu_configs/arm.py
> > > @@ -0,0 +1,13 @@
> > > +from ..qemu_config import QemuArchParams
> > > +
> > > +QEMU_ARCH = QemuArchParams(linux_arch='arm',
> > > + qemuconfig='''
> > > +CONFIG_ARCH_VIRT=y
> > > +CONFIG_SERIAL_AMBA_PL010=y
> > > +CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
> > > +CONFIG_SERIAL_AMBA_PL011=y
> > > +CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
>
> Similarly, if in a separate file and there were some multiple-file
> mechanism, these could mostly be shared between arm & arm64 (ARCH_VIRT
> being the only problem). Again, probably overcomplicating it at this
> point though.
Brendan Higgins
May 18, 2021, 4:52:15 PM5/18/21
to David Gow, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
Thanks, will fix.
Brendan Higgins
May 18, 2021, 5:08:55 PM5/18/21
to David Gow, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
Documentation changes before I document the new options to avoid
conflicts.
David Gow
May 20, 2021, 11:53:27 PM5/20/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Wed, May 19, 2021 at 4:43 AM Brendan Higgins
That sounds like a better plan. It looks like all_tests.config isn't
used anywhere, anyway. I might rename it and replace the
arch/um/.../kunit_defconfig version in another patch, then.
Yeah: I'm not thrilled with the name all_tests.config, but since it
doesn't appear to be being used anywhere, I might just rename it in
another patch.
> > (b) giving each architecture its own kunit_defconfig, possibly in
> > place of the qemuconfig member of QemuArchParams
> >
> > I'm leaning towards (b), which solves two different sources of
> > ugliness in one go, though it would appear to have the downside that
> > the default .kunitconfig could end up being architecture specific,
> > which isn't great.
>
> Yeah, I am not a fan of trying to solve that problem in this patchset.
> This is sounding more and more like what should be follow-on work to
> me.
Yeah, I'm not sure exactly what that should look like yet, anyway.
Let's keep things as they are in this patch. I'll put a follow-up
patch to use all_tests.config rather than the arch/um one (possibly as
part of my "default to ALL_TESTS" patchset), and if we think of
something better that is more architecture specific, we'll do that.
I don't think we need "qemu" in the name, as this is already part of
the QemuArchParams struct, and isn't a qemu config, but a kernel one.
Something along the lines of "kernel_config" (or just "kconfig") maybe?
Yeah: I think we can solve this if it actually becomes a problem. No
need to change anything here.
used anywhere, anyway. I might rename it and replace the
arch/um/.../kunit_defconfig version in another patch, then.
doesn't appear to be being used anywhere, I might just rename it in
another patch.
> > (b) giving each architecture its own kunit_defconfig, possibly in
> > place of the qemuconfig member of QemuArchParams
> >
> > I'm leaning towards (b), which solves two different sources of
> > ugliness in one go, though it would appear to have the downside that
> > the default .kunitconfig could end up being architecture specific,
> > which isn't great.
>
> Yeah, I am not a fan of trying to solve that problem in this patchset.
> This is sounding more and more like what should be follow-on work to
> me.
Let's keep things as they are in this patch. I'll put a follow-up
patch to use all_tests.config rather than the arch/um one (possibly as
part of my "default to ALL_TESTS" patchset), and if we think of
something better that is more architecture specific, we'll do that.
the QemuArchParams struct, and isn't a qemu config, but a kernel one.
Something along the lines of "kernel_config" (or just "kconfig") maybe?
need to change anything here.
Brendan Higgins
May 21, 2021, 4:57:39 PM5/21/21
to David Gow, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
talking about the name QemuArchParams, not the qemuconfig field within
that struct. Looking back it is entirely clear to me that that is what
you were talking about, but for some reason I had a brainfart in my
last response. So yes, your suggestion sounds very reasonable. Will
fix.
Brendan Higgins
May 26, 2021, 5:24:14 PM5/26/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
did this also allows new QemuConfigs to be added without making any
changes to kunit_tool.
I changed how Kconfigs are loaded; they are now merged inside of
kunit_tool instead of letting Kbuild do it.
I also made numerous nit fixes.
Finally, I added a new section to the kunit_tool documentation to
document the new command line flags I added.
[1] http://patches.linaro.org/patch/208336/
[2] https://lkml.org/lkml/2020/6/26/988
Brendan Higgins (3):
Documentation: Add kunit_shutdown to kernel-parameters.txt
kunit: tool: add support for QEMU
Documentation: kunit: document support for QEMU in kunit_tool
David Gow (1):
kunit: Add 'kunit_shutdown' option
.../admin-guide/kernel-parameters.txt | 8 +
Documentation/dev-tools/kunit/usage.rst | 50 +++--
lib/kunit/executor.c | 20 ++
tools/testing/kunit/kunit.py | 57 +++++-
tools/testing/kunit/kunit_config.py | 7 +-
tools/testing/kunit/kunit_kernel.py | 177 +++++++++++++++---
tools/testing/kunit/kunit.py | 57 +++++-
tools/testing/kunit/kunit_config.py | 7 +-
tools/testing/kunit/kunit_parser.py | 2 +-
tools/testing/kunit/kunit_tool_test.py | 18 +-
tools/testing/kunit/qemu_config.py | 16 ++
tools/testing/kunit/kunit_tool_test.py | 18 +-
tools/testing/kunit/qemu_configs/alpha.py | 10 +
tools/testing/kunit/qemu_configs/arm.py | 13 ++
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/i386.py | 10 +
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/i386.py | 10 +
tools/testing/kunit/qemu_configs/powerpc.py | 12 ++
tools/testing/kunit/qemu_configs/riscv.py | 31 +++
tools/testing/kunit/qemu_configs/riscv.py | 31 +++
tools/testing/kunit/qemu_configs/s390.py | 14 ++
tools/testing/kunit/qemu_configs/sparc.py | 10 +
tools/testing/kunit/qemu_configs/x86_64.py | 10 +
tools/testing/kunit/qemu_configs/sparc.py | 10 +
19 files changed, 471 insertions(+), 54 deletions(-)
create mode 100644 tools/testing/kunit/qemu_config.py
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
base-commit: d7eab3df8f39b116d934bc17f8070861e18cfb62
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
--
2.31.1.818.g46aad6cb9e-goog
Brendan Higgins
May 26, 2021, 5:24:17 PM5/26/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
From: David Gow <davi...@google.com>
Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.
Signed-off-by: David Gow <davi...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
Tested-By: Daniel Latypov <dlat...@google.com>
---
Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.
Signed-off-by: David Gow <davi...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Stephen Boyd <sb...@kernel.org>
Tested-By: Daniel Latypov <dlat...@google.com>
---
index 89a7d4024e878..dbbceaee12305 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -208,7 +208,7 @@ class LinuxSourceTree(object):
def run_kernel(self, args=None, build_dir='', filter_glob='', timeout=None) -> Iterator[str]:
if not args:
args = []
- args.extend(['mem=1G', 'console=tty'])
+ args.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt'])
def run_kernel(self, args=None, build_dir='', filter_glob='', timeout=None) -> Iterator[str]:
if not args:
args = []
- args.extend(['mem=1G', 'console=tty'])
+ args.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
self._ops.linux_bin(args, timeout, build_dir)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index e8bcc139702e2..8d8d4d70b39dd 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -49,7 +49,7 @@ class TestStatus(Enum):
kunit_start_re = re.compile(r'TAP version [0-9]+$')
kunit_end_re = re.compile('(List of all partitions:|'
- 'Kernel panic - not syncing: VFS:)')
+ 'Kernel panic - not syncing: VFS:|reboot: System halted)')
def isolate_kunit_output(kernel_output) -> Iterator[str]:
started = False
--
2.31.1.818.g46aad6cb9e-goog
args.append('kunit.filter_glob='+filter_glob)
self._ops.linux_bin(args, timeout, build_dir)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index e8bcc139702e2..8d8d4d70b39dd 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -49,7 +49,7 @@ class TestStatus(Enum):
kunit_start_re = re.compile(r'TAP version [0-9]+$')
kunit_end_re = re.compile('(List of all partitions:|'
- 'Kernel panic - not syncing: VFS:)')
+ 'Kernel panic - not syncing: VFS:|reboot: System halted)')
def isolate_kunit_output(kernel_output) -> Iterator[str]:
started = False
--
Brendan Higgins
May 26, 2021, 5:24:18 PM5/26/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Add kunit_shutdown, an option to specify that the kernel shutsdown after
running KUnit tests, to the kernel-parameters.txt documentation.
Reviewed-by: David Gow <davi...@google.com>
running KUnit tests, to the kernel-parameters.txt documentation.
---
Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index cb89dbdedc463..78c22bd55b92e 100644
1 file changed, 8 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2256,6 +2256,14 @@
0: force disabled
1: force enabled
+ kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after
+ running built-in tests. Tests configured as modules will
+ not be run.
+ Default: (flag not present) don't shutdown
+ poweroff: poweroff the kernel after running tests
+ halt: halt the kernel after running tests
+ reboot: reboot the kernel after running tests
+
kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.
Default is 0 (don't ignore, but inject #GP)
--
2.31.1.818.g46aad6cb9e-goog
1: force enabled
+ kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after
+ running built-in tests. Tests configured as modules will
+ not be run.
+ Default: (flag not present) don't shutdown
+ poweroff: poweroff the kernel after running tests
+ halt: halt the kernel after running tests
+ reboot: reboot the kernel after running tests
+
kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.
Default is 0 (don't ignore, but inject #GP)
--
Brendan Higgins
May 26, 2021, 5:24:20 PM5/26/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Add basic support to run QEMU via kunit_tool. Add support for i386,
x86_64, arm, arm64, and a bunch more.
Signed-off-by: Brendan Higgins <brendan...@google.com>
Tested-by: David Gow <davi...@google.com>
Changes since last revision:
x86_64, arm, arm64, and a bunch more.
Signed-off-by: Brendan Higgins <brendan...@google.com>
Tested-by: David Gow <davi...@google.com>
- I pulled out the QemuConfigs into their own files; the way in which I
did this also allows new QemuConfigs to be added without making any
changes to kunit_tool.
kunit_tool instead of letting Kbuild do it.
- I also made numerous nit fixes.
---
tools/testing/kunit/kunit.py | 57 ++++++-
tools/testing/kunit/kunit_config.py | 7 +-
tools/testing/kunit/kunit_kernel.py | 175 +++++++++++++++++---
tools/testing/kunit/kunit_config.py | 7 +-
tools/testing/kunit/kunit_tool_test.py | 18 +-
tools/testing/kunit/qemu_config.py | 16 ++
tools/testing/kunit/qemu_configs/alpha.py | 10 ++
tools/testing/kunit/qemu_config.py | 16 ++
tools/testing/kunit/qemu_configs/arm.py | 13 ++
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/i386.py | 10 ++
tools/testing/kunit/qemu_configs/arm64.py | 12 ++
tools/testing/kunit/qemu_configs/powerpc.py | 12 ++
tools/testing/kunit/qemu_configs/riscv.py | 31 ++++
tools/testing/kunit/qemu_configs/s390.py | 14 ++
tools/testing/kunit/qemu_configs/sparc.py | 10 ++
tools/testing/kunit/qemu_configs/x86_64.py | 10 ++
14 files changed, 354 insertions(+), 41 deletions(-)
tools/testing/kunit/qemu_configs/x86_64.py | 10 ++
create mode 100644 tools/testing/kunit/qemu_config.py
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
create mode 100644 tools/testing/kunit/qemu_configs/arm.py
create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
create mode 100644 tools/testing/kunit/qemu_configs/i386.py
create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
create mode 100644 tools/testing/kunit/qemu_configs/s390.py
create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
+ 'if you have downloaded the microblaze toolchain '
+ 'from the 0-day website to a directory in your '
+ 'home directory called `toolchains`).'),
+ metavar='cross_compile')
+
+ parser.add_argument('--qemu_config',
+ help=('Takes a path to a path to a file containing '
+ 'a QemuArchParams object.'),
+ type=str, metavar='qemu_config')
+
+ 'from the 0-day website to a directory in your '
+ 'home directory called `toolchains`).'),
+ metavar='cross_compile')
+
+ parser.add_argument('--qemu_config',
+ help=('Takes a path to a path to a file containing '
+ 'a QemuArchParams object.'),
+ type=str, metavar='qemu_config')
+
index dbbceaee12305..e1951fa600277 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -6,23 +6,31 @@
# Author: Felix Guo <felixgu...@gmail.com>
# Author: Brendan Higgins <brendan...@google.com>
+from __future__ import annotations
+import importlib.util
import logging
import subprocess
import os
import shutil
import signal
from typing import Iterator
+from typing import Optional
from contextlib import ExitStack
+from collections import namedtuple
+
import kunit_config
import kunit_parser
+import qemu_config
KCONFIG_PATH = '.config'
KUNITCONFIG_PATH = '.kunitconfig'
DEFAULT_KUNITCONFIG_PATH = 'arch/um/configs/kunit_defconfig'
# Author: Felix Guo <felixgu...@gmail.com>
# Author: Brendan Higgins <brendan...@google.com>
+from __future__ import annotations
+import importlib.util
import logging
import subprocess
import os
import shutil
import signal
from typing import Iterator
+from typing import Optional
from contextlib import ExitStack
+from collections import namedtuple
+
import kunit_config
import kunit_parser
+import qemu_config
KCONFIG_PATH = '.config'
KUNITCONFIG_PATH = '.kunitconfig'
DEFAULT_KUNITCONFIG_PATH = 'arch/um/configs/kunit_defconfig'
BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
OUTFILE_PATH = 'test.log'
+ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__))
+QEMU_CONFIGS_DIR = os.path.join(ABS_TOOL_PATH, 'qemu_configs')
OUTFILE_PATH = 'test.log'
+ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__))
+QEMU_CONFIGS_DIR = os.path.join(ABS_TOOL_PATH, 'qemu_configs')
+ self._qemu_arch = qemu_arch_params.qemu_arch
+ self._kernel_path = qemu_arch_params.kernel_path
+ self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
+ self._extra_qemu_params = qemu_arch_params.extra_qemu_params
+
+ def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None:
+ kconfig = kunit_config.Kconfig()
+ self._kernel_path = qemu_arch_params.kernel_path
+ self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
+ self._extra_qemu_params = qemu_arch_params.extra_qemu_params
+
+ def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None:
+ kconfig.parse_from_string(self._kconfig)
+ base_kunitconfig.merge_in_entries(kconfig)
def get_outfile_path(build_dir) -> str:
return get_file_path(build_dir, OUTFILE_PATH)
+def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations:
+ config_path = os.path.join(QEMU_CONFIGS_DIR, arch + '.py')
+ if arch == 'um':
+ return LinuxSourceTreeOperationsUml(cross_compile=cross_compile)
+ elif os.path.isfile(config_path):
+ return get_source_tree_ops_from_qemu_config(config_path, cross_compile)[1]
+ else:
+ raise ConfigError(arch + ' is not a valid arch')
+
+def get_source_tree_ops_from_qemu_config(config_path: str,
+ cross_compile: Optional[str]) -> tuple[
+ str, LinuxSourceTreeOperations]:
+ # The module name/path has very little to do with where the actual file
return get_file_path(build_dir, OUTFILE_PATH)
+def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations:
+ config_path = os.path.join(QEMU_CONFIGS_DIR, arch + '.py')
+ if arch == 'um':
+ return LinuxSourceTreeOperationsUml(cross_compile=cross_compile)
+ elif os.path.isfile(config_path):
+ return get_source_tree_ops_from_qemu_config(config_path, cross_compile)[1]
+ else:
+ raise ConfigError(arch + ' is not a valid arch')
+
+def get_source_tree_ops_from_qemu_config(config_path: str,
+ cross_compile: Optional[str]) -> tuple[
+ str, LinuxSourceTreeOperations]:
+ # exists (I learned this through experimentation and could not find it
+ # anywhere in the Python documentation).
+ #
+ # Bascially, we completely ignore the actual file location of the config
+ # we are loading and just tell Python that the module lives in the
+ # QEMU_CONFIGS_DIR for import purposes regardless of where it actually
+ # exists as a file.
+ module_path = '.' + os.path.join(os.path.basename(QEMU_CONFIGS_DIR), os.path.basename(config_path))
+ spec = importlib.util.spec_from_file_location(module_path, config_path)
kconfig_path = get_kconfig_path(build_dir)
if build_dir and not os.path.exists(build_dir):
os.mkdir(build_dir)
- self._kconfig.write_to_file(kconfig_path)
try:
+ self._ops.make_arch_qemuconfig(self._kconfig)
+ self._kconfig.write_to_file(kconfig_path)
self._ops.make_olddefconfig(build_dir, make_options)
except ConfigError as e:
logging.error(e)
@@ -184,6 +304,7 @@ class LinuxSourceTree(object):
if build_dir and not os.path.exists(build_dir):
os.mkdir(build_dir)
- self._kconfig.write_to_file(kconfig_path)
try:
+ self._ops.make_arch_qemuconfig(self._kconfig)
+ self._kconfig.write_to_file(kconfig_path)
self._ops.make_olddefconfig(build_dir, make_options)
except ConfigError as e:
logging.error(e)
if os.path.exists(kconfig_path):
existing_kconfig = kunit_config.Kconfig()
existing_kconfig.read_from_file(kconfig_path)
+ self._ops.make_arch_qemuconfig(self._kconfig)
if not self._kconfig.is_subset_of(existing_kconfig):
print('Regenerating .config ...')
os.remove(kconfig_path)
@@ -194,7 +315,7 @@ class LinuxSourceTree(object):
existing_kconfig = kunit_config.Kconfig()
existing_kconfig.read_from_file(kconfig_path)
+ self._ops.make_arch_qemuconfig(self._kconfig)
if not self._kconfig.is_subset_of(existing_kconfig):
print('Regenerating .config ...')
os.remove(kconfig_path)
print('Generating .config ...')
return self.build_config(build_dir, make_options)
- def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
+ def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
self._ops.make_allyesconfig(build_dir, make_options)
@@ -211,8 +332,8 @@ class LinuxSourceTree(object):
return self.build_config(build_dir, make_options)
- def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
+ def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
self._ops.make_allyesconfig(build_dir, make_options)
args.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt'])
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
if filter_glob:
args.append('kunit.filter_glob='+filter_glob)
diff --git a/tools/testing/kunit/qemu_config.py b/tools/testing/kunit/qemu_config.py
new file mode 100644
index 0000000000000..1672f6184e952
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_config.py
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Collection of configs for building non-UML kernels and running them on QEMU.
+#
+# Copyright (C) 2021, Google LLC.
+# Author: Brendan Higgins <brendan...@google.com>
+
+from collections import namedtuple
+
+
+QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
+ 'kconfig',
+#
+# Collection of configs for building non-UML kernels and running them on QEMU.
+#
+# Copyright (C) 2021, Google LLC.
+# Author: Brendan Higgins <brendan...@google.com>
+
+from collections import namedtuple
+
+
+QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
+ 'qemu_arch',
+ 'kernel_path',
+ 'kernel_command_line',
+ 'extra_qemu_params'])
+ 'kernel_path',
+ 'kernel_command_line',
+ 'extra_qemu_params'])
diff --git a/tools/testing/kunit/qemu_configs/alpha.py b/tools/testing/kunit/qemu_configs/alpha.py
new file mode 100644
index 0000000000000..5d0c0cff03bd2
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/alpha.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='alpha',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/alpha.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='alpha',
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='alpha',
+ kernel_path='arch/alpha/boot/vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
diff --git a/tools/testing/kunit/qemu_configs/arm.py b/tools/testing/kunit/qemu_configs/arm.py
new file mode 100644
index 0000000000000..b9c2a35e02962
+ kernel_path='arch/alpha/boot/vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
diff --git a/tools/testing/kunit/qemu_configs/arm.py b/tools/testing/kunit/qemu_configs/arm.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/arm.py
@@ -0,0 +1,13 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='arm',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/arm.py
@@ -0,0 +1,13 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='arm',
+CONFIG_ARCH_VIRT=y
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='arm',
+ kernel_path='arch/arm/boot/zImage',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt'])
diff --git a/tools/testing/kunit/qemu_configs/arm64.py b/tools/testing/kunit/qemu_configs/arm64.py
new file mode 100644
index 0000000000000..517c04459f47a
+ kernel_path='arch/arm/boot/zImage',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt'])
diff --git a/tools/testing/kunit/qemu_configs/arm64.py b/tools/testing/kunit/qemu_configs/arm64.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/arm64.py
@@ -0,0 +1,12 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='arm64',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/arm64.py
@@ -0,0 +1,12 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='arm64',
+CONFIG_SERIAL_AMBA_PL010=y
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='aarch64',
+ kernel_path='arch/arm64/boot/Image.gz',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt', '-cpu cortex-a57'])
diff --git a/tools/testing/kunit/qemu_configs/i386.py b/tools/testing/kunit/qemu_configs/i386.py
new file mode 100644
index 0000000000000..aed3ffd3937dd
+CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
+ qemu_arch='aarch64',
+ kernel_path='arch/arm64/boot/Image.gz',
+ kernel_command_line='console=ttyAMA0',
+ extra_qemu_params=['-machine virt', '-cpu cortex-a57'])
diff --git a/tools/testing/kunit/qemu_configs/i386.py b/tools/testing/kunit/qemu_configs/i386.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/i386.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='i386',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/i386.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='i386',
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
diff --git a/tools/testing/kunit/qemu_configs/powerpc.py b/tools/testing/kunit/qemu_configs/powerpc.py
new file mode 100644
index 0000000000000..35e9de24f0db7
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
diff --git a/tools/testing/kunit/qemu_configs/powerpc.py b/tools/testing/kunit/qemu_configs/powerpc.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/powerpc.py
@@ -0,0 +1,12 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='powerpc',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/powerpc.py
@@ -0,0 +1,12 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='powerpc',
+CONFIG_PPC64=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_HVC_CONSOLE=y''',
+ qemu_arch='ppc64',
+ kernel_path='vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-M pseries', '-cpu power8'])
diff --git a/tools/testing/kunit/qemu_configs/riscv.py b/tools/testing/kunit/qemu_configs/riscv.py
new file mode 100644
index 0000000000000..9e528087cd7c5
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_HVC_CONSOLE=y''',
+ qemu_arch='ppc64',
+ kernel_path='vmlinux',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=['-M pseries', '-cpu power8'])
diff --git a/tools/testing/kunit/qemu_configs/riscv.py b/tools/testing/kunit/qemu_configs/riscv.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/riscv.py
@@ -0,0 +1,31 @@
+from ..qemu_config import QemuArchParams
+import os
+import os.path
+import sys
+
+GITHUB_OPENSBI_URL = 'https://github.com/qemu/qemu/raw/master/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin'
+OPENSBI_FILE = os.path.basename(GITHUB_OPENSBI_URL)
+
+if not os.path.isfile(OPENSBI_FILE):
+ print('\n\nOpenSBI file is not in the current working directory.\n'
+ 'Would you like me to download it for you from:\n' + GITHUB_OPENSBI_URL + ' ?\n')
+ response = input('yes/[no]: ')
+ if response.strip() == 'yes':
+ os.system('wget ' + GITHUB_OPENSBI_URL)
+ else:
+ sys.exit()
+
+QEMU_ARCH = QemuArchParams(linux_arch='riscv',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/riscv.py
@@ -0,0 +1,31 @@
+from ..qemu_config import QemuArchParams
+import os
+import os.path
+import sys
+
+GITHUB_OPENSBI_URL = 'https://github.com/qemu/qemu/raw/master/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin'
+OPENSBI_FILE = os.path.basename(GITHUB_OPENSBI_URL)
+
+if not os.path.isfile(OPENSBI_FILE):
+ print('\n\nOpenSBI file is not in the current working directory.\n'
+ 'Would you like me to download it for you from:\n' + GITHUB_OPENSBI_URL + ' ?\n')
+ response = input('yes/[no]: ')
+ if response.strip() == 'yes':
+ os.system('wget ' + GITHUB_OPENSBI_URL)
+ else:
+ sys.exit()
+
+QEMU_ARCH = QemuArchParams(linux_arch='riscv',
+CONFIG_SOC_VIRT=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_OF_PLATFORM=y
+CONFIG_SERIAL_EARLYCON_RISCV_SBI=y''',
+ qemu_arch='riscv64',
+ kernel_path='arch/riscv/boot/Image',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine virt',
+ '-cpu rv64',
+ '-bios opensbi-riscv64-generic-fw_dynamic.bin'])
diff --git a/tools/testing/kunit/qemu_configs/s390.py b/tools/testing/kunit/qemu_configs/s390.py
new file mode 100644
index 0000000000000..e310bd5211135
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_OF_PLATFORM=y
+CONFIG_SERIAL_EARLYCON_RISCV_SBI=y''',
+ qemu_arch='riscv64',
+ kernel_path='arch/riscv/boot/Image',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine virt',
+ '-cpu rv64',
+ '-bios opensbi-riscv64-generic-fw_dynamic.bin'])
diff --git a/tools/testing/kunit/qemu_configs/s390.py b/tools/testing/kunit/qemu_configs/s390.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/s390.py
@@ -0,0 +1,14 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='s390',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/s390.py
@@ -0,0 +1,14 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='s390',
+CONFIG_EXPERT=y
+CONFIG_TUNE_ZEC12=y
+CONFIG_NUMA=y
+CONFIG_MODULES=y''',
+ qemu_arch='s390x',
+ kernel_path='arch/s390/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine s390-ccw-virtio',
+ '-cpu qemu',])
diff --git a/tools/testing/kunit/qemu_configs/sparc.py b/tools/testing/kunit/qemu_configs/sparc.py
new file mode 100644
index 0000000000000..27f474e7ad6e7
+CONFIG_TUNE_ZEC12=y
+CONFIG_NUMA=y
+CONFIG_MODULES=y''',
+ qemu_arch='s390x',
+ kernel_path='arch/s390/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[
+ '-machine s390-ccw-virtio',
+ '-cpu qemu',])
diff --git a/tools/testing/kunit/qemu_configs/sparc.py b/tools/testing/kunit/qemu_configs/sparc.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/sparc.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='sparc',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/sparc.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='sparc',
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='sparc',
+ kernel_path='arch/sparc/boot/zImage',
+ kernel_command_line='console=ttyS0 mem=256M',
+ extra_qemu_params=['-m 256'])
diff --git a/tools/testing/kunit/qemu_configs/x86_64.py b/tools/testing/kunit/qemu_configs/x86_64.py
new file mode 100644
index 0000000000000..77ab1aeee8a37
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='sparc',
+ kernel_path='arch/sparc/boot/zImage',
+ kernel_command_line='console=ttyS0 mem=256M',
+ extra_qemu_params=['-m 256'])
diff --git a/tools/testing/kunit/qemu_configs/x86_64.py b/tools/testing/kunit/qemu_configs/x86_64.py
new file mode 100644
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/x86_64.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='x86_64',
+ kconfig='''
+++ b/tools/testing/kunit/qemu_configs/x86_64.py
@@ -0,0 +1,10 @@
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='x86_64',
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
--
2.31.1.818.g46aad6cb9e-goog
+CONFIG_SERIAL_8250_CONSOLE=y''',
+ qemu_arch='x86_64',
+ kernel_path='arch/x86/boot/bzImage',
+ kernel_command_line='console=ttyS0',
+ extra_qemu_params=[''])
--
Brendan Higgins
May 26, 2021, 5:24:23 PM5/26/21
to sh...@kernel.org, davi...@google.com, linux-k...@vger.kernel.org, kuni...@googlegroups.com, linux-...@vger.kernel.org, cor...@lwn.net, linu...@vger.kernel.org, sb...@kernel.org, kees...@chromium.org, frowan...@gmail.com, dlat...@google.com, Brendan Higgins
Document QEMU support, what it does, and how to use it in kunit_tool.
Signed-off-by: Brendan Higgins <brendan...@google.com>
Signed-off-by: Brendan Higgins <brendan...@google.com>
---
Changes since last revision:
- Finally, I added a new section to the kunit_tool documentation to
Changes since last revision:
document the new command line flags I added.
---
Documentation/dev-tools/kunit/kunit-tool.rst | 48 +++++++++++++++++++
Documentation/dev-tools/kunit/usage.rst | 50 +++++++++++++++-----
2 files changed, 87 insertions(+), 11 deletions(-)
diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst
index 4247b7420e3ba..c7ff9afe407a5 100644
--- a/Documentation/dev-tools/kunit/kunit-tool.rst
+++ b/Documentation/dev-tools/kunit/kunit-tool.rst
@@ -145,6 +145,54 @@ to run KUnit resource tests, you could use:
This uses the standard glob format for wildcards.
+Running Tests on QEMU
+=====================
+
+kunit_tool supports running tests on QEMU as well as via UML (as mentioned
+elsewhere). The default way of running tests on QEMU requires two flags:
+
+``--arch``
+ Selects a collection of configs (Kconfig as well as QEMU configs
+ options, etc) that allow KUnit tests to be run on the specified
+ architecture in a minimal way; this is usually not much slower than
+ using UML. The architecture argument is the same as the name of the
+ option passed to the ``ARCH`` variable used by Kbuild. Not all
+ architectures are currently supported by this flag, but can be handled
+ by the ``--qemu_config`` discussed later. If ``um`` is passed (or this
+ this flag is ignored) the tests will run via UML. Non-UML architectures,
+ e.g. i386, x86_64, arm, um, etc. Non-UML run on QEMU.
+
+``--cross_compile``
+ Specifies the use of a toolchain by Kbuild. The argument passed here is
+ the same passed to the ``CROSS_COMPILE`` variable used by Kbuild. As a
+ reminder this will be the prefix for the toolchain binaries such as gcc
+ for example ``sparc64-linux-gnu-`` if you have the sparc toolchain
+ installed on your system, or
+ ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-``
+ if you have downloaded the microblaze toolchain from the 0-day website
+ to a directory in your home directory called ``toolchains``.
+
+In many cases it is likely that you may want to run an architecture which is
+not supported by the ``--arch`` flag, or you may want to just run KUnit tests
+on QEMU using a non-default configuration. For this use case, you can write
+your own QemuConfig. These QemuConfigs are written in Python. They must have an
+import line ``from ..qemu_config import QemuArchParams`` at the top of the file
+and the file must contain a variable called ``QEMU_ARCH`` that has an instance
+of ``QemuArchParams`` assigned to it. An example can be seen in
+``tools/testing/kunit/qemu_configs/x86_64.py``.
+
+Once you have a QemuConfig you can pass it into kunit_tool using the
+``--qemu_config`` flag; when used this flag replaces the ``--arch`` flag. If we
+were to do this with the ``x86_64.py`` example from above, the invocation would
+look something like this:
+
+.. code-block:: bash
+
+ ./tools/testing/kunit/kunit.py run \
+.. code-block:: bash
+
+ --timeout=60 \
+ --jobs=12 \
+ --qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py
+
Other Useful Options
====================
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 650f99590df57..888c341701da4 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -609,17 +609,45 @@ non-UML architectures:
None of these are reasons not to run your KUnit tests on real hardware; they are
only things to be aware of when doing so.
-The biggest impediment will likely be that certain KUnit features and
-infrastructure may not support your target environment. For example, at this
-time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside
-of UML. Unfortunately, there is no way around this. Using UML (or even just a
-particular architecture) allows us to make a lot of assumptions that make it
-possible to do things which might otherwise be impossible.
-
-particular architecture) allows us to make a lot of assumptions that make it
-possible to do things which might otherwise be impossible.
-Nevertheless, all core KUnit framework features are fully supported on all
-architectures, and using them is straightforward: all you need to do is to take
-your kunitconfig, your Kconfig options for the tests you would like to run, and
-merge them into whatever config your are using for your platform. That's it!
+Currently, the KUnit Wrapper (``tools/testing/kunit/kunit.py``) (aka
-your kunitconfig, your Kconfig options for the tests you would like to run, and
-merge them into whatever config your are using for your platform. That's it!
+kunit_tool) only fully supports running tests inside of UML and QEMU; however,
+this is only due to our own time limitations as humans working on KUnit. It is
+entirely possible to support other emulators and even actual hardware, but for
+now QEMU and UML is what is fully supported within the KUnit Wrapper. Again, to
+be clear, this is just the Wrapper. The actualy KUnit tests and the KUnit
+library they are written in is fully architecture agnostic and can be used in
+virtually any setup, you just won't have the benefit of typing a single command
+out of the box and having everything magically work perfectly.
+
+Again, all core KUnit framework features are fully supported on all
+architectures, and using them is straightforward: Most popular architectures
+are supported directly in the KUnit Wrapper via QEMU. Currently, supported
+architectures on QEMU include:
+
+* i386
+* x86_64
+* arm
+* arm64
+* alpha
+* powerpc
+* riscv
+* s390
+* sparc
+
+In order to run KUnit tests on one of these architectures via QEMU with the
+KUnit wrapper, all you need to do is specify the flags ``--arch`` and
+``--cross_compile`` when invoking the KUnit Wrapper. For example, we could run
+the default KUnit tests on ARM in the following manner (assuming we have an ARM
+toolchain installed):
+
+.. code-block:: bash
+
+ tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --arch=arm --cross_compile=arm-linux-gnueabihf-
+
+are supported directly in the KUnit Wrapper via QEMU. Currently, supported
+architectures on QEMU include:
+
+* i386
+* x86_64
+* arm
+* arm64
+* alpha
+* powerpc
+* riscv
+* s390
+* sparc
+
+In order to run KUnit tests on one of these architectures via QEMU with the
+KUnit wrapper, all you need to do is specify the flags ``--arch`` and
+``--cross_compile`` when invoking the KUnit Wrapper. For example, we could run
+the default KUnit tests on ARM in the following manner (assuming we have an ARM
+toolchain installed):
+
+.. code-block:: bash
+
+ tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --arch=arm --cross_compile=arm-linux-gnueabihf-
+
+Alternatively, if you want to run your tests on real hardware or in some other
+emulation environment, all you need to do is to take your kunitconfig, your
+Kconfig options for the tests you would like to run, and merge them into
+whatever config your are using for your platform. That's it!
For example, let's say you have the following kunitconfig:
--
2.31.1.818.g46aad6cb9e-goog
+emulation environment, all you need to do is to take your kunitconfig, your
+Kconfig options for the tests you would like to run, and merge them into
+whatever config your are using for your platform. That's it!
For example, let's say you have the following kunitconfig:
--
Daniel Latypov
May 26, 2021, 8:53:47 PM5/26/21
to Brendan Higgins, shuah, David Gow, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand
On Wed, May 26, 2021 at 2:24 PM Brendan Higgins
<brendan...@google.com> wrote:
>
> From: David Gow <davi...@google.com>
>
> Add a new kernel command-line option, 'kunit_shutdown', which allows the
> user to specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running KUnit tests
> on UML or a VM so that the UML/VM process exits cleanly immediately
> after running all tests without needing a special initramfs.
>
> Signed-off-by: David Gow <davi...@google.com>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
> Tested-By: Daniel Latypov <dlat...@google.com>
Reviewed-by: Daniel Latypov <dlat...@google.com>
<brendan...@google.com> wrote:
>
> From: David Gow <davi...@google.com>
>
> Add a new kernel command-line option, 'kunit_shutdown', which allows the
> user to specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running KUnit tests
> on UML or a VM so that the UML/VM process exits cleanly immediately
> after running all tests without needing a special initramfs.
>
> Signed-off-by: David Gow <davi...@google.com>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Stephen Boyd <sb...@kernel.org>
> Tested-By: Daniel Latypov <dlat...@google.com>
Looks good, one optional nit below.
Also, I tested this again on top of 5.13-r4 (7ac3a1c1ae51 ("Merge tag
'mtd/fixes-for-5.13-rc4' of
git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux"))
$ echo -e "\nCONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y"
>> .kunit/.kunitconfig
$ run_kunit --make_options=CC=/usr/bin/gcc-6
$ <run coverage generation steps>
Overall coverage rate:
lines......: 13.8% (17286 of 125320 lines)
functions..: 15.8% (1790 of 11336 functions)
So coverage still seems to be fixed, no need to comment out the
uml_abort() anymore and this sidesteps whatever weird issue I started
running into these past few weeks.
So that's very exciting.
David Gow
May 28, 2021, 2:17:14 AM5/28/21
to Brendan Higgins, Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK, KUnit Development, Linux Kernel Mailing List, Jonathan Corbet, open list:DOCUMENTATION, Stephen Boyd, Kees Cook, Frank Rowand, Daniel Latypov
On Thu, May 27, 2021 at 5:24 AM Brendan Higgins
<brendan...@google.com> wrote:
>
> Document QEMU support, what it does, and how to use it in kunit_tool.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> ---
> Changes since last revision:
>
> - Finally, I added a new section to the kunit_tool documentation to
> document the new command line flags I added.
>
Thanks! This looks good to me, modulo a couple of nitpicky
<brendan...@google.com> wrote:
>
> Document QEMU support, what it does, and how to use it in kunit_tool.
>
> Signed-off-by: Brendan Higgins <brendan...@google.com>
> ---
> Changes since last revision:
>
> - Finally, I added a new section to the kunit_tool documentation to
> document the new command line flags I added.
>
spelling/grammar notes below.
Reviewed-by: David Gow <davi...@google.com>
Cheers
-- David
> ---
> Documentation/dev-tools/kunit/kunit-tool.rst | 48 +++++++++++++++++++
> Documentation/dev-tools/kunit/usage.rst | 50 +++++++++++++++-----
> 2 files changed, 87 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst
> index 4247b7420e3ba..c7ff9afe407a5 100644
> --- a/Documentation/dev-tools/kunit/kunit-tool.rst
> +++ b/Documentation/dev-tools/kunit/kunit-tool.rst
> @@ -145,6 +145,54 @@ to run KUnit resource tests, you could use:
>
> This uses the standard glob format for wildcards.
>
> +Running Tests on QEMU
> +=====================
> +
> +kunit_tool supports running tests on QEMU as well as via UML (as mentioned
> +elsewhere). The default way of running tests on QEMU requires two flags:
> +
> +``--arch``
> + Selects a collection of configs (Kconfig as well as QEMU configs
> + options, etc) that allow KUnit tests to be run on the specified
> + architecture in a minimal way; this is usually not much slower than
> + using UML. The architecture argument is the same as the name of the
> + option passed to the ``ARCH`` variable used by Kbuild. Not all
> + architectures are currently supported by this flag, but can be handled
> + by the ``--qemu_config`` discussed later. If ``um`` is passed (or this
for the "KUnit Wrapper" here, and "KUnit Wrapper" didn't seem to be
used much. Then I looked at the rest of the documentation, and "KUnit
Wrapper" is used all over the place.
I think it'd be nice to consolidate on one (or, if we have to, two)
names for this, but this doc is probably not the place to do so.
> +this is only due to our own time limitations as humans working on KUnit. It is
> +entirely possible to support other emulators and even actual hardware, but for
> +now QEMU and UML is what is fully supported within the KUnit Wrapper. Again, to
> +be clear, this is just the Wrapper. The actualy KUnit tests and the KUnit
"for now only QEMU and UML are supported by the KUnit Wrapper. The
KUnit library and most KUnit tests are fully architecture agnostic..."
Also, "actualy" -> "actually".
Reply all
Reply to author
Forward
0 new messages