[PATCH] Documentation: kunit: Add CLI args for kunit_tool

[Sadiya Kazi]

Run_wrapper.rst was missing some command line arguments. Added
additional args in the file. Included all initial review comments.

Signed-off-by: Sadiya Kazi <sadiy...@google.com>
---
Documentation/dev-tools/kunit/run_wrapper.rst | 48 +++++++++++++++++++
1 file changed, 48 insertions(+)

diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index 5e560f2c5fca..91f5dda36e83 100644
--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst
@@ -257,3 +257,51 @@ command line arguments:
added or modified. Instead, enable all tests
which have satisfied dependencies by adding
``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
+- ``--kunitconfig``: Specifies the path to the ``.kunitconfig`` file.
+ This Kconfig fragment enables KUnit tests. The "/.kunitconfig" gets
+ appended to the path specified. For example, If a directory path "lib/kunit"
+ is given, the complete path will be "lib/kunit/.kunitconfig".
+
+- ``--kconfig_add``: Specifies additional configuration options to be
+ appended to the ``.kunitconfig`` file. For example, ``CONFIG_KASAN=y``.
+
+- ``--arch``: Runs tests with the specified architecture. The architecture
+ specified must match the string passed to the ARCH make parameter.
+ For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.
+ Default to 'um'.
+
+- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
+ same argument as passed to the ``CROSS_COMPILE`` variable used by
+ Kbuild. This will be the prefix for the toolchain
+ binaries such as GCC. For example:
+
+ - ``sparc64-linux-gnu`` if we have the sparc toolchain installed on
+ our system.
+
+ - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
+ if we have downloaded the microblaze toolchain from the 0-day
+ website to a specified path in our home directory called toolchains.
+
+- ``--qemu_config``: Specifies the path to the file containing a
+ custom qemu architecture definition. This should be a python file
+ containing a QemuArchParams object.
+
+- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
+
+- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.
+ By default, this is set to the number of cores on your system.
+
+- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
+ This does not include the time taken to build the tests.
+
+- ``--kernel_args``: Specifies the kernel command-line arguments. Might be repeated.
+
+- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
+ This is useful for debugging a non-hermetic test, one that
+ might pass/fail based on what ran before it.
+
+- ``--raw_output``: If set, generates unformatted output from kernel.
+ If set to ``--raw_output=kunit``, filters to just KUnit output.
+
+- ``--json``: If set, it stores the test results in a JSON format and prints to stdout or
+ saves to a file if a filename is specified.
--
2.37.0.170.g444d1eabd0-goog

[Maíra Canal]

On 7/19/22 06:22, 'Sadiya Kazi' via KUnit Development wrote:
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file. Included all initial review comments.
>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>
> ---
> Documentation/dev-tools/kunit/run_wrapper.rst | 48 +++++++++++++++++++
> 1 file changed, 48 insertions(+)
>
> diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
> index 5e560f2c5fca..91f5dda36e83 100644
> --- a/Documentation/dev-tools/kunit/run_wrapper.rst
> +++ b/Documentation/dev-tools/kunit/run_wrapper.rst
> @@ -257,3 +257,51 @@ command line arguments:
> added or modified. Instead, enable all tests
> which have satisfied dependencies by adding
> ``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
> +- ``--kunitconfig``: Specifies the path to the ``.kunitconfig`` file.
> + This Kconfig fragment enables KUnit tests. The "/.kunitconfig" gets
> + appended to the path specified. For example, If a directory path "lib/kunit"

You might want to change it for a lower-case "if", like: "For example,
if a directory path "lib/kunit"".

> + is given, the complete path will be "lib/kunit/.kunitconfig".

This explanation may imply that if I pass `--kunitconfig
lib/kunit/.kunitconfig` it wouldn't work, as "/.kunitconfig" is always
appended to the path. But this is not true, both `--kunitconfig
lib/kunit/.kunitconfig` and `--kunitconfig lib/kunit` are valid options.

> +
> +- ``--kconfig_add``: Specifies additional configuration options to be
> + appended to the ``.kunitconfig`` file. For example, ``CONFIG_KASAN=y``.

You could expand this example to:
```
./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y
```

> +
> +- ``--arch``: Runs tests with the specified architecture. The architecture

"Runs tests on the specified architecture."

> + specified must match the string passed to the ARCH make parameter.

ARCH is not a "make parameter", ARCH is a environment variable.
Therefor, ARCH is not "passed to make", as it a environment variable on
the shell. Check the ARCH documentation [1].

> + For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.
> + Default to 'um'.
> +
> +- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
> + same argument as passed to the ``CROSS_COMPILE`` variable used by> + Kbuild. This will be the prefix for the toolchain
> + binaries such as GCC. For example:
> +
> + - ``sparc64-linux-gnu`` if we have the sparc toolchain installed on
> + our system.
> +
> + - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
> + if we have downloaded the microblaze toolchain from the 0-day
> + website to a specified path in our home directory called toolchains.

This is a copy of the explanation on [2]. You may want to delete the
previous explanation or change this one.

> +
> +- ``--qemu_config``: Specifies the path to the file containing a

"Specifies the path to a file containing a"

> + custom qemu architecture definition. This should be a python file
> + containing a QemuArchParams object.

"containing a `QemuArchParams` object."

The qemu_config has also a pretty good documentation on [2].

> +
> +- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
> +
> +- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.
> + By default, this is set to the number of cores on your system.

This CLI argument is already explained on [3]. You may want to delete
the previous explanation or change this one.

> +
> +- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
> + This does not include the time taken to build the tests.

Same here.

> +
> +- ``--kernel_args``: Specifies the kernel command-line arguments. Might be repeated.
> +
> +- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
> + This is useful for debugging a non-hermetic test, one that
> + might pass/fail based on what ran before it.
> +
> +- ``--raw_output``: If set, generates unformatted output from kernel.
> + If set to ``--raw_output=kunit``, filters to just KUnit output.

This command line argument is already explain on the Parse Test Results
section [4].

> +
> +- ``--json``: If set, it stores the test results in a JSON format and prints to stdout or

"If set, it stores the test results in a JSON format and prints it to

`stdout` or"

> + saves to a file if a filename is specified.

It's great to see new documentation for KUnit!

[1] https://docs.kernel.org/kbuild/kbuild.html#arch
[2]
https://docs.kernel.org/dev-tools/kunit/run_wrapper.html#run-tests-on-qemu
[3]
https://docs.kernel.org/dev-tools/kunit/run_wrapper.html#run-tests-with-kunit-tool
[4]
https://docs.kernel.org/dev-tools/kunit/run_wrapper.html#parse-test-results

Best Regards,
- Maíra Canal

[David Gow]

On Tue, Jul 19, 2022 at 5:22 PM Sadiya Kazi <sadiy...@google.com> wrote:
>
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file. Included all initial review comments.
>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>
> ---

This looks pretty good, save for some minor nitpicks below. I also
echo Maíra's comments, though I'm not _too_ worried about having a
little bit of duplication if it means we get a complete (or
nearly-complete) list of arguments here, but still have them described
in the appropriate part of the rest of the documentation.

Cheers,
-- David

> Documentation/dev-tools/kunit/run_wrapper.rst | 48 +++++++++++++++++++
> 1 file changed, 48 insertions(+)
>
> diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
> index 5e560f2c5fca..91f5dda36e83 100644
> --- a/Documentation/dev-tools/kunit/run_wrapper.rst
> +++ b/Documentation/dev-tools/kunit/run_wrapper.rst
> @@ -257,3 +257,51 @@ command line arguments:
> added or modified. Instead, enable all tests
> which have satisfied dependencies by adding
> ``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
> +- ``--kunitconfig``: Specifies the path to the ``.kunitconfig`` file.
> + This Kconfig fragment enables KUnit tests. The "/.kunitconfig" gets
> + appended to the path specified. For example, If a directory path "lib/kunit"
> + is given, the complete path will be "lib/kunit/.kunitconfig".

This is definitely a bit confusing: it seems to describe "how it
works" more than "what is it, and when do you use it". Perhaps
something more like:

"Specifies a .kunitconfig file to use to build and run with a
predefined set of tests and their dependencies. (e.g., to run tests
for a given subsystem). This can be the path to a file, or to a
directory (in which case a file named ".kunitconfig" in said directory
will be used.)"

> +
> +- ``--kconfig_add``: Specifies additional configuration options to be
> + appended to the ``.kunitconfig`` file. For example, ``CONFIG_KASAN=y``.
> +
> +- ``--arch``: Runs tests with the specified architecture. The architecture
> + specified must match the string passed to the ARCH make parameter.
> + For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.
> + Default to 'um'.
> +
> +- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
> + same argument as passed to the ``CROSS_COMPILE`` variable used by
> + Kbuild. This will be the prefix for the toolchain
> + binaries such as GCC. For example:
> +
> + - ``sparc64-linux-gnu`` if we have the sparc toolchain installed on
> + our system.

This example should be ``sparc64-linux-gnu-`` (with the extra
hyphen-minus at the end). It's concatenated to produce, e.g.,
sparc64-linux-gnu-gcc.

> +
> + - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
> + if we have downloaded the microblaze toolchain from the 0-day
> + website to a specified path in our home directory called toolchains.
> +
> +- ``--qemu_config``: Specifies the path to the file containing a
> + custom qemu architecture definition. This should be a python file
> + containing a QemuArchParams object.
> +
> +- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
> +
> +- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.
> + By default, this is set to the number of cores on your system.
> +
> +- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
> + This does not include the time taken to build the tests.
> +
> +- ``--kernel_args``: Specifies the kernel command-line arguments. Might be repeated.

Nit: maybe "Specifies _additional_ kernel command-line arguments".
kunit_tool already sets some.

> +
> +- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
> + This is useful for debugging a non-hermetic test, one that
> + might pass/fail based on what ran before it.



> +
> +- ``--raw_output``: If set, generates unformatted output from kernel.
> + If set to ``--raw_output=kunit``, filters to just KUnit output.

--raw_output=kunit is the default if just --raw_output is set. You
need to specify --raw_output=all for the full kernel output.

> +
> +- ``--json``: If set, it stores the test results in a JSON format and prints to stdout or
> + saves to a file if a filename is specified.

Nit: let's remove the first 'it' to be consistent with the previous
entries. i.e., "If set, stores..."

> --
> 2.37.0.170.g444d1eabd0-goog
>

[Sadiya Kazi]

Run_wrapper.rst was missing some command line arguments. Added
additional args in the file.

Signed-off-by: Sadiya Kazi <sadiy...@google.com>
---

Changes since V1:
https://lore.kernel.org/linux-kselftest/20220719092214.99...@google.com/
- Addressed most of the review comments from Maira and David, except
removing the duplicate arguments as I felt its worth keeping them in
the reference documentation as well as in context. We can improve them
and differentiate their use cases in the future patches.


---
Documentation/dev-tools/kunit/run_wrapper.rst | 60 ++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)

diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index 5e560f2c5fca..600af7ac5f88 100644
--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst
@@ -233,7 +233,7 @@ Command-Line Arguments
======================

kunit_tool has a number of other command-line arguments which can
-be useful for our test environment. Below the most commonly used
+be useful for our test environment. Below are the most commonly used
command line arguments:

- ``--help``: Lists all available options. To list common options,
@@ -257,3 +257,61 @@ command line arguments:

added or modified. Instead, enable all tests
which have satisfied dependencies by adding
``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
+

+- ``--kunitconfig``: Specifies the path or the directory of the ``.kunitconfig``
+ file. For example:
+
+ - ``lib/kunit/.kunitconfig`` can be the path of the file.
+
+ - ``lib/kunit`` can be the directory in which the file is located.
+
+ This file is used to build and run with a predefined set of tests
+ and their dependencies. For example, to run tests for a given subsystem.

+
+- ``--kconfig_add``: Specifies additional configuration options to be
+ appended to the ``.kunitconfig`` file.

+ For example, ``./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y``.
+
+- ``--arch``: Runs tests on the specified architecture. The architecture
+ specified must match the Kbuild ARCH environment variable.

+ For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.

+ Default is `um`.

+
+- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
+ same argument as passed to the ``CROSS_COMPILE`` variable used by
+ Kbuild. This will be the prefix for the toolchain
+ binaries such as GCC. For example:
+

+ - ``sparc64-linux-gnu-`` if we have the sparc toolchain installed on
+ our system.

+
+ - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
+ if we have downloaded the microblaze toolchain from the 0-day
+ website to a specified path in our home directory called toolchains.
+

+- ``--qemu_config``: Specifies the path to a file containing a

+ custom qemu architecture definition. This should be a python file
+ containing a `QemuArchParams` object.
+
+- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
+
+- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.
+ By default, this is set to the number of cores on your system.
+
+- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
+ This does not include the time taken to build the tests.
+

+- ``--kernel_args``: Specifies additional kernel command-line arguments. Might be repeated.

+
+- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
+ This is useful for debugging a non-hermetic test, one that
+ might pass/fail based on what ran before it.
+

+- ``--raw_output``: If set, generates unformatted output from kernel. Possible options are:
+
+ - ``all``: To view the full kernel output, use ``--raw_output=all``.
+
+ - ``kunit``: This is the default option and filters to KUnit output. Use ``--raw_output`` or ``--raw_output=kunit``.
+
+- ``--json``: If set, stores the test results in a JSON format and prints to `stdout` or

+ saves to a file if a filename is specified.

--
2.37.0.170.g444d1eabd0-goog

[Maíra Canal]

Small nit pick: I would rather do:

```
./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y
```

> +
> +- ``--arch``: Runs tests on the specified architecture. The architecture
> + specified must match the Kbuild ARCH environment variable.
> + For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.
> + Default is `um`.
> +
> +- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
> + same argument as passed to the ``CROSS_COMPILE`` variable used by
> + Kbuild. This will be the prefix for the toolchain
> + binaries such as GCC. For example:
> +
> + - ``sparc64-linux-gnu-`` if we have the sparc toolchain installed on
> + our system.
> +
> + - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
> + if we have downloaded the microblaze toolchain from the 0-day
> + website to a specified path in our home directory called toolchains.
> +
> +- ``--qemu_config``: Specifies the path to a file containing a
> + custom qemu architecture definition. This should be a python file
> + containing a `QemuArchParams` object.

Nit: choose a standard for referring to qemu. Either "qemu" or "QEMU" is
great for me, but it is ideal that you chose one and stick with it.
Here, you used "qemu" and on the next argument, you used "QEMU".

> +
> +- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
> +
> +- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.
> + By default, this is set to the number of cores on your system.
> +
> +- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
> + This does not include the time taken to build the tests.
> +
> +- ``--kernel_args``: Specifies additional kernel command-line arguments. Might be repeated.
> +
> +- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
> + This is useful for debugging a non-hermetic test, one that
> + might pass/fail based on what ran before it.
> +
> +- ``--raw_output``: If set, generates unformatted output from kernel. Possible options are:
> +
> + - ``all``: To view the full kernel output, use ``--raw_output=all``.
> +
> + - ``kunit``: This is the default option and filters to KUnit output. Use ``--raw_output`` or ``--raw_output=kunit``.
> +
> +- ``--json``: If set, stores the test results in a JSON format and prints to `stdout` or
> + saves to a file if a filename is specified.

Anyway, the documentation is pretty good and informative! The small nits
I pointed out are optional. So,

Reviewed-by: Maíra Canal <maira...@riseup.net>

[David Gow]

On Thu, Jul 21, 2022 at 4:26 PM Sadiya Kazi <sadiy...@google.com> wrote:
>
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file.
>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>
> ---
> Changes since V1:
> https://lore.kernel.org/linux-kselftest/20220719092214.99...@google.com/
> - Addressed most of the review comments from Maira and David, except
> removing the duplicate arguments as I felt its worth keeping them in
> the reference documentation as well as in context. We can improve them
> and differentiate their use cases in the future patches.
>
>

Looks good to me. A couple of super-minor suggestions below, and a
note about how KASAN/UML support hasn't fully landed yet (but I still
like it as an example, so maybe leave it as-is).

None of these are deal breakers, though, and I'd be okay with this
going in as-is, as well.

Reviewed-by: David Gow <davi...@google.com>

Cheers,
-- David

Just FYI, this example won't work as-is until KASAN for UML is merged.
It's already sitting in uml/next, so this shouldn't be a problem:
https://git.kernel.org/pub/scm/linux/kernel/git/uml/linux.git/commit/?h=next&id=5b301409e8bc5d7fad2ee138be44c5c529dd0874

But if you want to try it on 5.19 or the current kunit branch, you'll
need to add, e.g., --arch=x86_64

> +- ``--arch``: Runs tests on the specified architecture. The architecture
> + specified must match the Kbuild ARCH environment variable.

I'm not quite sold on 'must match the Kbuild ARCH environment
variable'. That seems to imply to me that you need to set ARCH= _and_
use --arch. Instead, --arch itself sets ARCH=, so the values use the
same names.

> + For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.
> + Default is `um`.
> +
> +- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
> + same argument as passed to the ``CROSS_COMPILE`` variable used by
> + Kbuild. This will be the prefix for the toolchain
> + binaries such as GCC. For example:
> +
> + - ``sparc64-linux-gnu-`` if we have the sparc toolchain installed on
> + our system.
> +
> + - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
> + if we have downloaded the microblaze toolchain from the 0-day
> + website to a specified path in our home directory called toolchains.
> +
> +- ``--qemu_config``: Specifies the path to a file containing a
> + custom qemu architecture definition. This should be a python file
> + containing a `QemuArchParams` object.
> +
> +- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
> +

> +- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.4

> + By default, this is set to the number of cores on your system.
> +
> +- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
> + This does not include the time taken to build the tests.
> +
> +- ``--kernel_args``: Specifies additional kernel command-line arguments. Might be repeated.

Nit: maybe "can be repeated"? As it's the reader of this documentation
who will likely be doing the repeating. Or "may be repeated"?

[Brendan Higgins]

On Thu, Jul 21, 2022 at 7:54 PM David Gow <davi...@google.com> wrote:
>
> On Thu, Jul 21, 2022 at 4:26 PM Sadiya Kazi <sadiy...@google.com> wrote:

[...]

> > Documentation/dev-tools/kunit/run_wrapper.rst | 60 ++++++++++++++++++-
> > 1 file changed, 59 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
> > index 5e560f2c5fca..600af7ac5f88 100644
> > --- a/Documentation/dev-tools/kunit/run_wrapper.rst
> > +++ b/Documentation/dev-tools/kunit/run_wrapper.rst

[...]

Agreed, I was just about to reply with a comment to the same effect.

+1

[...]

[Brendan Higgins]

On Thu, Jul 21, 2022 at 4:26 AM Sadiya Kazi <sadiy...@google.com> wrote:
>
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file.
>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>

Aside from the small nits pointed out by Maíra and David, this looks good to me.

Reviewed-by: Brendan Higgins <brendan...@google.com>

[Daniel Latypov]

On Thu, Jul 21, 2022 at 1:26 AM 'Sadiya Kazi' via KUnit Development
<kuni...@googlegroups.com> wrote:
>
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file.
>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>

Reviewed-by: Daniel Latypov <dlat...@google.com>

Looks good!
A minor suggestion down below to go along with what everyone else has said.

> +- ``--qemu_config``: Specifies the path to a file containing a
> + custom qemu architecture definition. This should be a python file
> + containing a `QemuArchParams` object.
> +
> +- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".

Minor nit: I think ``-smp 8`` would be a bit better here.
It feels like it would fit what we did with other example arguments.

[Sadiya Kazi]

Run_wrapper.rst was missing some command line arguments. Added
additional args in the file.

Signed-off-by: Sadiya Kazi <sadiy...@google.com>

---
Changes since v2:
https://lore.kernel.org/linux-kselftest/20220721081026.12...@google.com/
-Added a code block for —kconfig_add argument to make the styling consistent
-Slightly changed the words for —arch argument
-Changed QEMU to qemu wherever applicable for the cli args
-Changed the style for ``-smp 8``
-Changed "Might be repeated" to "may be repeated” for kernel_args



---
Documentation/dev-tools/kunit/run_wrapper.rst | 61 ++++++++++++++++++-
1 file changed, 60 insertions(+), 1 deletion(-)

diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index 5e560f2c5fca..ed3715fef32d 100644
--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst

@@ -233,7 +233,7 @@ Command-Line Arguments
======================

kunit_tool has a number of other command-line arguments which can
-be useful for our test environment. Below the most commonly used
+be useful for our test environment. Below are the most commonly used
command line arguments:

- ``--help``: Lists all available options. To list common options,

@@ -257,3 +257,62 @@ command line arguments:

added or modified. Instead, enable all tests
which have satisfied dependencies by adding
``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
+
+- ``--kunitconfig``: Specifies the path or the directory of the ``.kunitconfig``
+ file. For example:
+
+ - ``lib/kunit/.kunitconfig`` can be the path of the file.
+
+ - ``lib/kunit`` can be the directory in which the file is located.
+
+ This file is used to build and run with a predefined set of tests
+ and their dependencies. For example, to run tests for a given subsystem.
+
+- ``--kconfig_add``: Specifies additional configuration options to be

+ appended to the ``.kunitconfig`` file. For example:
+ .. code-block::
+ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y
+

+- ``--arch``: Runs tests on the specified architecture. The architecture

+ argument is same as the Kbuild ARCH environment variable.
+ For example, i386, x86_64, arm, um, etc. Non-UML architectures run on qemu.

+ Default is `um`.
+
+- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
+ same argument as passed to the ``CROSS_COMPILE`` variable used by
+ Kbuild. This will be the prefix for the toolchain
+ binaries such as GCC. For example:
+
+ - ``sparc64-linux-gnu-`` if we have the sparc toolchain installed on
+ our system.
+
+ - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
+ if we have downloaded the microblaze toolchain from the 0-day
+ website to a specified path in our home directory called toolchains.
+

+- ``--qemu_config``: Specifies the path to a file containing a
+ custom qemu architecture definition. This should be a python file
+ containing a `QemuArchParams` object.
+

+- ``--qemu_args``: Specifies additional qemu arguments, for example, ``-smp 8``.

+
+- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.

+ By default, this is set to the number of cores on your system.
+
+- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
+ This does not include the time taken to build the tests.
+

+- ``--kernel_args``: Specifies additional kernel command-line arguments. May be repeated.

+
+- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
+ This is useful for debugging a non-hermetic test, one that
+ might pass/fail based on what ran before it.
+
+- ``--raw_output``: If set, generates unformatted output from kernel. Possible options are:
+
+ - ``all``: To view the full kernel output, use ``--raw_output=all``.
+
+ - ``kunit``: This is the default option and filters to KUnit output. Use ``--raw_output`` or ``--raw_output=kunit``.
+
+- ``--json``: If set, stores the test results in a JSON format and prints to `stdout` or
+ saves to a file if a filename is specified.
--

2.37.1.359.gd136c6c3e2-goog

[Bagas Sanjaya]

On Sun, Jul 24, 2022 at 06:47:59PM +0000, Sadiya Kazi wrote:
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file.
>

Hi Sadiya,

The patch description looks awkward. Better say:
"Many kunit_tool common command line arguments are missing from the
documentation. Document them."

> +- ``--kconfig_add``: Specifies additional configuration options to be
> + appended to the ``.kunitconfig`` file. For example:
> + .. code-block::
> + ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y
> +

Above generated new warning:

Documentation/dev-tools/kunit/run_wrapper.rst:262: WARNING: Unexpected indentation.

I have applied the fixup:

---- >8 ----

From cd4cf7c5bb2a7bfe3b006d7859e84de0814d7af4 Mon Sep 17 00:00:00 2001
From: Bagas Sanjaya <bagas...@gmail.com>
Date: Mon, 25 Jul 2022 15:54:24 +0700
Subject: [PATCH] fixup for "Documentation: kunit: Add CLI args for kunit_tool"

Pad --kconfig_add code block with blank line

Signed-off-by: Bagas Sanjaya <bagas...@gmail.com>
---
Documentation/dev-tools/kunit/run_wrapper.rst | 2 ++
1 file changed, 2 insertions(+)

diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index 479d16d1f17d60..d19a1b0d58595b 100644
--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst
@@ -258,7 +258,9 @@ command line arguments:

- ``--kconfig_add``: Specifies additional configuration options to be

appended to the ``.kunitconfig`` file. For example:
+
.. code-block::
+
./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y

- ``--arch``: Runs tests on the specified architecture. The architecture

--

Thanks.

--
An old man doll... just what I always wanted! - Clara

[kernel test robot]

Hi Sadiya,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on lwn/docs-next]
[also build test WARNING on linus/master v5.19-rc8 next-20220722]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url: https://github.com/intel-lab-lkp/linux/commits/Sadiya-Kazi/Documentation-kunit-Add-CLI-args-for-kunit_tool/20220725-025055
base: git://git.lwn.net/linux.git docs-next
reproduce: make htmldocs

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <l...@intel.com>

All warnings (new ones prefixed by >>):

>> Documentation/dev-tools/kunit/run_wrapper.rst:262: WARNING: Unexpected indentation.

vim +262 Documentation/dev-tools/kunit/run_wrapper.rst

226
227 - ``--help``: Lists all available options. To list common options,
228 place ``--help`` before the command. To list options specific to that
229 command, place ``--help`` after the command.
230
231 .. note:: Different commands (``config``, ``build``, ``run``, etc)
232 have different supported options.
233 - ``--build_dir``: Specifies kunit_tool build directory. It includes
234 the ``.kunitconfig``, ``.config`` files and compiled kernel.
235
236 - ``--make_options``: Specifies additional options to pass to make, when
237 compiling a kernel (using ``build`` or ``run`` commands). For example:
238 to enable compiler warnings, we can pass ``--make_options W=1``.
239
240 - ``--alltests``: Builds a UML kernel with all config options enabled
241 using ``make allyesconfig``. This allows us to run as many tests as
242 possible.
243
244 .. note:: It is slow and prone to breakage as new options are
245 added or modified. Instead, enable all tests
246 which have satisfied dependencies by adding
247 ``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
248
249 - ``--kunitconfig``: Specifies the path or the directory of the ``.kunitconfig``
250 file. For example:
251
252 - ``lib/kunit/.kunitconfig`` can be the path of the file.
253
254 - ``lib/kunit`` can be the directory in which the file is located.
255
256 This file is used to build and run with a predefined set of tests
257 and their dependencies. For example, to run tests for a given subsystem.
258
259 - ``--kconfig_add``: Specifies additional configuration options to be
260 appended to the ``.kunitconfig`` file. For example:
261 .. code-block::
> 262 ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y
263

--
0-DAY CI Kernel Test Service
https://01.org/lkp

[Maíra Canal]

Hi Sadiya,

On 7/24/22 15:47, Sadiya Kazi wrote:
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file.
>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>

Usually, when people send their Reviewed-by on a previous version, we
send the next version with their tags in chronological order [1].

> ---
> Changes since v2:
> https://lore.kernel.org/linux-kselftest/20220721081026.12...@google.com/
> -Added a code block for —kconfig_add argument to make the styling consistent
> -Slightly changed the words for —arch argument
> -Changed QEMU to qemu wherever applicable for the cli args
> -Changed the style for ``-smp 8``
> -Changed "Might be repeated" to "may be repeated” for kernel_args
>

Also, it is a common practice to keep the entire changelog here [1].

Other than the error reported-by kernel test robot, looks good to me.

Reviewed-by: Maíra Canal <maira...@riseup.net>

[1]
https://docs.kernel.org/process/submitting-patches.html#the-canonical-patch-format

Best Regards,
- Maíra Canal

>
>

[Sadiya Kazi]

Some kunit_tool command line arguments are missing in run_wrapper.rst.
Document them.

Reported-by: Bagas Sanjaya <bagas...@gmail.com>

Reported-by: kernel test robot <l...@intel.com>

Reviewed-by: David Gow <davi...@google.com>
Reviewed-by: Brendan Higgins <brendan...@google.com>
Reviewed-by: Daniel Latypov <dlat...@google.com>
Reviewed-by: Maíra Canal <maira...@riseup.net>

Signed-off-by: Sadiya Kazi <sadiy...@google.com>
---

Changes since v3:
https://lore.kernel.org/linux-kselftest/20220724184758.17...@google.com/
-Fixed the indention bug in the run_wrapper.rst file. Thanks for
catching that(Kernel test robot, Bagas).
-Updated the commit message.

Changes since v2:
https://lore.kernel.org/linux-kselftest/20220721081026.12...@google.com/
-Added a code block for —kconfig_add argument to make the styling consistent
-Slightly changed the words for —arch argument
-Changed QEMU to qemu wherever applicable for the cli args
-Changed the style for ``-smp 8``
-Changed "Might be repeated" to "may be repeated” for kernel_args

Changes since V1:
https://lore.kernel.org/linux-kselftest/20220719092214.99...@google.com/
- Addressed most of the review comments from Maira and David, except
removing the duplicate arguments as I felt its worth keeping them in
the reference documentation as well as in context. We can improve them
and differentiate their use cases in the future patches.

---
Documentation/dev-tools/kunit/run_wrapper.rst | 63 ++++++++++++++++++-
1 file changed, 62 insertions(+), 1 deletion(-)

diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index 5e560f2c5fca..cce203138fb7 100644

--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst
@@ -233,7 +233,7 @@ Command-Line Arguments
======================

kunit_tool has a number of other command-line arguments which can
-be useful for our test environment. Below the most commonly used
+be useful for our test environment. Below are the most commonly used
command line arguments:

- ``--help``: Lists all available options. To list common options,

@@ -257,3 +257,64 @@ command line arguments:

added or modified. Instead, enable all tests
which have satisfied dependencies by adding
``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
+
+- ``--kunitconfig``: Specifies the path or the directory of the ``.kunitconfig``
+ file. For example:
+
+ - ``lib/kunit/.kunitconfig`` can be the path of the file.
+
+ - ``lib/kunit`` can be the directory in which the file is located.
+
+ This file is used to build and run with a predefined set of tests
+ and their dependencies. For example, to run tests for a given subsystem.
+
+- ``--kconfig_add``: Specifies additional configuration options to be
+ appended to the ``.kunitconfig`` file. For example:
+

+ .. code-block::
+

--
2.37.1.359.gd136c6c3e2-goog

[Bagas Sanjaya]

On Tue, Jul 26, 2022 at 04:12:50AM +0000, Sadiya Kazi wrote:
> Some kunit_tool command line arguments are missing in run_wrapper.rst.
> Document them.
>
> Reported-by: Bagas Sanjaya <bagas...@gmail.com>
> Reported-by: kernel test robot <l...@intel.com>
> Reviewed-by: David Gow <davi...@google.com>
> Reviewed-by: Brendan Higgins <brendan...@google.com>
> Reviewed-by: Daniel Latypov <dlat...@google.com>
> Reviewed-by: Maíra Canal <maira...@riseup.net>
> Signed-off-by: Sadiya Kazi <sadiy...@google.com>

I think Reported-by should only be used when submitting patches that
fixes build warnings/failure pointed to the code that is already in the
tree (like mainline).

Otherwise, the documentation builds succesfully without new warnings.

Reviewed-by: Bagas Sanjaya <bagas...@gmail.com>
Tested-by: Bagas Sanjaya <bagas...@gmail.com>