[PATCH 1/2] kunit: Add param generator macro for zero terminated arrays

[Michal Wajdeczko]

The existing macro KUNIT_ARRAY_PARAM can produce parameter
generator function but only when we fully know the definition
of the array. However, there might be cases where we would like
to generate test params based on externaly defined array, which
is defined as zero-terminated array, like pci_driver.id_table.

Add helper macro KUNIT_ZERO_ARRAY_PARAM that can work with zero
terminated arrays and provide example how to use it.

$ ./tools/testing/kunit/kunit.py run \
--kunitconfig ./lib/kunit/.kunitconfig *.example_params*

[ ] Starting KUnit Kernel (1/1)...
[ ] ============================================================
[ ] ========================= example =========================
[ ] =================== example_params_test ===================
[ ] [SKIPPED] example value 3
[ ] [PASSED] example value 2
[ ] [PASSED] example value 1
[ ] [SKIPPED] example value 0
[ ] =============== [PASSED] example_params_test ===============
[ ] =================== example_params_test ===================
[ ] [SKIPPED] example value 3
[ ] [PASSED] example value 2
[ ] [PASSED] example value 1
[ ] =============== [PASSED] example_params_test ===============
[ ] ===================== [PASSED] example =====================
[ ] ============================================================
[ ] Testing complete. Ran 7 tests: passed: 4, skipped: 3

Signed-off-by: Michal Wajdeczko <michal.w...@intel.com>
Cc: David Gow <davi...@google.com>
Cc: Rae Moar <rm...@google.com>
---
include/kunit/test.h | 22 ++++++++++++++++++++++
lib/kunit/kunit-example-test.c | 2 ++
2 files changed, 24 insertions(+)

diff --git a/include/kunit/test.h b/include/kunit/test.h
index 20ed9f9275c9..280113ceb6a6 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -1514,6 +1514,28 @@ do { \
return NULL; \
}

+/**
+ * KUNIT_ZERO_ARRAY_PARAM() - Define test parameter generator from a zero terminated array.
+ * @name: prefix for the test parameter generator function.
+ * @array: zero terminated array of test parameters.
+ * @get_desc: function to convert param to description; NULL to use default
+ *
+ * Define function @name_gen_params which uses zero terminated @array to generate parameters.
+ */
+#define KUNIT_ZERO_ARRAY_PARAM(name, array, get_desc) \
+ static const void *name##_gen_params(const void *prev, char *desc) \
+ { \
+ typeof((array)[0]) *__prev = prev; \
+ typeof(__prev) __next = __prev ? __prev + 1 : (array); \
+ void (*__get_desc)(typeof(__next), char *) = get_desc; \
+ for (; memchr_inv(__next, 0, sizeof(*__next)); __prev = __next++) { \
+ if (__get_desc) \
+ __get_desc(__next, desc); \
+ return __next; \
+ } \
+ return NULL; \
+ }
+
// TODO(dlat...@google.com): consider eventually migrating users to explicitly
// include resource.h themselves if they need it.
#include <kunit/resource.h>
diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
index 6bb5c2ef6696..ad9ebcfd513e 100644
--- a/lib/kunit/kunit-example-test.c
+++ b/lib/kunit/kunit-example-test.c
@@ -202,6 +202,7 @@ static void example_param_get_desc(const struct example_param *p, char *desc)
}

KUNIT_ARRAY_PARAM(example, example_params_array, example_param_get_desc);
+KUNIT_ZERO_ARRAY_PARAM(example_zero, example_params_array, example_param_get_desc);

/*
* This test shows the use of params.
@@ -246,6 +247,7 @@ static struct kunit_case example_test_cases[] = {
KUNIT_CASE(example_all_expect_macros_test),
KUNIT_CASE(example_static_stub_test),
KUNIT_CASE_PARAM(example_params_test, example_gen_params),
+ KUNIT_CASE_PARAM(example_params_test, example_zero_gen_params),
KUNIT_CASE_SLOW(example_slow_test),
{}
};
--
2.25.1

[Michal Wajdeczko]

In some cases we may want to generate params based on existing
zero terminated array, but with some entries filtered out.
Extend macro KUNIT_ZERO_ARRAY_PARAM to accept filter function

and provide example how to use it.

$ ./tools/testing/kunit/kunit.py run \
--kunitconfig ./lib/kunit/.kunitconfig *.example_params*

[ ] Starting KUnit Kernel (1/1)...
[ ] ============================================================
[ ] ========================= example =========================
[ ] =================== example_params_test ===================
[ ] [SKIPPED] example value 3
[ ] [PASSED] example value 2
[ ] [PASSED] example value 1
[ ] [SKIPPED] example value 0
[ ] =============== [PASSED] example_params_test ===============
[ ] =================== example_params_test ===================
[ ] [SKIPPED] example value 3
[ ] [PASSED] example value 2
[ ] [PASSED] example value 1
[ ] =============== [PASSED] example_params_test ===============

[ ] =================== example_params_test ===================

[ ] [PASSED] example value 2
[ ] [PASSED] example value 1
[ ] =============== [PASSED] example_params_test ===============
[ ] ===================== [PASSED] example =====================
[ ] ============================================================

[ ] Testing complete. Ran 9 tests: passed: 6, skipped: 3

Signed-off-by: Michal Wajdeczko <michal.w...@intel.com>
Cc: David Gow <davi...@google.com>
Cc: Rae Moar <rm...@google.com>
---

include/kunit/test.h | 19 +++++++++++++++++--
lib/kunit/kunit-example-test.c | 9 +++++++++
2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/include/kunit/test.h b/include/kunit/test.h
index 280113ceb6a6..8a87d1ce37e0 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -1515,20 +1515,24 @@ do { \
}

/**
- * KUNIT_ZERO_ARRAY_PARAM() - Define test parameter generator from a zero terminated array.
+ * KUNIT_FILTERED_ZERO_ARRAY_PARAM() - Define test parameter generator from a zero terminated array.

* @name: prefix for the test parameter generator function.

* @array: zero terminated array of test parameters.

* @get_desc: function to convert param to description; NULL to use default

+ * @filter: function to filter out unwanted params (like duplicates); can be NULL
*

* Define function @name_gen_params which uses zero terminated @array to generate parameters.

*/
-#define KUNIT_ZERO_ARRAY_PARAM(name, array, get_desc) \
+#define KUNIT_FILTERED_ZERO_ARRAY_PARAM(name, array, get_desc, filter) \

static const void *name##_gen_params(const void *prev, char *desc) \

{ \

typeof((array)[0]) *__prev = prev; \

typeof(__prev) __next = __prev ? __prev + 1 : (array); \

void (*__get_desc)(typeof(__next), char *) = get_desc; \

+ bool (*__filter)(typeof(__prev), typeof(__next)) = filter; \

for (; memchr_inv(__next, 0, sizeof(*__next)); __prev = __next++) { \

+ if (__filter && !__filter(__prev, __next)) \
+ continue; \
if (__get_desc) \
__get_desc(__next, desc); \
return __next; \
@@ -1536,6 +1540,17 @@ do { \

return NULL; \
}

+/**
+ * KUNIT_ZERO_ARRAY_PARAM() - Define test parameter generator from a zero terminated array.
+ * @name: prefix for the test parameter generator function.
+ * @array: zero terminated array of test parameters.
+ * @get_desc: function to convert param to description; NULL to use default
+ *
+ * Define function @name_gen_params which uses zero terminated @array to generate parameters.
+ */
+#define KUNIT_ZERO_ARRAY_PARAM(name, array, get_desc) \

+ KUNIT_FILTERED_ZERO_ARRAY_PARAM(name, array, get_desc, NULL)

+
// TODO(dlat...@google.com): consider eventually migrating users to explicitly
// include resource.h themselves if they need it.
#include <kunit/resource.h>
diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c

index ad9ebcfd513e..a3268754392c 100644
--- a/lib/kunit/kunit-example-test.c
+++ b/lib/kunit/kunit-example-test.c
@@ -201,8 +201,16 @@ static void example_param_get_desc(const struct example_param *p, char *desc)
snprintf(desc, KUNIT_PARAM_DESC_SIZE, "example value %d", p->value);
}

+static bool example_param_filter(const struct example_param *prev,
+ const struct example_param *next)
+{
+ return next->value < 3;
+}
+
KUNIT_ARRAY_PARAM(example, example_params_array, example_param_get_desc);
KUNIT_ZERO_ARRAY_PARAM(example_zero, example_params_array, example_param_get_desc);
+KUNIT_FILTERED_ZERO_ARRAY_PARAM(example_filter, example_params_array, example_param_get_desc,
+ example_param_filter);

/*
* This test shows the use of params.

@@ -248,6 +256,7 @@ static struct kunit_case example_test_cases[] = {
KUNIT_CASE(example_static_stub_test),
KUNIT_CASE_PARAM(example_params_test, example_gen_params),
KUNIT_CASE_PARAM(example_params_test, example_zero_gen_params),
+ KUNIT_CASE_PARAM(example_params_test, example_filter_gen_params),
KUNIT_CASE_SLOW(example_slow_test),
{}
};
--
2.25.1

[Rae Moar]

Hello!

This overall looks good to me. I am not sure how many uses there are
for zero-terminated arrays for parameterized tests.

However, since it seems to be a well designed feature, I think it
should be good to add this to KUnit.

Thanks for including an example! Just one other comment below.

-Rae

> // TODO(dlat...@google.com): consider eventually migrating users to explicitly
> // include resource.h themselves if they need it.
> #include <kunit/resource.h>
> diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
> index 6bb5c2ef6696..ad9ebcfd513e 100644
> --- a/lib/kunit/kunit-example-test.c
> +++ b/lib/kunit/kunit-example-test.c
> @@ -202,6 +202,7 @@ static void example_param_get_desc(const struct example_param *p, char *desc)
> }
>
> KUNIT_ARRAY_PARAM(example, example_params_array, example_param_get_desc);
> +KUNIT_ZERO_ARRAY_PARAM(example_zero, example_params_array, example_param_get_desc);
>
> /*
> * This test shows the use of params.
> @@ -246,6 +247,7 @@ static struct kunit_case example_test_cases[] = {
> KUNIT_CASE(example_all_expect_macros_test),
> KUNIT_CASE(example_static_stub_test),
> KUNIT_CASE_PARAM(example_params_test, example_gen_params),
> + KUNIT_CASE_PARAM(example_params_test, example_zero_gen_params),

I would prefer if the name of the new params test was a bit different
just to differentiate them. Maybe: example_params_zero_test

> KUNIT_CASE_SLOW(example_slow_test),
> {}
> };
> --
> 2.25.1
>

[Rae Moar]

On Tue, Sep 26, 2023 at 6:02 PM Michal Wajdeczko
<michal.w...@intel.com> wrote:
>

> In some cases we may want to generate params based on existing
> zero terminated array, but with some entries filtered out.
> Extend macro KUNIT_ZERO_ARRAY_PARAM to accept filter function
> and provide example how to use it.

Hello!

I definitely understand the use case of wanting to filter params.
However, since this is a static filter, it seems this could be done in
the test file and rather than implemented as a new KUnit feature.

I would be interested to see David's thoughts on this. If we do decide
to implement this as a KUnit feature, I would also prefer if the
filtering ability is available for both the zero-terminated param
arrays and normal param arrays.

Otherwise I just have one comment below. Let me know what you think.

Thanks!
-Rae

Similar to last patch, I would probably prefer the test names to
differ slightly for the three param tests.

> KUNIT_CASE_SLOW(example_slow_test),
> {}
> };
> --
> 2.25.1
>
> --
> You received this message because you are subscribed to the Google Groups "KUnit Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kunit-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kunit-dev/20230926220208.1423-2-michal.wajdeczko%40intel.com.

[David Gow]

On Wed, 27 Sept 2023 at 06:02, Michal Wajdeczko

<michal.w...@intel.com> wrote:
>
> The existing macro KUNIT_ARRAY_PARAM can produce parameter
> generator function but only when we fully know the definition
> of the array. However, there might be cases where we would like
> to generate test params based on externaly defined array, which
> is defined as zero-terminated array, like pci_driver.id_table.

Hmm... I like the idea of this, but am a little wary of dealing with
zero-terminated arrays in a generic fashion. Some cases (pointers,
where we can just != NULL) are obvious,
but we could hit inconsistencies with things like padding, as things
like pci_driver.id_table seem to mostly be iterated over with things
like:
while (ids->vendor || ids->subvendor || ids->class_mask)

which not only ignores the padding, but also half of the fields. So
there may be a consistency issue there.

Though I suspect it's not likely to cause issues in practice.

Thoughts?
-- David

Are there any places where this might interact awkwardly with padding?
I _think_ it should be okay (variables with static lifetimes should
have padding initialised to zero), but there could be a case I'm
missing.

[David Gow]

On Wed, 27 Sept 2023 at 06:02, Michal Wajdeczko
<michal.w...@intel.com> wrote:
>

I tend to agree with Rae here that this is:
a) starting to get quite specific, and might be better served by a
per-test generator function than a generic helper here, and
b) probably would need to be implemented for both ZERO_ARRAY_PARAM
and the regular ARRAY_PARAM, for consistency's sake.

I'd probably err on the side of letting tests implement something like
this themselves, and then reconsider a helper macro once there are
several tests all using the same thing. If, for example, all the users
of this are just checking PCI ids, or something, it may make more
sense to just have a PCI-specific macro they can share.

Regardless, I'd love to see a real-world example of this being used,
which may make it seem less niche.

Thanks,
-- David

I think we'd probably want some more comments around these examples,
just to make it a little clearer what the difference between these
features are and what results we should expect. The purpose the the
'example_test' is basically documentation, so I think we'll want to
either make this a clearer example of how/why you'd want to use this,
or leave it out of the examples (and just document it in the
documentation, and possibly test it in another kunit test).

>
> /*
> * This test shows the use of params.
> @@ -248,6 +256,7 @@ static struct kunit_case example_test_cases[] = {
> KUNIT_CASE(example_static_stub_test),
> KUNIT_CASE_PARAM(example_params_test, example_gen_params),
> KUNIT_CASE_PARAM(example_params_test, example_zero_gen_params),
> + KUNIT_CASE_PARAM(example_params_test, example_filter_gen_params),
> KUNIT_CASE_SLOW(example_slow_test),
> {}
> };
> --
> 2.25.1
>

[Michal Wajdeczko]

It looks that most of the existing code is relying on empty
initialization with = { 0 } or = { }, the latter known to be zero
initialized, including padding, in C23, so if we want to be pedantic we
may allow to provide a pointer to a table specific "is_end()" function,
that will detect end of the parameters array, which we could still
default to memchr_inv if custom solution is not needed:

#define KUNIT_ZERO_ARRAY_PARAM(name, array, get_desc, is_end) \
...
bool (*__is_end)(typeof(__next)) = is_end; \
for (; __is_end ? __is_end(__next) : \
!!memchr_inv(__next, 0, sizeof(*__next)); \
__prev = __next++) { \


KUNIT_ZERO_ARRAY_PARAM(example_zero, example_params_array,
example_param_get_desc, NULL);
or

static bool example_param_valid(const struct example_param *next)
{
return next->value;
}

KUNIT_ZERO_ARRAY_PARAM(example_zero, example_params_array,
example_param_get_desc, example_param_valid);

then we shouldn't miss any case

Michal