[PATCH 1/2] kcov: use write memory barrier after memcpy() in kcov_move_area()

[Soham Bagchi]

KCOV Remote uses two separate memory buffers, one private to the kernel
space (kcov_remote_areas) and the second one shared between user and
kernel space (kcov->area). After every pair of kcov_remote_start() and
kcov_remote_stop(), the coverage data collected in the
kcov_remote_areas is copied to kcov->area so the user can read the
collected coverage data. This memcpy() is located in kcov_move_area().

The load/store pattern on the kernel-side [1] is:

```
/* dst_area === kcov->area, dst_area[0] is where the count is stored */
dst_len = READ_ONCE(*(unsigned long *)dst_area);
...
memcpy(dst_entries, src_entries, ...);
...
WRITE_ONCE(*(unsigned long *)dst_area, dst_len + entries_moved);
```

And for the user [2]:

```
/* cover is equivalent to kcov->area */
n = __atomic_load_n(&cover[0], __ATOMIC_RELAXED);
```

Without a write-memory barrier, the atomic load for the user can
potentially read fresh values of the count stored at cover[0],
but continue to read stale coverage data from the buffer itself.
Hence, we recommend adding a write-memory barrier between the
memcpy() and the WRITE_ONCE() in kcov_move_area().

[1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/kernel/kcov.c?h=master#n978
[2] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/dev-tools/kcov.rst#n364

Signed-off-by: Soham Bagchi <soham....@utah.edu>
---
kernel/kcov.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/kernel/kcov.c b/kernel/kcov.c
index 187ba1b80bd..f6ee6d7dc2c 100644
--- a/kernel/kcov.c
+++ b/kernel/kcov.c
@@ -978,6 +978,15 @@ static void kcov_move_area(enum kcov_mode mode, void *dst_area,
memcpy(dst_entries, src_entries, bytes_to_move);
entries_moved = bytes_to_move >> entry_size_log;

+ /*
+ * A write memory barrier is required here, to ensure
+ * that the writes from the memcpy() are visible before
+ * the count is updated. Without this, it is possible for
+ * a user to observe a new count value but stale
+ * coverage data.
+ */
+ smp_wmb();
+
switch (mode) {
case KCOV_MODE_TRACE_PC:
WRITE_ONCE(*(unsigned long *)dst_area, dst_len + entries_moved);
--
2.34.1

[Soham Bagchi]

Updating the KCOV documentation to use a load-acquire
operation for the first element of the shared memory
buffer between kernel-space and user-space.

The load-acquire pairs with the write memory barrier
used in kcov_move_area()

Signed-off-by: Soham Bagchi <soham....@utah.edu>
---

Documentation/dev-tools/kcov.rst | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Documentation/dev-tools/kcov.rst b/Documentation/dev-tools/kcov.rst
index 6611434e2dd..46450fb46fe 100644
--- a/Documentation/dev-tools/kcov.rst
+++ b/Documentation/dev-tools/kcov.rst
@@ -287,6 +287,11 @@ handle instance id.
The following program demonstrates using KCOV to collect coverage from both
local tasks spawned by the process and the global task that handles USB bus #1:

+The user-space code for KCOV should also use an acquire to fetch the count
+of coverage entries in the shared buffer. This acquire pairs with the
+corresponding write memory barrier (smp_wmb()) on the kernel-side in
+kcov_move_area().
+
.. code-block:: c

/* Same includes and defines as above. */
@@ -361,7 +366,7 @@ local tasks spawned by the process and the global task that handles USB bus #1:
*/
sleep(2);

- n = __atomic_load_n(&cover[0], __ATOMIC_RELAXED);
+ n = __atomic_load_n(&cover[0], __ATOMIC_ACQUIRE);
for (i = 0; i < n; i++)
printf("0x%lx\n", cover[i + 1]);
if (ioctl(fd, KCOV_DISABLE, 0))
--
2.34.1

[Marco Elver]

Reviewed-by: Marco Elver <el...@google.com>

> ---
> kernel/kcov.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/kernel/kcov.c b/kernel/kcov.c
> index 187ba1b80bd..f6ee6d7dc2c 100644
> --- a/kernel/kcov.c
> +++ b/kernel/kcov.c
> @@ -978,6 +978,15 @@ static void kcov_move_area(enum kcov_mode mode, void *dst_area,
> memcpy(dst_entries, src_entries, bytes_to_move);
> entries_moved = bytes_to_move >> entry_size_log;
>
> + /*
> + * A write memory barrier is required here, to ensure
> + * that the writes from the memcpy() are visible before
> + * the count is updated. Without this, it is possible for
> + * a user to observe a new count value but stale
> + * coverage data.
> + */
> + smp_wmb();
> +
> switch (mode) {
> case KCOV_MODE_TRACE_PC:
> WRITE_ONCE(*(unsigned long *)dst_area, dst_len + entries_moved);
> --
> 2.34.1
>

> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/kasan-dev/20250728184318.1839137-1-soham.bagchi%40utah.edu.

[Marco Elver]

On Mon, 28 Jul 2025 at 20:43, Soham Bagchi <soham....@utah.edu> wrote:
>

> Updating the KCOV documentation to use a load-acquire
> operation for the first element of the shared memory
> buffer between kernel-space and user-space.
>
> The load-acquire pairs with the write memory barrier
> used in kcov_move_area()
>
> Signed-off-by: Soham Bagchi <soham....@utah.edu>
> ---
> Documentation/dev-tools/kcov.rst | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/dev-tools/kcov.rst b/Documentation/dev-tools/kcov.rst
> index 6611434e2dd..46450fb46fe 100644
> --- a/Documentation/dev-tools/kcov.rst
> +++ b/Documentation/dev-tools/kcov.rst
> @@ -287,6 +287,11 @@ handle instance id.
> The following program demonstrates using KCOV to collect coverage from both
> local tasks spawned by the process and the global task that handles USB bus #1:
>
> +The user-space code for KCOV should also use an acquire to fetch the count
> +of coverage entries in the shared buffer. This acquire pairs with the
> +corresponding write memory barrier (smp_wmb()) on the kernel-side in
> +kcov_move_area().
> +

This new paragraph is misplaced.
You've added it after the "... handles USB bus #1:" part which clearly
should be right before the code (note the colon).

Why not add what you wrote here as a block-comment (similar in style
to comment above the sleep()) right above the __atomic_load_n below? I
think those details probably don't quite belong into the high level
text, but the detailed code example.

[Soham Bagchi]

Updating the KCOV documentation to use a load-acquire
operation for the first element of the shared memory
buffer between kernel-space and user-space.

The load-acquire pairs with the write memory barrier
used in kcov_move_area()

Signed-off-by: Soham Bagchi <soham....@utah.edu>
---

Changes in v2:
- note for load-acquire shifted to block comment
in code rather than in the preceding paragraphs

---
Documentation/dev-tools/kcov.rst | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Documentation/dev-tools/kcov.rst b/Documentation/dev-tools/kcov.rst

index 6611434e2dd..40a4b500073 100644
--- a/Documentation/dev-tools/kcov.rst
+++ b/Documentation/dev-tools/kcov.rst
@@ -361,7 +361,12 @@ local tasks spawned by the process and the global task that handles USB bus #1:

*/
sleep(2);

- n = __atomic_load_n(&cover[0], __ATOMIC_RELAXED);

+ /*
+ * The load to the coverage count should be an acquire to pair with
+ * pair with the corresponding write memory barrier (smp_wmb()) on
+ * the kernel-side in kcov_move_area().
+ */

[Marco Elver]

On Sun, 3 Aug 2025 at 20:06, Soham Bagchi <soham....@utah.edu> wrote:
>
> Updating the KCOV documentation to use a load-acquire
> operation for the first element of the shared memory
> buffer between kernel-space and user-space.
>
> The load-acquire pairs with the write memory barrier
> used in kcov_move_area()
>
> Signed-off-by: Soham Bagchi <soham....@utah.edu>

Reviewed-by: Marco Elver <el...@google.com>

> ---
>

> Changes in v2:

Btw, it is customary to send out the whole patch series on a version
bump, even if only one of the patches changed.
https://www.kernel.org/doc/html/latest/process/submitting-patches.html#explicit-in-reply-to-headers

[Andrew Morton]

On Mon, 4 Aug 2025 08:00:00 +0200 Marco Elver <el...@google.com> wrote:

> > The load-acquire pairs with the write memory barrier
> > used in kcov_move_area()
> >
> > Signed-off-by: Soham Bagchi <soham....@utah.edu>
>
> Reviewed-by: Marco Elver <el...@google.com>
>
> > ---
> >
> > Changes in v2:
>
> Btw, it is customary to send out the whole patch series on a version
> bump, even if only one of the patches changed.
> https://www.kernel.org/doc/html/latest/process/submitting-patches.html#explicit-in-reply-to-headers

Yes please, try to keep everything together. We look at a lot of
patches!

I queued this as a -fix against the original
https://lkml.kernel.org/r/20250728184318.183...@utah.edu

--- a/Documentation/dev-tools/kcov.rst~kcov-load-acquire-coverage-count-in-user-space-code-v2
+++ a/Documentation/dev-tools/kcov.rst
@@ -287,11 +287,6 @@ handle instance id.

The following program demonstrates using KCOV to collect coverage from both

local tasks spawned by the process and the global task that handles USB bus #1:

-The user-space code for KCOV should also use an acquire to fetch the count
-of coverage entries in the shared buffer. This acquire pairs with the
-corresponding write memory barrier (smp_wmb()) on the kernel-side in
-kcov_move_area().
-

.. code-block:: c

/* Same includes and defines as above. */

@@ -366,6 +361,11 @@ kcov_move_area().
*/
sleep(2);

+ /*
+ * The load to the coverage count should be an acquire to pair with
+ * pair with the corresponding write memory barrier (smp_wmb()) on
+ * the kernel-side in kcov_move_area().
+ */

n = __atomic_load_n(&cover[0], __ATOMIC_ACQUIRE);
for (i = 0; i < n; i++)
printf("0x%lx\n", cover[i + 1]);

_