[PATCH] task_work: kasan: record task_work_add() call stack

[Walter Wu]

Why record task_work_add() call stack?
Syzbot reports many use-after-free issues for task_work, see [1].
After see the free stack and the current auxiliary stack, we think
they are useless, we don't know where register the work, this work
may be the free call stack, so that we miss the root cause and
don't solve the use-after-free.

Add task_work_add() call stack into KASAN auxiliary stack in
order to improve KASAN report. It is useful for programmers
to solve use-after-free issues.

[1]: https://groups.google.com/g/syzkaller-bugs/search?q=kasan%20use-after-free%20task_work_run

Signed-off-by: Walter Wu <walter...@mediatek.com>
Suggested-by: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <ryabin...@gmail.com>
Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Konovalov <andre...@google.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Andrew Morton <ak...@linux-foundation.org>
Cc: Matthias Brugger <matthi...@gmail.com>
Cc: Jens Axboe <ax...@kernel.dk>
Cc: Oleg Nesterov <ol...@redhat.com>
---
kernel/task_work.c | 3 +++
mm/kasan/kasan.h | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/task_work.c b/kernel/task_work.c
index 9cde961875c0..f255294377da 100644
--- a/kernel/task_work.c
+++ b/kernel/task_work.c
@@ -55,6 +55,9 @@ int task_work_add(struct task_struct *task, struct callback_head *work,
break;
}

+ /* record the work call stack in order to print it in KASAN reports */
+ kasan_record_aux_stack(work);
+
return 0;
}

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3436c6bf7c0c..d300fe9415bd 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -146,7 +146,7 @@ struct kasan_alloc_meta {
struct kasan_track alloc_track;
#ifdef CONFIG_KASAN_GENERIC
/*
- * call_rcu() call stack is stored into struct kasan_alloc_meta.
+ * Auxiliary stack is stored into struct kasan_alloc_meta.
* The free stack is stored into struct kasan_free_meta.
*/
depot_stack_handle_t aux_stack[2];
--
2.18.0

[Dmitry Vyukov]

I think this call should be done _before_ we actually queue the work,
because this function may operate on non-current task.
Consider, we queue the work, the other task already executes it and
triggers use-after-free, now only now we record the stack.
Moreover, I think we can trigger use-after-free here ourselves while
recording the aux stack. We queued the work, and the work can cause
own free, so it's not necessary live by now.

[Walter Wu]

agree, what do you think below change?

--- a/kernel/task_work.c
+++ b/kernel/task_work.c
@@ -34,6 +34,9 @@ int task_work_add(struct task_struct *task, struct
callback_head *work,
{
struct callback_head *head;

+ /* record the work call stack in order to print it in KASAN reports
*/
+ kasan_record_aux_stack(work);

+
do {
head = READ_ONCE(task->task_works);
if (unlikely(head == &work_exited))
@@ -55,9 +58,6 @@ int task_work_add(struct task_struct *task, struct
callback_head *work,
break;
}

- /* record the work call stack in order to print it in KASAN reports
*/
- kasan_record_aux_stack(work);
-
return 0;

}

> Moreover, I think we can trigger use-after-free here ourselves while
> recording the aux stack. We queued the work, and the work can cause
> own free, so it's not necessary live by now.

Sorry, I don't fully know your meaning, do you mean we should add an
abort when detect use-after-free?

[Dmitry Vyukov]

This looks good to me.

> do {
> head = READ_ONCE(task->task_works);
> if (unlikely(head == &work_exited))
> @@ -55,9 +58,6 @@ int task_work_add(struct task_struct *task, struct
> callback_head *work,
> break;
> }
>
> - /* record the work call stack in order to print it in KASAN reports
> */
> - kasan_record_aux_stack(work);
> -
> return 0;
> }
>
> > Moreover, I think we can trigger use-after-free here ourselves while
> > recording the aux stack. We queued the work, and the work can cause
> > own free, so it's not necessary live by now.
>
> Sorry, I don't fully know your meaning, do you mean we should add an
> abort when detect use-after-free?

I meant that where we had the kasan_record_aux_stack(work) call in the
first version of the patch, work can be already freed. We must not
access work after queueing it.

> > > return 0;
> > > }
> > >
> > > diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> > > index 3436c6bf7c0c..d300fe9415bd 100644
> > > --- a/mm/kasan/kasan.h
> > > +++ b/mm/kasan/kasan.h
> > > @@ -146,7 +146,7 @@ struct kasan_alloc_meta {
> > > struct kasan_track alloc_track;
> > > #ifdef CONFIG_KASAN_GENERIC
> > > /*
> > > - * call_rcu() call stack is stored into struct kasan_alloc_meta.
> > > + * Auxiliary stack is stored into struct kasan_alloc_meta.
> > > * The free stack is stored into struct kasan_free_meta.
> > > */
> > > depot_stack_handle_t aux_stack[2];
> > > --
> > > 2.18.0
> >
>

> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/1615801102.24887.4.camel%40mtksdccf07.

[Walter Wu]

Got it. Now I must treat urgent issue, I will send v2 patch tomorrow.

Thanks for your review.

[Walter Wu]

Why record task_work_add() call stack?
Syzbot reports many use-after-free issues for task_work, see [1].
After see the free stack and the current auxiliary stack, we think
they are useless, we don't know where register the work, this work
may be the free call stack, so that we miss the root cause and
don't solve the use-after-free.

Add task_work_add() call stack into KASAN auxiliary stack in
order to improve KASAN report. It is useful for programmers
to solve use-after-free issues.

[1]: https://groups.google.com/g/syzkaller-bugs/search?q=kasan%20use-after-free%20task_work_run

Signed-off-by: Walter Wu <walter...@mediatek.com>
Suggested-by: Dmitry Vyukov <dvy...@google.com>

Cc: Andrey Konovalov <andre...@google.com>

Cc: Andrey Ryabinin <ryabin...@gmail.com>
Cc: Dmitry Vyukov <dvy...@google.com>

Cc: Alexander Potapenko <gli...@google.com>
Cc: Andrew Morton <ak...@linux-foundation.org>
Cc: Matthias Brugger <matthi...@gmail.com>
Cc: Jens Axboe <ax...@kernel.dk>
Cc: Oleg Nesterov <ol...@redhat.com>
---

v2: Fix kasan_record_aux_stack() calling sequence issue.
Thanks for Dmitry's suggestion

---
kernel/task_work.c | 3 +++
mm/kasan/kasan.h | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/task_work.c b/kernel/task_work.c

index 9cde961875c0..3d4852891fa8 100644
--- a/kernel/task_work.c
+++ b/kernel/task_work.c

@@ -34,6 +34,9 @@ int task_work_add(struct task_struct *task, struct callback_head *work,
{
struct callback_head *head;

+ /* record the work call stack in order to print it in KASAN reports */
+ kasan_record_aux_stack(work);
+

do {
head = READ_ONCE(task->task_works);
if (unlikely(head == &work_exited))

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3436c6bf7c0c..e4629a971a3c 100644

--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -146,7 +146,7 @@ struct kasan_alloc_meta {
struct kasan_track alloc_track;
#ifdef CONFIG_KASAN_GENERIC
/*
- * call_rcu() call stack is stored into struct kasan_alloc_meta.

+ * The auxiliary stack is stored into struct kasan_alloc_meta.

[Dmitry Vyukov]

On Tue, Mar 16, 2021 at 3:44 AM Walter Wu <walter...@mediatek.com> wrote:
>
> Why record task_work_add() call stack?
> Syzbot reports many use-after-free issues for task_work, see [1].
> After see the free stack and the current auxiliary stack, we think
> they are useless, we don't know where register the work, this work
> may be the free call stack, so that we miss the root cause and
> don't solve the use-after-free.
>
> Add task_work_add() call stack into KASAN auxiliary stack in
> order to improve KASAN report. It is useful for programmers
> to solve use-after-free issues.
>
> [1]: https://groups.google.com/g/syzkaller-bugs/search?q=kasan%20use-after-free%20task_work_run
>
> Signed-off-by: Walter Wu <walter...@mediatek.com>
> Suggested-by: Dmitry Vyukov <dvy...@google.com>
> Cc: Andrey Konovalov <andre...@google.com>
> Cc: Andrey Ryabinin <ryabin...@gmail.com>
> Cc: Dmitry Vyukov <dvy...@google.com>
> Cc: Alexander Potapenko <gli...@google.com>
> Cc: Andrew Morton <ak...@linux-foundation.org>
> Cc: Matthias Brugger <matthi...@gmail.com>
> Cc: Jens Axboe <ax...@kernel.dk>
> Cc: Oleg Nesterov <ol...@redhat.com>
> ---
>
> v2: Fix kasan_record_aux_stack() calling sequence issue.
> Thanks for Dmitry's suggestion

Reviewed-by: Dmitry Vyukov <dvy...@google.com>

> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20210316024410.19967-1-walter-zh.wu%40mediatek.com.

[Jens Axboe]

On 3/15/21 8:44 PM, Walter Wu wrote:
> Why record task_work_add() call stack?
> Syzbot reports many use-after-free issues for task_work, see [1].
> After see the free stack and the current auxiliary stack, we think
> they are useless, we don't know where register the work, this work
> may be the free call stack, so that we miss the root cause and
> don't solve the use-after-free.
>
> Add task_work_add() call stack into KASAN auxiliary stack in
> order to improve KASAN report. It is useful for programmers
> to solve use-after-free issues.

I think this is a very useful addition, especially as task_work
proliferates.

Reviewed-by: Jens Axboe <ax...@kernel.dk>

--
Jens Axboe

[Oleg Nesterov]

On 03/16, Walter Wu wrote:
>
> --- a/kernel/task_work.c
> +++ b/kernel/task_work.c
> @@ -34,6 +34,9 @@ int task_work_add(struct task_struct *task, struct callback_head *work,
> {
> struct callback_head *head;
>
> + /* record the work call stack in order to print it in KASAN reports */
> + kasan_record_aux_stack(work);
> +
> do {
> head = READ_ONCE(task->task_works);
> if (unlikely(head == &work_exited))

Acked-by: Oleg Nesterov <ol...@redhat.com>

[Andrey Konovalov]

On Tue, Mar 16, 2021 at 3:44 AM Walter Wu <walter...@mediatek.com> wrote:
>

Acked-by: Andrey Konovalov <andre...@google.com>