[syzbot] Monthly trace report (Oct 2024)

[syzbot]

Hello trace maintainers/developers,

This is a 31-day syzbot report for the trace subsystem.
All related reports/information can be found at:
https://syzkaller.appspot.com/upstream/s/trace

During the period, 1 new issues were detected and 0 were fixed.
In total, 10 issues are still open and 38 have been fixed so far.

Some of the still happening issues:

Ref Crashes Repro Title
<1> 34 Yes INFO: task hung in blk_trace_ioctl (4)
https://syzkaller.appspot.com/bug?extid=ed812ed461471ab17a0c
<2> 32 Yes WARNING in bpf_get_stack_raw_tp
https://syzkaller.appspot.com/bug?extid=ce35de20ed6652f60652
<3> 13 Yes WARNING in get_probe_ref
https://syzkaller.appspot.com/bug?extid=8672dcb9d10011c0a160
<4> 6 Yes INFO: task hung in blk_trace_remove (2)
https://syzkaller.appspot.com/bug?extid=2373f6be3e6de4f92562
<5> 4 Yes possible deadlock in __mod_timer (4)
https://syzkaller.appspot.com/bug?extid=83a876aef81c9a485ae8

---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

To disable reminders for individual bugs, reply with the following command:
#syz set <Ref> no-reminders

To change bug's subsystems, reply with:
#syz set <Ref> subsystems: new-subsystem

You may send multiple commands in a single email message.

[Steven Rostedt]

On Fri, 11 Oct 2024 08:25:29 -0700
syzbot <syzbot+list3bf2...@syzkaller.appspotmail.com> wrote:

> Hello trace maintainers/developers,
>
> This is a 31-day syzbot report for the trace subsystem.
> All related reports/information can be found at:
> https://syzkaller.appspot.com/upstream/s/trace
>
> During the period, 1 new issues were detected and 0 were fixed.
> In total, 10 issues are still open and 38 have been fixed so far.
>
> Some of the still happening issues:
>
> Ref Crashes Repro Title
> <1> 34 Yes INFO: task hung in blk_trace_ioctl (4)
> https://syzkaller.appspot.com/bug?extid=ed812ed461471ab17a0c

If you check the maintainers file, blktrace.c has:

BLOCK LAYER
M: Jens Axboe <ax...@kernel.dk>
L: linux...@vger.kernel.org

> <2> 32 Yes WARNING in bpf_get_stack_raw_tp
> https://syzkaller.appspot.com/bug?extid=ce35de20ed6652f60652

bpf_trace.c has:

M: Alexei Starovoitov <a...@kernel.org>
M: Daniel Borkmann <dan...@iogearbox.net>
M: Andrii Nakryiko <and...@kernel.org>
R: Martin KaFai Lau <marti...@linux.dev>
R: Eduard Zingerman <edd...@gmail.com>
R: Song Liu <so...@kernel.org>
R: Yonghong Song <yongho...@linux.dev>
R: John Fastabend <john.fa...@gmail.com>
R: KP Singh <kps...@kernel.org>
R: Stanislav Fomichev <s...@fomichev.me>
R: Hao Luo <hao...@google.com>
R: Jiri Olsa <jo...@kernel.org>
L: b...@vger.kernel.org

> <3> 13 Yes WARNING in get_probe_ref
> https://syzkaller.appspot.com/bug?extid=8672dcb9d10011c0a160
> <4> 6 Yes INFO: task hung in blk_trace_remove (2)
> https://syzkaller.appspot.com/bug?extid=2373f6be3e6de4f92562
> <5> 4 Yes possible deadlock in __mod_timer (4)
> https://syzkaller.appspot.com/bug?extid=83a876aef81c9a485ae8

None of these look like they are tracing infrastructure related.

-- Steve

[Jens Axboe]

On 10/11/24 10:00 AM, Steven Rostedt wrote:
> On Fri, 11 Oct 2024 08:25:29 -0700
> syzbot <syzbot+list3bf2...@syzkaller.appspotmail.com> wrote:
>
>> Hello trace maintainers/developers,
>>
>> This is a 31-day syzbot report for the trace subsystem.
>> All related reports/information can be found at:
>> https://syzkaller.appspot.com/upstream/s/trace
>>
>> During the period, 1 new issues were detected and 0 were fixed.
>> In total, 10 issues are still open and 38 have been fixed so far.
>>
>> Some of the still happening issues:
>>
>> Ref Crashes Repro Title
>> <1> 34 Yes INFO: task hung in blk_trace_ioctl (4)
>> https://syzkaller.appspot.com/bug?extid=ed812ed461471ab17a0c
>
> If you check the maintainers file, blktrace.c has:
>
> BLOCK LAYER
> M: Jens Axboe <ax...@kernel.dk>
> L: linux...@vger.kernel.org

If syzbot can test this one, that would be appreciated. Guess here is
that we're stuck faulting under the debugfs_mutex. I took a look at the
syzbot reproducer, and no not going to attempt to run that... I strongly
suspect that the interesting bits there are:

1) Memory pressure/swap
2) blktrace setup/teardown, obviously

I do wish that once syzbot had a reproducer, it would continue
condensing it down into the most basic reproducer. Once you get into
"let's setup wifi, bluetooth, and tons of other things!" they become
almost impossible to run. And like in this case, I highly doubt they are
either related or useful.


diff --git a/kernel/trace/blktrace.c b/kernel/trace/blktrace.c
index 8fd292d34d89..cefcad120495 100644
--- a/kernel/trace/blktrace.c
+++ b/kernel/trace/blktrace.c
@@ -617,8 +617,8 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev,
return ret;
}

-static int __blk_trace_setup(struct request_queue *q, char *name, dev_t dev,
- struct block_device *bdev, char __user *arg)
+int blk_trace_setup(struct request_queue *q, char *name, dev_t dev,
+ struct block_device *bdev, char __user *arg)
{
struct blk_user_trace_setup buts;
int ret;
@@ -627,28 +627,18 @@ static int __blk_trace_setup(struct request_queue *q, char *name, dev_t dev,
if (ret)
return -EFAULT;

+ mutex_lock(&q->debugfs_mutex);
ret = do_blk_trace_setup(q, name, dev, bdev, &buts);
+ mutex_unlock(&q->debugfs_mutex);
if (ret)
return ret;

if (copy_to_user(arg, &buts, sizeof(buts))) {
- __blk_trace_remove(q);
+ blk_trace_remove(q);
return -EFAULT;
}
- return 0;
-}

-int blk_trace_setup(struct request_queue *q, char *name, dev_t dev,
- struct block_device *bdev,
- char __user *arg)
-{
- int ret;
-
- mutex_lock(&q->debugfs_mutex);
- ret = __blk_trace_setup(q, name, dev, bdev, arg);
- mutex_unlock(&q->debugfs_mutex);
-
- return ret;
+ return 0;
}
EXPORT_SYMBOL_GPL(blk_trace_setup);

@@ -673,12 +663,14 @@ static int compat_blk_trace_setup(struct request_queue *q, char *name,
.pid = cbuts.pid,
};

+ mutex_lock(&q->debugfs_mutex);
ret = do_blk_trace_setup(q, name, dev, bdev, &buts);
+ mutex_unlock(&q->debugfs_mutex);
if (ret)
return ret;

if (copy_to_user(arg, &buts.name, ARRAY_SIZE(buts.name))) {
- __blk_trace_remove(q);
+ blk_trace_remove(q);
return -EFAULT;
}

@@ -732,12 +724,10 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg)
int ret, start = 0;
char b[BDEVNAME_SIZE];

- mutex_lock(&q->debugfs_mutex);
-
switch (cmd) {
case BLKTRACESETUP:
snprintf(b, sizeof(b), "%pg", bdev);
- ret = __blk_trace_setup(q, b, bdev->bd_dev, bdev, arg);
+ ret = blk_trace_setup(q, b, bdev->bd_dev, bdev, arg);
break;
#if defined(CONFIG_COMPAT) && defined(CONFIG_X86_64)
case BLKTRACESETUP32:
@@ -749,17 +739,16 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg)
start = 1;
fallthrough;
case BLKTRACESTOP:
- ret = __blk_trace_startstop(q, start);
+ ret = blk_trace_startstop(q, start);
break;
case BLKTRACETEARDOWN:
- ret = __blk_trace_remove(q);
+ ret = blk_trace_remove(q);
break;
default:
ret = -ENOTTY;
break;
}

- mutex_unlock(&q->debugfs_mutex);
return ret;
}


--
Jens Axboe

[Aleksandr Nogikh]

Hi Steven,

Thanks for the analysis!

On Fri, Oct 11, 2024 at 6:00 PM Steven Rostedt <ros...@goodmis.org> wrote:
>
> On Fri, 11 Oct 2024 08:25:29 -0700
> syzbot <syzbot+list3bf2...@syzkaller.appspotmail.com> wrote:
>
> > Hello trace maintainers/developers,
> >
> > This is a 31-day syzbot report for the trace subsystem.
> > All related reports/information can be found at:
> > https://syzkaller.appspot.com/upstream/s/trace
> >
> > During the period, 1 new issues were detected and 0 were fixed.
> > In total, 10 issues are still open and 38 have been fixed so far.
> >
> > Some of the still happening issues:
> >
> > Ref Crashes Repro Title
> > <1> 34 Yes INFO: task hung in blk_trace_ioctl (4)
> > https://syzkaller.appspot.com/bug?extid=ed812ed461471ab17a0c
>
> If you check the maintainers file, blktrace.c has:
>
> BLOCK LAYER
> M: Jens Axboe <ax...@kernel.dk>
> L: linux...@vger.kernel.org

Judging by MAINTAINERS and ./scripts/get_maintainer.pl,
kernel/trace/blktrace.c belongs to both "BLOCK LAYER" and "TRACING".

$ ./scripts/get_maintainer.pl kernel/trace/blktrace.c
< ... >
linux...@vger.kernel.org (open list:BLOCK LAYER)
linux-...@vger.kernel.org (open list:TRACING)

>
>
>
> > <2> 32 Yes WARNING in bpf_get_stack_raw_tp
> > https://syzkaller.appspot.com/bug?extid=ce35de20ed6652f60652
>
> bpf_trace.c has:
>
> M: Alexei Starovoitov <a...@kernel.org>
> M: Daniel Borkmann <dan...@iogearbox.net>
> M: Andrii Nakryiko <and...@kernel.org>
> R: Martin KaFai Lau <marti...@linux.dev>
> R: Eduard Zingerman <edd...@gmail.com>
> R: Song Liu <so...@kernel.org>
> R: Yonghong Song <yongho...@linux.dev>
> R: John Fastabend <john.fa...@gmail.com>
> R: KP Singh <kps...@kernel.org>
> R: Stanislav Fomichev <s...@fomichev.me>
> R: Hao Luo <hao...@google.com>
> R: Jiri Olsa <jo...@kernel.org>
> L: b...@vger.kernel.org

Same for kernel/trace/bpf_trace.c:

$ ./scripts/get_maintainer.pl kernel/trace/bpf_trace.c
< ... >
Matt Bobrowski <mattbo...@google.com> (maintainer:BPF [SECURITY &
LSM] (Security Audit and Enforc...)
Steven Rostedt <ros...@goodmis.org> (maintainer:TRACING)

>
> > <3> 13 Yes WARNING in get_probe_ref
> > https://syzkaller.appspot.com/bug?extid=8672dcb9d10011c0a160
> > <4> 6 Yes INFO: task hung in blk_trace_remove (2)
> > https://syzkaller.appspot.com/bug?extid=2373f6be3e6de4f92562
> > <5> 4 Yes possible deadlock in __mod_timer (4)
> > https://syzkaller.appspot.com/bug?extid=83a876aef81c9a485ae8
>
> None of these look like they are tracing infrastructure related.

Like get_maintainer.pl, syzbot relies on the MAINTAINERS file to
attribute bugs to the individual kernel subsystems. If several ones
are suitable, the bug is assigned several labels at once. It's now
actually the case for all open "trace" findings:

https://syzkaller.appspot.com/upstream/s/trace

(FWIW it's also possible to manually overwrite these labels and remove
specific bugs from the monthly reports).

I could make syzbot set "trace" only if there's no other good
candidate, but I wonder if that could hide the findings in the trace
infrastructure that manifested themselves in some specific traced
subsystem.

--
Aleksandr

[Steven Rostedt]

On Tue, 15 Oct 2024 12:05:29 +0200
Aleksandr Nogikh <nog...@google.com> wrote:

> > None of these look like they are tracing infrastructure related.
>
> Like get_maintainer.pl, syzbot relies on the MAINTAINERS file to
> attribute bugs to the individual kernel subsystems. If several ones
> are suitable, the bug is assigned several labels at once. It's now
> actually the case for all open "trace" findings:
>
> https://syzkaller.appspot.com/upstream/s/trace
>
> (FWIW it's also possible to manually overwrite these labels and remove
> specific bugs from the monthly reports).
>
> I could make syzbot set "trace" only if there's no other good
> candidate, but I wonder if that could hide the findings in the trace
> infrastructure that manifested themselves in some specific traced
> subsystem.
>

I don't mind being Cc'd to these bugs. What I do mind is that only the
tracing maintainers are Cc'd. I still care about these, because they do
depend on the tracing code, and it could be the tracing infrastructure's
fault. But if an error is in a file that is explicitly called out in the
maintainers file, such as, blktrace.c and bpf_trace.c, then PLEASE also Cc
the maintainers of those files!

I had to manually add those maintainers when I replied to the initial
email. That is something I shouldn't need to do.

-- Steve