[syzbot] upstream boot error: BUG: unable to handle kernel NULL pointer dereference in __dabt_svc

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: de10553fce40 Merge tag 'x86-apic-2023-04-24' of git://git...
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=14bdae68280000
kernel config: https://syzkaller.appspot.com/x/.config?x=975b8311f6b96bca
dashboard link: https://syzkaller.appspot.com/bug?extid=d692037148a8169fc9dd
compiler: arm-linux-gnueabi-gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+d69203...@syzkaller.appspotmail.com

8<--- cut here ---
Unable to handle kernel NULL pointer dereference at virtual address 000005fc when read
[000005fc] *pgd=80000080004003, *pmd=00000000
Internal error: Oops: 206 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.3.0-syzkaller #0
Hardware name: ARM-Versatile Express
Insufficient stack space to handle exception!
Task stack: [0xdf85c000..0xdf85e000]
IRQ stack: [0xdf804000..0xdf806000]
Overflow stack: [0x828ae000..0x828af000]
Internal error: kernel stack overflow: 0 [#2] PREEMPT SMP ARM
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.3.0-syzkaller #0
Hardware name: ARM-Versatile Express
PC is at __dabt_svc+0x14/0x60 arch/arm/kernel/entry-armv.S:210
LR is at vsnprintf+0x378/0x408 lib/vsprintf.c:2862
pc : [<80200a74>] lr : [<817ad5d8>] psr: 00000193
sp : df804028 ip : df805868 fp : df805864
r10: 00000060 r9 : ffffffff r8 : 00000010
r7 : 00000020 r6 : 00000004 r5 : ffffffff r4 : df805960
r3 : ffffffff r2 : 00000040 r1 : ffffffff r0 : 8264d250
Flags: nzcv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
Control: 30c5387d Table: 80003000 DAC: 00000000
Register r0 information:
8<--- cut here ---
Unable to handle kernel NULL pointer dereference at virtual address 000001ff when read
[000001ff] *pgd=80000080004003, *pmd=00000000
Internal error: Oops: 206 [#3] PREEMPT SMP ARM
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.3.0-syzkaller #0
Hardware name: ARM-Versatile Express
PC is at __find_vmap_area mm/vmalloc.c:841 [inline]
PC is at find_vmap_area mm/vmalloc.c:1862 [inline]
PC is at find_vm_area mm/vmalloc.c:2571 [inline]
PC is at vmalloc_dump_obj+0x38/0xb4 mm/vmalloc.c:4108
LR is at __raw_spin_lock include/linux/spinlock_api_smp.h:132 [inline]
LR is at _raw_spin_lock+0x18/0x58 kernel/locking/spinlock.c:154
pc : [<8046b2f0>] lr : [<817db0f4>] psr: 20000193
sp : 828aeef8 ip : 828aeee0 fp : 828aef0c
r10: 828f3980 r9 : 8241c964 r8 : 8264d41c
r7 : 60000193 r6 : 00000001 r5 : 8264e000 r4 : 00000207
r3 : 80216bd4 r2 : 00001d4c r1 : 00000000 r0 : 00000001
Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
Control: 30c5387d Table: 80003000 DAC: 00000000


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[Miguel Ojeda]

Hi syzbot engineers,

On Tue, Apr 25, 2023 at 10:06 PM syzbot
<syzbot+d69203...@syzkaller.appspotmail.com> wrote:
>
> HEAD commit: de10553fce40 Merge tag 'x86-apic-2023-04-24' of git://git...
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=14bdae68280000
> kernel config: https://syzkaller.appspot.com/x/.config?x=975b8311f6b96bca
> dashboard link: https://syzkaller.appspot.com/bug?extid=d692037148a8169fc9dd
> compiler: arm-linux-gnueabi-gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> userspace arch: arm

I am not sure what triggered the bot to consider Rust here -- the
config does not enable it.

What am I missing?

Cheers,
Miguel

[Sergey Senozhatsky]

On (23/04/25 13:06), syzbot wrote:
> 8<--- cut here ---
> Unable to handle kernel NULL pointer dereference at virtual address 000005fc when read
> [000005fc] *pgd=80000080004003, *pmd=00000000
> Internal error: Oops: 206 [#1] PREEMPT SMP ARM
> Modules linked in:
> CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.3.0-syzkaller #0
> Hardware name: ARM-Versatile Express

> Insufficient stack space to handle exception!

So much stuff is going on there.

Not sure if I can make sense out of this.

[Dmitry Vyukov]

Hi Miguel,

The crash is in lib/vsprintf.c and:

$ scripts/get_maintainer.pl -f lib/vsprintf.c
...
rust-fo...@vger.kernel.org (open list:RUST)
...

[Dmitry Vyukov]

+linux-arm-kernel@

I suspect this is some recent arch/arm related corruption.
There are also these similar boot crashes that started happening at
roughly the same time:
https://syzkaller.appspot.com/bug?id=4d697346183db2f86ba2f76acb7d66e7731f88df
https://syzkaller.appspot.com/bug?id=dcd98d67539fe4d0d28d2e655e510569eda6f4de

[Miguel Ojeda]

Hi Dmitry,

On Wed, Apr 26, 2023 at 8:34 AM Dmitry Vyukov <dvy...@google.com> wrote:
>
> The crash is in lib/vsprintf.c and:
>
> $ scripts/get_maintainer.pl -f lib/vsprintf.c
> ...
> rust-fo...@vger.kernel.org (open list:RUST)
> ...

Ah, yes, thanks!

For the moment it is fine, since there are not many reports nor
keyword instances, but perhaps in the future we could consider
filtering out `RUST` on the bot side if `CONFIG_RUST=n` and the match
was based on `K:` (via diff with `--no-keywords`?).

Cheers,
Miguel

[Dmitry Vyukov]

On Wed, 26 Apr 2023 at 12:09, Miguel Ojeda
<miguel.oje...@gmail.com> wrote:
>
> Hi Dmitry,
>
> On Wed, Apr 26, 2023 at 8:34 AM Dmitry Vyukov <dvy...@google.com> wrote:
> >
> > The crash is in lib/vsprintf.c and:
> >
> > $ scripts/get_maintainer.pl -f lib/vsprintf.c
> > ...
> > rust-fo...@vger.kernel.org (open list:RUST)
> > ...
>
> Ah, yes, thanks!
>
> For the moment it is fine, since there are not many reports nor
> keyword instances, but perhaps in the future we could consider

In which of the dozens of kernel testing systems? ;)
And also in heads of thousands of kernel developers and users?
All of them use get_maintainer.pl.

[Miguel Ojeda]

On Wed, Apr 26, 2023 at 12:12 PM Dmitry Vyukov <dvy...@google.com> wrote:
>
> In which of the dozens of kernel testing systems? ;)
> And also in heads of thousands of kernel developers and users?
> All of them use get_maintainer.pl.

I am aware, but `get_maintainer.pl` is fine as it is -- we still want
to know about things that touch things that mention Rust in general,
so that we can possibly be helpful to others, especially early on.

However, if a bot is testing the kernel with Rust actually disabled at
runtime, what I am saying is that the chance that it has something to
do with Rust is quite low, especially if matched via `K:` rather than
`F:`. Thus my request.

Now, it could be nice to have some logic like that in
`get_maintainer.pl` encoded for all bots to filter things out based on
the kernel config and the type of match; but otherwise, yes, the bots
would need to add the logic.

Cc'ing Joe in case this is already possible in `get_maintainer.pl` or
whether there could be a better approach.

Cheers,
Miguel

[Dmitry Vyukov]

I understand your intentions and they make sense.
But adding this logic to syzbot won't help thousands of users of
get_maintainer.pl and dozens of other testing systems. There also will
be a bit of get_maintainer.pl inside of syzbot code, so now all kernel
developers will need to be aware of it and also submit changes to
syzbot when they want to change maintainers logic.

I think this also equally applies to all other users of K:.
And a number of them had similar complaints re how K; works.

I am thinking if K: should actually apply just to patches and be
ignored for source files?
If there are files that belong to "rust" (or "bpf" or any other user
of K:), then I think these should be just listed explicitly in the
subsystem (that should be a limited set of files that can be
enumerated with wildcards).
It's also reasonable to apply K: to patches.
But if a random source file happened to mention "rust" somewhere once,
I am not sure you want to be CCed on all issues in that file.
Does it sound reasonable?

[Miguel Ojeda]

On Wed, Apr 26, 2023 at 12:43 PM Dmitry Vyukov <dvy...@google.com> wrote:
>
> I understand your intentions and they make sense.

Thanks! I am glad you agree it may have some value -- please see below
for details.

> But adding this logic to syzbot won't help thousands of users of
> get_maintainer.pl and dozens of other testing systems. There also will

I haven't said otherwise -- as I said, I think it would be nice to
have this be part of the kernel itself. :)

> be a bit of get_maintainer.pl inside of syzbot code, so now all kernel
> developers will need to be aware of it and also submit changes to
> syzbot when they want to change maintainers logic.
>
> I think this also equally applies to all other users of K:.
> And a number of them had similar complaints re how K; works.

Yeah, I would imagine so.

> I am thinking if K: should actually apply just to patches and be
> ignored for source files?

I considered that -- for things like Rust, it could make sense, but
perhaps somebody is already using `K:` to match files they do care
about, rather than `F:`. So we would need to ask others, but I think
it is fine.

> If there are files that belong to "rust" (or "bpf" or any other user
> of K:), then I think these should be just listed explicitly in the
> subsystem (that should be a limited set of files that can be
> enumerated with wildcards).

Yes, at least for Rust, modulo omissions, we match files explicitly
with `F:`. We have a couple unimportant omissions, e.g.
`.rustfmt.toml`, but I can send a patch.

Personally, I have always seen `F:` files (and `N:`-matched ones) as
having more weight than `K:`-matched ones, i.e. I saw `K:` as more of
a "it depends on what it matches -- discretion needed".

From a quick look, most `K:`-using subsystems seem to list `F:` and
`N:` as I would expect.

> It's also reasonable to apply K: to patches.

Yes, definitely, for Rust, that is our main use case, i.e. it is
mainly why we wanted to have the `K:` entry: to catch changes to
things that are tagged with "Rust" in C files (early on, at least).

It is particularly important for us, since we are also considering
having more of these annotations in the future.

> But if a random source file happened to mention "rust" somewhere once,
> I am not sure you want to be CCed on all issues in that file.
> Does it sound reasonable?

For Rust, yes, that would probably work for us. Not sure for all
subsystems using `K:`, though.

Having said that, I suggested including the kernel config too in this
decision (i.e. not for the patches case, but for testers finding
runtime issues), because it adds information: it leaves reports out
when something is not even enabled but matched via `K:`, but still
allows a Cc when matched via `K:` and enabled. It is, of course, still
potentially a false positive, but some subsystems may want to hear
about those.

For instance, for Rust, this would be fine early on, since we don't
expect many to have `RUST=y` to begin with, and thus the odd false
positive report via `K:` is fine. Later on, this heuristic may change,
and we may not change those matches anymore (especially since, by
then, the goal is that subsystems would be taking care of their own
Rust bits).

This is what I was suggesting to then put in `get_maintainer.pl`, e.g.
a `--bot` option (or `--runtime`, or `--config-based-filtering`, or
similar) option. Then the bots can add that option on their side.

Thanks again for considering this!

Cheers,
Miguel

[syzbot]

Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.