[syzbot] [bpf?] WARNING in push_jmp_history

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: c02d24a5af66 Add linux-next specific files for 20241003
git tree: linux-next
console+strace: https://syzkaller.appspot.com/x/log.txt?x=17382707980000
kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d
dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/641e642c9432/disk-c02d24a5.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/98aaf20c29e0/vmlinux-c02d24a5.xz
kernel image: https://storage.googleapis.com/syzbot-assets/c23099f2d86b/bzImage-c02d24a5.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+7e46cd...@syzkaller.appspotmail.com

------------[ cut here ]------------
virt_to_cache: Object is not a Slab page!
WARNING: CPU: 0 PID: 5232 at mm/slub.c:4655 virt_to_cache mm/slub.c:4655 [inline]
WARNING: CPU: 0 PID: 5232 at mm/slub.c:4655 __do_krealloc mm/slub.c:4753 [inline]
WARNING: CPU: 0 PID: 5232 at mm/slub.c:4655 krealloc_noprof+0x1b3/0x2e0 mm/slub.c:4838
Modules linked in:
CPU: 0 UID: 0 PID: 5232 Comm: syz-executor250 Not tainted 6.12.0-rc1-next-20241003-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:virt_to_cache mm/slub.c:4655 [inline]
RIP: 0010:__do_krealloc mm/slub.c:4753 [inline]
RIP: 0010:krealloc_noprof+0x1b3/0x2e0 mm/slub.c:4838
Code: 45 31 ff 45 31 f6 45 31 ed e9 21 ff ff ff c6 05 4e 2a 14 0e 01 90 48 c7 c7 24 f2 0b 8e 48 c7 c6 44 f2 0b 8e e8 3e 19 63 ff 90 <0f> 0b 90 90 e9 d9 fe ff ff f3 0f 1e fa 41 8b 45 08 f7 d0 a8 88 0f
RSP: 0018:ffffc90003c36ba8 EFLAGS: 00010246
RAX: 3f2bb101b90db800 RBX: 0000000000000000 RCX: ffff88802bb01e00
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff88807849c000 R08: ffffffff8155d412 R09: 1ffff110170c519a
R10: dffffc0000000000 R11: ffffed10170c519b R12: 0000000000004000
R13: 0000000000000201 R14: 0000000000100cc0 R15: dffffc0000000000
FS: 0000555587952380(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005594dac5c5d8 CR3: 00000000786d6000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
push_jmp_history+0x13c/0x5c0 kernel/bpf/verifier.c:3497
do_check+0x6716/0xfe40 kernel/bpf/verifier.c:18352
do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:21618
do_check_main kernel/bpf/verifier.c:21709 [inline]
bpf_check+0x18a25/0x1e320 kernel/bpf/verifier.c:22421
bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2846
__sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5634
__do_sys_bpf kernel/bpf/syscall.c:5741 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5739 [inline]
__x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5739
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fc05a9603e9
Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd106d44d8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ffd106d46b8 RCX: 00007fc05a9603e9
RDX: 0000000000000048 RSI: 00000000200054c0 RDI: 0000000000000005
RBP: 00007fc05a9d4610 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000001
R13: 00007ffd106d46a8 R14: 0000000000000001 R15: 0000000000000001
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Eduard Zingerman]

On Mon, 2024-10-07 at 11:35 -0700, syzbot wrote:
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: c02d24a5af66 Add linux-next specific files for 20241003
> git tree: linux-next
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=17382707980000
> kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d
> dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4
> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000

When I try this reproducer the bpf syscall never exits (waited for 5 minutes).
Here is the reproducer as a selftest:

SEC("kprobe")
__success
__naked void syzbot_bug(void)
{
asm volatile (
" r2 = *(u32 *)(r1 +140)\n" // 0
" r3 = *(u32 *)(r1 +76)\n" // 1
" r0 = r2\n" // 2
" if w0 > 0xffffff07 goto 1f\n" // 3
" if r3 <= r0 goto +16\n" // 4
" exit\n" // 5
"1: r6 = *(u16 *)(r1 +0)\n" // 6
" r7 = r6\n" // 7
"2: w7 += 447767737\n" // 8
" if w7 & 0x702000 goto 2b\n" // 9
" w7 &= 2024974\n" // 10
" r5 = r1\n" // 11
" r7 += r5\n" // 12
" if r7 s> 0x37d2 goto +0\n" // 13
" w3 *= w0\n" // 14
" r5 -= r7\n" // 15
" r4 = r5\n" // 16
" r0 += -458748\n" // 17
" if r3 < r4 goto 3f\n" // 18
" w0 >>= w0\n" // 19
"3: goto +0\n" // 20
" exit\n" // 21
::: __clobber_all);
}

(e.g. can be put to tools/testing/selftests/bpf/progs/verifier_and.c
or any other verifier_*.c).

Inserting a few printks shows that the following instructions are
verified in a loop:

... verification starts ...
[ 2.094272] do_check: env->insn_idx 0
[ 2.094345] do_check: env->insn_idx 1
[ 2.094417] do_check: env->insn_idx 2
[ 2.094486] do_check: env->insn_idx 3
[ 2.094585] do_check: env->insn_idx 4
[ 2.094675] do_check: env->insn_idx 5
[ 2.094748] do_check: env->insn_idx 21
[ 2.094879] do_check: env->insn_idx 6
[ 2.095005] do_check: env->insn_idx 7
[ 2.095074] do_check: env->insn_idx 8 <------ let's call this point LBL
[ 2.095156] do_check: env->insn_idx 9
[ 2.095264] do_check: env->insn_idx 8
[ 2.095372] do_check: env->insn_idx 9
[ 2.095497] do_check: env->insn_idx 8
[ 2.095579] do_check: env->insn_idx 9
[ 2.095716] do_check: env->insn_idx 8
[ 2.095892] do_check: env->insn_idx 9
[ 2.096070] do_check: env->insn_idx 8
[ 2.096151] do_check: env->insn_idx 9
[ 2.096314] do_check: env->insn_idx 8
[ 2.096402] do_check: env->insn_idx 9
[ 2.096570] do_check: env->insn_idx 8
[ 2.096646] do_check: env->insn_idx 9
[ 2.096840] do_check: env->insn_idx 8
[ 2.096921] do_check: env->insn_idx 9
[ 2.097040] do_check: env->insn_idx 10
[ 2.097113] do_check: env->insn_idx 11
[ 2.097195] do_check: env->insn_idx 12
[ 2.097417] do_check: env->insn_idx 13
[ 2.097521] do_check: env->insn_idx 14
[ 2.097597] do_check: env->insn_idx 15
[ 2.097688] do_check: env->insn_idx 16
[ 2.097774] do_check: env->insn_idx 17
[ 2.097866] do_check: env->insn_idx 18
[ 2.097990] do_check: env->insn_idx 19
[ 2.098050] do_check: env->insn_idx 20
[ 2.098119] do_check: env->insn_idx 21
[ 2.098195] do_check: env->insn_idx 20
[ 2.098347] do_check: env->insn_idx 21
[ 2.098414] do_check: env->insn_idx 14
[ 2.098556] do_check: env->insn_idx 15
[ 2.098629] do_check: env->insn_idx 16
[ 2.098700] do_check: env->insn_idx 17
[ 2.098767] do_check: env->insn_idx 18
[ 2.098842] do_check: env->insn_idx 8
[ 2.098984] do_check: env->insn_idx 9
[ 2.099108] do_check: env->insn_idx 8
[ 2.099171] do_check: env->insn_idx 9
[ 2.099304] do_check: env->insn_idx 8
[ 2.099368] do_check: env->insn_idx 9
[ 2.099505] do_check: env->insn_idx 8
[ 2.099568] do_check: env->insn_idx 9
[ 2.099703] do_check: env->insn_idx 8
[ 2.099774] do_check: env->insn_idx 9
[ 2.099921] do_check: env->insn_idx 8
[ 2.099984] do_check: env->insn_idx 9
[ 2.100133] do_check: env->insn_idx 8
[ 2.100200] do_check: env->insn_idx 9
[ 2.100349] do_check: env->insn_idx 8
[ 2.100413] do_check: env->insn_idx 9
[ 2.100503] do_check: env->insn_idx 10
[ 2.100566] do_check: env->insn_idx 11
[ 2.100636] do_check: env->insn_idx 12
[ 2.100813] do_check: env->insn_idx 13
[ 2.100909] do_check: env->insn_idx 14
[ 2.100972] do_check: env->insn_idx 15
[ 2.101047] do_check: env->insn_idx 16
[ 2.101117] do_check: env->insn_idx 17
[ 2.101185] do_check: env->insn_idx 18
[ 2.101250] do_check: env->insn_idx 14
[ 2.101389] do_check: env->insn_idx 15
[ 2.101462] do_check: env->insn_idx 16
[ 2.101531] do_check: env->insn_idx 17
[ 2.101598] do_check: env->insn_idx 18

... verification repeats from LBL ...

[...]

[syzbot]

syzbot has bisected this issue to:

commit d0a38fad51cc70ab3dd3c59b54d8079ac19220b9
Author: Feng Tang <feng...@intel.com>
Date: Wed Sep 11 06:45:34 2024 +0000

mm/slub: Improve redzone check and zeroing for krealloc()

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11ddbb80580000
start commit: c02d24a5af66 Add linux-next specific files for 20241003
git tree: linux-next
final oops: https://syzkaller.appspot.com/x/report.txt?x=13ddbb80580000
console output: https://syzkaller.appspot.com/x/log.txt?x=15ddbb80580000

kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d
dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000

Reported-by: syzbot+7e46cd...@syzkaller.appspotmail.com
Fixes: d0a38fad51cc ("mm/slub: Improve redzone check and zeroing for krealloc()")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Eduard Zingerman]

There are two issues demonstrated by this repro:
- one is mm/slub related;
- another one is verification taking forever.

About the mm/slub related. Applying the following patch with
additional logging on top of commit d0a38fad51cc identified by syzbot:

------- 8< ------------------------------------------------------------
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9a7ed527e47e..c1582a6d1d33 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3494,7 +3494,9 @@ static int push_jmp_history(struct bpf_verifier_env *env, struct bpf_verifier_st

cnt++;
alloc_size = kmalloc_size_roundup(size_mul(cnt, sizeof(*p)));
+ printk("push_jmp_history: #1 cur->jmp_history=%p\n", cur->jmp_history);
p = krealloc(cur->jmp_history, alloc_size, GFP_USER);
+ printk("push_jmp_history: #2 cur->jmp_history=%p\n", p);
if (!p)
return -ENOMEM;
cur->jmp_history = p;
diff --git a/mm/slub.c b/mm/slub.c
index e0fb0a26c796..3f5b080ac1f5 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -4627,7 +4627,7 @@ static inline struct kmem_cache *virt_to_cache(const void *obj)
struct slab *slab;

slab = virt_to_slab(obj);
- if (WARN_ONCE(!slab, "%s: Object is not a Slab page!\n", __func__))
+ if (WARN_ONCE(!slab, "%s: Object %p is not a Slab page!\n", __func__, obj))
return NULL;
return slab->slab_cache;
}
------------------------------------------------------------ >8 -------

Produces the following log:

l1: [ 2.942120] push_jmp_history: #2 cur->jmp_history=00000000a0f6f503
l2: [ 2.944445] push_jmp_history: #1 cur->jmp_history=00000000a0f6f503
l3: [ 2.944560] ------------[ cut here ]------------
l4: [ 2.944647] virt_to_cache: Object 00000000a0f6f503 is not a Slab page!
l5: [ 2.944765] WARNING: CPU: 0 PID: 145 at mm/slub.c:4630 krealloc_noprof (mm/slub.c:4630 mm/slub.c:4728 mm/slub.c:4813)
l6: [ 2.944906] Modules linked in:
l7: [ 2.945134] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-2.fc40 04/01/2014
l8: [ 2.945285] RIP: 0010:krealloc_noprof (mm/slub.c:4630 mm/slub.c:4728 mm/slub.c:4813)
...
l9: [ 2.952088] BUG: kernel NULL pointer dereference, address: 000000000000001c
l10: [ 2.952171] #PF: supervisor read access in kernel mode
l11: [ 2.952240] #PF: error_code(0x0000) - not-present page
l12: [ 2.952309] PGD 105d51067 P4D 105d51067 PUD 1013d0067 PMD 0
l13: [ 2.952402] Oops: Oops: 0000 [#1] PREEMPT SMP KASAN NOPTI
l14: [ 2.952611] Tainted: [W]=WARN
l15: [ 2.952664] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-2.fc40 04/01/2014
l16: [ 2.952794] RIP: 0010:krealloc_noprof (mm/slub.c:0 mm/slub.c:4729 mm/slub.c:4813)

Lines l{1,2,4} show that address 0xa0f6f503 was first allocated by
krealloc and then krealloc failed to recognize it as such.

The warning at l3 is reported by virt_to_cache() called from
__do_krealloc():


4715 static __always_inline __realloc_size(2) void *
4716 __do_krealloc(const void *p, size_t new_size, gfp_t flags)
4717 {
4718 void *ret;
4719 size_t ks;
4720 int orig_size = 0;
4721 struct kmem_cache *s;
4722
4723 /* Check for double-free. */
4724 if (likely(!ZERO_OR_NULL_PTR(p))) {
4725 if (!kasan_check_byte(p))
4726 return NULL;
4727
4728 s = virt_to_cache(p);
4729 orig_size = get_orig_size(s, (void *)p);
4730 ks = s->object_size;

When virt_to_cache() reports the warning it returns NULL.
Hence variable 's' at line 4728 is NULL and this causes null pointer
dereference at line 4730, reported at l9.

Lines 4725-4730 were changed by commit d0a38fad51cc identified by syzbot,
previously 'ks' was identified using other means.

Feng, this issue seem unrelated to BPF verifier, could you please take a look?

Best regards,
Eduard

[Vlastimil Babka]

On 10/8/24 11:41, Eduard Zingerman wrote:
> On Tue, 2024-10-08 at 01:43 -0700, syzbot wrote:
>> syzbot has bisected this issue to:
>>
>> commit d0a38fad51cc70ab3dd3c59b54d8079ac19220b9
>> Author: Feng Tang <feng...@intel.com>
>> Date: Wed Sep 11 06:45:34 2024 +0000
>>
>> mm/slub: Improve redzone check and zeroing for krealloc()
>>
>> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11ddbb80580000
>> start commit: c02d24a5af66 Add linux-next specific files for 20241003
>> git tree: linux-next
>> final oops: https://syzkaller.appspot.com/x/report.txt?x=13ddbb80580000
>> console output: https://syzkaller.appspot.com/x/log.txt?x=15ddbb80580000
>> kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d
>> dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4
>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000
>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000
>>
>> Reported-by: syzbot+7e46cd...@syzkaller.appspotmail.com
>> Fixes: d0a38fad51cc ("mm/slub: Improve redzone check and zeroing for krealloc()")
>>
>> For information about bisection process see: https://goo.gl/tpsmEJ#bisection
>
> There are two issues demonstrated by this repro:
> - one is mm/slub related;
> - another one is verification taking forever.
>
> About the mm/slub related. Applying the following patch with
> additional logging on top of commit d0a38fad51cc identified by syzbot:

The slab one is known from other reports and the problematic commit was
removed from -next since then.