[syzbot] [net?] general protection fault in put_page (4)

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 78d4f34e2115 Linux 6.13-rc3
git tree: upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=16445730580000
kernel config: https://syzkaller.appspot.com/x/.config?x=6c532525a32eb57d
dashboard link: https://syzkaller.appspot.com/bug?extid=38a095a81f30d82884c1
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=169b0b44580000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13f502df980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/7129ee07f8aa/disk-78d4f34e.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/c23c0af59a16/vmlinux-78d4f34e.xz
kernel image: https://storage.googleapis.com/syzbot-assets/031aecf04ea7/bzImage-78d4f34e.xz

The issue was bisected to:

commit b83fbca1b4c9c45628aa55d582c14825b0e71c2b
Author: Matthieu Baerts (NGI0) <mat...@kernel.org>
Date: Mon Sep 2 10:45:53 2024 +0000

mptcp: pm: reduce entries iterations on connect

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=163682df980000
final oops: https://syzkaller.appspot.com/x/report.txt?x=153682df980000
console output: https://syzkaller.appspot.com/x/log.txt?x=113682df980000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+38a095...@syzkaller.appspotmail.com
Fixes: b83fbca1b4c9 ("mptcp: pm: reduce entries iterations on connect")

Oops: general protection fault, probably for non-canonical address 0xdffffc0000000001: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000008-0x000000000000000f]
CPU: 1 UID: 0 PID: 5836 Comm: sshd Not tainted 6.13.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/25/2024
RIP: 0010:_compound_head include/linux/page-flags.h:242 [inline]
RIP: 0010:put_page+0x23/0x260 include/linux/mm.h:1552
Code: 90 90 90 90 90 90 90 55 41 57 41 56 53 49 89 fe 48 bd 00 00 00 00 00 fc ff df e8 f8 5e 12 f8 49 8d 5e 08 48 89 d8 48 c1 e8 03 <80> 3c 28 00 74 08 48 89 df e8 8f c7 78 f8 48 8b 1b 48 89 de 48 83
RSP: 0000:ffffc90003916c90 EFLAGS: 00010202
RAX: 0000000000000001 RBX: 0000000000000008 RCX: ffff888030458000
RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000
RBP: dffffc0000000000 R08: ffffffff898ca81d R09: 1ffff110054414ac
R10: dffffc0000000000 R11: ffffed10054414ad R12: 0000000000000007
R13: ffff88802a20a542 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f34f496e800(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9d6ec9ec28 CR3: 000000004d260000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
skb_page_unref include/linux/skbuff_ref.h:43 [inline]
__skb_frag_unref include/linux/skbuff_ref.h:56 [inline]
skb_release_data+0x483/0x8a0 net/core/skbuff.c:1119
skb_release_all net/core/skbuff.c:1190 [inline]
__kfree_skb+0x55/0x70 net/core/skbuff.c:1204
tcp_clean_rtx_queue net/ipv4/tcp_input.c:3436 [inline]
tcp_ack+0x2442/0x6bc0 net/ipv4/tcp_input.c:4032
tcp_rcv_state_process+0x8eb/0x44e0 net/ipv4/tcp_input.c:6805
tcp_v4_do_rcv+0x77d/0xc70 net/ipv4/tcp_ipv4.c:1939
tcp_v4_rcv+0x2dc0/0x37f0 net/ipv4/tcp_ipv4.c:2351
ip_protocol_deliver_rcu+0x22e/0x440 net/ipv4/ip_input.c:205
ip_local_deliver_finish+0x341/0x5f0 net/ipv4/ip_input.c:233
NF_HOOK+0x3a4/0x450 include/linux/netfilter.h:314
NF_HOOK+0x3a4/0x450 include/linux/netfilter.h:314
__netif_receive_skb_one_core net/core/dev.c:5672 [inline]
__netif_receive_skb+0x2bf/0x650 net/core/dev.c:5785
process_backlog+0x662/0x15b0 net/core/dev.c:6117
__napi_poll+0xcb/0x490 net/core/dev.c:6883
napi_poll net/core/dev.c:6952 [inline]
net_rx_action+0x89b/0x1240 net/core/dev.c:7074
handle_softirqs+0x2d4/0x9b0 kernel/softirq.c:561
__do_softirq kernel/softirq.c:595 [inline]
invoke_softirq kernel/softirq.c:435 [inline]
__irq_exit_rcu+0xf7/0x220 kernel/softirq.c:662
irq_exit_rcu+0x9/0x30 kernel/softirq.c:678
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1049 [inline]
sysvec_apic_timer_interrupt+0x57/0xc0 arch/x86/kernel/apic/apic.c:1049
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0033:0x7f34f4519ad5
Code: 85 d2 74 0d 0f 10 02 48 8d 54 24 20 0f 11 44 24 20 64 8b 04 25 18 00 00 00 85 c0 75 27 41 b8 08 00 00 00 b8 0f 01 00 00 0f 05 <48> 3d 00 f0 ff ff 76 75 48 8b 15 24 73 0d 00 f7 d8 64 89 02 48 83
RSP: 002b:00007ffec5b32ce0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: 00000000000668a0 RCX: 00007f34f4519ad5
RDX: 00007ffec5b32d00 RSI: 0000000000000004 RDI: 0000564f4bc6cae0
RBP: 0000564f4bc6b5a0 R08: 0000000000000008 R09: 0000000000000000
R10: 00007ffec5b32de8 R11: 0000000000000246 R12: 0000564f48ea8aa4
R13: 0000000000000001 R14: 0000564f48ea93e8 R15: 00007ffec5b32d68
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:_compound_head include/linux/page-flags.h:242 [inline]
RIP: 0010:put_page+0x23/0x260 include/linux/mm.h:1552
Code: 90 90 90 90 90 90 90 55 41 57 41 56 53 49 89 fe 48 bd 00 00 00 00 00 fc ff df e8 f8 5e 12 f8 49 8d 5e 08 48 89 d8 48 c1 e8 03 <80> 3c 28 00 74 08 48 89 df e8 8f c7 78 f8 48 8b 1b 48 89 de 48 83
RSP: 0000:ffffc90003916c90 EFLAGS: 00010202
RAX: 0000000000000001 RBX: 0000000000000008 RCX: ffff888030458000
RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000
RBP: dffffc0000000000 R08: ffffffff898ca81d R09: 1ffff110054414ac
R10: dffffc0000000000 R11: ffffed10054414ad R12: 0000000000000007
R13: ffff88802a20a542 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f34f496e800(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9d6ec9ec28 CR3: 000000004d260000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
----------------
Code disassembly (best guess):
0: 90 nop
1: 90 nop
2: 90 nop
3: 90 nop
4: 90 nop
5: 90 nop
6: 90 nop
7: 55 push %rbp
8: 41 57 push %r15
a: 41 56 push %r14
c: 53 push %rbx
d: 49 89 fe mov %rdi,%r14
10: 48 bd 00 00 00 00 00 movabs $0xdffffc0000000000,%rbp
17: fc ff df
1a: e8 f8 5e 12 f8 call 0xf8125f17
1f: 49 8d 5e 08 lea 0x8(%r14),%rbx
23: 48 89 d8 mov %rbx,%rax
26: 48 c1 e8 03 shr $0x3,%rax
* 2a: 80 3c 28 00 cmpb $0x0,(%rax,%rbp,1) <-- trapping instruction
2e: 74 08 je 0x38
30: 48 89 df mov %rbx,%rdi
33: e8 8f c7 78 f8 call 0xf878c7c7
38: 48 8b 1b mov (%rbx),%rbx
3b: 48 89 de mov %rbx,%rsi
3e: 48 rex.W
3f: 83 .byte 0x83


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Eric Dumazet]

I spent some time on this bug before releasing it, because I have
other syzbot reports probably
caused by the same issue, hinting at shinfo->nr_frags corruption.

I will hold these reports to avoid flooding the mailing list.

[Hillf Danton]

On Tue, Dec 17, 2024 at 6:03 PM

> syzbot found the following issue on:
>
> HEAD commit: 78d4f34e2115 Linux 6.13-rc3
> git tree: upstream

> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13f502df980000

#syz test

--- x/net/core/skbuff.c
+++ y/net/core/skbuff.c
@@ -1105,7 +1105,9 @@ static void skb_release_data(struct sk_b
int i;

if (!skb_data_unref(skb, shinfo))
- goto exit;
+ return;
+ if (test_and_set_bit(7, &shinfo->flags))
+ return;

if (skb_zcopy(skb)) {
bool skip_unref = shinfo->flags & SKBFL_MANAGED_FRAG_REFS;
--

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

net/core/skbuff.c:1109:26: error: incompatible pointer types passing '__u8 *' (aka 'unsigned char *') to parameter of type 'volatile unsigned long *' [-Werror,-Wincompatible-pointer-types]


Tested on:

commit: aef25be3 hexagon: Disable constant extender optimizati..
git tree: upstream

kernel config: https://syzkaller.appspot.com/x/.config?x=6c532525a32eb57d
dashboard link: https://syzkaller.appspot.com/bug?extid=38a095a81f30d82884c1
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=11fef60f980000

[Matthieu Baerts]

Hi Eric,

(...)

> I spent some time on this bug before releasing it, because I have
> other syzbot reports probably
> caused by the same issue, hinting at shinfo->nr_frags corruption.
>
> I will hold these reports to avoid flooding the mailing list.

Thank you for having released this bug report!

The bisected commit looks unrelated. I don't know if we can tell syzbot
to "skip this commit and try harder".

I'm trying to run a 'git bisect' on my side since this morning: the
issue seems to be older, between v6.10 and v6.11 if I'm not mistaken.
When using the same kernel config, I'm getting quite a few issues on
older commits (compilation, other warnings, etc.), plus the compilation
is slow on my laptop. I will update you if I can find anything useful.

Cheers,
Matt
--
Sponsored by the NGI0 Core fund.

[Hillf Danton]

On Tue, Dec 17, 2024 at 6:03 PM

> syzbot found the following issue on:
>
> HEAD commit: 78d4f34e2115 Linux 6.13-rc3
> git tree: upstream

> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13f502df980000

#syz test

--- x/net/core/skbuff.c
+++ y/net/core/skbuff.c

@@ -1099,13 +1099,22 @@ static void skb_free_head(struct sk_buff
}
}

+static DEFINE_SPINLOCK(skb_release_data_lock);
static void skb_release_data(struct sk_buff *skb, enum skb_drop_reason reason)
{
struct skb_shared_info *shinfo = skb_shinfo(skb);
int i;
+ unsigned long flg;

if (!skb_data_unref(skb, shinfo))
- goto exit;
+ return;

+ spin_lock_irqsave(&skb_release_data_lock, flg);
+ if ((1 << 6) & shinfo->flags) {
+ spin_unlock_irqrestore(&skb_release_data_lock, flg);
+ return;
+ }
+ shinfo->flags |= (1 << 6);
+ spin_unlock_irqrestore(&skb_release_data_lock, flg);

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
general protection fault in put_page

Oops: general protection fault, probably for non-canonical address 0xdffffc0000000001: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000008-0x000000000000000f]

CPU: 0 UID: 0 PID: 5976 Comm: kworker/0:4 Not tainted 6.13.0-rc3-syzkaller-00073-geabcdba3ad40-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/25/2024

Workqueue: rcu_gp srcu_invoke_callbacks

RIP: 0010:_compound_head include/linux/page-flags.h:242 [inline]
RIP: 0010:put_page+0x23/0x260 include/linux/mm.h:1552

Code: 90 90 90 90 90 90 90 55 41 57 41 56 53 49 89 fe 48 bd 00 00 00 00 00 fc ff df e8 98 5b 12 f8 49 8d 5e 08 48 89 d8 48 c1 e8 03 <80> 3c 28 00 74 08 48 89 df e8 3f bb 78 f8 48 8b 1b 48 89 de 48 83
RSP: 0018:ffffc90000006df0 EFLAGS: 00010202
RAX: 0000000000000001 RBX: 0000000000000008 RCX: ffff888027620000

RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000

RBP: dffffc0000000000 R08: ffffffff898caa7a R09: 1ffffffff2854734
R10: dffffc0000000000 R11: fffffbfff2854735 R12: 0000000000000007
R13: ffff88807a1efa42 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000001b3045ffff CR3: 000000007a6ec000 CR4: 00000000003526f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:

<IRQ>

skb_page_unref include/linux/skbuff_ref.h:43 [inline]
__skb_frag_unref include/linux/skbuff_ref.h:56 [inline]

skb_release_data+0x480/0x9a0 net/core/skbuff.c:1128
skb_release_all net/core/skbuff.c:1199 [inline]
__kfree_skb+0x55/0x70 net/core/skbuff.c:1213

tcp_clean_rtx_queue net/ipv4/tcp_input.c:3436 [inline]
tcp_ack+0x2442/0x6bc0 net/ipv4/tcp_input.c:4032
tcp_rcv_state_process+0x8eb/0x44e0 net/ipv4/tcp_input.c:6805
tcp_v4_do_rcv+0x77d/0xc70 net/ipv4/tcp_ipv4.c:1939
tcp_v4_rcv+0x2dc0/0x37f0 net/ipv4/tcp_ipv4.c:2351
ip_protocol_deliver_rcu+0x22e/0x440 net/ipv4/ip_input.c:205
ip_local_deliver_finish+0x341/0x5f0 net/ipv4/ip_input.c:233
NF_HOOK+0x3a4/0x450 include/linux/netfilter.h:314
NF_HOOK+0x3a4/0x450 include/linux/netfilter.h:314
__netif_receive_skb_one_core net/core/dev.c:5672 [inline]
__netif_receive_skb+0x2bf/0x650 net/core/dev.c:5785
process_backlog+0x662/0x15b0 net/core/dev.c:6117
__napi_poll+0xcb/0x490 net/core/dev.c:6883
napi_poll net/core/dev.c:6952 [inline]
net_rx_action+0x89b/0x1240 net/core/dev.c:7074
handle_softirqs+0x2d4/0x9b0 kernel/softirq.c:561

do_softirq+0x11b/0x1e0 kernel/softirq.c:462
</IRQ>
<TASK>
__local_bh_enable_ip+0x1bb/0x200 kernel/softirq.c:389
local_bh_enable include/linux/bottom_half.h:33 [inline]
srcu_invoke_callbacks+0x246/0x490 kernel/rcu/srcutree.c:1796
process_one_work kernel/workqueue.c:3229 [inline]
process_scheduled_works+0xa66/0x1840 kernel/workqueue.c:3310
worker_thread+0x870/0xd30 kernel/workqueue.c:3391
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:_compound_head include/linux/page-flags.h:242 [inline]
RIP: 0010:put_page+0x23/0x260 include/linux/mm.h:1552

Code: 90 90 90 90 90 90 90 55 41 57 41 56 53 49 89 fe 48 bd 00 00 00 00 00 fc ff df e8 98 5b 12 f8 49 8d 5e 08 48 89 d8 48 c1 e8 03 <80> 3c 28 00 74 08 48 89 df e8 3f bb 78 f8 48 8b 1b 48 89 de 48 83
RSP: 0018:ffffc90000006df0 EFLAGS: 00010202
RAX: 0000000000000001 RBX: 0000000000000008 RCX: ffff888027620000

RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000

RBP: dffffc0000000000 R08: ffffffff898caa7a R09: 1ffffffff2854734
R10: dffffc0000000000 R11: fffffbfff2854735 R12: 0000000000000007
R13: ffff88807a1efa42 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000001b3045ffff CR3: 000000007a6ec000 CR4: 00000000003526f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
----------------
Code disassembly (best guess):
0: 90 nop
1: 90 nop
2: 90 nop
3: 90 nop
4: 90 nop
5: 90 nop
6: 90 nop
7: 55 push %rbp
8: 41 57 push %r15
a: 41 56 push %r14
c: 53 push %rbx
d: 49 89 fe mov %rdi,%r14
10: 48 bd 00 00 00 00 00 movabs $0xdffffc0000000000,%rbp
17: fc ff df

1a: e8 98 5b 12 f8 call 0xf8125bb7

1f: 49 8d 5e 08 lea 0x8(%r14),%rbx
23: 48 89 d8 mov %rbx,%rax
26: 48 c1 e8 03 shr $0x3,%rax
* 2a: 80 3c 28 00 cmpb $0x0,(%rax,%rbp,1) <-- trapping instruction
2e: 74 08 je 0x38
30: 48 89 df mov %rbx,%rdi

33: e8 3f bb 78 f8 call 0xf878bb77

38: 48 8b 1b mov (%rbx),%rbx
3b: 48 89 de mov %rbx,%rsi
3e: 48 rex.W
3f: 83 .byte 0x83

Tested on:

commit: eabcdba3 Merge tag 'for-6.13-rc3-tag' of git://git.ker..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1292b730580000
kernel config: https://syzkaller.appspot.com/x/.config?x=6a2b862bf4a5409f

dashboard link: https://syzkaller.appspot.com/bug?extid=38a095a81f30d82884c1
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=1474b730580000

[Aleksandr Nogikh]

Hi Matthieu,

As of now, it's not yet supported. I've added a +1 mention to the
corresponding syzbot backlog issue:
https://github.com/google/syzkaller/issues/3491

I've also looked at the bisection log of this particular report and
the only suspicious part is that syzbot could have been too eager to
minimize the .config file. A different set of enabled options changed
the cash title from "general protection fault in put_page" to "BUG:
unable to handle kernel NULL pointer dereference in skb_release_data",
but the rest of the bisection log looks reasonable to me.

>
> I'm trying to run a 'git bisect' on my side since this morning: the
> issue seems to be older, between v6.10 and v6.11 if I'm not mistaken.
> When using the same kernel config, I'm getting quite a few issues on
> older commits (compilation, other warnings, etc.), plus the compilation
> is slow on my laptop. I will update you if I can find anything useful.

If you find the proper guilty commit, it would also really help debug
the bot's bisection result.

In case it may help you during the manual bisection, syzbot
cherry-picks this set of fix commits while doing the bisection:
https://github.com/google/syzkaller/blob/master/pkg/vcs/linux_patches.go#L60

--
Aleksandr

[Matthieu Baerts]

Hi Aleksandr,

Thank you for your reply!

Thank you for that!

> I've also looked at the bisection log of this particular report and
> the only suspicious part is that syzbot could have been too eager to
> minimize the .config file. A different set of enabled options changed
> the cash title from "general protection fault in put_page" to "BUG:
> unable to handle kernel NULL pointer dereference in skb_release_data",
> but the rest of the bisection log looks reasonable to me.

OK, thank you for having looked! The other title sounds better to
describe this issue.

>> I'm trying to run a 'git bisect' on my side since this morning: the
>> issue seems to be older, between v6.10 and v6.11 if I'm not mistaken.
>> When using the same kernel config, I'm getting quite a few issues on
>> older commits (compilation, other warnings, etc.), plus the compilation
>> is slow on my laptop. I will update you if I can find anything useful.
>
> If you find the proper guilty commit, it would also really help debug
> the bot's bisection result.

Will do if I can find something!

> In case it may help you during the manual bisection, syzbot
> cherry-picks this set of fix commits while doing the bisection:
> https://github.com/google/syzkaller/blob/master/pkg/vcs/linux_patches.go#L60

I appreciate the hint. It looks like all these commits are old enough to
be included in the range I'm testing for the moment.

I adapted my scripts to detect "other issues" and skip the commit.
Hopefully I can achieve something.