[syzbot] [net?] memory leak in tcp_md5_do_add

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+686628...@syzkaller.appspotmail.com

executing program
BUG: memory leak
unreferenced object 0xffff88810a86f7a0 (size 32):
comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
[<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
[<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
[<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
[<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
[<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
[<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd

BUG: memory leak
unreferenced object 0xffff88811225ccc0 (size 192):
comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
backtrace:
[<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
[<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
[<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
[<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
[<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
[<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
[<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
[<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
[<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
[<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
[<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd



---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Eric Dumazet]

Dmitry, please take a look at this bug, we need to fix it before your
patch series.

Thank you.

[Eric Dumazet]

On Fri, Sep 22, 2023 at 1:15 AM Dmitry Safonov <di...@arista.com> wrote:
>
> Hi Eric,
>
> On 9/21/23 18:01, Dmitry Safonov wrote:

> > Sure, seems reasonable to me to fix before merging something on top.
>
> It seems to me that it's related to a race between RCU grace period and
> kmemleak scan period. There seems to be a patch [1] that likely fixes
> that, albeit I couldn't verify it as all my attempts to reproduce syzbot
> issue produced only unrelated to TCP-MD5 log:
>

I doubt this, looking at the repro, which seems to abuse a not often
used feature of TCP (self connect)

# https://syzkaller.appspot.com/bug?id=323165b5fe193114de7a3a6a8bd16cf3a3c36ecf
# See https://goo.gl/kgGztJ for information about syzkaller reproducers.
#{"repeat":true,"procs":1,"slowdown":1,"sandbox":"none","sandbox_arg":0,"leak":true,"netdev":true,"close_fds":true,"usb":true}
r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
setsockopt$inet6_tcp_TCP_MD5SIG(r0, 0x6, 0xe,
&(0x7f0000000040)={@in6={{0xa, 0x0, 0x0, @local}}, 0x0, 0x0, 0x22,
0x0, "b05423587c18814d6b1a5f25671d09815a4687d637ffc958defc671aad3d4de8ac7d88560c759d600ab650c07ef0ef162b199da0d017fe6f0ae40cfb4e241cf9a990f20f6b8c2c070a61cfad8a2d2600"},
0xd8)
connect$inet6(r0, &(0x7f0000000180)={0xa, 0x4001, 0x0, @ipv4={'\x00',
'\xff\xff', @remote}}, 0x1c)
dup(0xffffffffffffffff)
setsockopt$SO_BINDTODEVICE(r0, 0x1, 0x19,
&(0x7f00000001c0)='ip6_vti0\x00', 0xff4a)



You could not have KMEMLEAK in the kernel, and run the repro a thousand times.

Then compare /proc/slabinfo before/after.

[Kuniyuki Iwashima]

From: Eric Dumazet <edum...@google.com>
Date: Fri, 22 Sep 2023 05:20:34 +0200

FWIW, I had the same report and another report for twsk and MD5.
syzkaller did not find repro though.

---8<---
BUG: memory leak
unreferenced object 0xffff888038513480 (size 192):
comm "syz-executor.0", pid 36537, jiffies 4295853096 (age 63.376s)

hex dump (first 32 bytes):

00 00 00 00 00 00 00 00 00 3e fc 43 80 88 ff ff .........>.C....
06 02 20 00 ac 14 14 aa 00 00 00 00 00 00 00 00 .. .............
backtrace:
[<0000000003e890c3>] __do_kmalloc_node mm/slab_common.c:984 [inline]
[<0000000003e890c3>] __kmalloc_node_track_caller+0x4b/0x130 mm/slab_common.c:1005
[<0000000026777435>] kmemdup+0x2c/0x60 mm/util.c:131
[<000000000318308e>] kmemdup include/linux/fortify-string.h:765 [inline]
[<000000000318308e>] tcp_time_wait_init net/ipv4/tcp_minisocks.c:261 [inline]
[<000000000318308e>] tcp_time_wait+0x25c/0x3b0 net/ipv4/tcp_minisocks.c:318
[<00000000bb86ba54>] tcp_rcv_state_process+0xb36/0x1990 net/ipv4/tcp_input.c:6668
[<00000000a26563d5>] tcp_v4_do_rcv+0x18b/0x4a0 net/ipv4/tcp_ipv4.c:1751
[<00000000b158e1f0>] sk_backlog_rcv include/net/sock.h:1115 [inline]
[<00000000b158e1f0>] __release_sock+0x177/0x1a0 net/core/sock.c:2982
[<000000000e8687d8>] __tcp_close+0x252/0x630 net/ipv4/tcp.c:2846
[<000000006b8a2f7d>] tcp_close+0x2d/0xc0 net/ipv4/tcp.c:2922
[<00000000d4c1915c>] inet_release+0x82/0xf0 net/ipv4/af_inet.c:433
[<00000000590c8ed6>] __sock_release+0x4b/0xf0 net/socket.c:657
[<00000000d49971a8>] sock_close+0x19/0x30 net/socket.c:1399
[<0000000097cacf4d>] __fput+0x1d0/0x4b0 fs/file_table.c:384
[<000000006a98802f>] __fput_sync+0x37/0x40 fs/file_table.c:465
[<00000000a6ebd3a7>] __do_sys_close fs/open.c:1572 [inline]
[<00000000a6ebd3a7>] __se_sys_close fs/open.c:1557 [inline]
[<00000000a6ebd3a7>] __x64_sys_close+0x4a/0xc0 fs/open.c:1557
[<000000004060032b>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<000000004060032b>] do_syscall_64+0x3c/0x90 arch/x86/entry/common.c:80
[<00000000e8d61c9b>] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
---8<---

In my syzkaller log, only this program had the MD5 operation.
I ran this overnight but had no luck for now.

---8<---
23:51:30 executing program 0:
r0 = socket$inet(0x2, 0x4000000000000001, 0x0)
setsockopt$inet_tcp_TCP_MD5SIG(r0, 0x6, 0xe, &(0x7f0000000780)={@in={{0x2, 0x0, @local}}, 0x0, 0x9, 0x6, 0x0, "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030cf00"}, 0xd8) (async)
bind$inet(r0, &(0x7f0000deb000)={0x2, 0x4e23, @multicast1}, 0x10) (async, rerun: 64)
sendto$inet(r0, 0x0, 0x0, 0x200007b9, &(0x7f0000000040)={0x2, 0x4e23, @local}, 0x10) (async, rerun: 64)
socket$inet6(0xa, 0x0, 0x0) (async)
getsockopt$EBT_SO_GET_INIT_ENTRIES(0xffffffffffffffff, 0x0, 0x83, &(0x7f0000000080)={'filter\x00', 0x0, 0x4, 0x1000, [0x0, 0x8, 0x1, 0x1, 0x0, 0x7fffffff], 0x4, &(0x7f0000000000)=[{}, {}, {}, {}], &(0x7f0000000880)=""/4096}, 0x0) (async, rerun: 32)
socket(0x0, 0x0, 0x0) (async, rerun: 32)
r1 = openat2(0xffffffffffffffff, &(0x7f0000000100)='./file0\x00', &(0x7f0000000140)={0x20000, 0x8, 0x14}, 0x18)
bind$inet(r1, &(0x7f0000000180)={0x2, 0x4e22, @remote}, 0x10) (async)
sendmsg$nl_route(0xffffffffffffffff, 0x0, 0x0)
---8<---

[ead...@sina.com]

From: Edward AD <ead...@sina.com>

please test memory leak in md5sig info and key

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ee3f96b16468

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 27140e5cdc06..292435dbec06 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1168,6 +1168,14 @@ struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
}
EXPORT_SYMBOL(tcp_v4_md5_lookup);

+static void tcp_md5sig_info_free(struct rcu_head *head)
+{
+ struct tcp_md5sig_info *md5sig =
+ container_of(head, struct tcp_md5sig_info, rcu);
+
+ kfree_sensitive(md5sig);
+}
+
static int tcp_md5sig_info_add(struct sock *sk, gfp_t gfp)
{
struct tcp_sock *tp = tcp_sk(sk);
@@ -1180,9 +1188,18 @@ static int tcp_md5sig_info_add(struct sock *sk, gfp_t gfp)
sk_gso_disable(sk);
INIT_HLIST_HEAD(&md5sig->head);
rcu_assign_pointer(tp->md5sig_info, md5sig);
+ call_rcu(&md5sig->rcu, tcp_md5sig_info_free);
return 0;
}

+static void tcp_md5sig_key_free(struct rcu_head *head)
+{
+ struct tcp_md5sig_key *key =
+ container_of(head, struct tcp_md5sig_key, rcu);
+
+ kfree_sensitive(key);
+}
+
/* This can be called on a newly created socket, from other files */
static int __tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
int family, u8 prefixlen, int l3index, u8 flags,
@@ -1234,6 +1251,7 @@ static int __tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
(IS_ENABLED(CONFIG_IPV6) && family == AF_INET6) ? sizeof(struct in6_addr) :
sizeof(struct in_addr));
hlist_add_head_rcu(&key->node, &md5sig->head);
+ call_rcu(&key->rcu, tcp_md5sig_key_free);
return 0;
}

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
general protection fault in __tcp_md5_do_lookup

general protection fault, probably for non-canonical address 0x17ff00000000011: 0000 [#1] PREEMPT SMP
CPU: 1 PID: 5495 Comm: dhcpcd-run-hook Not tainted 6.2.0-syzkaller-13115-gee3f96b16468-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/04/2023
RIP: 0010:__tcp_md5_do_lookup+0x77/0x2e0 net/ipv4/tcp_ipv4.c:1093
Code: ee bf 0a 00 00 00 e8 28 3e 30 fd 83 fd 0a 0f 84 31 01 00 00 e8 4a 45 30 fd 48 8b 1b 48 85 db 0f 84 09 01 00 00 e8 39 45 30 fd <44> 0f b6 63 11 89 ee 44 89 e7 e8 7a 3d 30 fd 41 39 ec 75 d6 e8 20
RSP: 0018:ffffc90000318cd0 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 017ff00000000000 RCX: 0000000000000100
RDX: ffff8881049ce340 RSI: ffffffff840bf0a7 RDI: 0000000000000004
RBP: 0000000000000002 R08: 0000000000000004 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000c00 R12: 0000000000000001
R13: 0000000000000000 R14: 0000000000000000 R15: ffff888114e81400
FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fffe533def8 CR3: 0000000115194000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
tcp_md5_do_lookup include/net/tcp.h:1700 [inline]
tcp_v4_md5_lookup+0x91/0xb0 net/ipv4/tcp_ipv4.c:1160
tcp_established_options+0x179/0x280 net/ipv4/tcp_output.c:927
tcp_current_mss+0x77/0xe0 net/ipv4/tcp_output.c:1834
tcp_send_loss_probe+0x1c/0x300 net/ipv4/tcp_output.c:2805
tcp_write_timer_handler net/ipv4/tcp_timer.c:616 [inline]
tcp_write_timer_handler+0x25c/0x3f0 net/ipv4/tcp_timer.c:594
tcp_write_timer+0x150/0x1b0 net/ipv4/tcp_timer.c:637
call_timer_fn+0x34/0x200 kernel/time/timer.c:1700
expire_timers kernel/time/timer.c:1751 [inline]
__run_timers.part.0+0x32a/0x440 kernel/time/timer.c:2022
__run_timers kernel/time/timer.c:2000 [inline]
run_timer_softirq+0x48/0xa0 kernel/time/timer.c:2035
__do_softirq+0xf5/0x302 kernel/softirq.c:571
invoke_softirq kernel/softirq.c:445 [inline]
__irq_exit_rcu kernel/softirq.c:650 [inline]
irq_exit_rcu+0xb0/0x110 kernel/softirq.c:662
sysvec_apic_timer_interrupt+0xa2/0xd0 arch/x86/kernel/apic/apic.c:1107
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:kmem_cache_free+0x145/0x250 mm/slab.c:3603
Code: 03 0f 85 f5 00 00 00 65 48 ff 08 e8 35 1e d5 ff 66 90 4c 89 f2 4c 89 ee 4c 89 e7 e8 05 fb ff ff f7 c5 00 02 00 00 74 01 fb 5b <5d> 41 5c 41 5d 41 5e 41 5f c3 65 8b 05 d6 cd a4 7e 89 c0 48 0f a3
RSP: 0018:ffffc900032cbb30 EFLAGS: 00000206
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000dbe
RDX: 000000000000003d RSI: ffffea000413b8c0 RDI: ffffffff85727a66
RBP: 0000000000000286 R08: ffffffff87572358 R09: 0000000000000000
R10: ffff8881419e5868 R11: 0000000000000000 R12: ffff888100044600
R13: ffff888104ee3e00 R14: ffffffff84948e1b R15: ffff888105254540
mt_free_rcu lib/maple_tree.c:176 [inline]
mt_destroy_walk+0x5db/0x610 lib/maple_tree.c:5594
mte_destroy_walk lib/maple_tree.c:5613 [inline]
__mt_destroy+0xc4/0xd0 lib/maple_tree.c:6415
do_vmi_align_munmap+0x511/0x5b0 mm/mmap.c:2399
do_vmi_munmap+0x160/0x200 mm/mmap.c:2452
mmap_region+0x107/0xd80 mm/mmap.c:2500
do_mmap+0x5f9/0x9c0 mm/mmap.c:1364
vm_mmap_pgoff+0x11a/0x1d0 mm/util.c:542
ksys_mmap_pgoff+0x251/0x2e0 mm/mmap.c:1410
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f40e9aafb74
Code: 63 08 44 89 e8 5b 41 5c 41 5d c3 41 89 ca 41 f7 c1 ff 0f 00 00 74 0c c7 05 f5 46 01 00 16 00 00 00 eb 17 b8 09 00 00 00 0f 05 <48> 3d 00 f0 ff ff 76 0c f7 d8 89 05 dc 46 01 00 48 83 c8 ff c3 0f
RSP: 002b:00007fffe533ded8 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
RAX: ffffffffffffffda RBX: 00007fffe533df18 RCX: 00007f40e9aafb74
RDX: 0000000000000005 RSI: 0000000000088000 RDI: 00007f40e99de000
RBP: 00007fffe533e240 R08: 0000000000000003 R09: 000000000000f000
R10: 0000000000000812 R11: 0000000000000246 R12: 00007f40e9a920c0
R13: 00007fffe533e2c8 R14: 000000000000ed80 R15: 0000000000000000
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:__tcp_md5_do_lookup+0x77/0x2e0 net/ipv4/tcp_ipv4.c:1093
Code: ee bf 0a 00 00 00 e8 28 3e 30 fd 83 fd 0a 0f 84 31 01 00 00 e8 4a 45 30 fd 48 8b 1b 48 85 db 0f 84 09 01 00 00 e8 39 45 30 fd <44> 0f b6 63 11 89 ee 44 89 e7 e8 7a 3d 30 fd 41 39 ec 75 d6 e8 20
RSP: 0018:ffffc90000318cd0 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 017ff00000000000 RCX: 0000000000000100
RDX: ffff8881049ce340 RSI: ffffffff840bf0a7 RDI: 0000000000000004
RBP: 0000000000000002 R08: 0000000000000004 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000c00 R12: 0000000000000001
R13: 0000000000000000 R14: 0000000000000000 R15: ffff888114e81400
FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fffe533def8 CR3: 0000000115194000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
----------------
Code disassembly (best guess):
0: ee out %al,(%dx)
1: bf 0a 00 00 00 mov $0xa,%edi
6: e8 28 3e 30 fd call 0xfd303e33
b: 83 fd 0a cmp $0xa,%ebp
e: 0f 84 31 01 00 00 je 0x145
14: e8 4a 45 30 fd call 0xfd304563
19: 48 8b 1b mov (%rbx),%rbx
1c: 48 85 db test %rbx,%rbx
1f: 0f 84 09 01 00 00 je 0x12e
25: e8 39 45 30 fd call 0xfd304563
* 2a: 44 0f b6 63 11 movzbl 0x11(%rbx),%r12d <-- trapping instruction
2f: 89 ee mov %ebp,%esi
31: 44 89 e7 mov %r12d,%edi
34: e8 7a 3d 30 fd call 0xfd303db3
39: 41 39 ec cmp %ebp,%r12d
3c: 75 d6 jne 0x14
3e: e8 .byte 0xe8
3f: 20 .byte 0x20


Tested on:

commit: ee3f96b1 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=13f49ab6680000
kernel config: https://syzkaller.appspot.com/x/.config?x=ba49a3c2ed724b44
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
patch: https://syzkaller.appspot.com/x/patch.diff?x=170df942680000

[ead...@sina.com]

From: Edward AD <ead...@sina.com>

please test memory leak in md5sig info and key

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ee3f96b16468

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c

index 27140e5cdc06..de5f0306b594 100644

--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1168,6 +1168,14 @@ struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
}
EXPORT_SYMBOL(tcp_v4_md5_lookup);

+static void tcp_md5sig_info_free(struct rcu_head *head)
+{
+ struct tcp_md5sig_info *md5sig =
+ container_of(head, struct tcp_md5sig_info, rcu);
+
+ kfree_sensitive(md5sig);
+}
+
static int tcp_md5sig_info_add(struct sock *sk, gfp_t gfp)
{
struct tcp_sock *tp = tcp_sk(sk);

@@ -1183,6 +1191,14 @@ static int tcp_md5sig_info_add(struct sock *sk, gfp_t gfp)

return 0;
}

+static void tcp_md5sig_key_free(struct rcu_head *head)
+{
+ struct tcp_md5sig_key *key =
+ container_of(head, struct tcp_md5sig_key, rcu);
+
+ kfree_sensitive(key);
+}
+
/* This can be called on a newly created socket, from other files */
static int __tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
int family, u8 prefixlen, int l3index, u8 flags,

@@ -1299,7 +1315,7 @@ int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr, int family,
return -ENOENT;
hlist_del_rcu(&key->node);
atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
- kfree_rcu(key, rcu);

+ call_rcu(&key->rcu, tcp_md5sig_key_free);
return 0;
}

EXPORT_SYMBOL(tcp_md5_do_del);
@@ -1316,7 +1332,7 @@ static void tcp_clear_md5_list(struct sock *sk)
hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
hlist_del_rcu(&key->node);
atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
- kfree_rcu(key, rcu);
+ call_rcu(&key->rcu, tcp_md5sig_key_free);
}
}

@@ -2318,7 +2334,7 @@ void tcp_v4_destroy_sock(struct sock *sk)
/* Clean up the MD5 key list, if any */
if (tp->md5sig_info) {
tcp_clear_md5_list(sk);
- kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu);
+ call_rcu(&tp->md5sig_info->rcu, tcp_md5sig_info_free);
tp->md5sig_info = NULL;
static_branch_slow_dec_deferred(&tcp_md5_needed);
}

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

memory leak in __register_sysctl_table

BUG: memory leak
unreferenced object 0xffff88810dc83400 (size 1024):
comm "syz-executor.0", pid 5328, jiffies 4294944213 (age 465.180s)

hex dump (first 32 bytes):

08 f8 3b 15 81 88 ff ff 00 00 00 00 00 00 00 00 ..;.............
00 00 00 00 00 00 00 00 ea ff ff ff ff ff ff ff ................
backtrace:
[<ffffffff81537378>] __do_kmalloc_node mm/slab_common.c:966 [inline]
[<ffffffff81537378>] __kmalloc+0x48/0x150 mm/slab_common.c:980
[<ffffffff8174e12f>] kmalloc include/linux/slab.h:584 [inline]
[<ffffffff8174e12f>] kzalloc include/linux/slab.h:720 [inline]
[<ffffffff8174e12f>] __register_sysctl_table+0x7f/0xad0 fs/proc/proc_sysctl.c:1343
[<ffffffff83d94653>] neigh_sysctl_register+0x143/0x210 net/core/neighbour.c:3876
[<ffffffff841e7c1c>] addrconf_sysctl_register+0x6c/0xf0 net/ipv6/addrconf.c:7131
[<ffffffff841e8143>] ipv6_add_dev+0x4a3/0x820 net/ipv6/addrconf.c:450
[<ffffffff841f2e89>] addrconf_notify+0x309/0xcc0 net/ipv6/addrconf.c:3552
[<ffffffff812c1665>] notifier_call_chain kernel/notifier.c:87 [inline]
[<ffffffff812c1665>] raw_notifier_call_chain+0x65/0xa0 kernel/notifier.c:455
[<ffffffff83d761d9>] call_netdevice_notifiers_info+0x79/0xd0 net/core/dev.c:1937
[<ffffffff83d90b35>] call_netdevice_notifiers_extack net/core/dev.c:1975 [inline]
[<ffffffff83d90b35>] call_netdevice_notifiers net/core/dev.c:1989 [inline]
[<ffffffff83d90b35>] register_netdevice+0x795/0x9a0 net/core/dev.c:10079
[<ffffffff842c95d5>] br_dev_newlink+0x25/0xd0 net/bridge/br_netlink.c:1534
[<ffffffff83da9828>] rtnl_newlink_create net/core/rtnetlink.c:3440 [inline]
[<ffffffff83da9828>] __rtnl_newlink+0xa58/0xdc0 net/core/rtnetlink.c:3657
[<ffffffff83da9bdc>] rtnl_newlink+0x4c/0x70 net/core/rtnetlink.c:3670
[<ffffffff83da35ff>] rtnetlink_rcv_msg+0x22f/0x5b0 net/core/rtnetlink.c:6174
[<ffffffff83ecd5e1>] netlink_rcv_skb+0x91/0x1d0 net/netlink/af_netlink.c:2574
[<ffffffff83ecc528>] netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline]
[<ffffffff83ecc528>] netlink_unicast+0x2b8/0x430 net/netlink/af_netlink.c:1365
[<ffffffff83ecca21>] netlink_sendmsg+0x381/0x710 net/netlink/af_netlink.c:1942

BUG: memory leak
unreferenced object 0xffff888115488100 (size 256):
comm "syz-executor.0", pid 5328, jiffies 4294944213 (age 465.180s)

hex dump (first 32 bytes):

ff 01 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................
00 18 0a 15 81 88 ff ff 00 00 00 00 00 00 00 00 ................
backtrace:
[<ffffffff81536c45>] kmalloc_trace+0x25/0x90 mm/slab_common.c:1061
[<ffffffff8422f7d5>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff8422f7d5>] kzalloc include/linux/slab.h:720 [inline]
[<ffffffff8422f7d5>] mca_alloc net/ipv6/mcast.c:880 [inline]
[<ffffffff8422f7d5>] __ipv6_dev_mc_inc+0x205/0x5a0 net/ipv6/mcast.c:936
[<ffffffff841e8184>] ipv6_add_dev+0x4e4/0x820 net/ipv6/addrconf.c:462
[<ffffffff841f2e89>] addrconf_notify+0x309/0xcc0 net/ipv6/addrconf.c:3552
[<ffffffff812c1665>] notifier_call_chain kernel/notifier.c:87 [inline]
[<ffffffff812c1665>] raw_notifier_call_chain+0x65/0xa0 kernel/notifier.c:455
[<ffffffff83d761d9>] call_netdevice_notifiers_info+0x79/0xd0 net/core/dev.c:1937
[<ffffffff83d90b35>] call_netdevice_notifiers_extack net/core/dev.c:1975 [inline]
[<ffffffff83d90b35>] call_netdevice_notifiers net/core/dev.c:1989 [inline]
[<ffffffff83d90b35>] register_netdevice+0x795/0x9a0 net/core/dev.c:10079
[<ffffffff82da1fa8>] bond_newlink drivers/net/bonding/bond_netlink.c:560 [inline]
[<ffffffff82da1fa8>] bond_newlink+0x48/0x90 drivers/net/bonding/bond_netlink.c:550
[<ffffffff83da9828>] rtnl_newlink_create net/core/rtnetlink.c:3440 [inline]
[<ffffffff83da9828>] __rtnl_newlink+0xa58/0xdc0 net/core/rtnetlink.c:3657
[<ffffffff83da9bdc>] rtnl_newlink+0x4c/0x70 net/core/rtnetlink.c:3670
[<ffffffff83da35ff>] rtnetlink_rcv_msg+0x22f/0x5b0 net/core/rtnetlink.c:6174
[<ffffffff83ecd5e1>] netlink_rcv_skb+0x91/0x1d0 net/netlink/af_netlink.c:2574
[<ffffffff83ecc528>] netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline]
[<ffffffff83ecc528>] netlink_unicast+0x2b8/0x430 net/netlink/af_netlink.c:1365
[<ffffffff83ecca21>] netlink_sendmsg+0x381/0x710 net/netlink/af_netlink.c:1942
[<ffffffff83d3ef06>] sock_sendmsg_nosec net/socket.c:722 [inline]
[<ffffffff83d3ef06>] sock_sendmsg+0x56/0xb0 net/socket.c:745
[<ffffffff83d42418>] __sys_sendto+0x138/0x1b0 net/socket.c:2145

BUG: memory leak
unreferenced object 0xffff888115488f00 (size 256):
comm "syz-executor.0", pid 5328, jiffies 4294944213 (age 465.180s)

hex dump (first 32 bytes):

ff 02 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................
00 18 0a 15 81 88 ff ff 00 81 48 15 81 88 ff ff ..........H.....
backtrace:
[<ffffffff81536c45>] kmalloc_trace+0x25/0x90 mm/slab_common.c:1061
[<ffffffff8422f7d5>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff8422f7d5>] kzalloc include/linux/slab.h:720 [inline]
[<ffffffff8422f7d5>] mca_alloc net/ipv6/mcast.c:880 [inline]
[<ffffffff8422f7d5>] __ipv6_dev_mc_inc+0x205/0x5a0 net/ipv6/mcast.c:936
[<ffffffff841e8193>] ipv6_add_dev+0x4f3/0x820 net/ipv6/addrconf.c:465
[<ffffffff841f2e89>] addrconf_notify+0x309/0xcc0 net/ipv6/addrconf.c:3552
[<ffffffff812c1665>] notifier_call_chain kernel/notifier.c:87 [inline]
[<ffffffff812c1665>] raw_notifier_call_chain+0x65/0xa0 kernel/notifier.c:455
[<ffffffff83d761d9>] call_netdevice_notifiers_info+0x79/0xd0 net/core/dev.c:1937
[<ffffffff83d90b35>] call_netdevice_notifiers_extack net/core/dev.c:1975 [inline]
[<ffffffff83d90b35>] call_netdevice_notifiers net/core/dev.c:1989 [inline]
[<ffffffff83d90b35>] register_netdevice+0x795/0x9a0 net/core/dev.c:10079
[<ffffffff82da1fa8>] bond_newlink drivers/net/bonding/bond_netlink.c:560 [inline]
[<ffffffff82da1fa8>] bond_newlink+0x48/0x90 drivers/net/bonding/bond_netlink.c:550
[<ffffffff83da9828>] rtnl_newlink_create net/core/rtnetlink.c:3440 [inline]
[<ffffffff83da9828>] __rtnl_newlink+0xa58/0xdc0 net/core/rtnetlink.c:3657
[<ffffffff83da9bdc>] rtnl_newlink+0x4c/0x70 net/core/rtnetlink.c:3670
[<ffffffff83da35ff>] rtnetlink_rcv_msg+0x22f/0x5b0 net/core/rtnetlink.c:6174
[<ffffffff83ecd5e1>] netlink_rcv_skb+0x91/0x1d0 net/netlink/af_netlink.c:2574
[<ffffffff83ecc528>] netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline]
[<ffffffff83ecc528>] netlink_unicast+0x2b8/0x430 net/netlink/af_netlink.c:1365
[<ffffffff83ecca21>] netlink_sendmsg+0x381/0x710 net/netlink/af_netlink.c:1942
[<ffffffff83d3ef06>] sock_sendmsg_nosec net/socket.c:722 [inline]
[<ffffffff83d3ef06>] sock_sendmsg+0x56/0xb0 net/socket.c:745
[<ffffffff83d42418>] __sys_sendto+0x138/0x1b0 net/socket.c:2145

Tested on:

commit: ee3f96b1 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

console output: https://syzkaller.appspot.com/x/log.txt?x=1378d476680000

kernel config: https://syzkaller.appspot.com/x/.config?x=ba49a3c2ed724b44
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=13abc196680000

[Dmitry Safonov]

Hi Eric,

On 9/21/23 18:01, Dmitry Safonov wrote:
> On 9/21/23 17:59, Eric Dumazet wrote:

> Sure, seems reasonable to me to fix before merging something on top.

It seems to me that it's related to a race between RCU grace period and
kmemleak scan period. There seems to be a patch [1] that likely fixes
that, albeit I couldn't verify it as all my attempts to reproduce syzbot
issue produced only unrelated to TCP-MD5 log:

> [ 263.201211] kmemleak: unreferenced object 0xffff9ceb047d9948 (size 192):
> [ 263.201781] kmemleak: comm "ip", pid 730, jiffies 4294937874 (age 257.270s)
> [ 263.202460] kmemleak: hex dump (first 32 bytes):
> [ 263.202921] kmemleak: 00 c8 e9 01 eb 9c ff ff e0 00 00 01 00 00 00 00 ................
> [ 263.203700] kmemleak: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 263.204484] kmemleak: backtrace:
> [ 263.204814] kmemleak: [<ffffffff830a2946>] kmalloc_trace+0x26/0x90
> [ 263.205440] kmemleak: [<ffffffff837e8310>] ____ip_mc_inc_group+0xa0/0x240
> [ 263.206134] kmemleak: [<ffffffff837e9a9b>] ip_mc_up+0x4b/0xb0
> [ 263.206725] kmemleak: [<ffffffff837e28fb>] inetdev_event+0xbb/0x5c0
> [ 263.207358] kmemleak: [<ffffffff82f3caf6>] notifier_call_chain+0x56/0xc0
> [ 263.208070] kmemleak: [<ffffffff836f1818>] __dev_notify_flags+0x58/0xf0
> [ 263.208784] kmemleak: [<ffffffff836f2210>] dev_change_flags+0x50/0x60
> [ 263.209471] kmemleak: [<ffffffff837e1718>] devinet_ioctl+0x378/0x770
> [ 263.210152] kmemleak: [<ffffffff837e34a7>] inet_ioctl+0x187/0x1d0
> [ 263.210805] kmemleak: [<ffffffff836c40ed>] sock_do_ioctl+0x3d/0x100
> [ 263.211482] kmemleak: [<ffffffff836c4293>] sock_ioctl+0xe3/0x2b0
> [ 263.212131] kmemleak: [<ffffffff8313cbec>] __x64_sys_ioctl+0x8c/0xc0
> [ 263.212789] kmemleak: [<ffffffff83a2ad75>] do_syscall_64+0x35/0x80
> [ 263.213438] kmemleak: [<ffffffff83c0006a>] entry_SYSCALL_64_after_hwframe+0x46/0xb0
> [ 263.214283] kmemleak: unreferenced object 0xffff9ceb03ad5400 (size 512):
> [ 263.214982] kmemleak: comm "ip", pid 730, jiffies 4294937874 (age 257.290s)
> [ 263.215728] kmemleak: hex dump (first 32 bytes):
> [ 263.216231] kmemleak: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................
> [ 263.217106] kmemleak: 80 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ................
> [ 263.218041] kmemleak: backtrace:
> [ 263.218438] kmemleak: [<ffffffff830a2946>] kmalloc_trace+0x26/0x90
> [ 263.219181] kmemleak: [<ffffffff8384b90b>] ipv6_add_addr+0x13b/0x6c0
> [ 263.219931] kmemleak: [<ffffffff8384d4b5>] add_addr+0x75/0x150
> [ 263.220627] kmemleak: [<ffffffff8385357d>] addrconf_notify+0x53d/0x730
> [ 263.221377] kmemleak: [<ffffffff82f3caf6>] notifier_call_chain+0x56/0xc0
> [ 263.222104] kmemleak: [<ffffffff836f1818>] __dev_notify_flags+0x58/0xf0
> [ 263.222844] kmemleak: [<ffffffff836f2210>] dev_change_flags+0x50/0x60
> [ 263.223581] kmemleak: [<ffffffff837e1718>] devinet_ioctl+0x378/0x770
> [ 263.224293] kmemleak: [<ffffffff837e34a7>] inet_ioctl+0x187/0x1d0
> [ 263.224961] kmemleak: [<ffffffff836c40ed>] sock_do_ioctl+0x3d/0x100
> [ 263.225660] kmemleak: [<ffffffff836c4293>] sock_ioctl+0xe3/0x2b0
> [ 263.226331] kmemleak: [<ffffffff8313cbec>] __x64_sys_ioctl+0x8c/0xc0
> [ 263.227039] kmemleak: [<ffffffff83a2ad75>] do_syscall_64+0x35/0x80
> [ 263.227747] kmemleak: [<ffffffff83c0006a>] entry_SYSCALL_64_after_hwframe+0x46/0xb0
> [ 263.228708] kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak)

This seems to be quite the same issue: inet6_ifa_finish_destroy()
destroys inet6_ifaddr with kfree_rcu().

[1]
https://lore.kernel.org/linux-mm/ZQA06490...@arm.com/T/#ma4a68fdc44793e2594c9e7cadefa8ea40da5807d

Thanks,
Dmitry

[Dmitry Safonov]

On 9/21/23 17:59, Eric Dumazet wrote:

Sure, seems reasonable to me to fix before merging something on top.

> Thank you.

Thanks,
Dmitry

[Dmitry Safonov]

On 9/22/23 04:20, Eric Dumazet wrote:
> On Fri, Sep 22, 2023 at 1:15 AM Dmitry Safonov <di...@arista.com> wrote:
>>
>> Hi Eric,
>>
>> On 9/21/23 18:01, Dmitry Safonov wrote:
>>> On 9/21/23 17:59, Eric Dumazet wrote:

[..]

>>>> Dmitry, please take a look at this bug, we need to fix it before your
>>>> patch series.
>>>
>>> Sure, seems reasonable to me to fix before merging something on top.
>>
>> It seems to me that it's related to a race between RCU grace period and
>> kmemleak scan period. There seems to be a patch [1] that likely fixes
>> that, albeit I couldn't verify it as all my attempts to reproduce syzbot
>> issue produced only unrelated to TCP-MD5 log:
>>
>
> I doubt this, looking at the repro, which seems to abuse a not often
> used feature of TCP (self connect)
>
> # https://syzkaller.appspot.com/bug?id=323165b5fe193114de7a3a6a8bd16cf3a3c36ecf
> # See https://goo.gl/kgGztJ for information about syzkaller reproducers.
> #{"repeat":true,"procs":1,"slowdown":1,"sandbox":"none","sandbox_arg":0,"leak":true,"netdev":true,"close_fds":true,"usb":true}
> r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
> setsockopt$inet6_tcp_TCP_MD5SIG(r0, 0x6, 0xe,
> &(0x7f0000000040)={@in6={{0xa, 0x0, 0x0, @local}}, 0x0, 0x0, 0x22,
> 0x0, "b05423587c18814d6b1a5f25671d09815a4687d637ffc958defc671aad3d4de8ac7d88560c759d600ab650c07ef0ef162b199da0d017fe6f0ae40cfb4e241cf9a990f20f6b8c2c070a61cfad8a2d2600"},
> 0xd8)
> connect$inet6(r0, &(0x7f0000000180)={0xa, 0x4001, 0x0, @ipv4={'\x00',
> '\xff\xff', @remote}}, 0x1c)
> dup(0xffffffffffffffff)
> setsockopt$SO_BINDTODEVICE(r0, 0x1, 0x19,
> &(0x7f00000001c0)='ip6_vti0\x00', 0xff4a)
>
>
>
> You could not have KMEMLEAK in the kernel, and run the repro a thousand times.
>
> Then compare /proc/slabinfo before/after.

Eric!

After some experiments, I'm still standing on the same RCU/kmemleak
race. Here's what I did:

1. I couldn't reproduce it on locally-build kernel with the same .config
in order to verify that the patch does fix this. Probably the toolchain
or any other small bits make a difference.

2. I can easily reproduce it with bzImage from syzcaller.

3. slabinfo does fluctuate for kmalloc-192 even without running any
reproducer, so it'd be hard to tell the difference.

4. I went running the reproducer in loop a hundred times:
# for i in $(seq 1 100) ; do ./repro ; dmesg -c >> ./dmesg.log ; done
the dmesg clean here is because pre-compiled ring-buffer is quite small
(I attach the full logs to this mail)

5. `cat /sys/kernel/debug/kmemleak` is empty, I presume it means that
the addresses that were reported later got removed from kmemleak
lists/tables on kfree().

6. Curious about addresses reported, I can see that some addresses were
reported multiple times, which means that slab got reused, rather than
leaked:
# grep tcp_md5_do_add -B9 dmesg.log | sed -n 's/.*unreferenced object
\([^ ]\+\) .*/\1/p' | sort | uniq -c | sort -n | tail -n 5
2 0xffff8880174b96c0
2 0xffff888017686900
2 0xffff8880179fd720
3 0xffff8880146a8600
3 0xffff888015c226c0

See the logs for 0xffff888015c226c0 address (you can view them in dmesg
that I attach):
https://gist.github.com/0x7f454c46/dcc7936392a51a789a235eb73df1598c

7. Well, OK, maybe at least one of the addresses reported was leaked?
In order to check that, I did:
# cat dmesg.log | grep tcp_md5_do_add -B9 | sed -n 's/.*unreferenced
object \([^ ]\+\) .*/\1/p' | sort | uniq > addresses
# dmesg -c ; for i in $(cat addresses) ; do echo "dump=$i" >
/sys/kernel/debug/kmemleak ; dmesg -c >> addresses.kmemleak ; done

I attach addresses.kmemleak: the slabs were reused or weren't allocated
(I presume, the "Unknown object at 0xffff888019cdcdc0" means that slab
is free).

8. Now that I verified that kmemleak was misreporting those addresses, I
went on my regular TCP-AO selftests, added one for TCP self-connect and
with ftrace I can clearly see that md5 keys/info is deallocated:
# cat trace
# tracer: function
#
# entries-in-buffer/entries-written: 5/5 #P:2
#
# _-----=> irqs-off/BH-disabled
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / _-=> migrate-disable
# |||| / delay
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
self-connect_ip-2125 [000] ...1. 6108.468401: tcp_md5_do_add
<-tcp_v4_parse_md5_keys
self-connect_ip-2125 [000] ...1. 6108.468727: __tcp_md5_do_add
<-tcp_md5_do_add
self-connect_ip-2125 [000] ...1. 6108.471276: tcp_clear_md5_list
<-tcp_v4_destroy_sock
kworker/u5:1-2108 [000] ..s1. 6108.475633:
tcp_md5sig_info_free_rcu <-rcu_core
<idle>-0 [000] ..s2. 6108.598342: tcp_md5_twsk_free_rcu
<-rcu_core

So, from source code point of view: the test opens socket, sends SYN,
receives SYN straight away and kernel decides that it's
simultaneous/fast open case. It sends SYN-ACK and establishes as normal.
On socket destruction, TCP-MD5 key gets destroyed the regular way.
Nothing seems special in this TCP self-connect case.

9. The only interesting part in this experiment is that now I have a TCP
self-connect selftest, that by its nature tests simultaneous open, which
as I expected works with TCP-MD5, but not with TCP-AO:
[ 3412.559472] TCP: AO hash mismatch for (127.0.0.1, 7010)->(127.0.0.1,
7010) SA
[ 4115.964926] TCP: AO hash mismatch for [::1]:7010->[::1]:7010 SA
L3index: 0

It was expected as tcp_inbound_ao_hash() has
: /* Fast-path */
: /* TODO: fix fastopen and simultaneous open (TCPF_SYN_RECV) */

Going to fix this for TCP-AO-v13.

Please, let me know if this explanation/investigation looks good to you,
so that I can proceed with v13.

Thanks,
Dmitry

[ead...@sina.com]

From: Edward AD <ead...@sina.com>

please test memory leak in md5sig info and key

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ee3f96b16468

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 0c3040a63ebd..f381e7c7a4d9 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4581,6 +4581,7 @@ int tcp_abort(struct sock *sk, int err)
bh_unlock_sock(sk);
local_bh_enable();
tcp_write_queue_purge(sk);
+ printk("%s, %p\n", __func__, sk);
if (!has_current_bpf_ctx())
release_sock(sk);
return 0;
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 27140e5cdc06..0045e276c5cf 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1253,6 +1253,7 @@ int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
md5sig = rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk));
rcu_assign_pointer(tp->md5sig_info, NULL);
kfree_rcu(md5sig, rcu);
+ printk("%s, %p \n", __func__, md5sig);
return -EUSERS;
}
}
@@ -1278,6 +1279,7 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr,
md5sig = rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk));
net_warn_ratelimited("Too many TCP-MD5 keys in the system\n");
rcu_assign_pointer(tp->md5sig_info, NULL);
+ printk("%s, %p \n", __func__, md5sig);
kfree_rcu(md5sig, rcu);
return -EUSERS;
}
@@ -1299,6 +1301,7 @@ int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr, int family,

return -ENOENT;
hlist_del_rcu(&key->node);
atomic_sub(sizeof(*key), &sk->sk_omem_alloc);

+ printk("%s, %p \n", __func__, key);
kfree_rcu(key, rcu);
return 0;
}
@@ -1316,6 +1319,7 @@ static void tcp_clear_md5_list(struct sock *sk)

hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
hlist_del_rcu(&key->node);
atomic_sub(sizeof(*key), &sk->sk_omem_alloc);

+ printk("%s, %p \n", __func__, key);
kfree_rcu(key, rcu);
}
}
@@ -2318,6 +2322,7 @@ void tcp_v4_destroy_sock(struct sock *sk)

/* Clean up the MD5 key list, if any */
if (tp->md5sig_info) {
tcp_clear_md5_list(sk);

+ printk("%s, %p \n", __func__, tp->md5sig_info);
kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu);

tp->md5sig_info = NULL;
static_branch_slow_dec_deferred(&tcp_md5_needed);

diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index ccfc8bbf7455..ae43546cb648 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1101,6 +1101,7 @@ void tcp_release_cb(struct sock *sk)
tcp_tsq_write(sk);
__sock_put(sk);
}
+ printk("%p, %s\n", sk, __func__);
/* Here begins the tricky part :
* We are called from release_sock() with :
* 1) BH disabled

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to apply patch:
checking file net/ipv4/tcp.c
Hunk #1 FAILED at 4581.
1 out of 1 hunk FAILED
checking file net/ipv4/tcp_ipv4.c
Hunk #1 succeeded at 1246 (offset -7 lines).
Hunk #2 succeeded at 1272 (offset -7 lines).
Hunk #3 succeeded at 1294 (offset -7 lines).
Hunk #4 succeeded at 1312 (offset -7 lines).
Hunk #5 succeeded at 2315 (offset -7 lines).
checking file net/ipv4/tcp_output.c
Hunk #1 succeeded at 1091 (offset -10 lines).



Tested on:

commit: ee3f96b1 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler:

patch: https://syzkaller.appspot.com/x/patch.diff?x=150dec86680000

[ead...@sina.com]

From: Edward AD <ead...@sina.com>

please test memory leak in md5sig info and key

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ee3f96b16468

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c

index 288693981b00..5d2587cd1920 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4704,6 +4704,7 @@ int tcp_abort(struct sock *sk, int err)

bh_unlock_sock(sk);
local_bh_enable();
tcp_write_queue_purge(sk);
+ printk("%s, %p\n", __func__, sk);

release_sock(sk);
return 0;
}
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c

index ea370afa70ed..6816445959e7 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1246,6 +1246,7 @@ int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,

md5sig = rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk));
rcu_assign_pointer(tp->md5sig_info, NULL);
kfree_rcu(md5sig, rcu);
+ printk("%s, %p \n", __func__, md5sig);
return -EUSERS;
}
}

@@ -1271,6 +1272,7 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr,

md5sig = rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk));
net_warn_ratelimited("Too many TCP-MD5 keys in the system\n");
rcu_assign_pointer(tp->md5sig_info, NULL);
+ printk("%s, %p \n", __func__, md5sig);
kfree_rcu(md5sig, rcu);
return -EUSERS;
}

@@ -1292,6 +1294,7 @@ int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr, int family,

return -ENOENT;
hlist_del_rcu(&key->node);
atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
+ printk("%s, %p \n", __func__, key);
kfree_rcu(key, rcu);
return 0;
}

@@ -1309,6 +1312,7 @@ static void tcp_clear_md5_list(struct sock *sk)

hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
hlist_del_rcu(&key->node);
atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
+ printk("%s, %p \n", __func__, key);
kfree_rcu(key, rcu);
}
}

@@ -2311,6 +2315,7 @@ void tcp_v4_destroy_sock(struct sock *sk)

/* Clean up the MD5 key list, if any */
if (tp->md5sig_info) {
tcp_clear_md5_list(sk);
+ printk("%s, %p \n", __func__, tp->md5sig_info);
kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu);
tp->md5sig_info = NULL;
static_branch_slow_dec_deferred(&tcp_md5_needed);
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c

index 71d01cf3c13e..1c0c50027c48 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1091,6 +1091,7 @@ void tcp_release_cb(struct sock *sk)

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

memory leak in tcp_md5_do_add

BUG: memory leak
unreferenced object 0xffff88810f927d20 (size 32):
comm "syz-executor.0", pid 5706, jiffies 4294944921 (age 114.940s)

hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:

[<ffffffff81536c45>] kmalloc_trace+0x25/0x90 mm/slab_common.c:1061
[<ffffffff840bf50e>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff840bf50e>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
[<ffffffff840bf50e>] tcp_md5_do_add+0x9e/0x160 net/ipv4/tcp_ipv4.c:1240
[<ffffffff84235fd0>] tcp_v6_parse_md5_keys+0x1b0/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff8409865d>] do_tcp_setsockopt+0x4dd/0x15a0 net/ipv4/tcp.c:3720
[<ffffffff840997b6>] tcp_setsockopt+0x96/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d4297e>] __sys_setsockopt+0x1ae/0x350 net/socket.c:2274
[<ffffffff83d42b43>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d42b43>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d42b43>] __x64_sys_setsockopt+0x23/0x30 net/socket.c:2282
[<ffffffff8497c758>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff8497c758>] do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80

[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd

BUG: memory leak

unreferenced object 0xffff888115272200 (size 32):
comm "syz-executor.0", pid 5708, jiffies 4294944921 (age 114.940s)

hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:

[<ffffffff81536c45>] kmalloc_trace+0x25/0x90 mm/slab_common.c:1061
[<ffffffff840bf50e>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff840bf50e>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
[<ffffffff840bf50e>] tcp_md5_do_add+0x9e/0x160 net/ipv4/tcp_ipv4.c:1240
[<ffffffff84235fd0>] tcp_v6_parse_md5_keys+0x1b0/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff8409865d>] do_tcp_setsockopt+0x4dd/0x15a0 net/ipv4/tcp.c:3720
[<ffffffff840997b6>] tcp_setsockopt+0x96/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d4297e>] __sys_setsockopt+0x1ae/0x350 net/socket.c:2274
[<ffffffff83d42b43>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d42b43>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d42b43>] __x64_sys_setsockopt+0x23/0x30 net/socket.c:2282
[<ffffffff8497c758>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff8497c758>] do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80

[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd

BUG: memory leak

unreferenced object 0xffff888114d14e40 (size 192):
comm "syz-executor.0", pid 5708, jiffies 4294944921 (age 114.940s)

hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
backtrace:

[<ffffffff81537378>] __do_kmalloc_node mm/slab_common.c:966 [inline]
[<ffffffff81537378>] __kmalloc+0x48/0x150 mm/slab_common.c:980

[<ffffffff83d453b5>] kmalloc include/linux/slab.h:584 [inline]
[<ffffffff83d453b5>] sock_kmalloc+0x65/0x90 net/core/sock.c:2635
[<ffffffff840bd4bd>] __tcp_md5_do_add+0x11d/0x300 net/ipv4/tcp_ipv4.c:1212
[<ffffffff840bf4d7>] tcp_md5_do_add+0x67/0x160 net/ipv4/tcp_ipv4.c:1254
[<ffffffff84235fd0>] tcp_v6_parse_md5_keys+0x1b0/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff8409865d>] do_tcp_setsockopt+0x4dd/0x15a0 net/ipv4/tcp.c:3720
[<ffffffff840997b6>] tcp_setsockopt+0x96/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d4297e>] __sys_setsockopt+0x1ae/0x350 net/socket.c:2274
[<ffffffff83d42b43>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d42b43>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d42b43>] __x64_sys_setsockopt+0x23/0x30 net/socket.c:2282
[<ffffffff8497c758>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff8497c758>] do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd

Tested on:

commit: ee3f96b1 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

console output: https://syzkaller.appspot.com/x/log.txt?x=13e9938a680000
kernel config: https://syzkaller.appspot.com/x/.config?x=ba49a3c2ed724b44
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
patch: https://syzkaller.appspot.com/x/patch.diff?x=11a20eb2680000

[ead...@sina.com]

From: Edward AD <ead...@sina.com>

please test memory leak in md5sig info and key

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ee3f96b16468

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 288693981b00..5d2587cd1920 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4704,6 +4704,7 @@ int tcp_abort(struct sock *sk, int err)
bh_unlock_sock(sk);
local_bh_enable();
tcp_write_queue_purge(sk);
+ printk("%s, %p\n", __func__, sk);
release_sock(sk);
return 0;
}
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c

index ea370afa70ed..562fb2b89494 100644

@@ -2310,9 +2314,12 @@ void tcp_v4_destroy_sock(struct sock *sk)
#ifdef CONFIG_TCP_MD5SIG

/* Clean up the MD5 key list, if any */
if (tp->md5sig_info) {

+ struct tcp_md5sig_info *md5sig;

tcp_clear_md5_list(sk);
- kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu);

- tp->md5sig_info = NULL;
+ md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
+ printk("%s, %p, %p \n", __func__, tp->md5sig_info, md5sig);
+ rcu_assign_pointer(tp->md5sig_info, NULL);
+ kfree_rcu(md5sig, rcu);
static_branch_slow_dec_deferred(&tcp_md5_needed);
}
#endif

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
memory leak in tcp_md5_do_add

BUG: memory leak

unreferenced object 0xffff888114cfae60 (size 32):
comm "syz-executor.0", pid 5701, jiffies 4294944895 (age 115.110s)

hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<ffffffff81536c45>] kmalloc_trace+0x25/0x90 mm/slab_common.c:1061
[<ffffffff840bf50e>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff840bf50e>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
[<ffffffff840bf50e>] tcp_md5_do_add+0x9e/0x160 net/ipv4/tcp_ipv4.c:1240

[<ffffffff84236000>] tcp_v6_parse_md5_keys+0x1b0/0x4a0 net/ipv6/tcp_ipv6.c:671

[<ffffffff8409865d>] do_tcp_setsockopt+0x4dd/0x15a0 net/ipv4/tcp.c:3720
[<ffffffff840997b6>] tcp_setsockopt+0x96/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d4297e>] __sys_setsockopt+0x1ae/0x350 net/socket.c:2274
[<ffffffff83d42b43>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d42b43>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d42b43>] __x64_sys_setsockopt+0x23/0x30 net/socket.c:2282
[<ffffffff8497c758>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff8497c758>] do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd

BUG: memory leak

unreferenced object 0xffff888115229840 (size 192):
comm "syz-executor.0", pid 5701, jiffies 4294944895 (age 115.110s)

hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
backtrace:
[<ffffffff81537378>] __do_kmalloc_node mm/slab_common.c:966 [inline]
[<ffffffff81537378>] __kmalloc+0x48/0x150 mm/slab_common.c:980
[<ffffffff83d453b5>] kmalloc include/linux/slab.h:584 [inline]
[<ffffffff83d453b5>] sock_kmalloc+0x65/0x90 net/core/sock.c:2635
[<ffffffff840bd4bd>] __tcp_md5_do_add+0x11d/0x300 net/ipv4/tcp_ipv4.c:1212
[<ffffffff840bf4d7>] tcp_md5_do_add+0x67/0x160 net/ipv4/tcp_ipv4.c:1254

[<ffffffff84236000>] tcp_v6_parse_md5_keys+0x1b0/0x4a0 net/ipv6/tcp_ipv6.c:671

[<ffffffff8409865d>] do_tcp_setsockopt+0x4dd/0x15a0 net/ipv4/tcp.c:3720
[<ffffffff840997b6>] tcp_setsockopt+0x96/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d4297e>] __sys_setsockopt+0x1ae/0x350 net/socket.c:2274
[<ffffffff83d42b43>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d42b43>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d42b43>] __x64_sys_setsockopt+0x23/0x30 net/socket.c:2282
[<ffffffff8497c758>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff8497c758>] do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd



Tested on:

commit: ee3f96b1 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

console output: https://syzkaller.appspot.com/x/log.txt?x=160385d6680000

kernel config: https://syzkaller.appspot.com/x/.config?x=ba49a3c2ed724b44
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=1619938a680000

[ead...@sina.com]

From: Edward AD <ead...@sina.com>

please test memory leak in md5sig info and key

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ee3f96b16468

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 288693981b00..5d2587cd1920 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4704,6 +4704,7 @@ int tcp_abort(struct sock *sk, int err)
bh_unlock_sock(sk);
local_bh_enable();
tcp_write_queue_purge(sk);
+ printk("%s, %p\n", __func__, sk);
release_sock(sk);
return 0;
}
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c

index ea370afa70ed..99310271505e 100644

@@ -2310,9 +2314,13 @@ void tcp_v4_destroy_sock(struct sock *sk)

#ifdef CONFIG_TCP_MD5SIG
/* Clean up the MD5 key list, if any */
if (tp->md5sig_info) {
+ struct tcp_md5sig_info *md5sig;
tcp_clear_md5_list(sk);
- kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu);
- tp->md5sig_info = NULL;
+ md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
+ printk("%s, %p, %p \n", __func__, tp->md5sig_info, md5sig);
+ rcu_assign_pointer(tp->md5sig_info, NULL);

+ synchronize_rcu();
+ kfree(md5sig);

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

BUG: scheduling while atomic in synchronize_rcu_expedited

tcp_clear_md5_list, ffff8881153f1f00
tcp_v4_destroy_sock, ffff8881157fe600, ffff8881157fe600
BUG: scheduling while atomic: swapper/0/0/0x00000103
Modules linked in:
Preemption disabled at:
[<ffffffff84991e6d>] schedule_preempt_disabled+0x1d/0x20 kernel/sched/core.c:6758
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.2.0-syzkaller-13115-gee3f96b16468-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/06/2023
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x72/0xa0 lib/dump_stack.c:106
__schedule_bug+0x90/0xa0 kernel/sched/core.c:5854
schedule_debug kernel/sched/core.c:5881 [inline]
__schedule+0xfc1/0x1310 kernel/sched/core.c:6516
schedule+0x59/0xa0 kernel/sched/core.c:6698
synchronize_rcu_expedited+0x295/0x3b0 kernel/rcu/tree_exp.h:1004
tcp_v4_destroy_sock+0x172/0x340 net/ipv4/tcp_ipv4.c:2322
inet_csk_destroy_sock+0x91/0x1d0 net/ipv4/inet_connection_sock.c:1195
tcp_done+0x134/0x1d0 net/ipv4/tcp.c:4656
tcp_write_err net/ipv4/tcp_timer.c:74 [inline]
tcp_write_timeout net/ipv4/tcp_timer.c:277 [inline]
tcp_retransmit_timer+0x41c/0xea0 net/ipv4/tcp_timer.c:510
tcp_write_timer_handler net/ipv4/tcp_timer.c:620 [inline]
tcp_write_timer_handler+0x229/0x3f0 net/ipv4/tcp_timer.c:594

tcp_write_timer+0x150/0x1b0 net/ipv4/tcp_timer.c:637
call_timer_fn+0x34/0x200 kernel/time/timer.c:1700
expire_timers kernel/time/timer.c:1751 [inline]
__run_timers.part.0+0x32a/0x440 kernel/time/timer.c:2022
__run_timers kernel/time/timer.c:2000 [inline]
run_timer_softirq+0x48/0xa0 kernel/time/timer.c:2035
__do_softirq+0xf5/0x302 kernel/softirq.c:571
invoke_softirq kernel/softirq.c:445 [inline]
__irq_exit_rcu kernel/softirq.c:650 [inline]
irq_exit_rcu+0xb0/0x110 kernel/softirq.c:662
sysvec_apic_timer_interrupt+0xa2/0xd0 arch/x86/kernel/apic/apic.c:1107
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645

RIP: 0010:native_irq_disable arch/x86/include/asm/irqflags.h:37 [inline]
RIP: 0010:arch_local_irq_disable arch/x86/include/asm/irqflags.h:72 [inline]
RIP: 0010:acpi_safe_halt+0x1b/0x20 drivers/acpi/processor_idle.c:113
Code: ed c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 65 48 8b 04 25 40 d3 02 00 48 8b 00 a8 08 75 0c 66 90 0f 00 2d 3f a3 4a 00 fb f4 <fa> c3 0f 1f 00 0f b6 47 08 3c 01 74 0b 3c 02 74 05 8b 7f 04 eb 9f
RSP: 0018:ffffffff85c03e10 EFLAGS: 00000246
RAX: 0000000000004000 RBX: 0000000000000001 RCX: 00000029d522e362
RDX: ffff88813bc00000 RSI: ffff88814115b000 RDI: ffff88814115b064
RBP: ffff88810492dc00 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000002 R11: 0000000000000000 R12: ffffffff85ec81a0
R13: ffff88814115b064 R14: 0000000000000001 R15: 0000000000000000
acpi_idle_enter+0xa3/0xf0 drivers/acpi/processor_idle.c:711
cpuidle_enter_state+0x77/0x2e0 drivers/cpuidle/cpuidle.c:267
cpuidle_enter+0x2d/0x40 drivers/cpuidle/cpuidle.c:388
cpuidle_idle_call kernel/sched/idle.c:215 [inline]
do_idle+0x1b1/0x210 kernel/sched/idle.c:282
cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:379
rest_init+0xc4/0xd0 init/main.c:732
arch_call_rest_init+0xe/0x20 init/main.c:894
start_kernel+0x600/0x9a0 init/main.c:1148
secondary_startup_64_no_verify+0xce/0xdb
</TASK>

----------------
Code disassembly (best guess):

0: ed in (%dx),%eax
1: c3 ret
2: 66 66 2e 0f 1f 84 00 data16 cs nopw 0x0(%rax,%rax,1)
9: 00 00 00 00
d: 66 90 xchg %ax,%ax
f: 65 48 8b 04 25 40 d3 mov %gs:0x2d340,%rax
16: 02 00
18: 48 8b 00 mov (%rax),%rax
1b: a8 08 test $0x8,%al
1d: 75 0c jne 0x2b
1f: 66 90 xchg %ax,%ax
21: 0f 00 2d 3f a3 4a 00 verw 0x4aa33f(%rip) # 0x4aa367
28: fb sti
29: f4 hlt
* 2a: fa cli <-- trapping instruction
2b: c3 ret
2c: 0f 1f 00 nopl (%rax)
2f: 0f b6 47 08 movzbl 0x8(%rdi),%eax
33: 3c 01 cmp $0x1,%al
35: 74 0b je 0x42
37: 3c 02 cmp $0x2,%al
39: 74 05 je 0x40
3b: 8b 7f 04 mov 0x4(%rdi),%edi
3e: eb 9f jmp 0xffffffdf

Tested on:

commit: ee3f96b1 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

console output: https://syzkaller.appspot.com/x/log.txt?x=178dec86680000

kernel config: https://syzkaller.appspot.com/x/.config?x=ba49a3c2ed724b44
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=179385e1680000

[syzbot]

Auto-closing this bug as obsolete.
No recent activity, existing reproducers are no longer triggering the issue.