[v5.15] INFO: rcu detected stall in cleanup_net

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: d54cfc420586 Linux 5.15.120
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=10a060b0a80000
kernel config: https://syzkaller.appspot.com/x/.config?x=c67dae6b27f0d6cd
dashboard link: https://syzkaller.appspot.com/bug?extid=f5d4d279dfe9b422732e
compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/4aa2cbf1e81c/disk-d54cfc42.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/2ebee78b098c/vmlinux-d54cfc42.xz
kernel image: https://storage.googleapis.com/syzbot-assets/4e17f9ac4b6a/bzImage-d54cfc42.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+f5d4d2...@syzkaller.appspotmail.com

rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P1240/1:b..l
(detected by 1, t=10502 jiffies, g=37373, q=456)
task:kworker/u4:5 state:R running task stack:22552 pid: 1240 ppid: 2 flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5026 [inline]
__schedule+0x12c4/0x4590 kernel/sched/core.c:6372
preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6776
irqentry_exit+0x53/0x80 kernel/entry/common.c:426
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:get_current arch/x86/include/asm/current.h:15 [inline]
RIP: 0010:write_comp_data kernel/kcov.c:217 [inline]
RIP: 0010:__sanitizer_cov_trace_const_cmp4+0x4/0x80 kernel/kcov.c:284
Code: 89 f8 89 f6 48 c7 44 0a 08 03 00 00 00 48 89 44 0a 10 48 89 74 0a 18 4c 89 44 0a 20 49 ff c1 4c 89 09 c3 0f 1f 00 4c 8b 04 24 <65> 48 8b 15 b4 6b 82 7e 65 8b 05 b5 6b 82 7e a9 00 01 ff 00 74 10
RSP: 0018:ffffc9000558fa58 EFLAGS: 00000246
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff88801d7d3b80
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc9000558fb70 R08: ffffffff88bc9fe1 R09: fffffbfff1f7a621
R10: 0000000000000000 R11: dffffc0000000001 R12: 00000000000750b4
R13: 1ffff1100a6c0202 R14: ffffffff917c8340 R15: dffffc0000000000
rcu_read_unlock include/linux/rcupdate.h:725 [inline]
inet_twsk_purge+0x8f1/0xa20 net/ipv4/inet_timewait_sock.c:299
ops_exit_list net/core/net_namespace.c:174 [inline]
cleanup_net+0x763/0xb60 net/core/net_namespace.c:596
process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2307
worker_thread+0xaca/0x1280 kernel/workqueue.c:2454
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g37373 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: Possible timer handling issue on cpu=0 timer-softirq=15753
rcu: rcu_preempt kthread starved for 10500 jiffies! g37373 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:I stack:27000 pid: 15 ppid: 2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5026 [inline]
__schedule+0x12c4/0x4590 kernel/sched/core.c:6372
schedule+0x11b/0x1f0 kernel/sched/core.c:6455
schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1884
rcu_gp_fqs_loop+0x2af/0xf70 kernel/rcu/tree.c:1959
rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2132
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 8246 Comm: syz-executor.2 Not tainted 5.15.120-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/03/2023
RIP: 0010:native_irq_disable arch/x86/include/asm/irqflags.h:40 [inline]
RIP: 0010:arch_local_irq_disable arch/x86/include/asm/irqflags.h:75 [inline]
RIP: 0010:arch_local_irq_save arch/x86/include/asm/irqflags.h:107 [inline]
RIP: 0010:lock_release+0x174/0x9a0 kernel/locking/lockdep.c:5638
Code: c7 84 24 b0 00 00 00 00 00 00 00 9c 8f 84 24 b0 00 00 00 42 80 3c 3b 00 74 08 4c 89 f7 e8 84 d9 66 00 48 8b 9c 24 b0 00 00 00 <fa> 48 c7 c7 60 0b 8b 8a e8 4f 28 b9 08 65 ff 05 e8 e3 9f 7e 48 8d
RSP: 0018:ffffc90000007b60 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000000046 RCX: ffffffff81629438
RDX: 0000000000000000 RSI: ffffffff8ad86260 RDI: ffffffff8ad86220
RBP: ffffc90000007ca0 R08: dffffc0000000000 R09: fffffbfff1bc7e0e
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000000f78
R13: ffffffff886ae4bb R14: ffffc90000007c10 R15: dffffc0000000000
FS: 000055555556d400(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f48901098c2 CR3: 000000007d47d000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<IRQ>
__raw_spin_unlock include/linux/spinlock_api_smp.h:150 [inline]
_raw_spin_unlock+0x12/0x40 kernel/locking/spinlock.c:186
spin_unlock include/linux/spinlock.h:403 [inline]
advance_sched+0x68b/0x940 net/sched/sch_taprio.c:765
__run_hrtimer kernel/time/hrtimer.c:1685 [inline]
__hrtimer_run_queues+0x598/0xcf0 kernel/time/hrtimer.c:1749
hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1811
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
__sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
sysvec_apic_timer_interrupt+0x8c/0xb0 arch/x86/kernel/apic/apic.c:1096
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:extract_string fs/d_path.c:20 [inline]
RIP: 0010:d_absolute_path+0x191/0x300 fs/d_path.c:237
Code: 18 48 8d 74 24 40 4c 89 fa e8 1b f3 ff ff 89 c3 bf 01 00 00 00 89 c6 e8 0d d6 9c ff 83 fb 02 0f 8d c4 00 00 00 48 8b 7c 24 08 <48> 89 f8 48 c1 e8 03 42 0f b6 04 28 84 c0 0f 85 14 01 00 00 8b 5c
RSP: 0018:ffffc90004a3f960 EFLAGS: 00000297
RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88804da88000
RDX: ffff88804da88000 RSI: 0000000000000001 RDI: ffffc90004a3f9c8
RBP: ffffc90004a3fa40 R08: ffffffff81e32df3 R09: fffffbfff1bc7e0e
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff88807ee76ffd
R13: dffffc0000000000 R14: 1ffff92000947f38 R15: ffffc90004a3f9c0
tomoyo_get_absolute_path security/tomoyo/realpath.c:101 [inline]
tomoyo_realpath_from_path+0x2fd/0x5e0 security/tomoyo/realpath.c:276
tomoyo_get_realpath security/tomoyo/file.c:151 [inline]
tomoyo_path_perm+0x273/0x6b0 security/tomoyo/file.c:822
tomoyo_path_symlink+0xda/0x110 security/tomoyo/tomoyo.c:199
security_path_symlink+0xd9/0x130 security/security.c:1179
do_symlinkat+0x129/0x600 fs/namei.c:4382
__do_sys_symlinkat fs/namei.c:4404 [inline]
__se_sys_symlinkat fs/namei.c:4401 [inline]
__x64_sys_symlinkat+0x95/0xa0 fs/namei.c:4401
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7f4890086bb7
Code: 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 0a 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffe988e4d68 EFLAGS: 00000202 ORIG_RAX: 000000000000010a
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4890086bb7
RDX: 00007f48900d253f RSI: 00000000ffffff9c RDI: 00007ffe988e4e30
RBP: 0000000000000000 R08: 0000000000000000 R09: 00007ffe988e4c00
R10: 00007ffe988e4ab7 R11: 0000000000000202 R12: 0000000000000001
R13: 0000000000000000 R14: 0000000000000001 R15: 00007ffe988e4e30
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to change bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 61fd484b2cf6 Linux 6.1.38
git tree: linux-6.1.y
console output: https://syzkaller.appspot.com/x/log.txt?x=14365922a80000
kernel config: https://syzkaller.appspot.com/x/.config?x=6a21f730857cfa53
dashboard link: https://syzkaller.appspot.com/bug?extid=e361cbd0ce11d6558816

compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:

disk image: https://storage.googleapis.com/syzbot-assets/eb0882a0450c/disk-61fd484b.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/483d02c4c64c/vmlinux-61fd484b.xz
kernel image: https://storage.googleapis.com/syzbot-assets/3b4bb7e6bcff/bzImage-61fd484b.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:

Reported-by: syzbot+e361cb...@syzkaller.appspotmail.com

rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:

rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P20877/1:b..l
(detected by 0, t=10502 jiffies, g=167369, q=113 ncpus=2)
task:kworker/u4:3 state:R running task stack:24536 pid:20877 ppid:2 flags:0x00004000

Workqueue: netns cleanup_net
Call Trace:
<TASK>

context_switch kernel/sched/core.c:5241 [inline]
__schedule+0x132c/0x4330 kernel/sched/core.c:6554
preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6866
irqentry_exit+0x53/0x80 kernel/entry/common.c:433
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:649
RIP: 0010:lock_acquire+0x26f/0x5a0 kernel/locking/lockdep.c:5673
Code: 2b 00 74 08 4c 89 f7 e8 5f 6d 76 00 f6 44 24 61 02 0f 85 84 01 00 00 41 f7 c7 00 02 00 00 74 01 fb 48 c7 44 24 40 0e 36 e0 45 <4b> c7 44 25 00 00 00 00 00 43 c7 44 25 09 00 00 00 00 43 c7 44 25
RSP: 0018:ffffc9000454f8e0 EFLAGS: 00000206
RAX: 0000000000000001 RBX: 1ffff920008a9f28 RCX: 1ffff920008a9ec8
RDX: dffffc0000000000 RSI: ffffffff8aebe4a0 RDI: ffffffff8b3cce60
RBP: ffffc9000454fa40 R08: dffffc0000000000 R09: fffffbfff2052845
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff920008a9f24
R13: dffffc0000000000 R14: ffffc9000454f940 R15: 0000000000000246
rcu_lock_acquire+0x2a/0x30 include/linux/rcupdate.h:306
rcu_read_lock include/linux/rcupdate.h:747 [inline]
inet_twsk_purge+0x124/0xad0 net/ipv4/inet_timewait_sock.c:295
ops_exit_list net/core/net_namespace.c:174 [inline]
cleanup_net+0x763/0xb60 net/core/net_namespace.c:601
process_one_work+0x8aa/0x11f0 kernel/workqueue.c:2289
worker_thread+0xa5f/0x1210 kernel/workqueue.c:2436
kthread+0x26e/0x300 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
</TASK>
rcu: rcu_preempt kthread starved for 10520 jiffies! g167369 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1

rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:

task:rcu_preempt state:R running task stack:26712 pid:16 ppid:2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5241 [inline]
__schedule+0x132c/0x4330 kernel/sched/core.c:6554
schedule+0xbf/0x180 kernel/sched/core.c:6630
schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1935
rcu_gp_fqs_loop+0x2c2/0x1010 kernel/rcu/tree.c:1661
rcu_gp_kthread+0xa3/0x3a0 kernel/rcu/tree.c:1860
kthread+0x26e/0x300 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306

</TASK>
rcu: Stack dump where RCU GP kthread last ran:

Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 2064 Comm: syz-executor.3 Not tainted 6.1.38-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/03/2023

RIP: 0010:__seqprop_assert include/linux/seqlock.h:269 [inline]
RIP: 0010:psi_group_change+0xb7/0x1310 kernel/sched/psi.c:751
Code: df 48 ba 00 00 00 00 00 fc ff df 48 03 7d 00 48 c7 c0 64 88 53 8e 48 c1 e8 03 0f b6 04 10 84 c0 48 89 3c 24 0f 85 e6 0b 00 00 <83> 3d 16 90 eb 0c 00 74 28 65 8b 05 69 80 9a 7e 85 c0 75 1d 65 8b
RSP: 0018:ffffc90003d671e8 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffffe8ffffd88080
RBP: ffffffff8c999848 R08: 000001fec39be0e2 R09: 0000000000000001
R10: ffffffff8fc97058 R11: ffffffff8fc97050 R12: 0000000000000001
R13: ffff88807cd8f000 R14: 0000000000000004 R15: 0000000000000001
FS: 00007f0d9e8a26c0(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000000020590030 CR3: 000000008f036000 CR4: 00000000003506e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>

<TASK>
psi_task_switch+0x43d/0x770 kernel/sched/psi.c:952
psi_sched_switch kernel/sched/stats.h:199 [inline]
__schedule+0x1ec8/0x4330 kernel/sched/core.c:6549
schedule+0xbf/0x180 kernel/sched/core.c:6630
schedule_timeout+0xac/0x300 kernel/time/timer.c:1911
unix_wait_for_peer+0x24b/0x330 net/unix/af_unix.c:1449
unix_dgram_sendmsg+0x1348/0x2050 net/unix/af_unix.c:2047
sock_sendmsg_nosec net/socket.c:716 [inline]
sock_sendmsg net/socket.c:736 [inline]
____sys_sendmsg+0x59e/0x8f0 net/socket.c:2482
___sys_sendmsg net/socket.c:2536 [inline]
__sys_sendmmsg+0x3ab/0x730 net/socket.c:2622
__do_sys_sendmmsg net/socket.c:2651 [inline]
__se_sys_sendmmsg net/socket.c:2648 [inline]
__x64_sys_sendmmsg+0x9c/0xb0 net/socket.c:2648

do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80

entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f0d9da7cb29
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f0d9e8a20c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 00007f0d9db9bf80 RCX: 00007f0d9da7cb29
RDX: 0000000000000318 RSI: 00000000200bd000 RDI: 0000000000000004
RBP: 00007f0d9dac847a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007f0d9db9bf80 R15: 00007fff2aa23038