INFO: rcu detected stall in __do_softirq (2)

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: 7472c402 Linux 4.19.108
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1625dd65e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=6d889e71eea7bde
dashboard link: https://syzkaller.appspot.com/bug?extid=dd71c5c0d713d4f48d9f
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+dd71c5...@syzkaller.appspotmail.com

rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P27877
rcu: (detected by 0, t=10503 jiffies, g=141957, q=496441)
syz-executor.2 R running task 25104 27877 9402 0x0000000c
Call Trace:
<IRQ>
sched_show_task kernel/sched/core.c:5338 [inline]
sched_show_task.cold+0x2e2/0x343 kernel/sched/core.c:5313
rcu_print_detail_task_stall_rnp+0xe7/0x120 kernel/rcu/tree_plugin.h:627
rcu_print_detail_task_stall kernel/rcu/tree_plugin.h:640 [inline]
print_other_cpu_stall kernel/rcu/tree.c:1418 [inline]
check_cpu_stall kernel/rcu/tree.c:1557 [inline]
__rcu_pending kernel/rcu/tree.c:3293 [inline]
rcu_pending kernel/rcu/tree.c:3336 [inline]
rcu_check_callbacks.cold+0x84a/0xddc kernel/rcu/tree.c:2682
update_process_times+0x2a/0x70 kernel/time/timer.c:1638
tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:168
tick_sched_timer+0x42/0x130 kernel/time/tick-sched.c:1278
__run_hrtimer kernel/time/hrtimer.c:1401 [inline]
__hrtimer_run_queues+0x2fc/0xd50 kernel/time/hrtimer.c:1463
hrtimer_interrupt+0x312/0x770 kernel/time/hrtimer.c:1521
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1067 [inline]
smp_apic_timer_interrupt+0x10c/0x550 arch/x86/kernel/apic/apic.c:1092
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
RIP: 0010:rcu_do_batch kernel/rcu/tree.c:2590 [inline]
RIP: 0010:invoke_rcu_callbacks kernel/rcu/tree.c:2897 [inline]
RIP: 0010:__rcu_process_callbacks kernel/rcu/tree.c:2864 [inline]
RIP: 0010:rcu_process_callbacks+0xb51/0x17f0 kernel/rcu/tree.c:2881
Code: 48 89 df ff 53 08 48 c7 c2 97 4f 56 81 be 01 00 00 00 48 c7 c7 20 29 b9 88 e8 8b 56 f9 ff 48 8b 84 24 88 00 00 00 48 89 04 24 <48> f7 d8 48 39 c5 7f 57 48 8b 44 24 10 80 38 00 0f 85 96 06 00 00
RSP: 0018:ffff8880ae607e58 EFLAGS: 00000283 ORIG_RAX: ffffffffffffff13
RAX: ffffffffffff87c4 RBX: ffff8880861a7d40 RCX: 1ffff11005dcc156
RDX: dffffc0000000000 RSI: 0000000000000002 RDI: 0000000000000286
RBP: 7fffffffffffffff R08: 0000000000000000 R09: 0000000000000001
R10: ffffed1015cc4732 R11: ffff8880ae623993 R12: ffff8880861a7d48
R13: ffffffff81a733d0 R14: ffff8880ae607ed0 R15: dffffc0000000000
__do_softirq+0x26c/0x93c kernel/softirq.c:292
do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1091
</IRQ>
do_softirq.part.0+0x10f/0x160 kernel/softirq.c:336
do_softirq kernel/softirq.c:328 [inline]
__local_bh_enable_ip+0x20e/0x270 kernel/softirq.c:189
local_bh_enable include/linux/bottom_half.h:32 [inline]
ipt_do_table+0xd0b/0x1840 net/ipv4/netfilter/ip_tables.c:361
ipt_mangle_out net/ipv4/netfilter/iptable_mangle.c:59 [inline]
iptable_mangle_hook+0x21f/0x510 net/ipv4/netfilter/iptable_mangle.c:84
nf_hook_entry_hookfn include/linux/netfilter.h:119 [inline]
nf_hook_slow+0xba/0x1e0 net/netfilter/core.c:511
nf_hook include/linux/netfilter.h:244 [inline]
__ip_local_out+0x3da/0x810 net/ipv4/ip_output.c:113
ip_local_out+0x26/0x170 net/ipv4/ip_output.c:122
ip_send_skb+0x3e/0xe0 net/ipv4/ip_output.c:1446
udp_send_skb.isra.0+0x6c2/0x11f0 net/ipv4/udp.c:845
udp_sendmsg+0x1df1/0x25d0 net/ipv4/udp.c:1132
inet_sendmsg+0x12e/0x590 net/ipv4/af_inet.c:798
sock_sendmsg_nosec net/socket.c:622 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:632
___sys_sendmsg+0x3e2/0x920 net/socket.c:2115
__sys_sendmmsg+0x195/0x470 net/socket.c:2210
__do_sys_sendmmsg net/socket.c:2239 [inline]
__se_sys_sendmmsg net/socket.c:2236 [inline]
__x64_sys_sendmmsg+0x99/0x100 net/socket.c:2236
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45c4a9
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f85d3bd7c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 00007f85d3bd86d4 RCX: 000000000045c4a9
RDX: 000000000800001d RSI: 0000000020007fc0 RDI: 0000000000000004
RBP: 000000000076bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000000008d2 R14: 00000000004cb3b9 R15: 000000000076bf2c
syz-executor.2 R running task 25104 27877 9402 0x8000000c
Call Trace:
<IRQ>
sched_show_task kernel/sched/core.c:5338 [inline]
sched_show_task.cold+0x2e2/0x343 kernel/sched/core.c:5313
rcu_print_detail_task_stall_rnp+0xe7/0x120 kernel/rcu/tree_plugin.h:627
rcu_print_detail_task_stall kernel/rcu/tree_plugin.h:642 [inline]
print_other_cpu_stall kernel/rcu/tree.c:1418 [inline]
check_cpu_stall kernel/rcu/tree.c:1557 [inline]
__rcu_pending kernel/rcu/tree.c:3293 [inline]
rcu_pending kernel/rcu/tree.c:3336 [inline]
rcu_check_callbacks.cold+0x8a8/0xddc kernel/rcu/tree.c:2682
update_process_times+0x2a/0x70 kernel/time/timer.c:1638
tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:168
tick_sched_timer+0x42/0x130 kernel/time/tick-sched.c:1278
__run_hrtimer kernel/time/hrtimer.c:1401 [inline]
__hrtimer_run_queues+0x2fc/0xd50 kernel/time/hrtimer.c:1463
hrtimer_interrupt+0x312/0x770 kernel/time/hrtimer.c:1521
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1067 [inline]
smp_apic_timer_interrupt+0x10c/0x550 arch/x86/kernel/apic/apic.c:1092
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
RIP: 0010:rcu_do_batch kernel/rcu/tree.c:2590 [inline]
RIP: 0010:invoke_rcu_callbacks kernel/rcu/tree.c:2897 [inline]
RIP: 0010:__rcu_process_callbacks kernel/rcu/tree.c:2864 [inline]
RIP: 0010:rcu_process_callbacks+0xb51/0x17f0 kernel/rcu/tree.c:2881
Code: 48 89 df ff 53 08 48 c7 c2 97 4f 56 81 be 01 00 00 00 48 c7 c7 20 29 b9 88 e8 8b 56 f9 ff 48 8b 84 24 88 00 00 00 48 89 04 24 <48> f7 d8 48 39 c5 7f 57 48 8b 44 24 10 80 38 00 0f 85 96 06 00 00
RSP: 0018:ffff8880ae607e58 EFLAGS: 00000283 ORIG_RAX: ffffffffffffff13
RAX: ffffffffffff87c4 RBX: ffff8880861a7d40 RCX: 1ffff11005dcc156
RDX: dffffc0000000000 RSI: 0000000000000002 RDI: 0000000000000286
RBP: 7fffffffffffffff R08: 0000000000000000 R09: 0000000000000001
R10: ffffed1015cc4732 R11: ffff8880ae623993 R12: ffff8880861a7d48
R13: ffffffff81a733d0 R14: ffff8880ae607ed0 R15: dffffc0000000000
__do_softirq+0x26c/0x93c kernel/softirq.c:292
do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1091
</IRQ>
do_softirq.part.0+0x10f/0x160 kernel/softirq.c:336
do_softirq kernel/softirq.c:328 [inline]
__local_bh_enable_ip+0x20e/0x270 kernel/softirq.c:189
local_bh_enable include/linux/bottom_half.h:32 [inline]
ipt_do_table+0xd0b/0x1840 net/ipv4/netfilter/ip_tables.c:361
ipt_mangle_out net/ipv4/netfilter/iptable_mangle.c:59 [inline]
iptable_mangle_hook+0x21f/0x510 net/ipv4/netfilter/iptable_mangle.c:84
nf_hook_entry_hookfn include/linux/netfilter.h:119 [inline]
nf_hook_slow+0xba/0x1e0 net/netfilter/core.c:511
nf_hook include/linux/netfilter.h:244 [inline]
__ip_local_out+0x3da/0x810 net/ipv4/ip_output.c:113
ip_local_out+0x26/0x170 net/ipv4/ip_output.c:122
ip_send_skb+0x3e/0xe0 net/ipv4/ip_output.c:1446
udp_send_skb.isra.0+0x6c2/0x11f0 net/ipv4/udp.c:845
udp_sendmsg+0x1df1/0x25d0 net/ipv4/udp.c:1132
inet_sendmsg+0x12e/0x590 net/ipv4/af_inet.c:798
sock_sendmsg_nosec net/socket.c:622 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:632
___sys_sendmsg+0x3e2/0x920 net/socket.c:2115
__sys_sendmmsg+0x195/0x470 net/socket.c:2210
__do_sys_sendmmsg net/socket.c:2239 [inline]
__se_sys_sendmmsg net/socket.c:2236 [inline]
__x64_sys_sendmmsg+0x99/0x100 net/socket.c:2236
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45c4a9
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f85d3bd7c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 00007f85d3bd86d4 RCX: 000000000045c4a9
RDX: 000000000800001d RSI: 0000000020007fc0 RDI: 0000000000000004
RBP: 000000000076bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000000008d2 R14: 00000000004cb3b9 R15: 000000000076bf2c
rcu: INFO: rcu_bh self-detected stall on CPU
rcu: 0-....: (25334 ticks this GP) idle=cae/1/0x4000000000000002 softirq=121618/132916 fqs=5184
rcu: (t=10502 jiffies g=-1043 q=4)
NMI backtrace for cpu 0
CPU: 0 PID: 27877 Comm: syz-executor.2 Not tainted 4.19.108-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x188/0x20d lib/dump_stack.c:118
nmi_cpu_backtrace.cold+0x63/0xa2 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x1a6/0x1eb lib/nmi_backtrace.c:62
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_dump_cpu_stacks+0x170/0x1bb kernel/rcu/tree.c:1340
print_cpu_stall kernel/rcu/tree.c:1478 [inline]
check_cpu_stall kernel/rcu/tree.c:1550 [inline]
__rcu_pending kernel/rcu/tree.c:3293 [inline]
rcu_pending kernel/rcu/tree.c:3336 [inline]
rcu_check_callbacks.cold+0x634/0xddc kernel/rcu/tree.c:2682
update_process_times+0x2a/0x70 kernel/time/timer.c:1638
tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:168
tick_sched_timer+0x42/0x130 kernel/time/tick-sched.c:1278
__run_hrtimer kernel/time/hrtimer.c:1401 [inline]
__hrtimer_run_queues+0x2fc/0xd50 kernel/time/hrtimer.c:1463
hrtimer_interrupt+0x312/0x770 kernel/time/hrtimer.c:1521
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1067 [inline]
smp_apic_timer_interrupt+0x10c/0x550 arch/x86/kernel/apic/apic.c:1092
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
RIP: 0010:rcu_cblist_dequeue+0x60/0xe0 kernel/rcu/rcu_segcblist.c:54
Code: fc ff df 48 89 f9 48 c1 e9 03 80 3c 11 00 75 64 48 ba 00 00 00 00 00 fc ff df 48 89 c1 48 83 6b 10 01 48 c1 e9 03 80 3c 11 00 <75> 36 48 8b 10 48 85 d2 48 89 13 74 06 48 83 c4 08 5b c3 48 8d 7b
RSP: 0018:ffff8880ae607e40 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
RAX: ffff8880a8a0f780 RBX: ffff8880ae607ed0 RCX: 1ffff11015141ef0
RDX: dffffc0000000000 RSI: 0000000000000002 RDI: ffff8880ae607ee0
RBP: 7fffffffffffffff R08: 0000000000000000 R09: 0000000000000001
R10: ffffed1015cc4732 R11: ffff8880ae623993 R12: ffff8880a8a0fa08
R13: ffffffff81a733d0 R14: ffff8880ae607ed0 R15: dffffc0000000000
rcu_do_batch kernel/rcu/tree.c:2582 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2897 [inline]
__rcu_process_callbacks kernel/rcu/tree.c:2864 [inline]
rcu_process_callbacks+0xbb8/0x17f0 kernel/rcu/tree.c:2881
__do_softirq+0x26c/0x93c kernel/softirq.c:292
do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1091
</IRQ>
do_softirq.part.0+0x10f/0x160 kernel/softirq.c:336
do_softirq kernel/softirq.c:328 [inline]
__local_bh_enable_ip+0x20e/0x270 kernel/softirq.c:189
local_bh_enable include/linux/bottom_half.h:32 [inline]
ipt_do_table+0xd0b/0x1840 net/ipv4/netfilter/ip_tables.c:361
ipt_mangle_out net/ipv4/netfilter/iptable_mangle.c:59 [inline]
iptable_mangle_hook+0x21f/0x510 net/ipv4/netfilter/iptable_mangle.c:84
nf_hook_entry_hookfn include/linux/netfilter.h:119 [inline]
nf_hook_slow+0xba/0x1e0 net/netfilter/core.c:511
nf_hook include/linux/netfilter.h:244 [inline]
__ip_local_out+0x3da/0x810 net/ipv4/ip_output.c:113
ip_local_out+0x26/0x170 net/ipv4/ip_output.c:122
ip_send_skb+0x3e/0xe0 net/ipv4/ip_output.c:1446
udp_send_skb.isra.0+0x6c2/0x11f0 net/ipv4/udp.c:845
udp_sendmsg+0x1df1/0x25d0 net/ipv4/udp.c:1132
inet_sendmsg+0x12e/0x590 net/ipv4/af_inet.c:798
sock_sendmsg_nosec net/socket.c:622 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:632
___sys_sendmsg+0x3e2/0x920 net/socket.c:2115
__sys_sendmmsg+0x195/0x470 net/socket.c:2210
__do_sys_sendmmsg net/socket.c:2239 [inline]
__se_sys_sendmmsg net/socket.c:2236 [inline]
__x64_sys_sendmmsg+0x99/0x100 net/socket.c:2236
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45c4a9
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f85d3bd7c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 00007f85d3bd86d4 RCX: 000000000045c4a9
RDX: 000000000800001d RSI: 0000000020007fc0 RDI: 0000000000000004
RBP: 000000000076bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000000008d2 R14: 00000000004cb3b9 R15: 000000000076bf2c


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.