INFO: rcu detected stall in sctp_generate_heartbeat_event

7 views
Skip to first unread message

syzbot

unread,
Dec 27, 2019, 10:40:07 AM12/27/19
to syzkaller...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: e1f7d50a Linux 4.14.160
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=17247615e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=46599517442ad9fb
dashboard link: https://syzkaller.appspot.com/bug?extid=906345f0d8cc08c096b5
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+906345...@syzkaller.appspotmail.com

INFO: rcu_preempt self-detected stall on CPU
0-...: (1 GPs behind) idle=102/140000000000002/0 softirq=29948/29951 fqs=4
(t=10500 jiffies g=8448 c=8447 q=426)
rcu_preempt kthread starved for 10436 jiffies! g8448 c8447 f0x0
RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1
rcu_preempt R running task 29152 8 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2808 [inline]
__schedule+0x7b8/0x1cd0 kernel/sched/core.c:3384
schedule+0x92/0x1c0 kernel/sched/core.c:3428
schedule_timeout+0x43e/0xe10 kernel/time/timer.c:1746
rcu_gp_kthread+0xbf4/0x1ec0 kernel/rcu/tree.c:2255
kthread+0x319/0x430 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
NMI backtrace for cpu 0
CPU: 0 PID: 12557 Comm: syz-executor.3 Not tainted 4.14.160-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x142/0x197 lib/dump_stack.c:58
nmi_cpu_backtrace.cold+0x57/0x94 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x141/0x189 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
rcu_dump_cpu_stacks+0x186/0x1d2 kernel/rcu/tree.c:1396
print_cpu_stall kernel/rcu/tree.c:1542 [inline]
check_cpu_stall kernel/rcu/tree.c:1610 [inline]
__rcu_pending kernel/rcu/tree.c:3390 [inline]
rcu_pending kernel/rcu/tree.c:3452 [inline]
rcu_check_callbacks.cold+0x43d/0xd0a kernel/rcu/tree.c:2792
update_process_times+0x31/0x70 kernel/time/timer.c:1590
tick_sched_handle+0x85/0x160 kernel/time/tick-sched.c:161
tick_sched_timer+0x43/0x130 kernel/time/tick-sched.c:1219
__run_hrtimer kernel/time/hrtimer.c:1220 [inline]
__hrtimer_run_queues+0x270/0xbc0 kernel/time/hrtimer.c:1284
hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1318
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:update_stack_state+0xf4/0x590 arch/x86/kernel/unwind_frame.c:225
RSP: 0018:ffff8880aec07168 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: 0000000000000000 RBX: ffff8880449075d8 RCX: 0000000000000005
RDX: 1ffff11015d80ffd RSI: ffff8880449075d9 RDI: ffff8880aec0730d
RBP: ffff8880aec07210 R08: 0000000000000001 R09: ffff8880aec07330
R10: ffff8880aec07300 R11: ffff8880aec07310 R12: ffff8880449075d8
R13: ffff8880aec072d8 R14: 1ffff11015d80e35 R15: ffff8880449075d9
unwind_next_frame.part.0+0x188/0x8b0 arch/x86/kernel/unwind_frame.c:333
unwind_next_frame+0x3b/0x50 arch/x86/kernel/unwind_frame.c:287
__save_stack_trace+0x6b/0xd0 arch/x86/kernel/stacktrace.c:44
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x45/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_kmalloc mm/kasan/kasan.c:551 [inline]
kasan_kmalloc+0xce/0xf0 mm/kasan/kasan.c:529
kmem_cache_alloc_node_trace+0x153/0x770 mm/slab.c:3661
__do_kmalloc_node mm/slab.c:3681 [inline]
__kmalloc_node_track_caller+0x3d/0x80 mm/slab.c:3696
__kmalloc_reserve.isra.0+0x40/0xe0 net/core/skbuff.c:137
__alloc_skb+0xcf/0x500 net/core/skbuff.c:205
alloc_skb include/linux/skbuff.h:980 [inline]
sctp_packet_transmit+0x351/0x3010 net/sctp/output.c:580
sctp_outq_flush+0xd13/0x31b0 net/sctp/outqueue.c:1187
sctp_outq_uncork+0x60/0x70 net/sctp/outqueue.c:761
sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1776 [inline]
sctp_side_effects net/sctp/sm_sideeffect.c:1179 [inline]
sctp_do_sm+0x417/0x4ae0 net/sctp/sm_sideeffect.c:1150
sctp_generate_heartbeat_event+0x1dd/0x400 net/sctp/sm_sideeffect.c:391
call_timer_fn+0x161/0x670 kernel/time/timer.c:1279
expire_timers kernel/time/timer.c:1318 [inline]
__run_timers kernel/time/timer.c:1636 [inline]
__run_timers kernel/time/timer.c:1604 [inline]
run_timer_softirq+0x5b7/0x1520 kernel/time/timer.c:1649
__do_softirq+0x244/0x9a0 kernel/softirq.c:288
invoke_softirq kernel/softirq.c:368 [inline]
irq_exit+0x160/0x1b0 kernel/softirq.c:409
exiting_irq arch/x86/include/asm/apic.h:648 [inline]
smp_apic_timer_interrupt+0x146/0x5e0 arch/x86/kernel/apic/apic.c:1102
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
</IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:779
[inline]
RIP: 0010:lock_acquire+0x1ea/0x430 kernel/locking/lockdep.c:3997
RSP: 0018:ffff888044907680 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff0fe2d29 RBX: ffff88807d4e0400 RCX: 00000000780ecdb0
RDX: dffffc0000000000 RSI: ffff88807d4e0ca8 RDI: 0000000000000282
RBP: ffff8880449076c8 R08: 0000000000000000 R09: ffff88807d4e0cc8
R10: ffff88807d4e0ca8 R11: ffff88807d4e0400 R12: ffffffff87f88960
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002
rcu_lock_acquire include/linux/rcupdate.h:242 [inline]
rcu_read_lock include/linux/rcupdate.h:629 [inline]
is_bpf_text_address+0x38/0x120 kernel/bpf/core.c:456
kernel_text_address+0x73/0xf0 kernel/extable.c:150
__kernel_text_address+0xd/0x40 kernel/extable.c:105
unwind_get_return_address arch/x86/kernel/unwind_frame.c:18 [inline]
unwind_get_return_address+0x61/0xa0 arch/x86/kernel/unwind_frame.c:13
__save_stack_trace+0x7b/0xd0 arch/x86/kernel/stacktrace.c:45
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x45/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_kmalloc mm/kasan/kasan.c:551 [inline]
kasan_kmalloc+0xce/0xf0 mm/kasan/kasan.c:529
kasan_slab_alloc+0xf/0x20 mm/kasan/kasan.c:489
kmem_cache_alloc+0x12e/0x780 mm/slab.c:3552
__split_vma+0xc9/0x6d0 mm/mmap.c:2585
split_vma+0x8d/0xc0 mm/mmap.c:2648
userfaultfd_register fs/userfaultfd.c:1455 [inline]
userfaultfd_ioctl+0x25c6/0x2fe0 fs/userfaultfd.c:1845
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:500 [inline]
do_vfs_ioctl+0x7ae/0x1060 fs/ioctl.c:684
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x45a919
RSP: 002b:00007fe2fe476c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 000000000045a919
RDX: 0000000020000040 RSI: 00000000c020aa00 RDI: 0000000000000003
RBP: 000000000075bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fe2fe4776d4
R13: 00000000004c6122 R14: 00000000004dc680 R15: 00000000ffffffff
INFO: rcu_sched detected stalls on CPUs/tasks:
0-...: (1 GPs behind) idle=102/140000000000001/0 softirq=29948/29951 fqs=5
(detected by 1, t=10594 jiffies, g=4823, c=4822, q=2)
Sending NMI from CPU 1 to CPUs 0:
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.004
msecs
NMI backtrace for cpu 0
CPU: 0 PID: 12557 Comm: syz-executor.3 Not tainted 4.14.160-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
task: ffff88807d4e0400 task.stack: ffff888044900000
RIP: 0010:debug_lockdep_rcu_enabled kernel/rcu/update.c:299 [inline]
RIP: 0010:rcu_read_lock_sched_held+0x0/0x130 kernel/rcu/update.c:109
RSP: 0018:ffff8880aec074a8 EFLAGS: 00000006
RAX: ffff88807d4e0400 RBX: 0000000000000000 RCX: 1ffffffff1066fb0
RDX: 0000000000010100 RSI: 0000000000000000 RDI: ffff88807d4e0c7c
RBP: ffff8880aec07568 R08: ffff88807d4e0400 R09: 0000000000000004
R10: 0000000000000000 R11: ffff88807d4e0400 R12: ffff88809baecfc8
R13: ffff8880aec25b40 R14: dffffc0000000000 R15: 0000000000000000
FS: 00007fe2fe477700(0000) GS:ffff8880aec00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b30121000 CR3: 000000007cc20000 CR4: 00000000001426f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1318
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:memcmp+0x48/0xc0 lib/string.c:861
RSP: 0018:ffff8880aec07710 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: 0000000000000000 RBX: ffff8880aec07818 RCX: 0000000000000007
RDX: 0000000000000000 RSI: ffff88809f5fcd28 RDI: ffff8880aec077b8
RBP: ffff8880aec07738 R08: 00000000cd22e513 R09: 0000000000000003
R10: 0000000000000000 R11: ffff88807d4e0400 R12: dffffc0000000000
R13: ffff8880aec07820 R14: 000000000000000d R15: ffff88809f5fccb0
find_stack lib/stackdepot.c:176 [inline]
depot_save_stack+0x11c/0x410 lib/stackdepot.c:225
save_stack+0xa9/0xd0 mm/kasan/kasan.c:453
set_track mm/kasan/kasan.c:459 [inline]
kasan_kmalloc mm/kasan/kasan.c:551 [inline]
kasan_kmalloc+0xce/0xf0 mm/kasan/kasan.c:529
kmem_cache_alloc_trace+0x152/0x790 mm/slab.c:3618
kmalloc include/linux/slab.h:488 [inline]
dst_cow_metrics_generic+0x4d/0x140 net/core/dst.c:210
ipv6_cow_metrics+0x78/0x180 net/ipv6/route.c:195
dst_metrics_write_ptr include/net/dst.h:146 [inline]
dst_metric_set include/net/dst.h:207 [inline]
icmp6_dst_alloc+0x3f6/0x580 net/ipv6/route.c:1772
ndisc_send_skb+0xc0f/0x11e0 net/ipv6/ndisc.c:463
ndisc_send_rs+0x12b/0x670 net/ipv6/ndisc.c:677
addrconf_rs_timer+0x289/0x5a0 net/ipv6/addrconf.c:3757
call_timer_fn+0x161/0x670 kernel/time/timer.c:1279
expire_timers kernel/time/timer.c:1318 [inline]
__run_timers kernel/time/timer.c:1636 [inline]
__run_timers kernel/time/timer.c:1604 [inline]
run_timer_softirq+0x5b7/0x1520 kernel/time/timer.c:1649
__do_softirq+0x244/0x9a0 kernel/softirq.c:288
invoke_softirq kernel/softirq.c:368 [inline]
irq_exit+0x160/0x1b0 kernel/softirq.c:409
exiting_irq arch/x86/include/asm/apic.h:648 [inline]
smp_apic_timer_interrupt+0x146/0x5e0 arch/x86/kernel/apic/apic.c:1102
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
</IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:779
[inline]
RIP: 0010:lock_acquire+0x1ea/0x430 kernel/locking/lockdep.c:3997
RSP: 0018:ffff888044907680 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff0fe2d29 RBX: ffff88807d4e0400 RCX: 00000000780ecdb0
RDX: dffffc0000000000 RSI: ffff88807d4e0ca8 RDI: 0000000000000282
RBP: ffff8880449076c8 R08: 0000000000000000 R09: ffff88807d4e0cc8
R10: ffff88807d4e0ca8 R11: ffff88807d4e0400 R12: ffffffff87f88960
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002
rcu_lock_acquire include/linux/rcupdate.h:242 [inline]
rcu_read_lock include/linux/rcupdate.h:629 [inline]
is_bpf_text_address+0x38/0x120 kernel/bpf/core.c:456
kernel_text_address+0x73/0xf0 kernel/extable.c:150
__kernel_text_address+0xd/0x40 kernel/extable.c:105
unwind_get_return_address arch/x86/kernel/unwind_frame.c:18 [inline]
unwind_get_return_address+0x61/0xa0 arch/x86/kernel/unwind_frame.c:13
__save_stack_trace+0x7b/0xd0 arch/x86/kernel/stacktrace.c:45
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x45/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_kmalloc mm/kasan/kasan.c:551 [inline]
kasan_kmalloc+0xce/0xf0 mm/kasan/kasan.c:529
kasan_slab_alloc+0xf/0x20 mm/kasan/kasan.c:489
kmem_cache_alloc+0x12e/0x780 mm/slab.c:3552
__split_vma+0xc9/0x6d0 mm/mmap.c:2585
split_vma+0x8d/0xc0 mm/mmap.c:2648
userfaultfd_register fs/userfaultfd.c:1455 [inline]
userfaultfd_ioctl+0x25c6/0x2fe0 fs/userfaultfd.c:1845
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:500 [inline]
do_vfs_ioctl+0x7ae/0x1060 fs/ioctl.c:684
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x45a919
RSP: 002b:00007fe2fe476c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 000000000045a919
RDX: 0000000020000040 RSI: 00000000c020aa00 RDI: 0000000000000003
RBP: 000000000075bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fe2fe4776d4
R13: 00000000004c6122 R14: 00000000004dc680 R15: 00000000ffffffff
Code: 06 00 74 14 9c 58 0f 1f 44 00 00 48 c1 e8 09 5d 48 83 f0 01 83 e0 01
c3 0f 0b 48 c7 c7 40 69 f1 87 e8 65 58 3a 00 eb d2 0f 1f 00 <48> c7 c0 0c
49 b2 88 55 48 ba 00 00 00 00 00 fc ff df 48 89 c1
IPVS: ftp: loaded support on port[0] = 21


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

syzbot

unread,
Dec 24, 2020, 6:25:15 AM12/24/20
to syzkaller...@googlegroups.com
Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.
Reply all
Reply to author
Forward
0 new messages