INFO: rcu detected stall in addrconf_dad_work

11 views
Skip to first unread message

syzbot

unread,
Sep 6, 2019, 9:25:07 PM9/6/19
to syzkaller...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 414510bc Linux 4.14.142
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1274eb0a600000
kernel config: https://syzkaller.appspot.com/x/.config?x=9aa0b2ccd827f416
dashboard link: https://syzkaller.appspot.com/bug?extid=360efe4e8b5dbf168f54
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=138fddd6600000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16c608b2600000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+360efe...@syzkaller.appspotmail.com

IPv6: ADDRCONF(NETDEV_CHANGE): hsr0: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): team0: link becomes ready
IPv6: ADDRCONF(NETDEV_UP): vxcan1: link is not ready
8021q: adding VLAN 0 to HW filter on device batadv0
INFO: rcu_preempt self-detected stall on CPU
1-...: (1 GPs behind) idle=a8e/140000000000001/0 softirq=11702/11703 fqs=27
(t=10500 jiffies g=1036 c=1035 q=31)
rcu_preempt kthread starved for 10446 jiffies! g1036 c1035 f0x0
RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=0
rcu_preempt I29776 8 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2807 [inline]
__schedule+0x7b8/0x1cd0 kernel/sched/core.c:3383
schedule+0x92/0x1c0 kernel/sched/core.c:3427
schedule_timeout+0x43e/0xe10 kernel/time/timer.c:1744
rcu_gp_kthread+0xbf4/0x1ec0 kernel/rcu/tree.c:2255
kthread+0x319/0x430 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
NMI backtrace for cpu 1
CPU: 1 PID: 6826 Comm: kworker/1:3 Not tainted 4.14.142 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Workqueue: ipv6_addrconf addrconf_dad_work
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x138/0x197 lib/dump_stack.c:53
nmi_cpu_backtrace.cold+0x57/0x94 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x141/0x189 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
rcu_dump_cpu_stacks+0x186/0x1d2 kernel/rcu/tree.c:1396
print_cpu_stall kernel/rcu/tree.c:1542 [inline]
check_cpu_stall kernel/rcu/tree.c:1610 [inline]
__rcu_pending kernel/rcu/tree.c:3390 [inline]
rcu_pending kernel/rcu/tree.c:3452 [inline]
rcu_check_callbacks.cold+0x43d/0xd0a kernel/rcu/tree.c:2792
update_process_times+0x31/0x70 kernel/time/timer.c:1588
tick_sched_handle+0x85/0x160 kernel/time/tick-sched.c:161
tick_sched_timer+0x43/0x130 kernel/time/tick-sched.c:1219
__run_hrtimer kernel/time/hrtimer.c:1220 [inline]
__hrtimer_run_queues+0x270/0xbc0 kernel/time/hrtimer.c:1284
hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1318
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
</IRQ>
RIP: 0010:__list_del_entry_valid+0x9c/0xf5 lib/list_debug.c:54
RSP: 0018:ffff888083c7f528 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: dffffc0000000000 RBX: ffff8880a8909438 RCX: 0000000000000000
RDX: ffff8880a8909438 RSI: ffff8880a89094d0 RDI: ffff8880a89094d8
RBP: ffff888083c7f540 R08: 0000000000000000 R09: ffff8880819f2d38
R10: ffff8880819f2d18 R11: ffff8880819f2380 R12: ffff8880a89094d0
R13: ffff8880a89094d0 R14: ffff8880a8909438 R15: ffff8880a89094c0
__list_del_entry include/linux/list.h:117 [inline]
list_move_tail include/linux/list.h:182 [inline]
hhf_dequeue+0x57f/0xa60 net/sched/sch_hhf.c:438
dequeue_skb net/sched/sch_generic.c:148 [inline]
qdisc_restart net/sched/sch_generic.c:241 [inline]
__qdisc_run+0x2b8/0xe00 net/sched/sch_generic.c:257
__dev_xmit_skb net/core/dev.c:3235 [inline]
__dev_queue_xmit+0x1571/0x25e0 net/core/dev.c:3493
dev_queue_xmit+0x18/0x20 net/core/dev.c:3558
neigh_resolve_output net/core/neighbour.c:1364 [inline]
neigh_resolve_output+0x4d8/0x870 net/core/neighbour.c:1344
neigh_output include/net/neighbour.h:500 [inline]
ip6_finish_output2+0x9ab/0x21b0 net/ipv6/ip6_output.c:120
ip6_finish_output+0x4f4/0xb50 net/ipv6/ip6_output.c:154
NF_HOOK_COND include/linux/netfilter.h:239 [inline]
ip6_output+0x20f/0x6d0 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:462 [inline]
NF_HOOK include/linux/netfilter.h:250 [inline]
ndisc_send_skb+0xb56/0x11e0 net/ipv6/ndisc.c:483
ndisc_send_ns+0x360/0x7e0 net/ipv6/ndisc.c:625
addrconf_dad_work+0xa40/0xff0 net/ipv6/addrconf.c:3996
process_one_work+0x863/0x1600 kernel/workqueue.c:2114
worker_thread+0x5d9/0x1050 kernel/workqueue.c:2248
kthread+0x319/0x430 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches

syzbot

unread,
Sep 7, 2019, 8:10:08 AM9/7/19
to syzkaller...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: e7d2672c Linux 4.19.71
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=12b6d776600000
kernel config: https://syzkaller.appspot.com/x/.config?x=47881f21d9108add
dashboard link: https://syzkaller.appspot.com/bug?extid=16f5801e71ba218f3bd3
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15f8d171600000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17bd185e600000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+16f580...@syzkaller.appspotmail.com

IPv6: ADDRCONF(NETDEV_UP): hsr0: link is not ready
IPv6: ADDRCONF(NETDEV_CHANGE): hsr0: link becomes ready
IPv6: ADDRCONF(NETDEV_UP): vxcan1: link is not ready
8021q: adding VLAN 0 to HW filter on device batadv0
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 1-...!: (1 GPs behind) idle=9d2/1/0x4000000000000002
softirq=13304/13305 fqs=28
rcu: (t=10500 jiffies g=5221 q=28)
rcu: rcu_preempt kthread starved for 10444 jiffies! g5221 f0x0
RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: RCU grace-period kthread stack dump:
rcu_preempt R running task 29424 10 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2826 [inline]
__schedule+0x866/0x1dc0 kernel/sched/core.c:3474
schedule+0x92/0x1c0 kernel/sched/core.c:3518
schedule_timeout+0x4db/0xfc0 kernel/time/timer.c:1804
rcu_gp_kthread+0xd5c/0x2190 kernel/rcu/tree.c:2202
kthread+0x354/0x420 kernel/kthread.c:246
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415
NMI backtrace for cpu 1
CPU: 1 PID: 7612 Comm: kworker/1:2 Not tainted 4.19.71 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Workqueue: ipv6_addrconf addrconf_dad_work
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x172/0x1f0 lib/dump_stack.c:113
nmi_cpu_backtrace.cold+0x63/0xa4 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x1b0/0x1f8 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_dump_cpu_stacks+0x189/0x1d5 kernel/rcu/tree.c:1340
print_cpu_stall kernel/rcu/tree.c:1478 [inline]
check_cpu_stall kernel/rcu/tree.c:1550 [inline]
__rcu_pending kernel/rcu/tree.c:3293 [inline]
rcu_pending kernel/rcu/tree.c:3336 [inline]
rcu_check_callbacks.cold+0x5e3/0xd90 kernel/rcu/tree.c:2682
update_process_times+0x32/0x80 kernel/time/timer.c:1636
tick_sched_handle+0xa2/0x190 kernel/time/tick-sched.c:164
tick_sched_timer+0x47/0x130 kernel/time/tick-sched.c:1274
__run_hrtimer kernel/time/hrtimer.c:1398 [inline]
__hrtimer_run_queues+0x33b/0xdc0 kernel/time/hrtimer.c:1460
hrtimer_interrupt+0x314/0x770 kernel/time/hrtimer.c:1518
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1067 [inline]
smp_apic_timer_interrupt+0x111/0x550 arch/x86/kernel/apic/apic.c:1092
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
</IRQ>
RIP: 0010:__sanitizer_cov_trace_pc+0x26/0x50 kernel/kcov.c:101
Code: 90 90 90 90 55 48 89 e5 48 8b 75 08 65 48 8b 04 25 40 ee 01 00 65 8b
15 28 87 96 7e 81 e2 00 01 1f 00 75 2b 8b 90 d0 12 00 00 <83> fa 02 75 20
48 8b 88 d8 12 00 00 8b 80 d4 12 00 00 48 8b 11 48
RSP: 0018:ffff888091fdede8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
RAX: ffff888091d6a4c0 RBX: ffff88808d0f94f8 RCX: ffffffff858e8a2e
RDX: 0000000000000000 RSI: ffffffff858e8a40 RDI: 0000000000000001
RBP: ffff888091fdede8 R08: ffff888091d6a4c0 R09: 0000000000000009
R10: ffff888091d6aea8 R11: 0000000025ee0a34 R12: dffffc0000000000
R13: ffff88808d0f9240 R14: ffff88808d0f9598 R15: ffff88808d0f9590
__list_add include/linux/list.h:63 [inline]
list_add_tail include/linux/list.h:93 [inline]
list_move_tail include/linux/list.h:183 [inline]
hhf_dequeue+0x670/0xa00 net/sched/sch_hhf.c:438
dequeue_skb net/sched/sch_generic.c:277 [inline]
qdisc_restart net/sched/sch_generic.c:380 [inline]
__qdisc_run+0x1e7/0x1960 net/sched/sch_generic.c:398
__dev_xmit_skb net/core/dev.c:3501 [inline]
__dev_queue_xmit+0x165c/0x2fe0 net/core/dev.c:3811
dev_queue_xmit+0x18/0x20 net/core/dev.c:3876
br_dev_queue_push_xmit+0x405/0x5d0 net/bridge/br_forward.c:56
NF_HOOK include/linux/netfilter.h:289 [inline]
NF_HOOK include/linux/netfilter.h:283 [inline]
br_forward_finish+0xfa/0x400 net/bridge/br_forward.c:69
NF_HOOK include/linux/netfilter.h:289 [inline]
NF_HOOK include/linux/netfilter.h:283 [inline]
__br_forward+0x647/0xb30 net/bridge/br_forward.c:113
deliver_clone+0x61/0xc0 net/bridge/br_forward.c:129
maybe_deliver net/bridge/br_forward.c:184 [inline]
maybe_deliver net/bridge/br_forward.c:172 [inline]
br_flood+0x4da/0x710 net/bridge/br_forward.c:226
br_dev_xmit+0xeeb/0x1490 net/bridge/br_device.c:103
__netdev_start_xmit include/linux/netdevice.h:4303 [inline]
netdev_start_xmit include/linux/netdevice.h:4312 [inline]
xmit_one net/core/dev.c:3257 [inline]
dev_hard_start_xmit+0x1a5/0x980 net/core/dev.c:3273
__dev_queue_xmit+0x2704/0x2fe0 net/core/dev.c:3843
dev_queue_xmit+0x18/0x20 net/core/dev.c:3876
neigh_resolve_output net/core/neighbour.c:1369 [inline]
neigh_resolve_output+0x5b7/0x980 net/core/neighbour.c:1349
neigh_output include/net/neighbour.h:501 [inline]
ip6_finish_output2+0xb7f/0x2550 net/ipv6/ip6_output.c:120
ip6_finish_output+0x574/0xbe0 net/ipv6/ip6_output.c:154
NF_HOOK_COND include/linux/netfilter.h:278 [inline]
ip6_output+0x235/0x7c0 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:447 [inline]
NF_HOOK include/linux/netfilter.h:289 [inline]
ndisc_send_skb+0xf3b/0x1460 net/ipv6/ndisc.c:491
ndisc_send_ns+0x3a9/0x850 net/ipv6/ndisc.c:633
addrconf_dad_work+0xb88/0x1150 net/ipv6/addrconf.c:4062
process_one_work+0x989/0x1750 kernel/workqueue.c:2153
worker_thread+0x98/0xe40 kernel/workqueue.c:2296
kthread+0x354/0x420 kernel/kthread.c:246
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415

syzbot

unread,
Dec 5, 2019, 10:59:01 PM12/5/19
to syzkaller...@googlegroups.com
syzbot suspects this bug was fixed by commit:

commit cc243e2427cef2a5dd7367cb0e0b846503350ffe
Author: Cong Wang <xiyou.w...@gmail.com>
Date: Sun Sep 8 20:40:51 2019 +0000

sch_hhf: ensure quantum and hhf_non_hh_weight are non-zero

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=13b9d941e00000
start commit: 414510bc Linux 4.14.142
git tree: linux-4.14.y
If the result looks correct, please mark the bug fixed by replying with:

#syz fix: sch_hhf: ensure quantum and hhf_non_hh_weight are non-zero

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

syzbot

unread,
Dec 7, 2019, 3:39:01 AM12/7/19
to syzkaller...@googlegroups.com
syzbot suspects this bug was fixed by commit:

commit a9e91767b921ee196ab693fa13a81b278f53ddac
Author: Cong Wang <xiyou.w...@gmail.com>
Date: Sun Sep 8 20:40:51 2019 +0000

sch_hhf: ensure quantum and hhf_non_hh_weight are non-zero

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=115ca1eae00000
start commit: e7d2672c Linux 4.19.71
git tree: linux-4.19.y
Reply all
Reply to author
Forward
0 new messages