BUG: workqueue lockup (3)

39 views
Skip to first unread message

syzbot

unread,
May 13, 2018, 8:48:03 PM5/13/18
to syzkaller-upst...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: a84880ef4352 Merge branch 'bpf-perf-rb-libbpf'
git tree: bpf-next
console output: https://syzkaller.appspot.com/x/log.txt?x=145c1157800000
kernel config: https://syzkaller.appspot.com/x/.config?x=10c4dc62055b68f5
dashboard link: https://syzkaller.appspot.com/bug?extid=51a45b2c02fda03d48c5
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
CC: [linux-...@vger.kernel.org linux-...@vger.kernel.org
vi...@zeniv.linux.org.uk]

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+51a45b...@syzkaller.appspotmail.com

BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 nice=0 stuck for 121s!
BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 121s!
BUG: workqueue lockup - pool cpus=0-1 flags=0x4 nice=0 stuck for 121s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=9/256
pending: defense_work_handler, defense_work_handler,
defense_work_handler, defense_work_handler, defense_work_handler,
defense_work_handler, defense_work_handler, perf_sched_delayed, cache_reap
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=5/256
pending: defense_work_handler, defense_work_handler, vmstat_shepherd,
cache_reap, check_corruption
workqueue events_unbound: flags=0x2
pwq 4: cpus=0-1 flags=0x4 nice=0 active=2/512
pending: fsnotify_connector_destroy_workfn, fsnotify_mark_destroy_workfn
workqueue events_power_efficient: flags=0x80
pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256
pending: do_cache_clean
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256
pending: gc_worker, neigh_periodic_work, neigh_periodic_work
workqueue mm_percpu_wq: flags=0x8
pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256
pending: vmstat_update
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256
pending: vmstat_update
workqueue writeback: flags=0x4e
pwq 4: cpus=0-1 flags=0x4 nice=0 active=9/256
pending: wb_workfn, wb_workfn, wb_workfn, wb_workfn, wb_workfn,
wb_workfn, wb_workfn, wb_workfn, wb_workfn
workqueue kblockd: flags=0x18
pwq 1: cpus=0 node=0 flags=0x0 nice=-20 active=1/256
pending: blk_mq_timeout_work
workqueue dm_bufio_cache: flags=0x8
pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256
pending: work_fn
workqueue ipv6_addrconf: flags=0x40008
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/1
pending: addrconf_verify_work
INFO: rcu_sched detected stalls on CPUs/tasks:
(detected by 1, t=125007 jiffies, g=19632, c=19631, q=163)
All QSes seen, last rcu_sched kthread activity 125014
(4294917194-4294792180), jiffies_till_next_fqs=3, root ->qsmask 0x0
syz-executor3 R running task 21224 8092 8091 0x00000008
Call Trace:
<IRQ>
sched_show_task.cold.86+0x27a/0x301 kernel/sched/core.c:5325
print_other_cpu_stall.cold.79+0x92f/0x9d2 kernel/rcu/tree.c:1481
check_cpu_stall.isra.61+0x706/0xf50 kernel/rcu/tree.c:1599
__rcu_pending kernel/rcu/tree.c:3356 [inline]
rcu_pending kernel/rcu/tree.c:3401 [inline]
rcu_check_callbacks+0x21b/0xad0 kernel/rcu/tree.c:2763
update_process_times+0x2d/0x70 kernel/time/timer.c:1636
tick_sched_handle+0x9f/0x180 kernel/time/tick-sched.c:164
tick_sched_timer+0x45/0x130 kernel/time/tick-sched.c:1274
__run_hrtimer kernel/time/hrtimer.c:1398 [inline]
__hrtimer_run_queues+0x3e3/0x10a0 kernel/time/hrtimer.c:1460
hrtimer_interrupt+0x2f3/0x750 kernel/time/hrtimer.c:1518
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1025 [inline]
smp_apic_timer_interrupt+0x15d/0x710 arch/x86/kernel/apic/apic.c:1050
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:863
</IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:783
[inline]
RIP: 0010:seqcount_lockdep_reader_access include/linux/seqlock.h:83 [inline]
RIP: 0010:read_seqcount_begin include/linux/seqlock.h:164 [inline]
RIP: 0010:read_seqbegin include/linux/seqlock.h:433 [inline]
RIP: 0010:read_seqbegin_or_lock include/linux/seqlock.h:529 [inline]
RIP: 0010:d_walk+0x840/0xc80 fs/dcache.c:1248
RSP: 0018:ffff8801b63279f8 EFLAGS: 00000293 ORIG_RAX: ffffffffffffff13
RAX: ffff8801b8c9e340 RBX: 0000000000000293 RCX: 1ffff10037193d7d
RDX: 0000000000000000 RSI: ffffffff81c6543b RDI: 0000000000000293
RBP: ffff8801b6327b78 R08: ffff8801b8c9eb78 R09: 0000000000000006
R10: ffff8801b8c9e340 R11: 0000000000000000 R12: 0000000000000200
R13: dffffc0000000000 R14: 1ffff10036c64f82 R15: ffff8801b6327b50
shrink_dcache_parent+0x179/0x230 fs/dcache.c:1486
vfs_rmdir+0x202/0x470 fs/namei.c:3850
do_rmdir+0x523/0x610 fs/namei.c:3911
__do_sys_rmdir fs/namei.c:3929 [inline]
__se_sys_rmdir fs/namei.c:3927 [inline]
__x64_sys_rmdir+0x36/0x40 fs/namei.c:3927
do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4556e7
RSP: 002b:00007ffee415f478 EFLAGS: 00000206 ORIG_RAX: 0000000000000054
RAX: ffffffffffffffda RBX: 0000000000000065 RCX: 00000000004556e7
RDX: 0000000000000000 RSI: 00007ffee4161220 RDI: 00007ffee4161220
RBP: 00007ffee4161220 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000006 R11: 0000000000000206 R12: 0000000001fb5940
R13: 0000000000000000 R14: 00000000000000a9 R15: 000000000001e736
rcu_sched kthread starved for 125570 jiffies! g19632 c19631 f0x2
RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1
RCU grace-period kthread stack dump:
rcu_sched R running task 23896 9 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2848 [inline]
__schedule+0x801/0x1e30 kernel/sched/core.c:3490
schedule+0xef/0x430 kernel/sched/core.c:3549
schedule_timeout+0x138/0x240 kernel/time/timer.c:1801
rcu_gp_kthread+0x6b5/0x1940 kernel/rcu/tree.c:2231
kthread+0x345/0x410 kernel/kthread.c:238
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

Dmitry Vyukov

unread,
May 14, 2018, 1:36:41 AM5/14/18
to syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation
#syz upstream
> --
> You received this message because you are subscribed to the Google Groups
> "syzkaller-upstream-moderation" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to syzkaller-upstream-m...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/syzkaller-upstream-moderation/000000000000892168056c1fd6f3%40google.com.
> For more options, visit https://groups.google.com/d/optout.

syzbot

unread,
May 14, 2018, 1:36:42 AM5/14/18
to Dmitry Vyukov, dvy...@google.com, syzkaller-upst...@googlegroups.com
> #syz upstream

Can't upstream this bug, the bug has fixing commits.

syzbot

unread,
May 19, 2018, 1:21:03 PM5/19/18
to dvy...@google.com, penguin...@i-love.sakura.ne.jp, syzkaller-upst...@googlegroups.com
syzbot has found a reproducer for the following crash on:

HEAD commit: 73fcb1a370c7 Merge branch 'akpm' (patches from Andrew)
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=126307cf800000
kernel config: https://syzkaller.appspot.com/x/.config?x=f3b4e30da84ec1ed
dashboard link: https://syzkaller.appspot.com/bug?extid=51a45b2c02fda03d48c5
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syzkaller repro:https://syzkaller.appspot.com/x/repro.syz?x=170a85a7800000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=10f84bcf800000
CC: []

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+51a45b...@syzkaller.appspotmail.com

BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 nice=0 stuck for 31s!
BUG: workqueue lockup - pool cpus=0-1 flags=0x4 nice=0 stuck for 30s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=5/256
pending: defense_work_handler, defense_work_handler,
defense_work_handler, vmstat_shepherd, cache_reap
workqueue events_power_efficient: flags=0x80
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256
pending: gc_worker, do_cache_clean
workqueue mm_percpu_wq: flags=0x8
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256
pending: vmstat_update
workqueue writeback: flags=0x4e
pwq 4: cpus=0-1 flags=0x4 nice=0 active=9/256
pending: wb_workfn, wb_workfn, wb_workfn, wb_workfn, wb_workfn,
wb_workfn, wb_workfn, wb_workfn, wb_workfn
workqueue dm_bufio_cache: flags=0x8
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256
Reply all
Reply to author
Forward
0 new messages