INFO: task hung in fsnotify_mark_destroy_workfn (2)

13 views
Skip to first unread message

syzbot

unread,
Sep 12, 2018, 12:22:02 AM9/12/18
to syzkaller-upst...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 3d0e7a9e00fd Merge tag 'md/4.19-rc2' of git://git.kernel.o..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=10020cd1400000
kernel config: https://syzkaller.appspot.com/x/.config?x=8f59875069d721b6
dashboard link: https://syzkaller.appspot.com/bug?extid=2712275f68c194ce24fd
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
CC: [amir...@gmail.com ja...@suse.cz
linux-...@vger.kernel.org linux-...@vger.kernel.org]

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+271227...@syzkaller.appspotmail.com

INFO: task kworker/u4:12:9048 blocked for more than 140 seconds.
Not tainted 4.19.0-rc2+ #6
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u4:12 D10792 9048 2 0x80000000
Workqueue: events_unbound fsnotify_mark_destroy_workfn
Call Trace:
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x86c/0x1ed0 kernel/sched/core.c:3473
schedule+0xfe/0x460 kernel/sched/core.c:3517
schedule_timeout+0x1cc/0x260 kernel/time/timer.c:1780
do_wait_for_common kernel/sched/completion.c:83 [inline]
__wait_for_common kernel/sched/completion.c:104 [inline]
wait_for_common kernel/sched/completion.c:115 [inline]
wait_for_completion+0x427/0x8a0 kernel/sched/completion.c:136
__synchronize_srcu+0x184/0x230 kernel/rcu/srcutree.c:934
synchronize_srcu+0x440/0x5ab kernel/rcu/srcutree.c:1012
fsnotify_mark_destroy_workfn+0x1be/0x510 fs/notify/mark.c:772
process_one_work+0xc90/0x1b90 kernel/workqueue.c:2153
worker_thread+0x17f/0x1390 kernel/workqueue.c:2296
kthread+0x35a/0x420 kernel/kthread.c:246
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:413

Showing all locks held in the system:
1 lock held by khungtaskd/984:
#0: 00000000b38e653c (rcu_read_lock){....}, at:
debug_show_all_locks+0xd0/0x424 kernel/locking/lockdep.c:4436
1 lock held by rsyslogd/5317:
#0: 00000000d292a055 (&f->f_pos_lock){+.+.}, at: __fdget_pos+0x1bb/0x200
fs/file.c:766
2 locks held by getty/5439:
#0: 0000000056010e53 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 0000000099b700ed (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5440:
#0: 00000000ebabf63b (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 0000000049a4581f (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5441:
#0: 0000000079d1f009 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 0000000047aed8e4 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5442:
#0: 00000000400826c6 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000dbe3dd22 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5443:
#0: 000000002edc9d22 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 0000000049e6e57a (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5444:
#0: 00000000c8e5c3c5 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000333eb678 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5445:
#0: 00000000a8e6e545 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000561394bf (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by kworker/u4:12/9048:
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
__write_once_size include/linux/compiler.h:215 [inline]
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
atomic64_set include/asm-generic/atomic-instrumented.h:40 [inline]
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
atomic_long_set include/asm-generic/atomic-long.h:59 [inline]
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
set_work_data kernel/workqueue.c:617 [inline]
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline]
#0: 000000002977e987 ((wq_completion)"events_unbound"){+.+.}, at:
process_one_work+0xb43/0x1b90 kernel/workqueue.c:2124
#1: 00000000233e961c ((reaper_work).work){+.+.}, at:
process_one_work+0xb9a/0x1b90 kernel/workqueue.c:2128
4 locks held by syz-executor6/30089:
1 lock held by syz-executor2/30088:

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 984 Comm: khungtaskd Not tainted 4.19.0-rc2+ #6
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
nmi_cpu_backtrace.cold.3+0x63/0xa2 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x1b3/0x1ed lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_all_cpu_backtrace include/linux/nmi.h:144 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:204 [inline]
watchdog+0xb3e/0x1050 kernel/hung_task.c:265
kthread+0x35a/0x420 kernel/kthread.c:246
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:413
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.860
msecs
CPU: 0 PID: 30089 Comm: syz-executor6 Not tainted 4.19.0-rc2+ #6
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
RIP: 0010:mark_lock+0x1f/0x1cb0 kernel/locking/lockdep.c:3095
Code: e8 a6 d1 61 00 eb 87 0f 1f 40 00 55 4c 8d 46 20 41 89 d1 49 89 fb 48
ba 00 00 00 00 00 fc ff df 48 89 e5 41 57 44 89 c9 41 56 <4c> 8d 7d d8 41
be 01 00 00 00 41 55 41 d3 e6 41 54 49 89 f4 53 48
RSP: 0018:ffff8801dae07660 EFLAGS: 00000046
RAX: 000000000000000c RBX: 0000000000000053 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: ffff8801bdd82ad0 RDI: ffff8801bdd82200
RBP: ffff8801dae07670 R08: ffff8801bdd82af0 R09: 0000000000000000
R10: ffff8801bdd82ad0 R11: ffff8801bdd82200 R12: ffff8801bdd82af2
R13: 0000000000000040 R14: ffff8801bdd82200 R15: 000000000000000c
FS: 00007faae8231700(0000) GS:ffff8801dae00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000018daa6000 CR4: 00000000001426f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
mark_irqflags kernel/locking/lockdep.c:3003 [inline]
__lock_acquire+0x1564/0x4ec0 kernel/locking/lockdep.c:3369
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x61/0x80 kernel/locking/spinlock.c:160
__run_hrtimer kernel/time/hrtimer.c:1400 [inline]
__hrtimer_run_queues+0x488/0x10d0 kernel/time/hrtimer.c:1460
hrtimer_interrupt+0x313/0x780 kernel/time/hrtimer.c:1518
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1029 [inline]
smp_apic_timer_interrupt+0x1a1/0x760 arch/x86/kernel/apic/apic.c:1054
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:864
</IRQ>
RIP: 0010:rcu_lockdep_current_cpu_online+0x189/0x2d0 kernel/rcu/tree.c:1079
Code: 49 c1 e8 03 43 80 3c 30 00 0f 85 c0 00 00 00 48 c7 c1 c0 0e 17 89 48
03 1c c1 48 8d 7b 20 48 89 f8 48 c1 e8 03 42 80 3c 30 00 <0f> 85 b7 00 00
00 48 8d 7b 28 48 8b 43 20 49 89 f8 49 c1 e8 03 43
RSP: 0018:ffff8801892072f8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
RAX: 1ffff1003b5c5b24 RBX: ffff8801dae2d900 RCX: ffffffff89170ec0
RDX: 0000000000040000 RSI: ffffffff83907258 RDI: ffff8801dae2d920
RBP: ffff880189207398 R08: 1ffffffff122e1d8 R09: ffffed003b5c4732
R10: ffffed003b5c4732 R11: ffff8801dae23993 R12: ffffffff895a1e00
R13: ffff880189207370 R14: dffffc0000000000 R15: ffffed0031240e66
rcu_read_lock_sched_held+0x96/0x120 kernel/rcu/update.c:114
trace_kvm_exit arch/x86/kvm/trace.h:225 [inline]
vmx_handle_exit+0x1144/0x17f0 arch/x86/kvm/vmx.c:10034
vcpu_enter_guest+0x14a9/0x62e0 arch/x86/kvm/x86.c:7630
vcpu_run arch/x86/kvm/x86.c:7693 [inline]
kvm_arch_vcpu_ioctl_run+0x375/0x16e0 arch/x86/kvm/x86.c:7870
kvm_vcpu_ioctl+0x72b/0x1150 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2590
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:501 [inline]
do_vfs_ioctl+0x1de/0x1720 fs/ioctl.c:685
ksys_ioctl+0xa9/0xd0 fs/ioctl.c:702
__do_sys_ioctl fs/ioctl.c:709 [inline]
__se_sys_ioctl fs/ioctl.c:707 [inline]
__x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:707
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x457099
Code: fd b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 cb b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007faae8230c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007faae82316d4 RCX: 0000000000457099
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000006
RBP: 00000000009300a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004cf730 R14: 00000000004c59b9 R15: 0000000000000000


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

Dmitry Vyukov

unread,
Oct 23, 2019, 3:21:47 AM10/23/19
to syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation
After https://github.com/google/syzkaller/commit/37dc03de04826cc0d5d1e3699832b0a3113d40af
this should be re-detected as "task hung in synchronize_rcu"

#syz invalid
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-upstream-moderation" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-upstream-m...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-upstream-moderation/00000000000097c43e0575a4ee0d%40google.com.
> For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages