possible deadlock in flush_work

9 views
Skip to first unread message

syzbot

unread,
Apr 13, 2019, 2:23:09 AM4/13/19
to syzkaller...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 1ec8f1f0 Linux 4.14.111
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=168a286b200000
kernel config: https://syzkaller.appspot.com/x/.config?x=fdadf290ea9fc6f9
dashboard link: https://syzkaller.appspot.com/bug?extid=50f018e02905b0881378
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+50f018...@syzkaller.appspotmail.com

======================================================
WARNING: possible circular locking dependency detected
4.14.111 #1 Not tainted
------------------------------------------------------
syz-executor.3/11661 is trying to acquire lock:
((&strp->work)){+.+.}, at: [<ffffffff813c8b54>] flush_work+0x84/0x730
kernel/workqueue.c:2878

but task is already holding lock:
(sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] lock_sock
include/net/sock.h:1462 [inline]
(sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] kcm_attach
net/kcm/kcmsock.c:1390 [inline]
(sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] kcm_attach_ioctl
net/kcm/kcmsock.c:1490 [inline]
(sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] kcm_ioctl+0x35d/0x1120
net/kcm/kcmsock.c:1701

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (sk_lock-AF_INET){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
lock_sock_nested+0xbd/0x110 net/core/sock.c:2768
lock_sock include/net/sock.h:1462 [inline]
strp_sock_lock+0x2e/0x40 net/strparser/strparser.c:451
do_strp_work net/strparser/strparser.c:415 [inline]
strp_work+0x49/0x110 net/strparser/strparser.c:434
process_one_work+0x868/0x1610 kernel/workqueue.c:2114
worker_thread+0x5d9/0x1050 kernel/workqueue.c:2248
kthread+0x31c/0x430 kernel/kthread.c:232
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:402

-> #0 ((&strp->work)){+.+.}:
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2c89/0x45e0 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
flush_work+0xae/0x730 kernel/workqueue.c:2881
__cancel_work_timer+0x2f0/0x480 kernel/workqueue.c:2956
cancel_work_sync+0x18/0x20 kernel/workqueue.c:2992
strp_done+0x58/0xe0 net/strparser/strparser.c:519
kcm_attach net/kcm/kcmsock.c:1429 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1490 [inline]
kcm_ioctl+0x8d9/0x1120 net/kcm/kcmsock.c:1701
sock_do_ioctl+0x6a/0xb0 net/socket.c:974
sock_ioctl+0x2ac/0x470 net/socket.c:1071
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:500 [inline]
do_vfs_ioctl+0x7b9/0x1070 fs/ioctl.c:684
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(sk_lock-AF_INET);
lock((&strp->work));
lock(sk_lock-AF_INET);
lock((&strp->work));

*** DEADLOCK ***

1 lock held by syz-executor.3/11661:
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] lock_sock
include/net/sock.h:1462 [inline]
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] kcm_attach
net/kcm/kcmsock.c:1390 [inline]
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>] kcm_attach_ioctl
net/kcm/kcmsock.c:1490 [inline]
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff8597674d>]
kcm_ioctl+0x35d/0x1120 net/kcm/kcmsock.c:1701

stack backtrace:
CPU: 1 PID: 11661 Comm: syz-executor.3 Not tainted 4.14.111 #1
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x138/0x19c lib/dump_stack.c:53
print_circular_bug.isra.0.cold+0x1cc/0x28f kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2c89/0x45e0 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
flush_work+0xae/0x730 kernel/workqueue.c:2881
__cancel_work_timer+0x2f0/0x480 kernel/workqueue.c:2956
cancel_work_sync+0x18/0x20 kernel/workqueue.c:2992
strp_done+0x58/0xe0 net/strparser/strparser.c:519
kcm_attach net/kcm/kcmsock.c:1429 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1490 [inline]
kcm_ioctl+0x8d9/0x1120 net/kcm/kcmsock.c:1701
sock_do_ioctl+0x6a/0xb0 net/socket.c:974
sock_ioctl+0x2ac/0x470 net/socket.c:1071
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:500 [inline]
do_vfs_ioctl+0x7b9/0x1070 fs/ioctl.c:684
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x458c29
RSP: 002b:00007ff4bd462c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000458c29
RDX: 0000000020002980 RSI: 00000000000089e0 RDI: 0000000000000005
RBP: 000000000073bfa0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ff4bd4636d4
R13: 00000000004c461a R14: 00000000004d7ec8 R15: 00000000ffffffff
kobject: 'loop3' (ffff8880a49eaf60): kobject_uevent_env
kobject: 'loop3' (ffff8880a49eaf60): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop4' (ffff8880a4a2c960): kobject_uevent_env
kobject: 'loop4' (ffff8880a4a2c960): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop3' (ffff8880a49eaf60): kobject_uevent_env
kobject: 'loop3' (ffff8880a49eaf60): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop4' (ffff8880a4a2c960): kobject_uevent_env
kobject: 'loop4' (ffff8880a4a2c960): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop3' (ffff8880a49eaf60): kobject_uevent_env
kobject: 'loop3' (ffff8880a49eaf60): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop4' (ffff8880a4a2c960): kobject_uevent_env
kobject: 'loop4' (ffff8880a4a2c960): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop3' (ffff8880a49eaf60): kobject_uevent_env
kobject: 'loop3' (ffff8880a49eaf60): fill_kobj_path: path
= '/devices/virtual/block/loop3'


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

syzbot

unread,
Dec 23, 2019, 1:00:08 AM12/23/19
to syzkaller...@googlegroups.com
syzbot has found a reproducer for the following crash on:

HEAD commit: e1f7d50a Linux 4.14.160
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=14d052c6e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=46599517442ad9fb
dashboard link: https://syzkaller.appspot.com/bug?extid=50f018e02905b0881378
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=17038cc6e00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=137bb90ae00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+50f018...@syzkaller.appspotmail.com

random: sshd: uninitialized urandom read (32 bytes read)
random: sshd: uninitialized urandom read (32 bytes read)
random: sshd: uninitialized urandom read (32 bytes read)
audit: type=1400 audit(1577080635.373:36): avc: denied { map } for
pid=7223 comm="syz-executor702" path="/root/syz-executor702617745"
dev="sda1" ino=16483 scontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023
tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1
======================================================
WARNING: possible circular locking dependency detected
4.14.160-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor702/7223 is trying to acquire lock:
((&strp->work)){+.+.}, at: [<ffffffff813ce074>] flush_work+0x84/0x730
kernel/workqueue.c:2884

but task is already holding lock:
(sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] lock_sock
include/net/sock.h:1462 [inline]
(sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] kcm_attach
net/kcm/kcmsock.c:1390 [inline]
(sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] kcm_attach_ioctl
net/kcm/kcmsock.c:1490 [inline]
(sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] kcm_ioctl+0x35d/0x1120
net/kcm/kcmsock.c:1701

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (sk_lock-AF_INET){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
lock_sock_nested+0xbd/0x110 net/core/sock.c:2770
lock_sock include/net/sock.h:1462 [inline]
strp_sock_lock+0x2e/0x40 net/strparser/strparser.c:451
do_strp_work net/strparser/strparser.c:415 [inline]
strp_work+0x43/0x100 net/strparser/strparser.c:434
process_one_work+0x863/0x1600 kernel/workqueue.c:2114
worker_thread+0x5d9/0x1050 kernel/workqueue.c:2248
kthread+0x319/0x430 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #0 ((&strp->work)){+.+.}:
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
flush_work+0xae/0x730 kernel/workqueue.c:2887
__cancel_work_timer+0x2f0/0x480 kernel/workqueue.c:2962
cancel_work_sync+0x18/0x20 kernel/workqueue.c:2998
strp_done+0x58/0xe0 net/strparser/strparser.c:519
kcm_attach net/kcm/kcmsock.c:1429 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1490 [inline]
kcm_ioctl+0x8d9/0x1120 net/kcm/kcmsock.c:1701
sock_do_ioctl+0x64/0xb0 net/socket.c:974
sock_ioctl+0x2a6/0x470 net/socket.c:1071
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:500 [inline]
do_vfs_ioctl+0x7ae/0x1060 fs/ioctl.c:684
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(sk_lock-AF_INET);
lock((&strp->work));
lock(sk_lock-AF_INET);
lock((&strp->work));

*** DEADLOCK ***

1 lock held by syz-executor702/7223:
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] lock_sock
include/net/sock.h:1462 [inline]
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] kcm_attach
net/kcm/kcmsock.c:1390 [inline]
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>] kcm_attach_ioctl
net/kcm/kcmsock.c:1490 [inline]
#0: (sk_lock-AF_INET){+.+.}, at: [<ffffffff85e04d0d>]
kcm_ioctl+0x35d/0x1120 net/kcm/kcmsock.c:1701

stack backtrace:
CPU: 1 PID: 7223 Comm: syz-executor702 Not tainted 4.14.160-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x142/0x197 lib/dump_stack.c:58
print_circular_bug.isra.0.cold+0x1cc/0x28f kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
flush_work+0xae/0x730 kernel/workqueue.c:2887
__cancel_work_timer+0x2f0/0x480 kernel/workqueue.c:2962
cancel_work_sync+0x18/0x20 kernel/workqueue.c:2998
strp_done+0x58/0xe0 net/strparser/strparser.c:519
kcm_attach net/kcm/kcmsock.c:1429 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1490 [inline]
kcm_ioctl+0x8d9/0x1120 net/kcm/kcmsock.c:1701
sock_do_ioctl+0x64/0xb0 net/socket.c:974
sock_ioctl+0x2a6/0x470 net/socket.c:1071
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:500 [inline]
do_vfs_ioctl+0x7ae/0x1060 fs/ioctl.c:684
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x440719
RSP: 002b:00007ffcb4582d48 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00000000004002c8 RCX: 0000000000440719
RDX: 0000000020000080 RSI: 00000000000089e0 RDI: 0000000000000004
RBP: 00000000006ca018 R08: 00000000004002c8 R09: 00000000004002c8
R10: 00000000004002c8 R11: 0000000000000246 R12: 0000000000401fa0
R13: 0000000000402030 R14:

Reply all
Reply to author
Forward
0 new messages