[syzbot] possible deadlock in ovl_fallocate

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 591cd61541b9 Add linux-next specific files for 20221207
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=1509b2f5880000
kernel config: https://syzkaller.appspot.com/x/.config?x=8b2d3e63e054c24f
dashboard link: https://syzkaller.appspot.com/bug?extid=38a94e1872470e3450a6
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/bc862c01ec56/disk-591cd615.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/8f9b93f8ed2f/vmlinux-591cd615.xz
kernel image: https://storage.googleapis.com/syzbot-assets/9d5cb636d548/bzImage-591cd615.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+38a94e...@syzkaller.appspotmail.com

============================================
WARNING: possible recursive locking detected
6.1.0-rc8-next-20221207-syzkaller #0 Not tainted
--------------------------------------------
syz-executor.5/21743 is trying to acquire lock:
ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: ovl_remove_privs_unlocked fs/overlayfs/file.c:519 [inline]
ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: ovl_fallocate+0x15c/0x270 fs/overlayfs/file.c:546

but task is already holding lock:
ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: ovl_fallocate+0x9f/0x270 fs/overlayfs/file.c:535

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----
lock(&ovl_i_mutex_key[depth]#2);
lock(&ovl_i_mutex_key[depth]#2);

*** DEADLOCK ***

May be due to missing lock nesting notation

2 locks held by syz-executor.5/21743:
#0: ffff88801cc94460 (sb_writers#14){.+.+}-{0:0}, at: ksys_fallocate fs/open.c:346 [inline]
#0: ffff88801cc94460 (sb_writers#14){.+.+}-{0:0}, at: __do_sys_fallocate fs/open.c:354 [inline]
#0: ffff88801cc94460 (sb_writers#14){.+.+}-{0:0}, at: __se_sys_fallocate fs/open.c:352 [inline]
#0: ffff88801cc94460 (sb_writers#14){.+.+}-{0:0}, at: __x64_sys_fallocate+0xd3/0x140 fs/open.c:352
#1: ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
#1: ffff888058fdd340 (&ovl_i_mutex_key[depth]#2){+.+.}-{3:3}, at: ovl_fallocate+0x9f/0x270 fs/overlayfs/file.c:535

stack backtrace:
CPU: 1 PID: 21743 Comm: syz-executor.5 Not tainted 6.1.0-rc8-next-20221207-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
print_deadlock_bug kernel/locking/lockdep.c:2990 [inline]
check_deadlock kernel/locking/lockdep.c:3033 [inline]
validate_chain kernel/locking/lockdep.c:3818 [inline]
__lock_acquire.cold+0x116/0x3a7 kernel/locking/lockdep.c:5055
lock_acquire kernel/locking/lockdep.c:5668 [inline]
lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
down_write+0x94/0x220 kernel/locking/rwsem.c:1562
inode_lock include/linux/fs.h:756 [inline]
ovl_remove_privs_unlocked fs/overlayfs/file.c:519 [inline]
ovl_fallocate+0x15c/0x270 fs/overlayfs/file.c:546
vfs_fallocate+0x48b/0xe00 fs/open.c:323
ksys_fallocate fs/open.c:346 [inline]
__do_sys_fallocate fs/open.c:354 [inline]
__se_sys_fallocate fs/open.c:352 [inline]
__x64_sys_fallocate+0xd3/0x140 fs/open.c:352
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fbb9608c0d9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 f1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fbb96dc6168 EFLAGS: 00000246 ORIG_RAX: 000000000000011d
RAX: ffffffffffffffda RBX: 00007fbb961abf80 RCX: 00007fbb9608c0d9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00007fbb960e7ae9 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000001000f4 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffee8cc3bef R14: 00007fbb96dc6300 R15: 0000000000022000
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: 591cd61541b9 Add linux-next specific files for 20221207
git tree: linux-next

console+strace: https://syzkaller.appspot.com/x/log.txt?x=1044bbad880000

kernel config: https://syzkaller.appspot.com/x/.config?x=8b2d3e63e054c24f
dashboard link: https://syzkaller.appspot.com/bug?extid=38a94e1872470e3450a6
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15236fe3880000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=178313f3880000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/bc862c01ec56/disk-591cd615.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/8f9b93f8ed2f/vmlinux-591cd615.xz
kernel image: https://storage.googleapis.com/syzbot-assets/9d5cb636d548/bzImage-591cd615.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+38a94e...@syzkaller.appspotmail.com

============================================
WARNING: possible recursive locking detected
6.1.0-rc8-next-20221207-syzkaller #0 Not tainted
--------------------------------------------

syz-executor169/5069 is trying to acquire lock:
ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: ovl_remove_privs_unlocked fs/overlayfs/file.c:519 [inline]
ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: ovl_fallocate+0x15c/0x270 fs/overlayfs/file.c:546

but task is already holding lock:

ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: ovl_fallocate+0x9f/0x270 fs/overlayfs/file.c:535

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----

lock(&ovl_i_mutex_key[depth]);
lock(&ovl_i_mutex_key[depth]);

*** DEADLOCK ***

May be due to missing lock nesting notation

2 locks held by syz-executor169/5069:
#0: ffff88802b1d6460 (sb_writers#9){.+.+}-{0:0}, at: ksys_fallocate fs/open.c:346 [inline]
#0: ffff88802b1d6460 (sb_writers#9){.+.+}-{0:0}, at: __do_sys_fallocate fs/open.c:354 [inline]
#0: ffff88802b1d6460 (sb_writers#9){.+.+}-{0:0}, at: __se_sys_fallocate fs/open.c:352 [inline]
#0: ffff88802b1d6460 (sb_writers#9){.+.+}-{0:0}, at: __x64_sys_fallocate+0xd3/0x140 fs/open.c:352
#1: ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
#1: ffff8880750fa480 (&ovl_i_mutex_key[depth]){+.+.}-{3:3}, at: ovl_fallocate+0x9f/0x270 fs/overlayfs/file.c:535

stack backtrace:
CPU: 1 PID: 5069 Comm: syz-executor169 Not tainted 6.1.0-rc8-next-20221207-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
print_deadlock_bug kernel/locking/lockdep.c:2990 [inline]
check_deadlock kernel/locking/lockdep.c:3033 [inline]
validate_chain kernel/locking/lockdep.c:3818 [inline]
__lock_acquire.cold+0x116/0x3a7 kernel/locking/lockdep.c:5055
lock_acquire kernel/locking/lockdep.c:5668 [inline]
lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
down_write+0x94/0x220 kernel/locking/rwsem.c:1562
inode_lock include/linux/fs.h:756 [inline]
ovl_remove_privs_unlocked fs/overlayfs/file.c:519 [inline]
ovl_fallocate+0x15c/0x270 fs/overlayfs/file.c:546
vfs_fallocate+0x48b/0xe00 fs/open.c:323
ksys_fallocate fs/open.c:346 [inline]
__do_sys_fallocate fs/open.c:354 [inline]
__se_sys_fallocate fs/open.c:352 [inline]
__x64_sys_fallocate+0xd3/0x140 fs/open.c:352
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

RIP: 0033:0x7fa3d1605c29
Code: 28 c3 e8 2a 14 00 00 66 2e 0f 1f 84 00 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffccc67e118 EFLAGS: 00000246 ORIG_RAX: 000000000000011d
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fa3d1605c29

RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000003

RBP: 00007fa3d15c9dd0 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000001000f4 R11: 0000000000000246 R12: 00007fa3d15c9e60
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
</TASK>

[Hillf Danton]

On 07 Dec 2022 12:47:34 -0800

> syzbot has found a reproducer for the following issue on:
>
> HEAD commit: 591cd61541b9 Add linux-next specific files for 20221207
> git tree: linux-next

> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=178313f3880000

Replace lock with debug info.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 591cd61541b9

--- x/fs/overlayfs/file.c
+++ y/fs/overlayfs/file.c
@@ -516,11 +516,10 @@ static int ovl_remove_privs_unlocked(str
struct inode *inode = file_inode(file);
int err;

- inode_lock(inode);
+ lockdep_assert_held_write(&inode->i_rwsem);
/* Update mode */
ovl_copyattr(inode);
err = file_remove_privs(file);
- inode_unlock(inode);

return err;
}
--

[Miklos Szeredi]

Removed offending commit from overlayfs-next (this was a duplicate fix).

Thanks,
Miklos

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+38a94e...@syzkaller.appspotmail.com

Tested on:

commit: 591cd615 Add linux-next specific files for 20221207
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
console output: https://syzkaller.appspot.com/x/log.txt?x=1030277b880000

kernel config: https://syzkaller.appspot.com/x/.config?x=8b2d3e63e054c24f
dashboard link: https://syzkaller.appspot.com/bug?extid=38a94e1872470e3450a6
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=12e0851d880000

Note: testing is done by a robot and is best-effort only.

[syzbot]

Auto-closing this bug as obsolete.
No recent activity, existing reproducers are no longer triggering the issue.