[v6.1] possible deadlock in f2fs_handle_error

0 views
Skip to first unread message

syzbot

unread,
Mar 11, 2023, 6:05:49 AM3/11/23
to syzkaller...@googlegroups.com
Hello,

syzbot found the following issue on:

HEAD commit: 11585e2f8b9d Linux 6.1.17
git tree: linux-6.1.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1303540cc80000
kernel config: https://syzkaller.appspot.com/x/.config?x=537251fed575b8fd
dashboard link: https://syzkaller.appspot.com/bug?extid=ce62f2505197acb97356
compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/cb636e909679/disk-11585e2f.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/a81b3e460dd4/vmlinux-11585e2f.xz
kernel image: https://storage.googleapis.com/syzbot-assets/1408be0676c0/bzImage-11585e2f.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+ce62f2...@syzkaller.appspotmail.com

loop4: detected capacity change from 0 to 131072
F2FS-fs (loop4): invalid crc value
F2FS-fs (loop4): Found nat_bits in checkpoint
F2FS-fs (loop4): f2fs_check_nid_range: out-of-range nid=2, run fsck to fix.
======================================================
WARNING: possible circular locking dependency detected
6.1.17-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.4/10081 is trying to acquire lock:
ffff88801f31c088 (&sbi->sb_lock){++++}-{3:3}, at: f2fs_down_write fs/f2fs/f2fs.h:2205 [inline]
ffff88801f31c088 (&sbi->sb_lock){++++}-{3:3}, at: f2fs_handle_error+0xeb/0x230 fs/f2fs/super.c:3898

but task is already holding lock:
ffff8880752ce130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: f2fs_down_read fs/f2fs/f2fs.h:2180 [inline]
ffff8880752ce130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: __f2fs_build_free_nids fs/f2fs/node.c:2479 [inline]
ffff8880752ce130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: f2fs_build_free_nids+0x72a/0x1190 fs/f2fs/node.c:2527

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #5 (&nm_i->nat_tree_lock){++++}-{3:3}:
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
down_read+0x39/0x50 kernel/locking/rwsem.c:1520
f2fs_down_read fs/f2fs/f2fs.h:2180 [inline]
f2fs_get_node_info+0xef/0x12c0 fs/f2fs/node.c:560
__write_node_page+0x71f/0x2710 fs/f2fs/node.c:1612
f2fs_sync_node_pages+0xec0/0x16d0 fs/f2fs/node.c:2016
block_operations fs/f2fs/checkpoint.c:1270 [inline]
f2fs_write_checkpoint+0xe51/0x1cd0 fs/f2fs/checkpoint.c:1650
__write_checkpoint_sync fs/f2fs/checkpoint.c:1767 [inline]
__checkpoint_and_complete_reqs+0xd6/0x3b0 fs/f2fs/checkpoint.c:1786
issue_checkpoint_thread+0xd6/0x260 fs/f2fs/checkpoint.c:1817
kthread+0x268/0x300 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306

-> #4 (&sbi->node_change){++++}-{3:3}:
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
down_read+0x39/0x50 kernel/locking/rwsem.c:1520
f2fs_vm_page_mkwrite+0x763/0x1410 fs/f2fs/file.c:116
do_page_mkwrite+0x1a1/0x5f0 mm/memory.c:2973
do_shared_fault mm/memory.c:4614 [inline]
do_fault mm/memory.c:4682 [inline]
handle_pte_fault mm/memory.c:4950 [inline]
__handle_mm_fault mm/memory.c:5092 [inline]
handle_mm_fault+0x22f0/0x5140 mm/memory.c:5213
do_user_addr_fault arch/x86/mm/fault.c:1428 [inline]
handle_page_fault arch/x86/mm/fault.c:1519 [inline]
exc_page_fault+0x655/0x860 arch/x86/mm/fault.c:1575
asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:570

-> #3 (mapping.invalidate_lock#4){.+.+}-{3:3}:
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
down_read+0x39/0x50 kernel/locking/rwsem.c:1520
filemap_invalidate_lock_shared include/linux/fs.h:811 [inline]
f2fs_vm_page_mkwrite+0x543/0x1410 fs/f2fs/file.c:104
do_page_mkwrite+0x1a1/0x5f0 mm/memory.c:2973
do_shared_fault mm/memory.c:4614 [inline]
do_fault mm/memory.c:4682 [inline]
handle_pte_fault mm/memory.c:4950 [inline]
__handle_mm_fault mm/memory.c:5092 [inline]
handle_mm_fault+0x22f0/0x5140 mm/memory.c:5213
do_user_addr_fault arch/x86/mm/fault.c:1428 [inline]
handle_page_fault arch/x86/mm/fault.c:1519 [inline]
exc_page_fault+0x655/0x860 arch/x86/mm/fault.c:1575
asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:570

-> #2 (sb_pagefaults#2){.+.+}-{0:0}:
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write+0x58/0x1c0 include/linux/fs.h:1832
sb_start_pagefault include/linux/fs.h:1936 [inline]
f2fs_vm_page_mkwrite+0x477/0x1410 fs/f2fs/file.c:99
do_page_mkwrite+0x1a1/0x5f0 mm/memory.c:2973
do_shared_fault mm/memory.c:4614 [inline]
do_fault mm/memory.c:4682 [inline]
handle_pte_fault mm/memory.c:4950 [inline]
__handle_mm_fault mm/memory.c:5092 [inline]
handle_mm_fault+0x22f0/0x5140 mm/memory.c:5213
do_user_addr_fault arch/x86/mm/fault.c:1428 [inline]
handle_page_fault arch/x86/mm/fault.c:1519 [inline]
exc_page_fault+0x655/0x860 arch/x86/mm/fault.c:1575
asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:570

-> #1 (&mm->mmap_lock#2){++++}-{3:3}:
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
__might_fault+0xb2/0x110 mm/memory.c:5641
_copy_to_user+0x26/0x130 lib/usercopy.c:36
copy_to_user include/linux/uaccess.h:169 [inline]
f2fs_ioc_get_encryption_pwsalt fs/f2fs/file.c:2353 [inline]
__f2fs_ioctl+0xb718/0xbdc0 fs/f2fs/file.c:4155
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:870 [inline]
__se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (&sbi->sb_lock){++++}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3098 [inline]
check_prevs_add kernel/locking/lockdep.c:3217 [inline]
validate_chain+0x1667/0x58e0 kernel/locking/lockdep.c:3832
__lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5056
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
down_write+0x36/0x60 kernel/locking/rwsem.c:1573
f2fs_down_write fs/f2fs/f2fs.h:2205 [inline]
f2fs_handle_error+0xeb/0x230 fs/f2fs/super.c:3898
scan_nat_page fs/f2fs/node.c:2383 [inline]
__f2fs_build_free_nids fs/f2fs/node.c:2489 [inline]
f2fs_build_free_nids+0xcae/0x1190 fs/f2fs/node.c:2527
f2fs_fill_super+0x53f9/0x7cb0 fs/f2fs/super.c:4308
mount_bdev+0x26d/0x3a0 fs/super.c:1414
legacy_get_tree+0xeb/0x180 fs/fs_context.c:610
vfs_get_tree+0x88/0x270 fs/super.c:1544
do_new_mount+0x28b/0xad0 fs/namespace.c:3040
do_mount fs/namespace.c:3383 [inline]
__do_sys_mount fs/namespace.c:3591 [inline]
__se_sys_mount+0x2d5/0x3c0 fs/namespace.c:3568
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

Chain exists of:
&sbi->sb_lock --> &sbi->node_change --> &nm_i->nat_tree_lock

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&nm_i->nat_tree_lock);
lock(&sbi->node_change);
lock(&nm_i->nat_tree_lock);
lock(&sbi->sb_lock);

*** DEADLOCK ***

3 locks held by syz-executor.4/10081:
#0: ffff8880541f00e0 (&type->s_umount_key#62/1){+.+.}-{3:3}, at: alloc_super+0x217/0x930 fs/super.c:228
#1: ffff8880752ce2c8 (&nm_i->build_lock){+.+.}-{3:3}, at: f2fs_build_free_nids+0xc8/0x1190 fs/f2fs/node.c:2526
#2: ffff8880752ce130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: f2fs_down_read fs/f2fs/f2fs.h:2180 [inline]
#2: ffff8880752ce130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: __f2fs_build_free_nids fs/f2fs/node.c:2479 [inline]
#2: ffff8880752ce130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: f2fs_build_free_nids+0x72a/0x1190 fs/f2fs/node.c:2527

stack backtrace:
CPU: 1 PID: 10081 Comm: syz-executor.4 Not tainted 6.1.17-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
check_noncircular+0x2fa/0x3b0 kernel/locking/lockdep.c:2178
check_prev_add kernel/locking/lockdep.c:3098 [inline]
check_prevs_add kernel/locking/lockdep.c:3217 [inline]
validate_chain+0x1667/0x58e0 kernel/locking/lockdep.c:3832
__lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5056
lock_acquire+0x23a/0x630 kernel/locking/lockdep.c:5669
down_write+0x36/0x60 kernel/locking/rwsem.c:1573
f2fs_down_write fs/f2fs/f2fs.h:2205 [inline]
f2fs_handle_error+0xeb/0x230 fs/f2fs/super.c:3898
scan_nat_page fs/f2fs/node.c:2383 [inline]
__f2fs_build_free_nids fs/f2fs/node.c:2489 [inline]
f2fs_build_free_nids+0xcae/0x1190 fs/f2fs/node.c:2527
f2fs_fill_super+0x53f9/0x7cb0 fs/f2fs/super.c:4308
mount_bdev+0x26d/0x3a0 fs/super.c:1414
legacy_get_tree+0xeb/0x180 fs/fs_context.c:610
vfs_get_tree+0x88/0x270 fs/super.c:1544
do_new_mount+0x28b/0xad0 fs/namespace.c:3040
do_mount fs/namespace.c:3383 [inline]
__do_sys_mount fs/namespace.c:3591 [inline]
__se_sys_mount+0x2d5/0x3c0 fs/namespace.c:3568
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fe4eea8d62a
Code: 48 c7 c2 b8 ff ff ff f7 d8 64 89 02 b8 ff ff ff ff eb d2 e8 b8 04 00 00 0f 1f 84 00 00 00 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fe4ef8d8f88 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5
RAX: ffffffffffffffda RBX: 00000000000105fb RCX: 00007fe4eea8d62a
RDX: 0000000020010600 RSI: 0000000020010640 RDI: 00007fe4ef8d8fe0
RBP: 00007fe4ef8d9020 R08: 00007fe4ef8d9020 R09: 0000000001008050
R10: 0000000001008050 R11: 0000000000000202 R12: 0000000020010600
R13: 0000000020010640 R14: 00007fe4ef8d8fe0 R15: 0000000020000000
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

syzbot

unread,
Jul 9, 2023, 7:05:40 AM7/9/23
to syzkaller...@googlegroups.com
Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.
Reply all
Reply to author
Forward
0 new messages