[v5.15] possible deadlock in ntfs_readpage

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 2ddbd0f967b3 Linux 5.15.102
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1367e048c80000
kernel config: https://syzkaller.appspot.com/x/.config?x=fec083380faceb1e
dashboard link: https://syzkaller.appspot.com/bug?extid=49cf7fd77f8792cd05d5
compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/156d2aa91f3c/disk-2ddbd0f9.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/f0e97f5be5fb/vmlinux-2ddbd0f9.xz
kernel image: https://storage.googleapis.com/syzbot-assets/20d0a55a041d/bzImage-2ddbd0f9.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+49cf7f...@syzkaller.appspotmail.com

======================================================
WARNING: possible circular locking dependency detected
5.15.102-syzkaller #0 Not tainted
------------------------------------------------------
kworker/u4:2/10660 is trying to acquire lock:
ffff8880761f2200 (&rl->lock){++++}-{3:3}, at: ntfs_read_block fs/ntfs/aops.c:248 [inline]
ffff8880761f2200 (&rl->lock){++++}-{3:3}, at: ntfs_readpage+0x123e/0x2210 fs/ntfs/aops.c:435

but task is already holding lock:
ffff8880761f5410 (&ni->mrec_lock){+.+.}-{3:3}, at: map_mft_record+0x4a/0x850 fs/ntfs/mft.c:154

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&ni->mrec_lock){+.+.}-{3:3}:
lock_acquire+0x1ff/0x570 kernel/locking/lockdep.c:5622
__mutex_lock_common+0x1da/0x25a0 kernel/locking/mutex.c:596
__mutex_lock kernel/locking/mutex.c:729 [inline]
mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
map_mft_record+0x4a/0x850 fs/ntfs/mft.c:154
ntfs_truncate+0x263/0x2860 fs/ntfs/inode.c:2383
ntfs_truncate_vfs fs/ntfs/inode.c:2862 [inline]
ntfs_setattr+0x2c7/0x3a0 fs/ntfs/inode.c:2914
notify_change+0xdad/0x1060 fs/attr.c:426
do_truncate+0x217/0x300 fs/open.c:65
handle_truncate fs/namei.c:3136 [inline]
do_open fs/namei.c:3484 [inline]
path_openat+0x2866/0x2ee0 fs/namei.c:3615
do_filp_open+0x21c/0x460 fs/namei.c:3642
do_sys_openat2+0x13b/0x500 fs/open.c:1211
do_sys_open fs/open.c:1227 [inline]
__do_sys_openat fs/open.c:1243 [inline]
__se_sys_openat fs/open.c:1238 [inline]
__x64_sys_openat+0x243/0x290 fs/open.c:1238
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x61/0xcb

-> #0 (&rl->lock){++++}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain+0x1646/0x58b0 kernel/locking/lockdep.c:3787
__lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5011
lock_acquire+0x1ff/0x570 kernel/locking/lockdep.c:5622
down_read+0x3b/0x50 kernel/locking/rwsem.c:1488
ntfs_read_block fs/ntfs/aops.c:248 [inline]
ntfs_readpage+0x123e/0x2210 fs/ntfs/aops.c:435
do_read_cache_page+0x752/0x1040
read_mapping_page include/linux/pagemap.h:515 [inline]
ntfs_map_page+0x25/0x390 fs/ntfs/aops.h:75
ntfs_sync_mft_mirror+0x253/0x19e0 fs/ntfs/mft.c:480
write_mft_record_nolock+0x1164/0x16c0 fs/ntfs/mft.c:787
write_mft_record fs/ntfs/mft.h:95 [inline]
__ntfs_write_inode+0x7b4/0xdd0 fs/ntfs/inode.c:3050
write_inode fs/fs-writeback.c:1475 [inline]
__writeback_single_inode+0x808/0x1210 fs/fs-writeback.c:1680
writeback_sb_inodes+0xbd8/0x1ac0 fs/fs-writeback.c:1905
wb_writeback+0x4b8/0xeb0 fs/fs-writeback.c:2079
wb_do_writeback fs/fs-writeback.c:2222 [inline]
wb_workfn+0x49a/0x10e0 fs/fs-writeback.c:2263
process_one_work+0x90d/0x1270 kernel/workqueue.c:2306
worker_thread+0xaca/0x1280 kernel/workqueue.c:2453
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 <unknown>:298

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&ni->mrec_lock);
lock(&rl->lock);
lock(&ni->mrec_lock);
lock(&rl->lock);

*** DEADLOCK ***

3 locks held by kworker/u4:2/10660:
#0: ffff8881427b4138 ((wq_completion)writeback){+.+.}-{0:0}, at: process_one_work+0x794/0x1270 kernel/workqueue.c:2279
#1: ffffc90002e7fd20 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_one_work+0x7da/0x1270 kernel/workqueue.c:2281
#2: ffff8880761f5410 (&ni->mrec_lock){+.+.}-{3:3}, at: map_mft_record+0x4a/0x850 fs/ntfs/mft.c:154

stack backtrace:
CPU: 1 PID: 10660 Comm: kworker/u4:2 Not tainted 5.15.102-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023
Workqueue: writeback wb_workfn (flush-7:1)
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
check_noncircular+0x2f8/0x3b0 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain+0x1646/0x58b0 kernel/locking/lockdep.c:3787
__lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5011
lock_acquire+0x1ff/0x570 kernel/locking/lockdep.c:5622
down_read+0x3b/0x50 kernel/locking/rwsem.c:1488
ntfs_read_block fs/ntfs/aops.c:248 [inline]
ntfs_readpage+0x123e/0x2210 fs/ntfs/aops.c:435
do_read_cache_page+0x752/0x1040
read_mapping_page include/linux/pagemap.h:515 [inline]
ntfs_map_page+0x25/0x390 fs/ntfs/aops.h:75
ntfs_sync_mft_mirror+0x253/0x19e0 fs/ntfs/mft.c:480
write_mft_record_nolock+0x1164/0x16c0 fs/ntfs/mft.c:787
write_mft_record fs/ntfs/mft.h:95 [inline]
__ntfs_write_inode+0x7b4/0xdd0 fs/ntfs/inode.c:3050
write_inode fs/fs-writeback.c:1475 [inline]
__writeback_single_inode+0x808/0x1210 fs/fs-writeback.c:1680
writeback_sb_inodes+0xbd8/0x1ac0 fs/fs-writeback.c:1905
wb_writeback+0x4b8/0xeb0 fs/fs-writeback.c:2079
wb_do_writeback fs/fs-writeback.c:2222 [inline]
wb_workfn+0x49a/0x10e0 fs/fs-writeback.c:2263
process_one_work+0x90d/0x1270 kernel/workqueue.c:2306
worker_thread+0xaca/0x1280 kernel/workqueue.c:2453
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 <unknown>:298
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: 8020ae3c051d Linux 5.15.103
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=11341dbec80000
kernel config: https://syzkaller.appspot.com/x/.config?x=d4215fb4040f8f8d

dashboard link: https://syzkaller.appspot.com/bug?extid=49cf7fd77f8792cd05d5
compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=107c9c6ec80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17397bc2c80000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/857e17de0f0a/disk-8020ae3c.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/9efc49fcd441/vmlinux-8020ae3c.xz
kernel image: https://storage.googleapis.com/syzbot-assets/f14c38b6bfa7/bzImage-8020ae3c.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/c4a74e783e7a/mount_3.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+49cf7f...@syzkaller.appspotmail.com

======================================================
WARNING: possible circular locking dependency detected

5.15.103-syzkaller #0 Not tainted
------------------------------------------------------
kworker/u4:4/1124 is trying to acquire lock:
ffff888074984b40 (&rl->lock){++++}-{3:3}, at: ntfs_read_block fs/ntfs/aops.c:248 [inline]
ffff888074984b40 (&rl->lock){++++}-{3:3}, at: ntfs_readpage+0x123e/0x2210 fs/ntfs/aops.c:435

but task is already holding lock:

ffff8880749889d0 (&ni->mrec_lock){+.+.}-{3:3}, at: map_mft_record+0x4a/0x850 fs/ntfs/mft.c:154

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&ni->mrec_lock){+.+.}-{3:3}:
lock_acquire+0x1ff/0x570 kernel/locking/lockdep.c:5622
__mutex_lock_common+0x1da/0x25a0 kernel/locking/mutex.c:596
__mutex_lock kernel/locking/mutex.c:729 [inline]
mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
map_mft_record+0x4a/0x850 fs/ntfs/mft.c:154
ntfs_truncate+0x263/0x2860 fs/ntfs/inode.c:2383
ntfs_truncate_vfs fs/ntfs/inode.c:2862 [inline]
ntfs_setattr+0x2c7/0x3a0 fs/ntfs/inode.c:2914

notify_change+0xd4d/0x1000 fs/attr.c:488
do_truncate+0x21c/0x300 fs/open.c:65
handle_truncate fs/namei.c:3195 [inline]
do_open fs/namei.c:3542 [inline]
path_openat+0x28a0/0x2f20 fs/namei.c:3672
do_filp_open+0x21c/0x460 fs/namei.c:3699

do_sys_openat2+0x13b/0x500 fs/open.c:1211
do_sys_open fs/open.c:1227 [inline]

__do_sys_creat fs/open.c:1301 [inline]
__se_sys_creat fs/open.c:1295 [inline]
__x64_sys_creat+0x11f/0x160 fs/open.c:1295

do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x61/0xcb

-> #0 (&rl->lock){++++}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain+0x1646/0x58b0 kernel/locking/lockdep.c:3787
__lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5011
lock_acquire+0x1ff/0x570 kernel/locking/lockdep.c:5622
down_read+0x3b/0x50 kernel/locking/rwsem.c:1488
ntfs_read_block fs/ntfs/aops.c:248 [inline]
ntfs_readpage+0x123e/0x2210 fs/ntfs/aops.c:435
do_read_cache_page+0x752/0x1040
read_mapping_page include/linux/pagemap.h:515 [inline]
ntfs_map_page+0x25/0x390 fs/ntfs/aops.h:75
ntfs_sync_mft_mirror+0x253/0x19e0 fs/ntfs/mft.c:480
write_mft_record_nolock+0x1164/0x16c0 fs/ntfs/mft.c:787
write_mft_record fs/ntfs/mft.h:95 [inline]
__ntfs_write_inode+0x7b4/0xdd0 fs/ntfs/inode.c:3050
write_inode fs/fs-writeback.c:1475 [inline]
__writeback_single_inode+0x808/0x1210 fs/fs-writeback.c:1680
writeback_sb_inodes+0xbd8/0x1ac0 fs/fs-writeback.c:1905

__writeback_inodes_wb+0x114/0x400 fs/fs-writeback.c:1976
wb_writeback+0x4cc/0xeb0 fs/fs-writeback.c:2081
wb_check_background_flush fs/fs-writeback.c:2147 [inline]
wb_do_writeback fs/fs-writeback.c:2235 [inline]
wb_workfn+0xcf2/0x10e0 fs/fs-writeback.c:2263

process_one_work+0x90d/0x1270 kernel/workqueue.c:2306
worker_thread+0xaca/0x1280 kernel/workqueue.c:2453
kthread+0x3f6/0x4f0 kernel/kthread.c:319

ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&ni->mrec_lock);
lock(&rl->lock);
lock(&ni->mrec_lock);
lock(&rl->lock);

*** DEADLOCK ***

4 locks held by kworker/u4:4/1124:
#0: ffff8880149ba938 ((wq_completion)writeback){+.+.}-{0:0}, at: process_one_work+0x794/0x1270 kernel/workqueue.c:2279
#1: ffffc90004d17d20 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_one_work+0x7da/0x1270 kernel/workqueue.c:2281
#2: ffff88801e84a0e0 (&type->s_umount_key#65){.+.+}-{3:3}, at: trylock_super+0x1b/0xf0 fs/super.c:418
#3: ffff8880749889d0 (&ni->mrec_lock){+.+.}-{3:3}, at: map_mft_record+0x4a/0x850 fs/ntfs/mft.c:154

stack backtrace:
CPU: 0 PID: 1124 Comm: kworker/u4:4 Not tainted 5.15.103-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023

Workqueue: writeback wb_workfn (flush-7:4)

Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
check_noncircular+0x2f8/0x3b0 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain+0x1646/0x58b0 kernel/locking/lockdep.c:3787
__lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5011
lock_acquire+0x1ff/0x570 kernel/locking/lockdep.c:5622
down_read+0x3b/0x50 kernel/locking/rwsem.c:1488
ntfs_read_block fs/ntfs/aops.c:248 [inline]
ntfs_readpage+0x123e/0x2210 fs/ntfs/aops.c:435
do_read_cache_page+0x752/0x1040
read_mapping_page include/linux/pagemap.h:515 [inline]
ntfs_map_page+0x25/0x390 fs/ntfs/aops.h:75
ntfs_sync_mft_mirror+0x253/0x19e0 fs/ntfs/mft.c:480
write_mft_record_nolock+0x1164/0x16c0 fs/ntfs/mft.c:787
write_mft_record fs/ntfs/mft.h:95 [inline]
__ntfs_write_inode+0x7b4/0xdd0 fs/ntfs/inode.c:3050
write_inode fs/fs-writeback.c:1475 [inline]
__writeback_single_inode+0x808/0x1210 fs/fs-writeback.c:1680
writeback_sb_inodes+0xbd8/0x1ac0 fs/fs-writeback.c:1905

__writeback_inodes_wb+0x114/0x400 fs/fs-writeback.c:1976
wb_writeback+0x4cc/0xeb0 fs/fs-writeback.c:2081
wb_check_background_flush fs/fs-writeback.c:2147 [inline]
wb_do_writeback fs/fs-writeback.c:2235 [inline]
wb_workfn+0xcf2/0x10e0 fs/fs-writeback.c:2263

process_one_work+0x90d/0x1270 kernel/workqueue.c:2306
worker_thread+0xaca/0x1280 kernel/workqueue.c:2453
kthread+0x3f6/0x4f0 kernel/kthread.c:319

ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>