[syzbot] [ntfs3?] possible deadlock in ntfs_set_state (2)

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 716f4aaa7b48 Merge tag 'vfs-6.8-rc5.fixes' of git://git.ke..
git tree: upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=100fd062180000
kernel config: https://syzkaller.appspot.com/x/.config?x=1d7c92dd8d5c7a1e
dashboard link: https://syzkaller.appspot.com/bug?extid=c2ada45c23d98d646118
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=11fcbd48180000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17f6e642180000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/ca4bf59e5a18/disk-716f4aaa.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/3d7ade517e63/vmlinux-716f4aaa.xz
kernel image: https://storage.googleapis.com/syzbot-assets/e13f7054c0c1/bzImage-716f4aaa.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/00ba9c2f3dd0/mount_0.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+c2ada4...@syzkaller.appspotmail.com

loop0: detected capacity change from 0 to 4096
ntfs3: loop0: Different NTFS sector size (4096) and media sector size (512).
ntfs3: loop0: ino=5, "/" ntfs_iget5
============================================
WARNING: possible recursive locking detected
6.8.0-rc4-syzkaller-00003-g716f4aaa7b48 #0 Not tainted
--------------------------------------------
syz-executor354/5071 is trying to acquire lock:
ffff888070ee0100 (&ni->ni_lock#3){+.+.}-{3:3}, at: ntfs_set_state+0x1ff/0x6c0 fs/ntfs3/fsntfs.c:947

but task is already holding lock:
ffff888070de3c00 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_trylock fs/ntfs3/ntfs_fs.h:1141 [inline]
ffff888070de3c00 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_write_inode+0x1bc/0x1010 fs/ntfs3/frecord.c:3265

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----
lock(&ni->ni_lock#3);
lock(&ni->ni_lock#3);

*** DEADLOCK ***

May be due to missing lock nesting notation

3 locks held by syz-executor354/5071:
#0: ffff88802223a420 (sb_writers#9){.+.+}-{0:0}, at: do_sys_ftruncate+0x25c/0x390 fs/open.c:191
#1: ffff888070de3ea0 (&sb->s_type->i_mutex_key#15){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:802 [inline]
#1: ffff888070de3ea0 (&sb->s_type->i_mutex_key#15){+.+.}-{3:3}, at: do_truncate+0x20c/0x310 fs/open.c:64
#2: ffff888070de3c00 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_trylock fs/ntfs3/ntfs_fs.h:1141 [inline]
#2: ffff888070de3c00 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_write_inode+0x1bc/0x1010 fs/ntfs3/frecord.c:3265

stack backtrace:
CPU: 0 PID: 5071 Comm: syz-executor354 Not tainted 6.8.0-rc4-syzkaller-00003-g716f4aaa7b48 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
check_deadlock kernel/locking/lockdep.c:3062 [inline]
validate_chain+0x15c0/0x58e0 kernel/locking/lockdep.c:3856
__lock_acquire+0x1345/0x1fd0 kernel/locking/lockdep.c:5137
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
__mutex_lock_common kernel/locking/mutex.c:608 [inline]
__mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
ntfs_set_state+0x1ff/0x6c0 fs/ntfs3/fsntfs.c:947
ntfs_iget5+0x3f0/0x3b70 fs/ntfs3/inode.c:535
ni_update_parent+0x943/0xdd0 fs/ntfs3/frecord.c:3218
ni_write_inode+0xde9/0x1010 fs/ntfs3/frecord.c:3324
ntfs_truncate fs/ntfs3/file.c:410 [inline]
ntfs3_setattr+0x950/0xb40 fs/ntfs3/file.c:703
notify_change+0xb9f/0xe70 fs/attr.c:499
do_truncate+0x220/0x310 fs/open.c:66
do_sys_ftruncate+0x2f7/0x390 fs/open.c:194
do_syscall_64+0xfb/0x240
entry_SYSCALL_64_after_hwframe+0x6f/0x77
RIP: 0033:0x7fd0ca446639
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 61 17 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fff0baab678 EFLAGS: 00000246 ORIG_RAX: 000000000000004d
RAX: ffffffffffffffda RBX: 00007fff0baab848 RCX: 00007fd0ca446639
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004
RBP: 00007fd0ca4d8610 R08: 0000000000000000 R09: 00007fff0baab848
R10: 000000000001f20a R11: 0000000000000246 R12: 0000000000000001
R13: 00007fff0baab838 R14: 0000000000000001 R15: 0000000000000001
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Edward Adam Davis]

please test deadlock in ntfs_set_state

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

diff --git a/fs/ntfs3/frecord.c b/fs/ntfs3/frecord.c
index 3b42938a9d3b..1dfc933b58ad 100644
--- a/fs/ntfs3/frecord.c
+++ b/fs/ntfs3/frecord.c
@@ -3210,7 +3210,6 @@ static bool ni_update_parent(struct ntfs_inode *ni, struct NTFS_DUP_INFO *dup,

/* Check simple case when parent inode equals current inode. */
if (ino_get(&fname->home) == ni->vfs_inode.i_ino) {
- ntfs_set_state(sbi, NTFS_DIRTY_ERROR);
continue;
}

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
possible deadlock in ntfs_set_state

loop0: detected capacity change from 0 to 4096
ntfs3: loop0: Different NTFS sector size (4096) and media sector size (512).
ntfs3: loop0: ino=5, "/" ntfs_iget5
============================================
WARNING: possible recursive locking detected

6.8.0-rc4-syzkaller-00005-gc664e16bb1ba-dirty #0 Not tainted
--------------------------------------------
syz-executor.0/5514 is trying to acquire lock:
ffff888068bf8100 (&ni->ni_lock#3){+.+.}-{3:3}, at: ntfs_set_state+0x1ff/0x6c0 fs/ntfs3/fsntfs.c:947

but task is already holding lock:

ffff888068bf9e80 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_trylock fs/ntfs3/ntfs_fs.h:1141 [inline]
ffff888068bf9e80 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_write_inode+0x1bc/0x1010 fs/ntfs3/frecord.c:3264

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----
lock(&ni->ni_lock#3);
lock(&ni->ni_lock#3);

*** DEADLOCK ***

May be due to missing lock nesting notation

3 locks held by syz-executor.0/5514:
#0: ffff88802ec00420 (sb_writers#13){.+.+}-{0:0}, at: do_sys_ftruncate+0x25c/0x390 fs/open.c:191
#1: ffff888068bfa120 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:802 [inline]
#1: ffff888068bfa120 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: do_truncate+0x20c/0x310 fs/open.c:64
#2: ffff888068bf9e80 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_trylock fs/ntfs3/ntfs_fs.h:1141 [inline]
#2: ffff888068bf9e80 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_write_inode+0x1bc/0x1010 fs/ntfs3/frecord.c:3264

stack backtrace:
CPU: 0 PID: 5514 Comm: syz-executor.0 Not tainted 6.8.0-rc4-syzkaller-00005-gc664e16bb1ba-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
check_deadlock kernel/locking/lockdep.c:3062 [inline]
validate_chain+0x15c0/0x58e0 kernel/locking/lockdep.c:3856
__lock_acquire+0x1345/0x1fd0 kernel/locking/lockdep.c:5137
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
__mutex_lock_common kernel/locking/mutex.c:608 [inline]
__mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
ntfs_set_state+0x1ff/0x6c0 fs/ntfs3/fsntfs.c:947
ntfs_iget5+0x3f0/0x3b70 fs/ntfs3/inode.c:535

ni_update_parent+0x934/0xdc0 fs/ntfs3/frecord.c:3217
ni_write_inode+0xde9/0x1010 fs/ntfs3/frecord.c:3323

ntfs_truncate fs/ntfs3/file.c:410 [inline]
ntfs3_setattr+0x950/0xb40 fs/ntfs3/file.c:703
notify_change+0xb9f/0xe70 fs/attr.c:499
do_truncate+0x220/0x310 fs/open.c:66
do_sys_ftruncate+0x2f7/0x390 fs/open.c:194
do_syscall_64+0xfb/0x240
entry_SYSCALL_64_after_hwframe+0x6f/0x77

RIP: 0033:0x7f8b46a7dda9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f8b478530c8 EFLAGS: 00000246 ORIG_RAX: 000000000000004d
RAX: ffffffffffffffda RBX: 00007f8b46babf80 RCX: 00007f8b46a7dda9

RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004

RBP: 00007f8b46aca47a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007f8b46babf80 R15: 00007ffcddaf9d18
</TASK>
ntfs3: loop0: Mark volume as dirty due to NTFS errors


Tested on:

commit: c664e16b Merge tag 'docs-6.8-fixes2' of git://git.lwn...
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
console output: https://syzkaller.appspot.com/x/log.txt?x=152c5114180000

kernel config: https://syzkaller.appspot.com/x/.config?x=1d7c92dd8d5c7a1e
dashboard link: https://syzkaller.appspot.com/bug?extid=c2ada45c23d98d646118
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=15ff6de0180000

[Hillf Danton]

On Mon, 12 Feb 2024 23:12:22 -0800

This report looks false positive but raises the question -- what made lockedp
pull the wrong trigger? Because of the correct lock_class_key in mutex_init()
instead of &ni->ni_lock?

[Boqun Feng]

this is a mutex_lock_nested() with a subkey 0.

> >
> > but task is already holding lock:
> > ffff888070de3c00 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_trylock fs/ntfs3/ntfs_fs.h:1141 [inline]
> > ffff888070de3c00 (&ni->ni_lock#3){+.+.}-{3:3}, at: ni_write_inode+0x1bc/0x1010 fs/ntfs3/frecord.c:3265

These two are try locks.

> >
> This report looks false positive but raises the question -- what made lockedp

This is not a false positive by lockdep locking rules, basically it
reported deadlock cases as the follow:

mutex_trylock(A1);
mutex_trylock(A2);
mutex_lock(A1 /* or A2 */);

Two things to notice here: 1) these two trylock()s not resulting in
real deadlock cases must be because they are on different lock
instances, 2) deadlock detectors work on lock classes, so although the
mutex_lock() above may be on a different instance (say A3), currently
there is no way for lockdep to tell that. In this case, users need to
use subkeys to tell lockdep mutex_lock() and mutex_trylock() are on
different sets of instannces (i.e. sub classes). Note that subkey == 0
means the main class.

Regards,
Boqun

[Hillf Danton]

On Sun, 25 Feb 2024 20:23:56 -0800 Boqun Feng wrote:
> On Tue, Feb 13, 2024 at 07:41:50PM +0800, Hillf Danton wrote:
> > On Mon, 12 Feb 2024 23:12:22 -0800
> > > HEAD commit: 716f4aaa7b48 Merge tag 'vfs-6.8-rc5.fixes' of git://git.ke..
> > > git tree: upstream
> > > console+strace: https://syzkaller.appspot.com/x/log.txt?x=100fd062180000
> > > kernel config: https://syzkaller.appspot.com/x/.config?x=1d7c92dd8d5c7a1e
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=c2ada45c23d98d646118
> > > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=11fcbd48180000
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17f6e642180000
> > >

Test non-zero subkey.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 716f4aaa7b48

--- x/fs/ntfs3/fsntfs.c
+++ y/fs/ntfs3/fsntfs.c
@@ -944,7 +944,7 @@ int ntfs_set_state(struct ntfs_sb_info *
if (!ni)
return -EINVAL;

- mutex_lock_nested(&ni->ni_lock, NTFS_INODE_MUTEX_DIRTY);
+ mutex_lock_nested(&ni->ni_lock, NTFS_INODE_MUTEX_NORMAL);

attr = ni_find_attr(ni, NULL, NULL, ATTR_VOL_INFO, NULL, 0, NULL, &mi);
if (!attr) {
--

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to checkout kernel repo https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git on commit 716f4aaa7b48: failed to run ["git" "fetch" "--force" "--tags" "4d52a57a3858a6eee0d0b25cc3a0c9533f747d8f" "716f4aaa7b48"]: exit status 128
fatal: couldn't find remote ref 716f4aaa7b48



Tested on:

commit: [unknown
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 716f4aaa7b48

kernel config: https://syzkaller.appspot.com/x/.config?x=1d7c92dd8d5c7a1e
dashboard link: https://syzkaller.appspot.com/bug?extid=c2ada45c23d98d646118
compiler:

patch: https://syzkaller.appspot.com/x/patch.diff?x=172e5f4a180000

[Hillf Danton]

Test non-zero subkey against the mainline tree.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

--- x/fs/ntfs3/fsntfs.c
+++ y/fs/ntfs3/fsntfs.c
@@ -944,7 +944,7 @@ int ntfs_set_state(struct ntfs_sb_info *
if (!ni)
return -EINVAL;

- mutex_lock_nested(&ni->ni_lock, NTFS_INODE_MUTEX_DIRTY);

+ mutex_lock_nested(&ni->ni_lock, (1 + NTFS_INODE_MUTEX_PARENT2));

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+c2ada4...@syzkaller.appspotmail.com

Tested on:

commit: d206a76d Linux 6.8-rc6

git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

console output: https://syzkaller.appspot.com/x/log.txt?x=10dec3aa180000
kernel config: https://syzkaller.appspot.com/x/.config?x=fad652894fc96962
dashboard link: https://syzkaller.appspot.com/bug?extid=c2ada45c23d98d646118

compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=163e9a9a180000

Note: testing is done by a robot and is best-effort only.