[syzbot] [ext4?] KASAN: slab-use-after-free Read in fsnotify

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 6ebf211bb11d Add linux-next specific files for 20240410
git tree: linux-next
console+strace: https://syzkaller.appspot.com/x/log.txt?x=12be955d180000
kernel config: https://syzkaller.appspot.com/x/.config?x=16ca158ef7e08662
dashboard link: https://syzkaller.appspot.com/bug?extid=5e3f9b2a67b45f16d4e6
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13c91175180000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1621af9d180000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/b050f81f73ed/disk-6ebf211b.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/412c9b9a536e/vmlinux-6ebf211b.xz
kernel image: https://storage.googleapis.com/syzbot-assets/016527216c47/bzImage-6ebf211b.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/75ad050c9945/mount_0.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+5e3f9b...@syzkaller.appspotmail.com

Quota error (device loop0): do_check_range: Getting block 0 out of range 1-5
EXT4-fs error (device loop0): ext4_release_dquot:6905: comm kworker/u8:4: Failed to release dquot type 1
==================================================================
BUG: KASAN: slab-use-after-free in fsnotify+0x2a4/0x1f70 fs/notify/fsnotify.c:539
Read of size 8 at addr ffff88802f1dce80 by task kworker/u8:4/62

CPU: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.9.0-rc3-next-20240410-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
Workqueue: events_unbound quota_release_workfn
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
print_address_description mm/kasan/report.c:377 [inline]
print_report+0x169/0x550 mm/kasan/report.c:488
kasan_report+0x143/0x180 mm/kasan/report.c:601
fsnotify+0x2a4/0x1f70 fs/notify/fsnotify.c:539
fsnotify_sb_error include/linux/fsnotify.h:456 [inline]
__ext4_error+0x255/0x3b0 fs/ext4/super.c:843
ext4_release_dquot+0x326/0x450 fs/ext4/super.c:6903
quota_release_workfn+0x39f/0x650 fs/quota/dquot.c:840
process_one_work kernel/workqueue.c:3218 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3299
worker_thread+0x86d/0xd70 kernel/workqueue.c:3380
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>

Allocated by task 5085:
kasan_save_stack mm/kasan/common.c:47 [inline]
kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
poison_kmalloc_redzone mm/kasan/common.c:370 [inline]
__kasan_kmalloc+0x98/0xb0 mm/kasan/common.c:387
kasan_kmalloc include/linux/kasan.h:211 [inline]
kmalloc_trace_noprof+0x19c/0x2b0 mm/slub.c:4109
kmalloc_noprof include/linux/slab.h:660 [inline]
kzalloc_noprof include/linux/slab.h:775 [inline]
fsnotify_attach_info_to_sb fs/notify/mark.c:600 [inline]
fsnotify_add_mark_list fs/notify/mark.c:692 [inline]
fsnotify_add_mark_locked+0x3b2/0xe60 fs/notify/mark.c:777
fanotify_add_new_mark fs/notify/fanotify/fanotify_user.c:1267 [inline]
fanotify_add_mark+0xbbd/0x1330 fs/notify/fanotify/fanotify_user.c:1334
do_fanotify_mark+0xbcc/0xd90 fs/notify/fanotify/fanotify_user.c:1896
__do_sys_fanotify_mark fs/notify/fanotify/fanotify_user.c:1919 [inline]
__se_sys_fanotify_mark fs/notify/fanotify/fanotify_user.c:1915 [inline]
__x64_sys_fanotify_mark+0xb5/0xd0 fs/notify/fanotify/fanotify_user.c:1915
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f

Freed by task 5085:
kasan_save_stack mm/kasan/common.c:47 [inline]
kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
kasan_save_free_info+0x40/0x50 mm/kasan/generic.c:579
poison_slab_object+0xe0/0x150 mm/kasan/common.c:240
__kasan_slab_free+0x37/0x60 mm/kasan/common.c:256
kasan_slab_free include/linux/kasan.h:184 [inline]
slab_free_hook mm/slub.c:2190 [inline]
slab_free mm/slub.c:4393 [inline]
kfree+0x149/0x350 mm/slub.c:4514
fsnotify_sb_delete+0x686/0x6f0 fs/notify/fsnotify.c:106
generic_shutdown_super+0xa5/0x2d0 fs/super.c:632
kill_block_super+0x44/0x90 fs/super.c:1675
ext4_kill_sb+0x68/0xa0 fs/ext4/super.c:7323
deactivate_locked_super+0xc4/0x130 fs/super.c:472
cleanup_mnt+0x426/0x4c0 fs/namespace.c:1267
task_work_run+0x24f/0x310 kernel/task_work.c:180
exit_task_work include/linux/task_work.h:38 [inline]
do_exit+0xa1b/0x27e0 kernel/exit.c:878
do_group_exit+0x207/0x2c0 kernel/exit.c:1027
__do_sys_exit_group kernel/exit.c:1038 [inline]
__se_sys_exit_group kernel/exit.c:1036 [inline]
__x64_sys_exit_group+0x3f/0x40 kernel/exit.c:1036
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f

The buggy address belongs to the object at ffff88802f1dce80
which belongs to the cache kmalloc-32 of size 32
The buggy address is located 0 bytes inside of
freed 32-byte region [ffff88802f1dce80, ffff88802f1dcea0)

The buggy address belongs to the physical page:
page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x2f1dc
flags: 0xfff80000000000(node=0|zone=1|lastcpupid=0xfff)
page_type: 0xffffefff(slab)
raw: 00fff80000000000 ffff888015041500 ffffea000096b540 dead000000000004
raw: 0000000000000000 0000000080400040 00000001ffffefff 0000000000000000
page dumped because: kasan: bad access detected
page_owner tracks the page as allocated
page last allocated via order 0, migratetype Unmovable, gfp_mask 0x12cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_NORETRY), pid 1, tgid -625457465 (swapper/0), ts 1, free_ts 0
set_page_owner include/linux/page_owner.h:32 [inline]
post_alloc_hook+0x1f3/0x230 mm/page_alloc.c:1468
prep_new_page mm/page_alloc.c:1476 [inline]
get_page_from_freelist+0x2ce2/0x2d90 mm/page_alloc.c:3438
__alloc_pages_noprof+0x256/0x6c0 mm/page_alloc.c:4696
__alloc_pages_node_noprof include/linux/gfp.h:244 [inline]
alloc_pages_node_noprof include/linux/gfp.h:271 [inline]
alloc_slab_page+0x5f/0x120 mm/slub.c:2259
allocate_slab+0x5a/0x2e0 mm/slub.c:2422
new_slab mm/slub.c:2475 [inline]
___slab_alloc+0xcd1/0x14b0 mm/slub.c:3624
__slab_alloc+0x58/0xa0 mm/slub.c:3714
__slab_alloc_node mm/slub.c:3767 [inline]
slab_alloc_node mm/slub.c:3945 [inline]
__do_kmalloc_node mm/slub.c:4077 [inline]
kmalloc_node_track_caller_noprof+0x286/0x440 mm/slub.c:4098
kstrdup+0x3a/0x80 mm/util.c:62
kobject_set_name_vargs+0x61/0x120 lib/kobject.c:274
kobject_add_varg lib/kobject.c:368 [inline]
kobject_init_and_add+0xde/0x190 lib/kobject.c:457
sysfs_slab_add+0x7a/0x290 mm/slub.c:6877
slab_sysfs_init+0x66/0x170 mm/slub.c:6961
do_one_initcall+0x248/0x880 init/main.c:1263
do_initcall_level+0x157/0x210 init/main.c:1325
do_initcalls+0x3f/0x80 init/main.c:1341
page_owner free stack trace missing

Memory state around the buggy address:
ffff88802f1dcd80: 00 00 05 fc fc fc fc fc 00 00 00 00 fc fc fc fc
ffff88802f1dce00: 00 00 00 06 fc fc fc fc fa fb fb fb fc fc fc fc
>ffff88802f1dce80: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc
^
ffff88802f1dcf00: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc
ffff88802f1dcf80: fa fb fb fb fc fc fc fc 00 00 00 fc fc fc fc fc
==================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Jan Kara]

Amir, I believe this happens on umount when the filesystem calls
fsnotify_sb_error() after calling fsnotify_sb_delete(). In theory these two
calls can even run in parallel and fsnotify() can be holding
fsnotify_sb_info pointer while fsnotify_sb_delete() is freeing it so we
need to figure out some proper synchronization for that...

Honza

--
Jan Kara <ja...@suse.com>
SUSE Labs, CR

[Amir Goldstein]

Is it really needed to handle any for non SB_ACTIVE sb?
How about something like this?
Is that enough? or more synchronization is needed?

#syz test: https://github.com/amir73il/linux fsnotify-fixes

Thanks,
Amir.

[Gabriel Krisman Bertazi]

I think it should be fine to exclude volumes being teared down. Cc'ing
Khazhy, who sponsored this work at the time and owned the use-case.

--
Gabriel Krisman Bertazi

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

viperboard
[ 7.499011][ T1] usbcore: registered new interface driver dln2
[ 7.500804][ T1] usbcore: registered new interface driver pn533_usb
[ 7.507181][ T1] nfcsim 0.2 initialized
[ 7.508964][ T1] usbcore: registered new interface driver port100
[ 7.511844][ T1] usbcore: registered new interface driver nfcmrvl
[ 7.519814][ T1] Loading iSCSI transport class v2.0-870.
[ 7.539126][ T1] virtio_scsi virtio0: 1/0/0 default/read/poll queues
[ 7.550224][ T1] ------------[ cut here ]------------
[ 7.551264][ T1] refcount_t: decrement hit 0; leaking memory.
[ 7.552627][ T1] WARNING: CPU: 0 PID: 1 at lib/refcount.c:31 refcount_warn_saturate+0xfa/0x1d0
[ 7.554218][ T1] Modules linked in:
[ 7.554791][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.9.0-rc3-syzkaller-00014-geb06a4b6cca5 #0
[ 7.556609][ T1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
[ 7.558128][ T1] RIP: 0010:refcount_warn_saturate+0xfa/0x1d0
[ 7.559937][ T1] Code: b2 00 00 00 e8 87 70 e7 fc 5b 5d c3 cc cc cc cc e8 7b 70 e7 fc c6 05 0c 5d e5 0a 01 90 48 c7 c7 40 4b 1f 8c e8 17 ee a9 fc 90 <0f> 0b 90 90 eb d9 e8 5b 70 e7 fc c6 05 e9 5c e5 0a 01 90 48 c7 c7
[ 7.563097][ T1] RSP: 0000:ffffc90000066e18 EFLAGS: 00010246
[ 7.564240][ T1] RAX: 6f40bd285f2a6000 RBX: ffff888147ed00fc RCX: ffff8880166d0000
[ 7.565743][ T1] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 7.567236][ T1] RBP: 0000000000000004 R08: ffffffff815800a2 R09: fffffbfff1c39af8
[ 7.568531][ T1] R10: dffffc0000000000 R11: fffffbfff1c39af8 R12: ffffea000503edc0
[ 7.570021][ T1] R13: ffffea000503edc8 R14: 1ffffd4000a07db9 R15: 0000000000000000
[ 7.571764][ T1] FS: 0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
[ 7.573270][ T1] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7.574232][ T1] CR2: ffff88823ffff000 CR3: 000000000e134000 CR4: 00000000003506f0
[ 7.575566][ T1] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7.576737][ T1] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 7.578004][ T1] Call Trace:
[ 7.578712][ T1] <TASK>
[ 7.579189][ T1] ? __warn+0x163/0x4e0
[ 7.580052][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.580896][ T1] ? report_bug+0x2b3/0x500
[ 7.581593][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.582383][ T1] ? handle_bug+0x3e/0x70
[ 7.583169][ T1] ? exc_invalid_op+0x1a/0x50
[ 7.584335][ T1] ? asm_exc_invalid_op+0x1a/0x20
[ 7.585285][ T1] ? __warn_printk+0x292/0x360
[ 7.586058][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.586882][ T1] ? refcount_warn_saturate+0xf9/0x1d0
[ 7.587666][ T1] __free_pages_ok+0xc60/0xd90
[ 7.588339][ T1] make_alloc_exact+0xa3/0xf0
[ 7.589220][ T1] vring_alloc_queue_split+0x20a/0x600
[ 7.590378][ T1] ? __pfx_vring_alloc_queue_split+0x10/0x10
[ 7.591429][ T1] ? vp_find_vqs+0x4c/0x4e0
[ 7.592174][ T1] ? virtscsi_probe+0x3ea/0xf60
[ 7.592853][ T1] ? virtio_dev_probe+0x991/0xaf0
[ 7.593892][ T1] ? really_probe+0x2b8/0xad0
[ 7.594581][ T1] ? driver_probe_device+0x50/0x430
[ 7.595325][ T1] vring_create_virtqueue_split+0xc6/0x310
[ 7.596200][ T1] ? ret_from_fork+0x4b/0x80
[ 7.597705][ T1] ? __pfx_vring_create_virtqueue_split+0x10/0x10
[ 7.599051][ T1] vring_create_virtqueue+0xca/0x110
[ 7.599905][ T1] ? __pfx_vp_notify+0x10/0x10
[ 7.600626][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.601770][ T1] setup_vq+0xe9/0x2d0
[ 7.602429][ T1] ? __pfx_vp_notify+0x10/0x10
[ 7.603153][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.604003][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.604758][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.605623][ T1] vp_setup_vq+0xbf/0x330
[ 7.606388][ T1] ? __pfx_vp_config_changed+0x10/0x10
[ 7.607239][ T1] ? ioread16+0x2f/0x90
[ 7.608129][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.609047][ T1] vp_find_vqs_msix+0x8b2/0xc80
[ 7.609880][ T1] vp_find_vqs+0x4c/0x4e0
[ 7.610578][ T1] virtscsi_init+0x8db/0xd00
[ 7.611247][ T1] ? __pfx_virtscsi_init+0x10/0x10
[ 7.612058][ T1] ? __pfx_default_calc_sets+0x10/0x10
[ 7.612860][ T1] ? scsi_host_alloc+0xa57/0xea0
[ 7.613671][ T1] ? vp_get+0xfd/0x140
[ 7.614243][ T1] virtscsi_probe+0x3ea/0xf60
[ 7.614969][ T1] ? __pfx_virtscsi_probe+0x10/0x10
[ 7.615847][ T1] ? kernfs_add_one+0x156/0x8b0
[ 7.616678][ T1] ? virtio_no_restricted_mem_acc+0x9/0x10
[ 7.617627][ T1] ? virtio_features_ok+0x10c/0x270
[ 7.618392][ T1] virtio_dev_probe+0x991/0xaf0
[ 7.619262][ T1] ? __pfx_virtio_dev_probe+0x10/0x10
[ 7.620216][ T1] really_probe+0x2b8/0xad0
[ 7.621124][ T1] __driver_probe_device+0x1a2/0x390
[ 7.621980][ T1] driver_probe_device+0x50/0x430
[ 7.622710][ T1] __driver_attach+0x45f/0x710
[ 7.623413][ T1] ? __pfx___driver_attach+0x10/0x10
[ 7.624299][ T1] bus_for_each_dev+0x239/0x2b0
[ 7.625118][ T1] ? __pfx___driver_attach+0x10/0x10
[ 7.625993][ T1] ? __pfx_bus_for_each_dev+0x10/0x10
[ 7.626858][ T1] ? do_raw_spin_unlock+0x13c/0x8b0
[ 7.627837][ T1] bus_add_driver+0x347/0x620
[ 7.628735][ T1] driver_register+0x23a/0x320
[ 7.629982][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.631006][ T1] virtio_scsi_init+0x65/0xe0
[ 7.631802][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.632612][ T1] do_one_initcall+0x248/0x880
[ 7.633404][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.634540][ T1] ? __pfx_do_one_initcall+0x10/0x10
[ 7.635786][ T1] ? lockdep_hardirqs_on_prepare+0x43d/0x780
[ 7.636889][ T1] ? __pfx_parse_args+0x10/0x10
[ 7.637652][ T1] ? do_initcalls+0x1c/0x80
[ 7.638456][ T1] ? rcu_is_watching+0x15/0xb0
[ 7.639227][ T1] do_initcall_level+0x157/0x210
[ 7.640192][ T1] do_initcalls+0x3f/0x80
[ 7.640818][ T1] kernel_init_freeable+0x435/0x5d0
[ 7.641593][ T1] ? __pfx_kernel_init_freeable+0x10/0x10
[ 7.642546][ T1] ? __pfx_lockdep_hardirqs_on_prepare+0x10/0x10
[ 7.643506][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.644295][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.645313][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.646036][ T1] kernel_init+0x1d/0x2b0
[ 7.646660][ T1] ret_from_fork+0x4b/0x80
[ 7.647368][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.648542][ T1] ret_from_fork_asm+0x1a/0x30
[ 7.649812][ T1] </TASK>
[ 7.650346][ T1] Kernel panic - not syncing: kernel: panic_on_warn set ...
[ 7.651620][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.9.0-rc3-syzkaller-00014-geb06a4b6cca5 #0
[ 7.653389][ T1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
[ 7.655321][ T1] Call Trace:
[ 7.655801][ T1] <TASK>
[ 7.656252][ T1] dump_stack_lvl+0x241/0x360
[ 7.657006][ T1] ? __pfx_dump_stack_lvl+0x10/0x10
[ 7.657825][ T1] ? __pfx__printk+0x10/0x10
[ 7.658542][ T1] ? _printk+0xd5/0x120
[ 7.659343][ T1] ? vscnprintf+0x5d/0x90
[ 7.659705][ T1] panic+0x349/0x860
[ 7.659705][ T1] ? __warn+0x172/0x4e0
[ 7.659705][ T1] ? __pfx_panic+0x10/0x10
[ 7.659705][ T1] ? show_trace_log_lvl+0x4e6/0x520
[ 7.659705][ T1] ? ret_from_fork_asm+0x1a/0x30
[ 7.659705][ T1] __warn+0x346/0x4e0
[ 7.659705][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.659705][ T1] report_bug+0x2b3/0x500
[ 7.659705][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.659705][ T1] handle_bug+0x3e/0x70
[ 7.659705][ T1] exc_invalid_op+0x1a/0x50
[ 7.659705][ T1] asm_exc_invalid_op+0x1a/0x20
[ 7.659705][ T1] RIP: 0010:refcount_warn_saturate+0xfa/0x1d0
[ 7.659705][ T1] Code: b2 00 00 00 e8 87 70 e7 fc 5b 5d c3 cc cc cc cc e8 7b 70 e7 fc c6 05 0c 5d e5 0a 01 90 48 c7 c7 40 4b 1f 8c e8 17 ee a9 fc 90 <0f> 0b 90 90 eb d9 e8 5b 70 e7 fc c6 05 e9 5c e5 0a 01 90 48 c7 c7
[ 7.659705][ T1] RSP: 0000:ffffc90000066e18 EFLAGS: 00010246
[ 7.659705][ T1] RAX: 6f40bd285f2a6000 RBX: ffff888147ed00fc RCX: ffff8880166d0000
[ 7.659705][ T1] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 7.659705][ T1] RBP: 0000000000000004 R08: ffffffff815800a2 R09: fffffbfff1c39af8
[ 7.659705][ T1] R10: dffffc0000000000 R11: fffffbfff1c39af8 R12: ffffea000503edc0
[ 7.659705][ T1] R13: ffffea000503edc8 R14: 1ffffd4000a07db9 R15: 0000000000000000
[ 7.659705][ T1] ? __warn_printk+0x292/0x360
[ 7.659705][ T1] ? refcount_warn_saturate+0xf9/0x1d0
[ 7.659705][ T1] __free_pages_ok+0xc60/0xd90
[ 7.659705][ T1] make_alloc_exact+0xa3/0xf0
[ 7.659705][ T1] vring_alloc_queue_split+0x20a/0x600
[ 7.659705][ T1] ? __pfx_vring_alloc_queue_split+0x10/0x10
[ 7.659705][ T1] ? vp_find_vqs+0x4c/0x4e0
[ 7.659705][ T1] ? virtscsi_probe+0x3ea/0xf60
[ 7.659705][ T1] ? virtio_dev_probe+0x991/0xaf0
[ 7.659705][ T1] ? really_probe+0x2b8/0xad0
[ 7.659705][ T1] ? driver_probe_device+0x50/0x430
[ 7.659705][ T1] vring_create_virtqueue_split+0xc6/0x310
[ 7.659705][ T1] ? ret_from_fork+0x4b/0x80
[ 7.659705][ T1] ? __pfx_vring_create_virtqueue_split+0x10/0x10
[ 7.659705][ T1] vring_create_virtqueue+0xca/0x110
[ 7.659705][ T1] ? __pfx_vp_notify+0x10/0x10
[ 7.659705][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.659705][ T1] setup_vq+0xe9/0x2d0
[ 7.659705][ T1] ? __pfx_vp_notify+0x10/0x10
[ 7.659705][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.659705][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.659705][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.659705][ T1] vp_setup_vq+0xbf/0x330
[ 7.659705][ T1] ? __pfx_vp_config_changed+0x10/0x10
[ 7.659705][ T1] ? ioread16+0x2f/0x90
[ 7.659705][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.709861][ T1] vp_find_vqs_msix+0x8b2/0xc80
[ 7.709861][ T1] vp_find_vqs+0x4c/0x4e0
[ 7.709861][ T1] virtscsi_init+0x8db/0xd00
[ 7.709861][ T1] ? __pfx_virtscsi_init+0x10/0x10
[ 7.709861][ T1] ? __pfx_default_calc_sets+0x10/0x10
[ 7.709861][ T1] ? scsi_host_alloc+0xa57/0xea0
[ 7.709861][ T1] ? vp_get+0xfd/0x140
[ 7.709861][ T1] virtscsi_probe+0x3ea/0xf60
[ 7.709861][ T1] ? __pfx_virtscsi_probe+0x10/0x10
[ 7.709861][ T1] ? kernfs_add_one+0x156/0x8b0
[ 7.709861][ T1] ? virtio_no_restricted_mem_acc+0x9/0x10
[ 7.709861][ T1] ? virtio_features_ok+0x10c/0x270
[ 7.709861][ T1] virtio_dev_probe+0x991/0xaf0
[ 7.709861][ T1] ? __pfx_virtio_dev_probe+0x10/0x10
[ 7.709861][ T1] really_probe+0x2b8/0xad0
[ 7.709861][ T1] __driver_probe_device+0x1a2/0x390
[ 7.709861][ T1] driver_probe_device+0x50/0x430
[ 7.709861][ T1] __driver_attach+0x45f/0x710
[ 7.709861][ T1] ? __pfx___driver_attach+0x10/0x10
[ 7.709861][ T1] bus_for_each_dev+0x239/0x2b0
[ 7.709861][ T1] ? __pfx___driver_attach+0x10/0x10
[ 7.709861][ T1] ? __pfx_bus_for_each_dev+0x10/0x10
[ 7.709861][ T1] ? do_raw_spin_unlock+0x13c/0x8b0
[ 7.709861][ T1] bus_add_driver+0x347/0x620
[ 7.709861][ T1] driver_register+0x23a/0x320
[ 7.709861][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.709861][ T1] virtio_scsi_init+0x65/0xe0
[ 7.709861][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.709861][ T1] do_one_initcall+0x248/0x880
[ 7.709861][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.709861][ T1] ? __pfx_do_one_initcall+0x10/0x10
[ 7.709861][ T1] ? lockdep_hardirqs_on_prepare+0x43d/0x780
[ 7.709861][ T1] ? __pfx_parse_args+0x10/0x10
[ 7.709861][ T1] ? do_initcalls+0x1c/0x80
[ 7.709861][ T1] ? rcu_is_watching+0x15/0xb0
[ 7.709861][ T1] do_initcall_level+0x157/0x210
[ 7.709861][ T1] do_initcalls+0x3f/0x80
[ 7.709861][ T1] kernel_init_freeable+0x435/0x5d0
[ 7.709861][ T1] ? __pfx_kernel_init_freeable+0x10/0x10
[ 7.709861][ T1] ? __pfx_lockdep_hardirqs_on_prepare+0x10/0x10
[ 7.709861][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.709861][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.709861][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.709861][ T1] kernel_init+0x1d/0x2b0
[ 7.709861][ T1] ret_from_fork+0x4b/0x80
[ 7.709861][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.709861][ T1] ret_from_fork_asm+0x1a/0x30
[ 7.709861][ T1] </TASK>
[ 7.709861][ T1] Kernel Offset: disabled
[ 7.709861][ T1] Rebooting in 86400 seconds..


syzkaller build log:
go env (err=<nil>)
GO111MODULE='auto'
GOARCH='amd64'
GOBIN=''
GOCACHE='/syzkaller/.cache/go-build'
GOENV='/syzkaller/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/syzkaller/jobs-2/linux/gopath/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/syzkaller/jobs-2/linux/gopath'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.4'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/syzkaller/jobs-2/linux/gopath/src/github.com/google/syzkaller/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1437292946=/tmp/go-build -gno-record-gcc-switches'

git status (err=<nil>)
HEAD detached at 56086b24b
nothing to commit, working tree clean


tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
Makefile:31: run command via tools/syz-env for best compatibility, see:
Makefile:32: https://github.com/google/syzkaller/blob/master/docs/contributing.md#using-syz-env
go list -f '{{.Stale}}' ./sys/syz-sysgen | grep -q false || go install ./sys/syz-sysgen
make .descriptions
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
Makefile:31: run command via tools/syz-env for best compatibility, see:
Makefile:32: https://github.com/google/syzkaller/blob/master/docs/contributing.md#using-syz-env
bin/syz-sysgen
touch .descriptions
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=56086b24bdfd822d3b227edb3064db443cd8c971 -X 'github.com/google/syzkaller/prog.gitRevisionDate=20240409-083312'" "-tags=syz_target syz_os_linux syz_arch_amd64 " -o ./bin/linux_amd64/syz-fuzzer github.com/google/syzkaller/syz-fuzzer
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=56086b24bdfd822d3b227edb3064db443cd8c971 -X 'github.com/google/syzkaller/prog.gitRevisionDate=20240409-083312'" "-tags=syz_target syz_os_linux syz_arch_amd64 " -o ./bin/linux_amd64/syz-execprog github.com/google/syzkaller/tools/syz-execprog
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=56086b24bdfd822d3b227edb3064db443cd8c971 -X 'github.com/google/syzkaller/prog.gitRevisionDate=20240409-083312'" "-tags=syz_target syz_os_linux syz_arch_amd64 " -o ./bin/linux_amd64/syz-stress github.com/google/syzkaller/tools/syz-stress
mkdir -p ./bin/linux_amd64
gcc -o ./bin/linux_amd64/syz-executor executor/executor.cc \
-m64 -O2 -pthread -Wall -Werror -Wparentheses -Wunused-const-variable -Wframe-larger-than=16384 -Wno-stringop-overflow -Wno-array-bounds -Wno-format-overflow -Wno-unused-but-set-variable -Wno-unused-command-line-argument -static-pie -fpermissive -w -DGOOS_linux=1 -DGOARCH_amd64=1 \
-DHOSTGOOS_linux=1 -DGIT_REVISION=\"56086b24bdfd822d3b227edb3064db443cd8c971\"


Error text is too large and was truncated, full error text is at:
https://syzkaller.appspot.com/x/error.txt?x=1523635d180000


Tested on:

commit: eb06a4b6 fsnotify: do not handle events on a shutting ..
git tree: https://github.com/amir73il/linux fsnotify-fixes
kernel config: https://syzkaller.appspot.com/x/.config?x=3335b6cc973feecf

dashboard link: https://syzkaller.appspot.com/bug?extid=5e3f9b2a67b45f16d4e6
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.

[Khazhy Kumykov]

Hmm, so we're releasing dquots after already shutting down the
filesystem? Is that expected? This "Failed to release dquot type"
error message only appears if we have an open handle from
ext4_journal_start (although this filesystem was mounted without a
journal, so we hit ext4_get_nojournal()...)

> In theory these two
> >> calls can even run in parallel and fsnotify() can be holding
> >> fsnotify_sb_info pointer while fsnotify_sb_delete() is freeing it so we
> >> need to figure out some proper synchronization for that...
> >
> > Is it really needed to handle any for non SB_ACTIVE sb?
>
> I think it should be fine to exclude volumes being teared down. Cc'ing
> Khazhy, who sponsored this work at the time and owned the use-case.

In terms of real-world use case... not sure there is one - you'll
notice the errors when you fsck/mount again later on, and any users
who care about notifs should be gone by the time we're unmounting. But
it seems weird to me that we can get write errors shutting everything
down.
>
> --
> Gabriel Krisman Bertazi

[Amir Goldstein]

Not sure what this is about.
Let's try again after rebase to current master:

[Amir Goldstein]

That's what I thought.
Anyway, my naive fix breaks IN_UNMOUNT event as well as some other
fanotify tests, so I will need to work on another fix.

Thanks,
Amir.

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

viperboard

[ 7.815991][ T1] usbcore: registered new interface driver dln2
[ 7.818221][ T1] usbcore: registered new interface driver pn533_usb
[ 7.823865][ T1] nfcsim 0.2 initialized
[ 7.825314][ T1] usbcore: registered new interface driver port100
[ 7.828562][ T1] usbcore: registered new interface driver nfcmrvl
[ 7.837679][ T1] Loading iSCSI transport class v2.0-870.
[ 7.856316][ T1] virtio_scsi virtio0: 1/0/0 default/read/poll queues
[ 7.867881][ T1] ------------[ cut here ]------------
[ 7.869240][ T1] refcount_t: decrement hit 0; leaking memory.
[ 7.870647][ T1] WARNING: CPU: 0 PID: 1 at lib/refcount.c:31 refcount_warn_saturate+0xfa/0x1d0
[ 7.872673][ T1] Modules linked in:
[ 7.873450][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.9.0-rc3-syzkaller-00222-ga4170c0055a4 #0
[ 7.875073][ T1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
[ 7.876781][ T1] RIP: 0010:refcount_warn_saturate+0xfa/0x1d0
[ 7.877963][ T1] Code: b2 00 00 00 e8 e7 63 e7 fc 5b 5d c3 cc cc cc cc e8 db 63 e7 fc c6 05 6c d2 e4 0a 01 90 48 c7 c7 c0 30 1f 8c e8 77 e1 a9 fc 90 <0f> 0b 90 90 eb d9 e8 bb 63 e7 fc c6 05 49 d2 e4 0a 01 90 48 c7 c7
[ 7.881853][ T1] RSP: 0000:ffffc90000066e18 EFLAGS: 00010246
[ 7.883056][ T1] RAX: 7664671302135400 RBX: ffff8880202afc6c RCX: ffff8880166d0000
[ 7.885276][ T1] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 7.887369][ T1] RBP: 0000000000000004 R08: ffffffff815880a2 R09: fffffbfff1c39b48
[ 7.888750][ T1] R10: dffffc0000000000 R11: fffffbfff1c39b48 R12: ffffea000502ddc0
[ 7.890131][ T1] R13: ffffea000502ddc8 R14: 1ffffd4000a05bb9 R15: 0000000000000000
[ 7.891813][ T1] FS: 0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
[ 7.893900][ T1] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7.895917][ T1] CR2: ffff88823ffff000 CR3: 000000000e134000 CR4: 00000000003506f0
[ 7.897448][ T1] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7.898766][ T1] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 7.900462][ T1] Call Trace:
[ 7.901245][ T1] <TASK>
[ 7.901998][ T1] ? __warn+0x163/0x4e0
[ 7.902783][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.904160][ T1] ? report_bug+0x2b3/0x500
[ 7.905119][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.906474][ T1] ? handle_bug+0x3e/0x70
[ 7.907513][ T1] ? exc_invalid_op+0x1a/0x50
[ 7.908463][ T1] ? asm_exc_invalid_op+0x1a/0x20
[ 7.909468][ T1] ? __warn_printk+0x292/0x360
[ 7.910957][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 7.912558][ T1] ? refcount_warn_saturate+0xf9/0x1d0
[ 7.913803][ T1] __free_pages_ok+0xc60/0xd90
[ 7.914583][ T1] make_alloc_exact+0xa3/0xf0
[ 7.915425][ T1] vring_alloc_queue_split+0x20a/0x600
[ 7.916903][ T1] ? __pfx_vring_alloc_queue_split+0x10/0x10
[ 7.917937][ T1] ? vp_find_vqs+0x4c/0x4e0
[ 7.918930][ T1] ? virtscsi_probe+0x3ea/0xf60
[ 7.919998][ T1] ? virtio_dev_probe+0x991/0xaf0
[ 7.921234][ T1] ? really_probe+0x2b8/0xad0
[ 7.921893][ T1] ? driver_probe_device+0x50/0x430
[ 7.922882][ T1] vring_create_virtqueue_split+0xc6/0x310
[ 7.924087][ T1] ? ret_from_fork+0x4b/0x80
[ 7.924906][ T1] ? __pfx_vring_create_virtqueue_split+0x10/0x10
[ 7.926406][ T1] vring_create_virtqueue+0xca/0x110
[ 7.927871][ T1] ? __pfx_vp_notify+0x10/0x10
[ 7.929037][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.929986][ T1] setup_vq+0xe9/0x2d0
[ 7.930782][ T1] ? __pfx_vp_notify+0x10/0x10
[ 7.931882][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.932643][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.933619][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.934781][ T1] vp_setup_vq+0xbf/0x330
[ 7.935911][ T1] ? __pfx_vp_config_changed+0x10/0x10
[ 7.937334][ T1] ? ioread16+0x2f/0x90
[ 7.938489][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 7.939978][ T1] vp_find_vqs_msix+0x8b2/0xc80
[ 7.940989][ T1] vp_find_vqs+0x4c/0x4e0
[ 7.941655][ T1] virtscsi_init+0x8db/0xd00
[ 7.942427][ T1] ? __pfx_virtscsi_init+0x10/0x10
[ 7.943724][ T1] ? __pfx_default_calc_sets+0x10/0x10
[ 7.944616][ T1] ? scsi_host_alloc+0xa57/0xea0
[ 7.946041][ T1] ? vp_get+0xfd/0x140
[ 7.946895][ T1] virtscsi_probe+0x3ea/0xf60
[ 7.947740][ T1] ? __pfx_virtscsi_probe+0x10/0x10
[ 7.948718][ T1] ? kernfs_add_one+0x156/0x8b0
[ 7.949578][ T1] ? virtio_no_restricted_mem_acc+0x9/0x10
[ 7.950761][ T1] ? virtio_features_ok+0x10c/0x270
[ 7.951946][ T1] virtio_dev_probe+0x991/0xaf0
[ 7.953286][ T1] ? __pfx_virtio_dev_probe+0x10/0x10
[ 7.954435][ T1] really_probe+0x2b8/0xad0
[ 7.955311][ T1] __driver_probe_device+0x1a2/0x390
[ 7.956686][ T1] driver_probe_device+0x50/0x430
[ 7.957771][ T1] __driver_attach+0x45f/0x710
[ 7.958769][ T1] ? __pfx___driver_attach+0x10/0x10
[ 7.959612][ T1] bus_for_each_dev+0x239/0x2b0
[ 7.960946][ T1] ? __pfx___driver_attach+0x10/0x10
[ 7.962251][ T1] ? __pfx_bus_for_each_dev+0x10/0x10
[ 7.964315][ T1] ? do_raw_spin_unlock+0x13c/0x8b0
[ 7.966148][ T1] bus_add_driver+0x347/0x620
[ 7.967112][ T1] driver_register+0x23a/0x320
[ 7.968628][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.969941][ T1] virtio_scsi_init+0x65/0xe0
[ 7.970833][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.972203][ T1] do_one_initcall+0x248/0x880
[ 7.973355][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 7.974515][ T1] ? __pfx_do_one_initcall+0x10/0x10
[ 7.975602][ T1] ? lockdep_hardirqs_on_prepare+0x43d/0x780
[ 7.977269][ T1] ? __pfx_parse_args+0x10/0x10
[ 7.978148][ T1] ? do_initcalls+0x1c/0x80
[ 7.979254][ T1] ? rcu_is_watching+0x15/0xb0
[ 7.980350][ T1] do_initcall_level+0x157/0x210
[ 7.981349][ T1] do_initcalls+0x3f/0x80
[ 7.982742][ T1] kernel_init_freeable+0x435/0x5d0
[ 7.984461][ T1] ? __pfx_kernel_init_freeable+0x10/0x10
[ 7.986325][ T1] ? __pfx_lockdep_hardirqs_on_prepare+0x10/0x10
[ 7.987513][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.988803][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.990065][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.991335][ T1] kernel_init+0x1d/0x2b0
[ 7.992855][ T1] ret_from_fork+0x4b/0x80
[ 7.994528][ T1] ? __pfx_kernel_init+0x10/0x10
[ 7.996074][ T1] ret_from_fork_asm+0x1a/0x30
[ 7.997423][ T1] </TASK>
[ 7.998234][ T1] Kernel panic - not syncing: kernel: panic_on_warn set ...
[ 7.999604][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.9.0-rc3-syzkaller-00222-ga4170c0055a4 #0
[ 8.001827][ T1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
[ 8.004098][ T1] Call Trace:
[ 8.005064][ T1] <TASK>
[ 8.005501][ T1] dump_stack_lvl+0x241/0x360
[ 8.006050][ T1] ? __pfx_dump_stack_lvl+0x10/0x10
[ 8.006050][ T1] ? __pfx__printk+0x10/0x10
[ 8.006050][ T1] ? _printk+0xd5/0x120
[ 8.006050][ T1] ? vscnprintf+0x5d/0x90
[ 8.006050][ T1] panic+0x349/0x860
[ 8.006050][ T1] ? __warn+0x172/0x4e0
[ 8.006050][ T1] ? __pfx_panic+0x10/0x10
[ 8.006050][ T1] ? show_trace_log_lvl+0x4e6/0x520
[ 8.006050][ T1] ? ret_from_fork_asm+0x1a/0x30
[ 8.006050][ T1] __warn+0x346/0x4e0
[ 8.006050][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 8.006050][ T1] report_bug+0x2b3/0x500
[ 8.006050][ T1] ? refcount_warn_saturate+0xfa/0x1d0
[ 8.006050][ T1] handle_bug+0x3e/0x70
[ 8.006050][ T1] exc_invalid_op+0x1a/0x50
[ 8.006050][ T1] asm_exc_invalid_op+0x1a/0x20
[ 8.006050][ T1] RIP: 0010:refcount_warn_saturate+0xfa/0x1d0
[ 8.006050][ T1] Code: b2 00 00 00 e8 e7 63 e7 fc 5b 5d c3 cc cc cc cc e8 db 63 e7 fc c6 05 6c d2 e4 0a 01 90 48 c7 c7 c0 30 1f 8c e8 77 e1 a9 fc 90 <0f> 0b 90 90 eb d9 e8 bb 63 e7 fc c6 05 49 d2 e4 0a 01 90 48 c7 c7
[ 8.006050][ T1] RSP: 0000:ffffc90000066e18 EFLAGS: 00010246
[ 8.006050][ T1] RAX: 7664671302135400 RBX: ffff8880202afc6c RCX: ffff8880166d0000
[ 8.006050][ T1] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 8.006050][ T1] RBP: 0000000000000004 R08: ffffffff815880a2 R09: fffffbfff1c39b48
[ 8.006050][ T1] R10: dffffc0000000000 R11: fffffbfff1c39b48 R12: ffffea000502ddc0
[ 8.006050][ T1] R13: ffffea000502ddc8 R14: 1ffffd4000a05bb9 R15: 0000000000000000
[ 8.006050][ T1] ? __warn_printk+0x292/0x360
[ 8.006050][ T1] ? refcount_warn_saturate+0xf9/0x1d0
[ 8.006050][ T1] __free_pages_ok+0xc60/0xd90
[ 8.006050][ T1] make_alloc_exact+0xa3/0xf0
[ 8.006050][ T1] vring_alloc_queue_split+0x20a/0x600
[ 8.006050][ T1] ? __pfx_vring_alloc_queue_split+0x10/0x10
[ 8.006050][ T1] ? vp_find_vqs+0x4c/0x4e0
[ 8.006050][ T1] ? virtscsi_probe+0x3ea/0xf60
[ 8.006050][ T1] ? virtio_dev_probe+0x991/0xaf0
[ 8.006050][ T1] ? really_probe+0x2b8/0xad0
[ 8.006050][ T1] ? driver_probe_device+0x50/0x430
[ 8.006050][ T1] vring_create_virtqueue_split+0xc6/0x310
[ 8.006050][ T1] ? ret_from_fork+0x4b/0x80
[ 8.055954][ T1] ? __pfx_vring_create_virtqueue_split+0x10/0x10
[ 8.055954][ T1] vring_create_virtqueue+0xca/0x110
[ 8.055954][ T1] ? __pfx_vp_notify+0x10/0x10
[ 8.055954][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 8.055954][ T1] setup_vq+0xe9/0x2d0
[ 8.055954][ T1] ? __pfx_vp_notify+0x10/0x10
[ 8.055954][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 8.055954][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 8.055954][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 8.055954][ T1] vp_setup_vq+0xbf/0x330
[ 8.055954][ T1] ? __pfx_vp_config_changed+0x10/0x10
[ 8.055954][ T1] ? ioread16+0x2f/0x90
[ 8.055954][ T1] ? __pfx_virtscsi_ctrl_done+0x10/0x10
[ 8.055954][ T1] vp_find_vqs_msix+0x8b2/0xc80
[ 8.055954][ T1] vp_find_vqs+0x4c/0x4e0
[ 8.055954][ T1] virtscsi_init+0x8db/0xd00
[ 8.055954][ T1] ? __pfx_virtscsi_init+0x10/0x10
[ 8.055954][ T1] ? __pfx_default_calc_sets+0x10/0x10
[ 8.055954][ T1] ? scsi_host_alloc+0xa57/0xea0
[ 8.055954][ T1] ? vp_get+0xfd/0x140
[ 8.055954][ T1] virtscsi_probe+0x3ea/0xf60
[ 8.055954][ T1] ? __pfx_virtscsi_probe+0x10/0x10
[ 8.055954][ T1] ? kernfs_add_one+0x156/0x8b0
[ 8.055954][ T1] ? virtio_no_restricted_mem_acc+0x9/0x10
[ 8.055954][ T1] ? virtio_features_ok+0x10c/0x270
[ 8.055954][ T1] virtio_dev_probe+0x991/0xaf0
[ 8.055954][ T1] ? __pfx_virtio_dev_probe+0x10/0x10
[ 8.055954][ T1] really_probe+0x2b8/0xad0
[ 8.055954][ T1] __driver_probe_device+0x1a2/0x390
[ 8.055954][ T1] driver_probe_device+0x50/0x430
[ 8.055954][ T1] __driver_attach+0x45f/0x710
[ 8.055954][ T1] ? __pfx___driver_attach+0x10/0x10
[ 8.055954][ T1] bus_for_each_dev+0x239/0x2b0
[ 8.055954][ T1] ? __pfx___driver_attach+0x10/0x10
[ 8.055954][ T1] ? __pfx_bus_for_each_dev+0x10/0x10
[ 8.055954][ T1] ? do_raw_spin_unlock+0x13c/0x8b0
[ 8.055954][ T1] bus_add_driver+0x347/0x620
[ 8.055954][ T1] driver_register+0x23a/0x320
[ 8.055954][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 8.055954][ T1] virtio_scsi_init+0x65/0xe0
[ 8.055954][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 8.055954][ T1] do_one_initcall+0x248/0x880
[ 8.055954][ T1] ? __pfx_virtio_scsi_init+0x10/0x10
[ 8.055954][ T1] ? __pfx_do_one_initcall+0x10/0x10
[ 8.105903][ T1] ? lockdep_hardirqs_on_prepare+0x43d/0x780
[ 8.105903][ T1] ? __pfx_parse_args+0x10/0x10
[ 8.105903][ T1] ? do_initcalls+0x1c/0x80
[ 8.105903][ T1] ? rcu_is_watching+0x15/0xb0
[ 8.105903][ T1] do_initcall_level+0x157/0x210
[ 8.105903][ T1] do_initcalls+0x3f/0x80
[ 8.105903][ T1] kernel_init_freeable+0x435/0x5d0
[ 8.105903][ T1] ? __pfx_kernel_init_freeable+0x10/0x10
[ 8.105903][ T1] ? __pfx_lockdep_hardirqs_on_prepare+0x10/0x10
[ 8.105903][ T1] ? __pfx_kernel_init+0x10/0x10
[ 8.105903][ T1] ? __pfx_kernel_init+0x10/0x10
[ 8.105903][ T1] ? __pfx_kernel_init+0x10/0x10
[ 8.105903][ T1] kernel_init+0x1d/0x2b0
[ 8.105903][ T1] ret_from_fork+0x4b/0x80
[ 8.105903][ T1] ? __pfx_kernel_init+0x10/0x10
[ 8.105903][ T1] ret_from_fork_asm+0x1a/0x30
[ 8.105903][ T1] </TASK>
[ 8.105903][ T1] Kernel Offset: disabled
[ 8.105903][ T1] Rebooting in 86400 seconds..

GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1668308411=/tmp/go-build -gno-record-gcc-switches'

git status (err=<nil>)
HEAD detached at 56086b24b
nothing to commit, working tree clean


tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
Makefile:31: run command via tools/syz-env for best compatibility, see:
Makefile:32: https://github.com/google/syzkaller/blob/master/docs/contributing.md#using-syz-env
go list -f '{{.Stale}}' ./sys/syz-sysgen | grep -q false || go install ./sys/syz-sysgen
make .descriptions
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
Makefile:31: run command via tools/syz-env for best compatibility, see:
Makefile:32: https://github.com/google/syzkaller/blob/master/docs/contributing.md#using-syz-env
bin/syz-sysgen
touch .descriptions
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=56086b24bdfd822d3b227edb3064db443cd8c971 -X 'github.com/google/syzkaller/prog.gitRevisionDate=20240409-083312'" "-tags=syz_target syz_os_linux syz_arch_amd64 " -o ./bin/linux_amd64/syz-fuzzer github.com/google/syzkaller/syz-fuzzer
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=56086b24bdfd822d3b227edb3064db443cd8c971 -X 'github.com/google/syzkaller/prog.gitRevisionDate=20240409-083312'" "-tags=syz_target syz_os_linux syz_arch_amd64 " -o ./bin/linux_amd64/syz-execprog github.com/google/syzkaller/tools/syz-execprog
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=56086b24bdfd822d3b227edb3064db443cd8c971 -X 'github.com/google/syzkaller/prog.gitRevisionDate=20240409-083312'" "-tags=syz_target syz_os_linux syz_arch_amd64 " -o ./bin/linux_amd64/syz-stress github.com/google/syzkaller/tools/syz-stress
mkdir -p ./bin/linux_amd64
gcc -o ./bin/linux_amd64/syz-executor executor/executor.cc \
-m64 -O2 -pthread -Wall -Werror -Wparentheses -Wunused-const-variable -Wframe-larger-than=16384 -Wno-stringop-overflow -Wno-array-bounds -Wno-format-overflow -Wno-unused-but-set-variable -Wno-unused-command-line-argument -static-pie -fpermissive -w -DGOOS_linux=1 -DGOARCH_amd64=1 \
-DHOSTGOOS_linux=1 -DGIT_REVISION=\"56086b24bdfd822d3b227edb3064db443cd8c971\"


Error text is too large and was truncated, full error text is at:

https://syzkaller.appspot.com/x/error.txt?x=122f8f97180000


Tested on:

commit: a4170c00 fsnotify: do not handle events on a shutting ..

git tree: https://github.com/amir73il/linux fsnotify-fixes

kernel config: https://syzkaller.appspot.com/x/.config?x=9995779c8305f57e

dashboard link: https://syzkaller.appspot.com/bug?extid=5e3f9b2a67b45f16d4e6
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

[Aleksandr Nogikh]

FWIW all latest v6.9 release candidates just do not boot with the syzbot config.
There's a "mm,page_owner: Fix refcount imbalance" series that
addresses it, but it has not apparently reached torvalds yet.

> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bug...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/000000000000774dfe0615e71b29%40google.com.

[Hillf Danton]

On Thu, 11 Apr 2024 01:11:20 -0700

> syzbot found the following issue on:
>
> HEAD commit: 6ebf211bb11d Add linux-next specific files for 20240410
> git tree: linux-next

> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1621af9d180000

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 6ebf211bb11d

--- x/fs/notify/fsnotify.c
+++ y/fs/notify/fsnotify.c
@@ -101,8 +101,8 @@ void fsnotify_sb_delete(struct super_blo
wait_var_event(fsnotify_sb_watched_objects(sb),
!atomic_long_read(fsnotify_sb_watched_objects(sb)));
WARN_ON(fsnotify_sb_has_priority_watchers(sb, FSNOTIFY_PRIO_CONTENT));
- WARN_ON(fsnotify_sb_has_priority_watchers(sb,
- FSNOTIFY_PRIO_PRE_CONTENT));
+ WARN_ON(fsnotify_sb_has_priority_watchers(sb, FSNOTIFY_PRIO_PRE_CONTENT));
+ synchronize_srcu(&fsnotify_mark_srcu);
kfree(sbinfo);
}

@@ -499,7 +499,7 @@ int fsnotify(__u32 mask, const void *dat
{
const struct path *path = fsnotify_data_path(data, data_type);
struct super_block *sb = fsnotify_data_sb(data, data_type);
- struct fsnotify_sb_info *sbinfo = fsnotify_sb_info(sb);
+ struct fsnotify_sb_info *sbinfo;
struct fsnotify_iter_info iter_info = {};
struct mount *mnt = NULL;
struct inode *inode2 = NULL;
@@ -529,6 +529,8 @@ int fsnotify(__u32 mask, const void *dat
inode2_type = FSNOTIFY_ITER_TYPE_PARENT;
}

+ iter_info.srcu_idx = srcu_read_lock(&fsnotify_mark_srcu);
+ sbinfo = fsnotify_sb_info(sb);
/*
* Optimization: srcu_read_lock() has a memory barrier which can
* be expensive. It protects walking the *_fsnotify_marks lists.
@@ -539,8 +541,10 @@ int fsnotify(__u32 mask, const void *dat
if ((!sbinfo || !sbinfo->sb_marks) &&
(!mnt || !mnt->mnt_fsnotify_marks) &&
(!inode || !inode->i_fsnotify_marks) &&
- (!inode2 || !inode2->i_fsnotify_marks))
- return 0;
+ (!inode2 || !inode2->i_fsnotify_marks)) {
+ ret = 0;
+ goto out;
+ }

marks_mask = sb->s_fsnotify_mask;
if (mnt)
@@ -558,10 +562,10 @@ int fsnotify(__u32 mask, const void *dat
* Otherwise, return if none of the marks care about this type of event.
*/
test_mask = (mask & ALL_FSNOTIFY_EVENTS);
- if (!(test_mask & marks_mask))
- return 0;
-
- iter_info.srcu_idx = srcu_read_lock(&fsnotify_mark_srcu);
+ if (!(test_mask & marks_mask)) {
+ ret = 0;
+ goto out;
+ }

if (sbinfo) {
iter_info.marks[FSNOTIFY_ITER_TYPE_SB] =
--

[Amir Goldstein]

See comment above. This kills the optimization.
It is not worth letting all the fsnotify hooks suffer the consequence
for the edge case of calling fsnotify hook during fs shutdown.

Also, fsnotify_sb_info(sb) in fsnotify_sb_has_priority_watchers()
is also not protected and using srcu_read_lock() there completely
nullifies the purpose of fsnotify_sb_info.

Here is a simplified fix for fsnotify_sb_error() rebased on the
pending mm fixes for this syzbot boot failure:

#syz test: https://github.com/amir73il/linux fsnotify-fixes

Jan,

I think that all the functions called from fs shutdown context
should observe that SB_ACTIVE is cleared but wasn't sure?

Thanks,
Amir.

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
KASAN: slab-use-after-free Read in fsnotify

Quota error (device loop0): do_check_range: Getting block 0 out of range 1-5
EXT4-fs error (device loop0): ext4_release_dquot:6905: comm kworker/u8:4: Failed to release dquot type 1
==================================================================

BUG: KASAN: slab-use-after-free in fsnotify+0x3e5/0x1f00 fs/notify/fsnotify.c:541
Read of size 8 at addr ffff88802d133300 by task kworker/u8:4/62

CPU: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.9.0-rc3-next-20240410-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024

Workqueue: events_unbound quota_release_workfn
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
print_address_description mm/kasan/report.c:377 [inline]
print_report+0x169/0x550 mm/kasan/report.c:488
kasan_report+0x143/0x180 mm/kasan/report.c:601

fsnotify+0x3e5/0x1f00 fs/notify/fsnotify.c:541

fsnotify_sb_error include/linux/fsnotify.h:456 [inline]
__ext4_error+0x255/0x3b0 fs/ext4/super.c:843
ext4_release_dquot+0x326/0x450 fs/ext4/super.c:6903
quota_release_workfn+0x39f/0x650 fs/quota/dquot.c:840
process_one_work kernel/workqueue.c:3218 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3299
worker_thread+0x86d/0xd70 kernel/workqueue.c:3380
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>

Allocated by task 5511:

kasan_save_stack mm/kasan/common.c:47 [inline]
kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
poison_kmalloc_redzone mm/kasan/common.c:370 [inline]
__kasan_kmalloc+0x98/0xb0 mm/kasan/common.c:387
kasan_kmalloc include/linux/kasan.h:211 [inline]
kmalloc_trace_noprof+0x19c/0x2b0 mm/slub.c:4109
kmalloc_noprof include/linux/slab.h:660 [inline]
kzalloc_noprof include/linux/slab.h:775 [inline]
fsnotify_attach_info_to_sb fs/notify/mark.c:600 [inline]
fsnotify_add_mark_list fs/notify/mark.c:692 [inline]
fsnotify_add_mark_locked+0x3b2/0xe60 fs/notify/mark.c:777
fanotify_add_new_mark fs/notify/fanotify/fanotify_user.c:1267 [inline]
fanotify_add_mark+0xbbd/0x1330 fs/notify/fanotify/fanotify_user.c:1334
do_fanotify_mark+0xbcc/0xd90 fs/notify/fanotify/fanotify_user.c:1896
__do_sys_fanotify_mark fs/notify/fanotify/fanotify_user.c:1919 [inline]
__se_sys_fanotify_mark fs/notify/fanotify/fanotify_user.c:1915 [inline]
__x64_sys_fanotify_mark+0xb5/0xd0 fs/notify/fanotify/fanotify_user.c:1915
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f

Freed by task 5442:

kasan_save_stack mm/kasan/common.c:47 [inline]
kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
kasan_save_free_info+0x40/0x50 mm/kasan/generic.c:579
poison_slab_object+0xe0/0x150 mm/kasan/common.c:240
__kasan_slab_free+0x37/0x60 mm/kasan/common.c:256
kasan_slab_free include/linux/kasan.h:184 [inline]
slab_free_hook mm/slub.c:2190 [inline]
slab_free mm/slub.c:4393 [inline]
kfree+0x149/0x350 mm/slub.c:4514

fsnotify_sb_delete+0x692/0x6f0 fs/notify/fsnotify.c:106

generic_shutdown_super+0xa5/0x2d0 fs/super.c:632
kill_block_super+0x44/0x90 fs/super.c:1675
ext4_kill_sb+0x68/0xa0 fs/ext4/super.c:7323
deactivate_locked_super+0xc4/0x130 fs/super.c:472
cleanup_mnt+0x426/0x4c0 fs/namespace.c:1267
task_work_run+0x24f/0x310 kernel/task_work.c:180

resume_user_mode_work include/linux/resume_user_mode.h:50 [inline]
exit_to_user_mode_loop kernel/entry/common.c:114 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
__syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
syscall_exit_to_user_mode+0x168/0x370 kernel/entry/common.c:218
do_syscall_64+0x102/0x240 arch/x86/entry/common.c:89
entry_SYSCALL_64_after_hwframe+0x77/0x7f

The buggy address belongs to the object at ffff88802d133300

which belongs to the cache kmalloc-32 of size 32
The buggy address is located 0 bytes inside of

freed 32-byte region [ffff88802d133300, ffff88802d133320)

The buggy address belongs to the physical page:

page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff88802d133180 pfn:0x2d133
flags: 0xfff80000000200(workingset|node=0|zone=1|lastcpupid=0xfff)
page_type: 0xffffefff(slab)
raw: 00fff80000000200 ffff888015041500 ffffea00007c7590 ffffea0001aa96d0
raw: ffff88802d133180 000000000040003d 00000001ffffefff 0000000000000000

page dumped because: kasan: bad access detected
page_owner tracks the page as allocated

page last allocated via order 0, migratetype Unmovable, gfp_mask 0x12cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_NORETRY), pid 1, tgid 177503641 (swapper/0), ts 1, free_ts 13042593494

set_page_owner include/linux/page_owner.h:32 [inline]
post_alloc_hook+0x1f3/0x230 mm/page_alloc.c:1468
prep_new_page mm/page_alloc.c:1476 [inline]
get_page_from_freelist+0x2ce2/0x2d90 mm/page_alloc.c:3438
__alloc_pages_noprof+0x256/0x6c0 mm/page_alloc.c:4696
__alloc_pages_node_noprof include/linux/gfp.h:244 [inline]
alloc_pages_node_noprof include/linux/gfp.h:271 [inline]
alloc_slab_page+0x5f/0x120 mm/slub.c:2259
allocate_slab+0x5a/0x2e0 mm/slub.c:2422
new_slab mm/slub.c:2475 [inline]
___slab_alloc+0xcd1/0x14b0 mm/slub.c:3624
__slab_alloc+0x58/0xa0 mm/slub.c:3714
__slab_alloc_node mm/slub.c:3767 [inline]
slab_alloc_node mm/slub.c:3945 [inline]
__do_kmalloc_node mm/slub.c:4077 [inline]
kmalloc_node_track_caller_noprof+0x286/0x440 mm/slub.c:4098

__do_krealloc mm/slab_common.c:1190 [inline]
krealloc_noprof+0x7d/0x120 mm/slab_common.c:1223
add_sysfs_param+0x137/0x7f0 kernel/params.c:663
kernel_add_sysfs_param+0xb4/0x130 kernel/params.c:817
param_sysfs_builtin+0x16e/0x1f0 kernel/params.c:856
param_sysfs_builtin_init+0x31/0x40 kernel/params.c:990

do_one_initcall+0x248/0x880 init/main.c:1263
do_initcall_level+0x157/0x210 init/main.c:1325
do_initcalls+0x3f/0x80 init/main.c:1341

page last free pid 782 tgid 782 stack trace:
reset_page_owner include/linux/page_owner.h:25 [inline]
free_pages_prepare mm/page_alloc.c:1088 [inline]
free_unref_page+0xd22/0xea0 mm/page_alloc.c:2601
vfree+0x186/0x2e0 mm/vmalloc.c:3338
delayed_vfree_work+0x56/0x80 mm/vmalloc.c:3259

process_one_work kernel/workqueue.c:3218 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3299
worker_thread+0x86d/0xd70 kernel/workqueue.c:3380
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

Memory state around the buggy address:

ffff88802d133200: 00 00 00 00 fc fc fc fc 00 00 00 00 fc fc fc fc
ffff88802d133280: 00 00 00 00 fc fc fc fc 00 00 00 00 fc fc fc fc
>ffff88802d133300: fa fb fb fb fc fc fc fc 00 00 00 00 fc fc fc fc
^
ffff88802d133380: 00 00 00 fc fc fc fc fc 00 00 00 04 fc fc fc fc
ffff88802d133400: 00 00 00 fc fc fc fc fc 00 00 00 00 fc fc fc fc
==================================================================


Tested on:

commit: 6ebf211b Add linux-next specific files for 20240410
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
console output: https://syzkaller.appspot.com/x/log.txt?x=10c3ac7d180000
kernel config: https://syzkaller.appspot.com/x/.config?x=16ca158ef7e08662

dashboard link: https://syzkaller.appspot.com/bug?extid=5e3f9b2a67b45f16d4e6
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=165b0243180000

[Hillf Danton]

On Fri, 12 Apr 2024 23:42:19 -0700 Amir Goldstein

> On Sat, Apr 13, 2024 at 4:41=E2=80=AFAM Hillf Danton <hda...@sina.com> wrote:
> >
> > On Thu, 11 Apr 2024 01:11:20 -0700
> > > syzbot found the following issue on:
> > >

> > > HEAD commit: 6ebf211bb11d Add linux-next specific files for 20240410
> > > git tree: linux-next
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=3D1621af9d180000

> >
> > #syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 6ebf211bb11d
> >
> > --- x/fs/notify/fsnotify.c
> > +++ y/fs/notify/fsnotify.c
> > @@ -101,8 +101,8 @@ void fsnotify_sb_delete(struct super_blo
> > wait_var_event(fsnotify_sb_watched_objects(sb),
> > !atomic_long_read(fsnotify_sb_watched_objects(sb)));
> > WARN_ON(fsnotify_sb_has_priority_watchers(sb, FSNOTIFY_PRIO_CONTENT));
> > - WARN_ON(fsnotify_sb_has_priority_watchers(sb,
> > - FSNOTIFY_PRIO_PRE_CONTENT));
> > + WARN_ON(fsnotify_sb_has_priority_watchers(sb, FSNOTIFY_PRIO_PRE_CONTENT));
> > + synchronize_srcu(&fsnotify_mark_srcu);
> > kfree(sbinfo);
> > }
> >
> > @@ -499,7 +499,7 @@ int fsnotify(__u32 mask, const void *dat
> > {

> > const struct path *path =3D fsnotify_data_path(data, data_type);
> > struct super_block *sb =3D fsnotify_data_sb(data, data_type);
> > - struct fsnotify_sb_info *sbinfo =3D fsnotify_sb_info(sb);

> > + struct fsnotify_sb_info *sbinfo;
> > struct fsnotify_iter_info iter_info = {};

> > struct mount *mnt =3D NULL;
> > struct inode *inode2 =3D NULL;

> > @@ -529,6 +529,8 @@ int fsnotify(__u32 mask, const void *dat

> > inode2_type =3D FSNOTIFY_ITER_TYPE_PARENT;
> > }
> >
> > + iter_info.srcu_idx =3D srcu_read_lock(&fsnotify_mark_srcu);
> > + sbinfo =3D fsnotify_sb_info(sb);

> > /*
> > * Optimization: srcu_read_lock() has a memory barrier which can
> > * be expensive. It protects walking the *_fsnotify_marks lists.
>
>
> See comment above. This kills the optimization.
> It is not worth letting all the fsnotify hooks suffer the consequence
> for the edge case of calling fsnotify hook during fs shutdown.

Say nothing before reading your fix.

>
> Also, fsnotify_sb_info(sb) in fsnotify_sb_has_priority_watchers()
> is also not protected and using srcu_read_lock() there completely
> nullifies the purpose of fsnotify_sb_info.
>
> Here is a simplified fix for fsnotify_sb_error() rebased on the
> pending mm fixes for this syzbot boot failure:
>
> #syz test: https://github.com/amir73il/linux fsnotify-fixes

Feel free to post your patch at lore because not everyone has
access to sites like github.

>
> Jan,
>
> I think that all the functions called from fs shutdown context
> should observe that SB_ACTIVE is cleared but wasn't sure?

If you composed fix based on SB_ACTIVE that is cleared in
generic_shutdown_super() with &sb->s_umount held for write,
I wonder what simpler serialization than srcu you could
find/create in fsnotify.

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+5e3f9b...@syzkaller.appspotmail.com

Tested on:

commit: 9e589786 fsnotify: do not handle events on a shutting ..

git tree: https://github.com/amir73il/linux fsnotify-fixes

console output: https://syzkaller.appspot.com/x/log.txt?x=11cdb161180000
kernel config: https://syzkaller.appspot.com/x/.config?x=9995779c8305f57e

dashboard link: https://syzkaller.appspot.com/bug?extid=5e3f9b2a67b45f16d4e6
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=14f52d33180000

Note: testing is done by a robot and is best-effort only.

[Amir Goldstein]

As far as I can tell there is no need for serialisation.

The problem is that fsnotify_sb_error() can be called from the
context of ->put_super() call from generic_shutdown_super().

It's true that in the repro the thread calling fsnotify_sb_error()
in the worker thread running quota deferred work from put_super()
but I think there are sufficient barriers for this worker thread to
observer the cleared SB_ACTIVE flag.

Anyway, according to syzbot, repro does not trigger the UAF
with my last fix.

To be clear, any fsnotify_sb_error() that is a result of a user operation
would be holding an active reference to sb so cannot race with
fsnotify_sb_delete(), but I am not sure that same is true for ext4
worker threads.

Jan,

You wrote that "In theory these two calls can even run in parallel

and fsnotify() can be holding fsnotify_sb_info pointer while

fsnotify_sb_delete() is freeing".

Can you give an example of this case?

Thanks,
Amir.

[Hillf Danton]

On Sat, 13 Apr 2024 12:32:32 +0300 Amir Goldstein

> On Sat, Apr 13, 2024 at 11:45=E2=80=AFAM Hillf Danton <hda...@sina.com> wrote:
> >
> > If you composed fix based on SB_ACTIVE that is cleared in
> > generic_shutdown_super() with &sb->s_umount held for write,
> > I wonder what simpler serialization than srcu you could
> > find/create in fsnotify.
>
> As far as I can tell there is no need for serialisation.
>
> The problem is that fsnotify_sb_error() can be called from the
> context of ->put_super() call from generic_shutdown_super().
>
> It's true that in the repro the thread calling fsnotify_sb_error()
> in the worker thread running quota deferred work from put_super()
> but I think there are sufficient barriers for this worker thread to
> observer the cleared SB_ACTIVE flag.
>

do_exit quota_release_workfn
--- ---
cleanup_mnt() ext4_release_dquot()
__super_lock_excl(s); __ext4_error()
deactivate_locked_super(s); fsnotify_sb_error()
ext4_kill_sb()
kill_block_super()
generic_shutdown_super()
if (!(sb->s_flags & SB_ACTIVE))
return;
sb->s_flags &= ~SB_ACTIVE;
fsnotify_sb_delete()
fsnotify()

Because of no sync like taking &sb->s_umount in the worker context,
checking SB_ACTIVE added in your fix is unable to close the race.

> Anyway, according to syzbot, repro does not trigger the UAF
> with my last fix.
>

[Jan Kara]

Yeah, basically what Hilf writes:

Task 1 Task 2
umount() some delayed work, transaction
commit, whatever is still running
before ext4_put_super() completes
... ext4_error()
fsnotify_sb_error()
fsnotify()
fetches fsnotify_sb_info
generic_shutdown_super()
fsnotify_sb_delete()
frees fsnotify_sb_info

Honza

[Amir Goldstein]

OK, so what do you say about Hillf's fix patch?

Maybe it is ok to let go of the optimization in fsnotify(), considering
that we now have stronger optimizations in the inline hooks and
in __fsnotify_parent()?

I think that Hillf's patch is missing setting s_fsnotify_info to NULL?

@@ -101,8 +101,8 @@ void fsnotify_sb_delete(struct super_blo
wait_var_event(fsnotify_sb_watched_objects(sb),
!atomic_long_read(fsnotify_sb_watched_objects(sb)));
WARN_ON(fsnotify_sb_has_priority_watchers(sb, FSNOTIFY_PRIO_CONTENT));

+ WRITE_ONCE(sb->s_fsnotify_info, NULL);
+ synchronize_srcu(&fsnotify_mark_srcu);
kfree(sbinfo);
}

Thanks,
Amir.

[Amir Goldstein]

I reworked Hillf's patch so it won't break the optimizations for
the common case and added setting s_fsnotify_info to NULL (attached).

Let's see what syzbot has to say:

#syz test: https://github.com/amir73il/linux fsnotify-fixes

Thanks,
Amir.

[Jan Kara]

So I had a look into this. Yes, something like this should work. We'll see
whether synchronize_srcu() won't slow down umount too much. If someone will
complain, we'll have to find a better solution.

[Amir Goldstein]

Actually, kfree_rcu(sbinfo) may be enough.
We do not actually access sbinfo during mark iteration and
event handling, we only access it to get to the sb connector.

Something like the attached patch?

Thanks,
Amir.

[Jan Kara]

Hum, thinking about this some more - what if we just freed sb_info from
destroy_super_work()? By then we definitely are not getting fsnotify()
calls for the superblock so all the problems are solved.

[Amir Goldstein]

> > > > Maybe it is ok to let go of the optimization in fsnotify(), considering
> > > > that we now have stronger optimizations in the inline hooks and
> > > > in __fsnotify_parent()?
> > > >
> > > > I think that Hillf's patch is missing setting s_fsnotify_info to NULL?
> > > >
> > > > @@ -101,8 +101,8 @@ void fsnotify_sb_delete(struct super_blo
> > > > wait_var_event(fsnotify_sb_watched_objects(sb),
> > > > !atomic_long_read(fsnotify_sb_watched_objects(sb)));
> > > > WARN_ON(fsnotify_sb_has_priority_watchers(sb, FSNOTIFY_PRIO_CONTENT));
> > > > + WRITE_ONCE(sb->s_fsnotify_info, NULL);
> > > > + synchronize_srcu(&fsnotify_mark_srcu);
> > > > kfree(sbinfo);
> > > > }
> > >
> > > So I had a look into this. Yes, something like this should work. We'll see
> > > whether synchronize_srcu() won't slow down umount too much. If someone will
> > > complain, we'll have to find a better solution.
> > >
> >
> > Actually, kfree_rcu(sbinfo) may be enough.
> > We do not actually access sbinfo during mark iteration and
> > event handling, we only access it to get to the sb connector.
> >
> > Something like the attached patch?
>
> Hum, thinking about this some more - what if we just freed sb_info from
> destroy_super_work()? By then we definitely are not getting fsnotify()
> calls for the superblock so all the problems are solved.
>

Considering that this is the solution for security_sb_free()
I don't see why not have fsnotify_sb_free().
I'll prepare a patch.

Thanks!
Amir.

[Hillf Danton]

On Tue, 16 Apr 2024 19:32:11 +0200 Jan Kara <ja...@suse.cz>

>
> Hum, thinking about this some more - what if we just freed sb_info from
> destroy_super_work()? By then we definitely are not getting fsnotify()
> calls for the superblock so all the problems are solved.
>

Sounds better :)

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to apply patch:
checking file fs/notify/fsnotify.c
Hunk #1 FAILED at 103.
Hunk #2 succeeded at 510 (offset 4 lines).
Hunk #3 succeeded at 540 (offset 4 lines).
Hunk #4 succeeded at 569 (offset 4 lines).
1 out of 4 hunks FAILED
checking file include/linux/fsnotify_backend.h



Tested on:

commit: 2f012b22 fsnotify: fix UAF from FS_ERROR event on a sh..

git tree: https://github.com/amir73il/linux fsnotify-fixes

kernel config: https://syzkaller.appspot.com/x/.config?x=16ca158ef7e08662

dashboard link: https://syzkaller.appspot.com/bug?extid=5e3f9b2a67b45f16d4e6
compiler:

patch: https://syzkaller.appspot.com/x/patch.diff?x=10103657180000