KASAN: use-after-free Read in fuse_dev_do_read

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: 54eda9df17f3 Merge tag 'pci-v4.19-fixes-1' of git://git.ke..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=109a3d9e400000
kernel config: https://syzkaller.appspot.com/x/.config?x=b8f349d23d3c4835
dashboard link: https://syzkaller.appspot.com/bug?extid=4e975615ca01f2277bdd
compiler: gcc (GCC) 8.0.1 20180413 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+4e9756...@syzkaller.appspotmail.com

==================================================================
BUG: KASAN: use-after-free in constant_test_bit
arch/x86/include/asm/bitops.h:328 [inline]
BUG: KASAN: use-after-free in fuse_dev_do_read.isra.27+0x1659/0x1920
fs/fuse/dev.c:1318
Read of size 8 at addr ffff8801cbd4ea30 by task syz-executor0/28821

CPU: 1 PID: 28821 Comm: syz-executor0 Not tainted 4.19.0-rc3+ #11
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
print_address_description.cold.8+0x9/0x1ff mm/kasan/report.c:256
kasan_report_error mm/kasan/report.c:354 [inline]
kasan_report.cold.9+0x242/0x309 mm/kasan/report.c:412
__asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433
constant_test_bit arch/x86/include/asm/bitops.h:328 [inline]
fuse_dev_do_read.isra.27+0x1659/0x1920 fs/fuse/dev.c:1318
fuse_dev_read+0x1a9/0x250 fs/fuse/dev.c:1360
call_read_iter include/linux/fs.h:1801 [inline]
new_sync_read fs/read_write.c:406 [inline]
__vfs_read+0x6ac/0x9b0 fs/read_write.c:418
vfs_read+0x17f/0x3c0 fs/read_write.c:452
ksys_read+0x101/0x260 fs/read_write.c:578
__do_sys_read fs/read_write.c:588 [inline]
__se_sys_read fs/read_write.c:586 [inline]
__x64_sys_read+0x73/0xb0 fs/read_write.c:586
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4572d9
Code: fd b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 cb b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007faaeefe4c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
RAX: ffffffffffffffda RBX: 00007faaeefe56d4 RCX: 00000000004572d9
RDX: 0000000000001000 RSI: 00000000200040c0 RDI: 0000000000000003
RBP: 00000000009300a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004d4508 R14: 00000000004c8d44 R15: 0000000000000000

Allocated by task 28825:
save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
kasan_kmalloc+0xc7/0xe0 mm/kasan/kasan.c:553
kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:490
kmem_cache_alloc+0x12e/0x730 mm/slab.c:3554
__fuse_request_alloc+0x27/0xf0 fs/fuse/dev.c:58
fuse_request_alloc fs/fuse/dev.c:89 [inline]
__fuse_get_req+0x1f7/0x9e0 fs/fuse/dev.c:164
fuse_get_req fs/fuse/dev.c:194 [inline]
fuse_simple_request+0x28/0x730 fs/fuse/dev.c:549
fuse_statfs+0x368/0x8a0 fs/fuse/inode.c:442
statfs_by_dentry+0x136/0x210 fs/statfs.c:64
vfs_statfs+0x47/0x2e0 fs/statfs.c:74
user_statfs+0xbd/0x150 fs/statfs.c:89
__do_sys_statfs+0x7f/0xf0 fs/statfs.c:179
__se_sys_statfs fs/statfs.c:176 [inline]
__x64_sys_statfs+0x54/0x80 fs/statfs.c:176
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

Freed by task 28825:
save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
__kasan_slab_free+0x102/0x150 mm/kasan/kasan.c:521
kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:528
__cache_free mm/slab.c:3498 [inline]
kmem_cache_free+0x83/0x290 mm/slab.c:3756
fuse_request_free+0x8b/0xa0 fs/fuse/dev.c:104
fuse_put_request+0x2a6/0x350 fs/fuse/dev.c:304
fuse_simple_request+0x453/0x730 fs/fuse/dev.c:571
fuse_statfs+0x368/0x8a0 fs/fuse/inode.c:442
statfs_by_dentry+0x136/0x210 fs/statfs.c:64
vfs_statfs+0x47/0x2e0 fs/statfs.c:74
user_statfs+0xbd/0x150 fs/statfs.c:89
__do_sys_statfs+0x7f/0xf0 fs/statfs.c:179
__se_sys_statfs fs/statfs.c:176 [inline]
__x64_sys_statfs+0x54/0x80 fs/statfs.c:176
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

The buggy address belongs to the object at ffff8801cbd4ea00
which belongs to the cache fuse_request of size 448
The buggy address is located 48 bytes inside of
448-byte region [ffff8801cbd4ea00, ffff8801cbd4ebc0)
The buggy address belongs to the page:
page:ffffea00072f5380 count:1 mapcount:0 mapping:ffff8801d4ad4840 index:0x0
flags: 0x2fffc0000000100(slab)
raw: 02fffc0000000100 ffffea000717eb48 ffff8801d4ad5c48 ffff8801d4ad4840
raw: 0000000000000000 ffff8801cbd4e000 0000000100000008 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
ffff8801cbd4e900: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
ffff8801cbd4e980: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> ffff8801cbd4ea00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff8801cbd4ea80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8801cbd4eb00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

[Kirill Tkhai]

Hi,

On 13.09.2018 11:00, syzbot wrote:
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit:    54eda9df17f3 Merge tag 'pci-v4.19-fixes-1' of git://git.ke..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=109a3d9e400000

what are requests id written here:

write$FUSE_INIT(r0, &(0x7f0000000100)={0x50, 0x0, 0x1}, 0x50)
write$FUSE_STATFS(r0, &(0x7f0000000040)={0x60, 0x0, 0x2, {{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xfffffffffffffffa}}}, 0x60)

?

In case of parallel thread called write with id, we just queued to processing
in fuse_dev_do_read(), we may bump to this stack (despite read has not finished
yet, syzbot may write just everything, any id).

fuse_dev_do_read() fuse_dev_do_write()
list_move_tail(&req->list, &fpq->processing); ...
spin_unlock(&fpq->lock); ...
... request_end(fc, req);
... fuse_put_request(fc, req);
if (test_bit(FR_INTERRUPTED, &req->flags))
queue_interrupt(fiq, req);

We should keep req refcount in fuse_dev_do_read(), till we haven't finished to use it.
Something like this (compile tested only):

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 11ea2c4a38ab..675caed3e655 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -1311,12 +1311,14 @@ static ssize_t fuse_dev_do_read(struct fuse_dev *fud, struct file *file,
goto out_end;
}
list_move_tail(&req->list, &fpq->processing);
+ __fuse_get_request(req);
spin_unlock(&fpq->lock);
set_bit(FR_SENT, &req->flags);
/* matches barrier in request_wait_answer() */
smp_mb__after_atomic();
if (test_bit(FR_INTERRUPTED, &req->flags))
queue_interrupt(fiq, req);
+ fuse_put_request(fc, req);

return reqsize;

[Dmitry Vyukov]

On Thu, Sep 13, 2018 at 10:51 AM, Kirill Tkhai <ktk...@virtuozzo.com> wrote:
> Hi,
>
> On 13.09.2018 11:00, syzbot wrote:
>> Hello,
>>
>> syzbot found the following crash on:
>>
>> HEAD commit: 54eda9df17f3 Merge tag 'pci-v4.19-fixes-1' of git://git.ke..
>> git tree: upstream
>> console output: https://syzkaller.appspot.com/x/log.txt?x=109a3d9e400000
>
> what are requests id written here:
>
> write$FUSE_INIT(r0, &(0x7f0000000100)={0x50, 0x0, 0x1}, 0x50)
> write$FUSE_STATFS(r0, &(0x7f0000000040)={0x60, 0x0, 0x2, {{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xfffffffffffffffa}}}, 0x60)
>
> ?

Hi Kirill,

It may be easier to contrast these numbers with struct definitions here:
https://github.com/google/syzkaller/blob/master/sys/linux/fuse.txt

But passing this fragment through syskaller's tools/syz-prog2c gives
me the following (exact binary layout:

int main(void)
{
syscall(__NR_mmap, 0x20000000, 0x1000000, 3, 0x32, -1, 0);

*(uint32_t*)0x20000100 = 0x50;
*(uint32_t*)0x20000104 = 0;
*(uint64_t*)0x20000108 = 1;
*(uint32_t*)0x20000110 = 7;
*(uint32_t*)0x20000114 = 0x1b;
*(uint32_t*)0x20000118 = 0;
*(uint32_t*)0x2000011c = 0;
*(uint16_t*)0x20000120 = 0;
*(uint16_t*)0x20000122 = 0;
*(uint32_t*)0x20000124 = 0;
*(uint32_t*)0x20000128 = 0;
*(uint32_t*)0x2000012c = 0;
*(uint32_t*)0x20000130 = 0;
*(uint32_t*)0x20000134 = 0;
*(uint32_t*)0x20000138 = 0;
*(uint32_t*)0x2000013c = 0;
*(uint32_t*)0x20000140 = 0;
*(uint32_t*)0x20000144 = 0;
*(uint32_t*)0x20000148 = 0;
*(uint32_t*)0x2000014c = 0;
syscall(__NR_write, -1, 0x20000100, 0x50);
*(uint32_t*)0x20000040 = 0x60;
*(uint32_t*)0x20000044 = 0;
*(uint64_t*)0x20000048 = 2;
*(uint64_t*)0x20000050 = 0;
*(uint64_t*)0x20000058 = 0;
*(uint64_t*)0x20000060 = 0;
*(uint64_t*)0x20000068 = 0;
*(uint64_t*)0x20000070 = 0;
*(uint32_t*)0x20000078 = 0;
*(uint32_t*)0x2000007c = 0;
*(uint32_t*)0x20000080 = 0xfffffffa;
*(uint32_t*)0x20000084 = 0;
*(uint32_t*)0x20000088 = 0;
*(uint32_t*)0x2000008c = 0;
*(uint32_t*)0x20000090 = 0;
*(uint32_t*)0x20000094 = 0;
*(uint32_t*)0x20000098 = 0;
*(uint32_t*)0x2000009c = 0;
syscall(__NR_write, -1, 0x20000040, 0x60);
return 0;
}


For request IDs syzkaller just chooses random numbers in 1:8 range:

type fuse_out[PAYLOAD] {
len len[parent, int32]
err flags[fuse_errors, int32]
unique fuse_unique
payload PAYLOAD
} [packed]

type fuse_unique int64[1:8]

> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bug...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/5882d989-8e9d-0ae1-1b2d-3ba431e97eb6%40virtuozzo.com.
> For more options, visit https://groups.google.com/d/optout.

[syzbot]

syzbot has found a reproducer for the following crash on:

HEAD commit: 10dc890d4228 Merge tag 'pinctrl-v4.19-3' of git://git.kern..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1631cfbe400000
kernel config: https://syzkaller.appspot.com/x/.config?x=5fa12be50bca08d8

dashboard link: https://syzkaller.appspot.com/bug?extid=4e975615ca01f2277bdd
compiler: gcc (GCC) 8.0.1 20180413 (experimental)

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15ffb766400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+4e9756...@syzkaller.appspotmail.com

8021q: adding VLAN 0 to HW filter on device team0
8021q: adding VLAN 0 to HW filter on device team0
8021q: adding VLAN 0 to HW filter on device team0
8021q: adding VLAN 0 to HW filter on device team0

==================================================================
BUG: KASAN: use-after-free in constant_test_bit
arch/x86/include/asm/bitops.h:328 [inline]
BUG: KASAN: use-after-free in fuse_dev_do_read.isra.27+0x1659/0x1920
fs/fuse/dev.c:1318

Read of size 8 at addr ffff8801d8702630 by task syz-executor1/7794

CPU: 0 PID: 7794 Comm: syz-executor1 Not tainted 4.19.0-rc4+ #26

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
print_address_description.cold.8+0x9/0x1ff mm/kasan/report.c:256
kasan_report_error mm/kasan/report.c:354 [inline]
kasan_report.cold.9+0x242/0x309 mm/kasan/report.c:412
__asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433
constant_test_bit arch/x86/include/asm/bitops.h:328 [inline]
fuse_dev_do_read.isra.27+0x1659/0x1920 fs/fuse/dev.c:1318
fuse_dev_read+0x1a9/0x250 fs/fuse/dev.c:1360

call_read_iter include/linux/fs.h:1802 [inline]

new_sync_read fs/read_write.c:406 [inline]
__vfs_read+0x6ac/0x9b0 fs/read_write.c:418
vfs_read+0x17f/0x3c0 fs/read_write.c:452
ksys_read+0x101/0x260 fs/read_write.c:578
__do_sys_read fs/read_write.c:588 [inline]
__se_sys_read fs/read_write.c:586 [inline]
__x64_sys_read+0x73/0xb0 fs/read_write.c:586
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

RIP: 0033:0x457679
Code: 1d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7

48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff

ff 0f 83 eb b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f6a5aeedc78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
RAX: ffffffffffffffda RBX: 00007f6a5aeee6d4 RCX: 0000000000457679
RDX: 0000000000001000 RSI: 0000000020001000 RDI: 0000000000000003
RBP: 000000000072bf00 R08: 0000000000000000 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff

R13: 00000000004d4ad0 R14: 00000000004c31e5 R15: 0000000000000000

Allocated by task 7801:

save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
kasan_kmalloc+0xc7/0xe0 mm/kasan/kasan.c:553
kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:490
kmem_cache_alloc+0x12e/0x730 mm/slab.c:3554
__fuse_request_alloc+0x27/0xf0 fs/fuse/dev.c:58

fuse_request_alloc+0x18/0x20 fs/fuse/dev.c:89
fuse_fill_super+0x12bf/0x1ea0 fs/fuse/inode.c:1157
mount_nodev+0x6b/0x110 fs/super.c:1204
fuse_mount+0x2c/0x40 fs/fuse/inode.c:1213
mount_fs+0xae/0x31d fs/super.c:1261
vfs_kern_mount.part.35+0xdc/0x4f0 fs/namespace.c:961
vfs_kern_mount fs/namespace.c:951 [inline]
do_new_mount fs/namespace.c:2457 [inline]
do_mount+0x581/0x31f0 fs/namespace.c:2787
ksys_mount+0x12d/0x140 fs/namespace.c:3003
__do_sys_mount fs/namespace.c:3017 [inline]
__se_sys_mount fs/namespace.c:3014 [inline]
__x64_sys_mount+0xbe/0x150 fs/namespace.c:3014
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

Freed by task 7801:

save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
__kasan_slab_free+0x102/0x150 mm/kasan/kasan.c:521
kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:528
__cache_free mm/slab.c:3498 [inline]
kmem_cache_free+0x83/0x290 mm/slab.c:3756
fuse_request_free+0x8b/0xa0 fs/fuse/dev.c:104
fuse_put_request+0x2a6/0x350 fs/fuse/dev.c:304

request_end+0xba/0xaa0 fs/fuse/dev.c:414
fuse_dev_do_write+0x192e/0x36e0 fs/fuse/dev.c:1915
fuse_dev_write+0x19a/0x240 fs/fuse/dev.c:1939
call_write_iter include/linux/fs.h:1808 [inline]
new_sync_write fs/read_write.c:474 [inline]
__vfs_write+0x6b8/0x9f0 fs/read_write.c:487
vfs_write+0x1fc/0x560 fs/read_write.c:549
ksys_write+0x101/0x260 fs/read_write.c:598
__do_sys_write fs/read_write.c:610 [inline]
__se_sys_write fs/read_write.c:607 [inline]
__x64_sys_write+0x73/0xb0 fs/read_write.c:607
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

The buggy address belongs to the object at ffff8801d8702600

which belongs to the cache fuse_request of size 448
The buggy address is located 48 bytes inside of

448-byte region [ffff8801d8702600, ffff8801d87027c0)

The buggy address belongs to the page:

page:ffffea000761c080 count:1 mapcount:0 mapping:ffff8801d4a0e240 index:0x0
flags: 0x2fffc0000000100(slab)
raw: 02fffc0000000100 ffffea0006ec0e88 ffffea0006ee91c8 ffff8801d4a0e240
raw: 0000000000000000 ffff8801d8702000 0000000100000008 0000000000000000

page dumped because: kasan: bad access detected

Memory state around the buggy address:

ffff8801d8702500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8801d8702580: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
> ffff8801d8702600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff8801d8702680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8801d8702700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================

[Kirill Tkhai]

Fix from my previous message makes the use-after-free does not reproduce
with the reproducer in my setup.

I can prepare the patch, but before this some comments from Miklos would
be welcome.

Miklos, what you think about this?

Kirill
---

[Miklos Szeredi]

On Mon, Sep 24, 2018 at 2:06 PM, Kirill Tkhai <ktk...@virtuozzo.com> wrote:

> Fix from my previous message makes the use-after-free does not reproduce
> with the reproducer in my setup.

Excellent.

>
> I can prepare the patch, but before this some comments from Miklos would
> be welcome.
>
> Miklos, what you think about this?

I like the patch.

We could optimize away the get/put by moving the set_bit/test_bit part
inside the fpq->lock-ed region and only get the refcount for the
(unlikely) interrupted case. OTOH it's probably not worth the extra
complexity, so let's stay with this simpler fix.

Thanks,
Miklos