KASAN: use-after-free Write in irq_bypass_register_consumer

[syzbot]

Hello,

syzkaller hit the following crash on
cd4175b11685b11c40e31a03e05084cc212b0649
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master
compiler: gcc (GCC) 7.1.1 20170620
.config is attached
Raw console output is attached.

syzkaller reproducer is attached. See https://goo.gl/kgGztJ
for information about syzkaller reproducers


BUG: KASAN: use-after-free in __list_add include/linux/list.h:63 [inline]
BUG: KASAN: use-after-free in list_add include/linux/list.h:78 [inline]
BUG: KASAN: use-after-free in irq_bypass_register_consumer+0x4b4/0x500
virt/lib/irqbypass.c:217
Write of size 8 at addr ffff8801c994a240 by task syz-executor0/4607

CPU: 0 PID: 4607 Comm: syz-executor0 Not tainted 4.14.0-rc1+ #8
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:16 [inline]
dump_stack+0x194/0x257 lib/dump_stack.c:52
print_address_description+0x73/0x250 mm/kasan/report.c:252
kasan_report_error mm/kasan/report.c:351 [inline]
kasan_report+0x24e/0x340 mm/kasan/report.c:409
__asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:435
__list_add include/linux/list.h:63 [inline]
list_add include/linux/list.h:78 [inline]
irq_bypass_register_consumer+0x4b4/0x500 virt/lib/irqbypass.c:217
kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:417 [inline]
kvm_irqfd+0x137a/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572
kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2991
kvm_vm_compat_ioctl+0x2ed/0x3e0
arch/x86/kvm/../../../virt/kvm/kvm_main.c:3135
C_SYSC_ioctl fs/compat_ioctl.c:1593 [inline]
compat_SyS_ioctl+0x1da/0x3300 fs/compat_ioctl.c:1540
do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline]
do_fast_syscall_32+0x3f2/0xeed arch/x86/entry/common.c:391
entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124
RIP: 0023:0xf7f5ac79
RSP: 002b:00000000f7f5605c EFLAGS: 00000296 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 0000000000000009 RCX: 000000004020ae76
RDX: 0000000020025fe0 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000

Allocated by task 4607:
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x43/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:551
kmem_cache_alloc_trace+0x136/0x750 mm/slab.c:3627
kmalloc include/linux/slab.h:493 [inline]
kzalloc include/linux/slab.h:666 [inline]
kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:296 [inline]
kvm_irqfd+0x16c/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572
kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2991
kvm_vm_compat_ioctl+0x2ed/0x3e0
arch/x86/kvm/../../../virt/kvm/kvm_main.c:3135
C_SYSC_ioctl fs/compat_ioctl.c:1593 [inline]
compat_SyS_ioctl+0x1da/0x3300 fs/compat_ioctl.c:1540
do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline]
do_fast_syscall_32+0x3f2/0xeed arch/x86/entry/common.c:391
entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124

Freed by task 23:
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x43/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_slab_free+0x71/0xc0 mm/kasan/kasan.c:524
__cache_free mm/slab.c:3503 [inline]
kfree+0xca/0x250 mm/slab.c:3820
irqfd_shutdown+0x13c/0x1a0 arch/x86/kvm/../../../virt/kvm/eventfd.c:148
process_one_work+0xbfa/0x1bd0 kernel/workqueue.c:2119
worker_thread+0x223/0x1860 kernel/workqueue.c:2253
kthread+0x39c/0x470 kernel/kthread.c:231
ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431

The buggy address belongs to the object at ffff8801c994a0c0
which belongs to the cache kmalloc-512 of size 512
The buggy address is located 384 bytes inside of
512-byte region [ffff8801c994a0c0, ffff8801c994a2c0)
The buggy address belongs to the page:
page:ffffea0007265280 count:1 mapcount:0 mapping:ffff8801c994a0c0
index:0xffff8801c994ad40
flags: 0x200000000000100(slab)
raw: 0200000000000100 ffff8801c994a0c0 ffff8801c994ad40 0000000100000001
raw: ffffea0007265320 ffffea00072630e0 ffff8801dac00940 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
ffff8801c994a100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8801c994a180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff8801c994a200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff8801c994a280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
ffff8801c994a300: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
==================================================================


---
This bug is generated by a dumb bot. It may contain errors.
See https://goo.gl/tpsmEJ for details.
Direct all questions to syzk...@googlegroups.com.

syzbot will keep track of this bug report.
Once a fix for this bug is committed, please reply to this email with:
#syz fix: exact-commit-title
To mark this as a duplicate of another syzbot report, please reply with:
#syz dup: exact-subject-of-another-report
If it's a one-off invalid bug report, please reply with:
#syz invalid
Note: if the crash happens again, it will cause creation of a new bug
report.

[Dmitry Vyukov]

On Mon, Oct 30, 2017 at 10:12 PM, syzbot
<bot+7ad82526b98c64c900...@syzkaller.appspotmail.com>
wrote:

> Hello,
>
> syzkaller hit the following crash on
> cd4175b11685b11c40e31a03e05084cc212b0649
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master
> compiler: gcc (GCC) 7.1.1 20170620
> .config is attached
> Raw console output is attached.
>
> syzkaller reproducer is attached. See https://goo.gl/kgGztJ
> for information about syzkaller reproducers

This also happens on more recent commits, including upstream
19e12196da601f7b80b5639adbcecf60fae13389 (Oct 29).

> --
> You received this message because you are subscribed to the Google Groups
> "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to syzkaller-bug...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/syzkaller-bugs/001a1143e716efd3a9055cc8696b%40google.com.
> For more options, visit https://groups.google.com/d/optout.

[syzbot]

syzkaller has found reproducer for the following crash on
82bcf1def3b5f1251177ad47c44f7e17af039b4b
git://git.cmpxchg.org/linux-mmots.git/master

compiler: gcc (GCC) 7.1.1 20170620
.config is attached
Raw console output is attached.

C reproducer is attached

syzkaller reproducer is attached. See https://goo.gl/kgGztJ
for information about syzkaller reproducers

==================================================================
BUG: KASAN: use-after-free in __list_add include/linux/list.h:64 [inline]
BUG: KASAN: use-after-free in list_add include/linux/list.h:79 [inline]

BUG: KASAN: use-after-free in irq_bypass_register_consumer+0x4b4/0x500
virt/lib/irqbypass.c:217

Write of size 8 at addr ffff8801cdf51180 by task syzkaller436086/15031

CPU: 1 PID: 15031 Comm: syzkaller436086 Not tainted 4.15.0-rc2-mm1+ #39

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:

__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x194/0x257 lib/dump_stack.c:53

print_address_description+0x73/0x250 mm/kasan/report.c:252
kasan_report_error mm/kasan/report.c:351 [inline]

kasan_report+0x25b/0x340 mm/kasan/report.c:409
__asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:435
__list_add include/linux/list.h:64 [inline]
list_add include/linux/list.h:79 [inline]

irq_bypass_register_consumer+0x4b4/0x500 virt/lib/irqbypass.c:217
kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:417 [inline]

kvm_irqfd+0x137f/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572
kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2992
vfs_ioctl fs/ioctl.c:46 [inline]
do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:686
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
entry_SYSCALL_64_fastpath+0x1f/0x96
RIP: 0033:0x44d379
RSP: 002b:00007fc5ff9a9d08 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007fc5ff9aa700 RCX: 000000000044d379
RDX: 0000000020080fe0 RSI: 000000004020ae76 RDI: 0000000000000005
RBP: 00000000007ff900 R08: 00007fc5ff9aa700 R09: 00007fc5ff9aa700
R10: 00007fc5ff9aa700 R11: 0000000000000246 R12: 0000000000000000
R13: 00000000007ff8ff R14: 00007fc5ff9aa9c0 R15: 0000000000000000

Allocated by task 15031:

save_stack+0x43/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:551

kmem_cache_alloc_trace+0x136/0x750 mm/slab.c:3614
kmalloc include/linux/slab.h:516 [inline]
kzalloc include/linux/slab.h:705 [inline]

kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:296 [inline]
kvm_irqfd+0x16c/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572

kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2992
vfs_ioctl fs/ioctl.c:46 [inline]
do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:686
SYSC_ioctl fs/ioctl.c:701 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
entry_SYSCALL_64_fastpath+0x1f/0x96

Freed by task 1402:

save_stack+0x43/0xd0 mm/kasan/kasan.c:447
set_track mm/kasan/kasan.c:459 [inline]
kasan_slab_free+0x71/0xc0 mm/kasan/kasan.c:524

__cache_free mm/slab.c:3492 [inline]
kfree+0xca/0x250 mm/slab.c:3807
irqfd_shutdown+0x13c/0x1a0 arch/x86/kvm/../../../virt/kvm/eventfd.c:148
process_one_work+0xbfd/0x1bc0 kernel/workqueue.c:2113
worker_thread+0x223/0x1990 kernel/workqueue.c:2247
kthread+0x37a/0x440 kernel/kthread.c:238
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:524

The buggy address belongs to the object at ffff8801cdf51000

which belongs to the cache kmalloc-512 of size 512
The buggy address is located 384 bytes inside of

512-byte region [ffff8801cdf51000, ffff8801cdf51200)

The buggy address belongs to the page:

page:00000000d08a0d63 count:1 mapcount:0 mapping:00000000d54c7be6 index:0x0
flags: 0x2fffc0000000100(slab)
raw: 02fffc0000000100 ffff8801cdf51000 0000000000000000 0000000100000006
raw: ffffea00073a7660 ffffea000737f3a0 ffff8801dac00940 0000000000000000

page dumped because: kasan: bad access detected

Memory state around the buggy address:

ffff8801cdf51080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8801cdf51100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff8801cdf51180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff8801cdf51200: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
ffff8801cdf51280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================

[Tianyu Lan]

I reproduced the issue. Will have a look.

--
Best regards
Tianyu Lan

2017-12-15 18:14 GMT+08:00 syzbot
<bot+7ad82526b98c64c900...@syzkaller.appspotmail.com>:

[Lan, Tianyu]

The root cause is that kvm_irqfd_assign() and kvm_irqfd_deassign() can't
be run in parallel. Some data structure(e.g, irqfd->consumer) will be
crashed because irqfd may be freed in deassign path before they are used
in assign path. The other data maybe used in deassign path before
initialization. Syzbot test hit such case. Add mutx between
kvm_irqfd_assign() and kvm_irqfd_deassign() can fix such issue. Will
send patch to fix it.

On 12/16/2017 12:53 PM, Tianyu Lan wrote:
> I reproduced the issue. Will have a look.
>
> -- Best regards Tianyu Lan 2017-12-15 18:14 GMT+08:00 syzbot
> <bot+7ad82526b98c64c900...@syzkaller.appspotmail.com>:
>> syzkaller has found reproducer for the following crash on
>> 82bcf1def3b5f1251177ad47c44f7e17af039b4b
>> git://git.cmpxchg.org/linux-mmots.git/master
>> compiler: gcc (GCC) 7.1.1 20170620
>> .config is attached
>> Raw console output is attached.
>> C reproducer is attached

>> syzkaller reproducer is attached. Seehttps://goo.gl/kgGztJ

[Eric Biggers]

Tianyu, are you still planning to send a patch for this? This is still
happening.

- Eric

[Tianyu Lan]

Yes, I sent out fix patch and wait for review.
 https://patchwork.kernel.org/patch/10128925/

[Eric Biggers]

This is still happening. Can you please resend your patch?

Eric

[Eric Biggers]

Still happening; it's pretty easily reproducible, using the program below.
(Sometimes it crashes in different ways too. It also spams the log, ignore
that.) Tianyu's patch at https://patchwork.kernel.org/patch/10128925/ fixes it,
though the locking is difficult to understand so I'm not sure it's correct yet.
Probably the comment that says "do not drop the file until the irqfd is fully
initialized, otherwise we might race against the EPOLLHUP" would be obsoleted
and should be removed? Can the KVM maintainers please review the patch too?

#include <fcntl.h>
#include <linux/kvm.h>
#include <pthread.h>
#include <sys/eventfd.h>
#include <sys/ioctl.h>

static int dev, vm, event;

static void *irqfd_thrproc(void *_flags)
{
struct kvm_irqfd irqfd = { .fd = event, .flags = (long)_flags };

for (;;)
ioctl(vm, KVM_IRQFD, &irqfd);
}

int main()
{
pthread_t t;

event = eventfd(0, 0);
dev = open("/dev/kvm", 0);
vm = ioctl(dev, KVM_CREATE_VM, 0);
ioctl(vm, KVM_CREATE_IRQCHIP, 0);
pthread_create(&t, NULL, irqfd_thrproc, (void *)(long)0);
irqfd_thrproc((void *)(long)KVM_IRQFD_FLAG_DEASSIGN);
}

[Dmitry Vyukov]

Tentative duplicates of this bug:

BUG: unable to handle kernel NULL pointer dereference in
irq_bypass_register_consumer
KASAN: use-after-free Read in irq_bypass_unregister_consumer
general protection fault in eventfd_ctx_remove_wait_queue
KASAN: use-after-free Read in irq_bypass_register_consumer

https://syzkaller.appspot.com/bug?id=45591ae3053c59fb50169401fb61cb596735f9d1