KASAN: use-after-free Read in idr_for_each (2)

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 472e5b05 pipe: remove pipe_wait() and fix wakeup race with..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=15ae0d47900000
kernel config: https://syzkaller.appspot.com/x/.config?x=89ab6a0c48f30b49
dashboard link: https://syzkaller.appspot.com/bug?extid=12056a09a0311d758e60
compiler: gcc (GCC) 10.1.0-syz 20200507
userspace arch: i386

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+12056a...@syzkaller.appspotmail.com

==================================================================
BUG: KASAN: use-after-free in radix_tree_next_slot include/linux/radix-tree.h:421 [inline]
BUG: KASAN: use-after-free in idr_for_each+0x206/0x220 lib/idr.c:202
Read of size 8 at addr ffff88804eb9cb30 by task kworker/u4:8/13668

CPU: 1 PID: 13668 Comm: kworker/u4:8 Not tainted 5.9.0-rc7-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound io_ring_exit_work
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x198/0x1fd lib/dump_stack.c:118
print_address_description.constprop.0.cold+0xae/0x497 mm/kasan/report.c:383
__kasan_report mm/kasan/report.c:513 [inline]
kasan_report.cold+0x1f/0x37 mm/kasan/report.c:530
radix_tree_next_slot include/linux/radix-tree.h:421 [inline]
idr_for_each+0x206/0x220 lib/idr.c:202
io_destroy_buffers fs/io_uring.c:7889 [inline]
io_ring_ctx_free fs/io_uring.c:7904 [inline]
io_ring_exit_work+0x363/0x6d0 fs/io_uring.c:7979
process_one_work+0x94c/0x1670 kernel/workqueue.c:2269
worker_thread+0x64c/0x1120 kernel/workqueue.c:2415
kthread+0x3b5/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294

Allocated by task 17016:
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_set_track mm/kasan/common.c:56 [inline]
__kasan_kmalloc.constprop.0+0xbf/0xd0 mm/kasan/common.c:461
slab_post_alloc_hook mm/slab.h:518 [inline]
slab_alloc mm/slab.c:3316 [inline]
kmem_cache_alloc+0x13a/0x3f0 mm/slab.c:3486
radix_tree_node_alloc.constprop.0+0x7c/0x350 lib/radix-tree.c:275
idr_get_free+0x4c5/0x940 lib/radix-tree.c:1505
idr_alloc_u32+0x170/0x2d0 lib/idr.c:46
idr_alloc+0xc2/0x130 lib/idr.c:87
io_provide_buffers fs/io_uring.c:3768 [inline]
io_issue_sqe+0x48d2/0x5c50 fs/io_uring.c:5906
__io_queue_sqe+0x280/0x1160 fs/io_uring.c:6178
io_queue_sqe+0x692/0xfa0 fs/io_uring.c:6257
io_submit_sqe fs/io_uring.c:6327 [inline]
io_submit_sqes+0x1759/0x23f0 fs/io_uring.c:6521
__do_sys_io_uring_enter+0xeac/0x1bd0 fs/io_uring.c:8349
do_syscall_32_irqs_on arch/x86/entry/common.c:78 [inline]
__do_fast_syscall_32+0x60/0x90 arch/x86/entry/common.c:137
do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:160
entry_SYSENTER_compat_after_hwframe+0x4d/0x5c

Freed by task 16:
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_set_track+0x1c/0x30 mm/kasan/common.c:56
kasan_set_free_info+0x1b/0x30 mm/kasan/generic.c:355
__kasan_slab_free+0xd8/0x120 mm/kasan/common.c:422
__cache_free mm/slab.c:3422 [inline]
kmem_cache_free.part.0+0x74/0x1e0 mm/slab.c:3697
rcu_do_batch kernel/rcu/tree.c:2430 [inline]
rcu_core+0x5ca/0x1130 kernel/rcu/tree.c:2658
__do_softirq+0x1f8/0xb23 kernel/softirq.c:298

Last call_rcu():
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_record_aux_stack+0x82/0xb0 mm/kasan/generic.c:346
__call_rcu kernel/rcu/tree.c:2896 [inline]
call_rcu+0x15e/0x7c0 kernel/rcu/tree.c:2970
radix_tree_node_free lib/radix-tree.c:309 [inline]
delete_node+0x591/0x8c0 lib/radix-tree.c:572
__radix_tree_delete+0x190/0x370 lib/radix-tree.c:1378
radix_tree_delete_item+0xe7/0x230 lib/radix-tree.c:1429
__io_remove_buffers fs/io_uring.c:3666 [inline]
__io_remove_buffers fs/io_uring.c:3645 [inline]
__io_destroy_buffers+0x161/0x200 fs/io_uring.c:7883
idr_for_each+0x113/0x220 lib/idr.c:208
io_destroy_buffers fs/io_uring.c:7889 [inline]
io_ring_ctx_free fs/io_uring.c:7904 [inline]
io_ring_exit_work+0x363/0x6d0 fs/io_uring.c:7979
process_one_work+0x94c/0x1670 kernel/workqueue.c:2269
worker_thread+0x64c/0x1120 kernel/workqueue.c:2415
kthread+0x3b5/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294

Second to last call_rcu():
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_record_aux_stack+0x82/0xb0 mm/kasan/generic.c:346
__call_rcu kernel/rcu/tree.c:2896 [inline]
call_rcu+0x15e/0x7c0 kernel/rcu/tree.c:2970
radix_tree_node_free lib/radix-tree.c:309 [inline]
radix_tree_shrink lib/radix-tree.c:535 [inline]
delete_node+0x37a/0x8c0 lib/radix-tree.c:553
__radix_tree_delete+0x190/0x370 lib/radix-tree.c:1378
radix_tree_delete_item+0xe7/0x230 lib/radix-tree.c:1429
free_pid+0xa1/0x260 kernel/pid.c:151
__change_pid+0x1c7/0x2d0 kernel/pid.c:352
__unhash_process kernel/exit.c:77 [inline]
__exit_signal kernel/exit.c:147 [inline]
release_task+0xd29/0x14d0 kernel/exit.c:198
wait_task_zombie kernel/exit.c:1088 [inline]
wait_consider_task+0x2fd2/0x3b70 kernel/exit.c:1315
do_wait_thread kernel/exit.c:1378 [inline]
do_wait+0x376/0xa00 kernel/exit.c:1449
kernel_wait4+0x14c/0x260 kernel/exit.c:1621
do_syscall_32_irqs_on arch/x86/entry/common.c:78 [inline]
__do_fast_syscall_32+0x60/0x90 arch/x86/entry/common.c:137
do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:160
entry_SYSENTER_compat_after_hwframe+0x4d/0x5c

The buggy address belongs to the object at ffff88804eb9cb00
which belongs to the cache radix_tree_node of size 576
The buggy address is located 48 bytes inside of
576-byte region [ffff88804eb9cb00, ffff88804eb9cd40)
The buggy address belongs to the page:
page:00000000a35d3b6e refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff88804eb9cffb pfn:0x4eb9c
flags: 0xfffe0000000200(slab)
raw: 00fffe0000000200 ffffea00013ab388 ffffea0002927748 ffff8880aa06f000
raw: ffff88804eb9cffb ffff88804eb9c000 0000000100000005 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
ffff88804eb9ca00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff88804eb9ca80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff88804eb9cb00: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff88804eb9cb80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff88804eb9cc00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: c84e1efa Merge tag 'asm-generic-fixes-5.10-2' of git://git..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1251d759500000
kernel config: https://syzkaller.appspot.com/x/.config?x=cb8d1a3819ba4356

dashboard link: https://syzkaller.appspot.com/bug?extid=12056a09a0311d758e60
compiler: gcc (GCC) 10.1.0-syz 20200507

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1126cce9500000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1173d2e9500000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+12056a...@syzkaller.appspotmail.com

==================================================================

BUG: KASAN: use-after-free in radix_tree_next_slot include/linux/radix-tree.h:422 [inline]

BUG: KASAN: use-after-free in idr_for_each+0x206/0x220 lib/idr.c:202

Read of size 8 at addr ffff888032eb2c40 by task kworker/u4:4/186

CPU: 1 PID: 186 Comm: kworker/u4:4 Not tainted 5.10.0-rc5-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound io_ring_exit_work
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]

dump_stack+0x107/0x163 lib/dump_stack.c:118
print_address_description.constprop.0.cold+0xae/0x4c8 mm/kasan/report.c:385
__kasan_report mm/kasan/report.c:545 [inline]
kasan_report.cold+0x1f/0x37 mm/kasan/report.c:562
radix_tree_next_slot include/linux/radix-tree.h:422 [inline]
idr_for_each+0x206/0x220 lib/idr.c:202
io_destroy_buffers fs/io_uring.c:8275 [inline]
io_ring_ctx_free fs/io_uring.c:8298 [inline]
io_ring_exit_work+0x3f7/0x7a0 fs/io_uring.c:8375
process_one_work+0x933/0x15a0 kernel/workqueue.c:2272
worker_thread+0x64c/0x1120 kernel/workqueue.c:2418
kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296

Allocated by task 10961:

kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_set_track mm/kasan/common.c:56 [inline]

__kasan_kmalloc.constprop.0+0xc2/0xd0 mm/kasan/common.c:461
slab_post_alloc_hook mm/slab.h:526 [inline]
slab_alloc_node mm/slub.c:2891 [inline]
slab_alloc mm/slub.c:2899 [inline]
kmem_cache_alloc+0x122/0x460 mm/slub.c:2904
radix_tree_node_alloc.constprop.0+0x7c/0x350 lib/radix-tree.c:274
idr_get_free+0x4c5/0x940 lib/radix-tree.c:1504

idr_alloc_u32+0x170/0x2d0 lib/idr.c:46
idr_alloc+0xc2/0x130 lib/idr.c:87

io_provide_buffers fs/io_uring.c:4032 [inline]
io_issue_sqe+0x2fc4/0x3d10 fs/io_uring.c:6012
__io_queue_sqe+0x132/0xda0 fs/io_uring.c:6232
io_queue_sqe+0x623/0x11f0 fs/io_uring.c:6298
io_submit_sqe fs/io_uring.c:6367 [inline]
io_submit_sqes+0x15e1/0x28a0 fs/io_uring.c:6596
__do_sys_io_uring_enter+0xc90/0x1ab0 fs/io_uring.c:8983
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9

Freed by task 8546:

kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_set_track+0x1c/0x30 mm/kasan/common.c:56
kasan_set_free_info+0x1b/0x30 mm/kasan/generic.c:355

__kasan_slab_free+0x102/0x140 mm/kasan/common.c:422
slab_free_hook mm/slub.c:1544 [inline]
slab_free_freelist_hook+0x5d/0x150 mm/slub.c:1577
slab_free mm/slub.c:3142 [inline]
kmem_cache_free+0x82/0x350 mm/slub.c:3158
rcu_do_batch kernel/rcu/tree.c:2476 [inline]
rcu_core+0x5df/0xe80 kernel/rcu/tree.c:2711
__do_softirq+0x2a0/0x9f6 kernel/softirq.c:298

Last call_rcu():
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48

kasan_record_aux_stack+0xc0/0xf0 mm/kasan/generic.c:346
__call_rcu kernel/rcu/tree.c:2953 [inline]
call_rcu+0xbb/0x700 kernel/rcu/tree.c:3027
radix_tree_node_free lib/radix-tree.c:308 [inline]
delete_node+0x591/0x8c0 lib/radix-tree.c:571
__radix_tree_delete+0x190/0x370 lib/radix-tree.c:1377
radix_tree_delete_item+0xe7/0x230 lib/radix-tree.c:1428
__io_remove_buffers fs/io_uring.c:3930 [inline]
__io_remove_buffers fs/io_uring.c:3909 [inline]
__io_destroy_buffers+0x161/0x200 fs/io_uring.c:8269
idr_for_each+0x113/0x220 lib/idr.c:208
io_destroy_buffers fs/io_uring.c:8275 [inline]
io_ring_ctx_free fs/io_uring.c:8298 [inline]
io_ring_exit_work+0x3f7/0x7a0 fs/io_uring.c:8375
process_one_work+0x933/0x15a0 kernel/workqueue.c:2272
worker_thread+0x64c/0x1120 kernel/workqueue.c:2418
kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296

The buggy address belongs to the object at ffff888032eb2c00

which belongs to the cache radix_tree_node of size 576

The buggy address is located 64 bytes inside of
576-byte region [ffff888032eb2c00, ffff888032eb2e40)

The buggy address belongs to the page:

page:00000000102f3139 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x32eb0
head:00000000102f3139 order:2 compound_mapcount:0 compound_pincount:0
flags: 0xfff00000010200(slab|head)
raw: 00fff00000010200 dead000000000100 dead000000000122 ffff88801004db40
raw: 0000000000000000 0000000000170017 00000001ffffffff 0000000000000000

page dumped because: kasan: bad access detected

Memory state around the buggy address:

ffff888032eb2b00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff888032eb2b80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff888032eb2c00: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff888032eb2c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff888032eb2d00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================

[Hillf Danton]

On Sat, 28 Nov 2020 09:19:21 -0800

Matthew, can you shed any light on the link between the use of idr
routines and the UAF reported?

Looks like it is not safe to have idr_remove() called in the
idr_for_each loop without rcu_read_lock().

>
> Last call_rcu():
> kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
> kasan_record_aux_stack+0xc0/0xf0 mm/kasan/generic.c:346
> __call_rcu kernel/rcu/tree.c:2953 [inline]
> call_rcu+0xbb/0x700 kernel/rcu/tree.c:3027
> radix_tree_node_free lib/radix-tree.c:308 [inline]
> delete_node+0x591/0x8c0 lib/radix-tree.c:571
> __radix_tree_delete+0x190/0x370 lib/radix-tree.c:1377
> radix_tree_delete_item+0xe7/0x230 lib/radix-tree.c:1428
> __io_remove_buffers fs/io_uring.c:3930 [inline]
> __io_remove_buffers fs/io_uring.c:3909 [inline]
> __io_destroy_buffers+0x161/0x200 fs/io_uring.c:8269
> idr_for_each+0x113/0x220 lib/idr.c:208
> io_destroy_buffers fs/io_uring.c:8275 [inline]
> io_ring_ctx_free fs/io_uring.c:8298 [inline]
> io_ring_exit_work+0x3f7/0x7a0 fs/io_uring.c:8375

But more weired, this work is scheduled more than once, given
a reproducer.

> process_one_work+0x933/0x15a0 kernel/workqueue.c:2272
> worker_thread+0x64c/0x1120 kernel/workqueue.c:2418
> kthread+0x3b1/0x4a0 kernel/kthread.c:292
> ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296
>
> The buggy address belongs to the object at ffff888032eb2c00
> which belongs to the cache radix_tree_node of size 576
> The buggy address is located 64 bytes inside of
> 576-byte region [ffff888032eb2c00, ffff888032eb2e40)
> The buggy address belongs to the page:
> page:00000000102f3139 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x32eb0
> head:00000000102f3139 order:2 compound_mapcount:0 compound_pincount:0
> flags: 0xfff00000010200(slab|head)
> raw: 00fff00000010200 dead000000000100 dead000000000122 ffff88801004db40
> raw: 0000000000000000 0000000000170017 00000001ffffffff 0000000000000000
> page dumped because: kasan: bad access detected
>
> Memory state around the buggy address:
> ffff888032eb2b00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff888032eb2b80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> >ffff888032eb2c00: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ^
> ffff888032eb2c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff888032eb2d00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==================================================================

Is it making sense to destroy idr after freeing entries without
idr_remove() invoked for every entry, as an alternative to
rcu_read_lock()?

--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -8261,18 +8261,21 @@ static int io_eventfd_unregister(struct
return -ENXIO;
}

-static int __io_destroy_buffers(int id, void *p, void *data)
+static void io_destroy_buffers(struct io_ring_ctx *ctx)
{
- struct io_ring_ctx *ctx = data;
- struct io_buffer *buf = p;
+ struct io_buffer *buf;
+ int id;

- __io_remove_buffers(ctx, buf, id, -1U);
- return 0;
-}
+ idr_for_each_entry(&ctx->io_buffer_idr, buf, id) {
+ while (!list_empty(&buf->list)) {
+ struct io_buffer *nxt;

-static void io_destroy_buffers(struct io_ring_ctx *ctx)
-{
- idr_for_each(&ctx->io_buffer_idr, __io_destroy_buffers, ctx);
+ nxt = list_first_entry(&buf->list, struct io_buffer, list);
+ list_del(&nxt->list);
+ kfree(nxt);
+ }
+ kfree(buf);
+ }
idr_destroy(&ctx->io_buffer_idr);
}

[Matthew Wilcox]

On Sun, Nov 29, 2020 at 07:34:29PM +0800, Hillf Danton wrote:
> > radix_tree_next_slot include/linux/radix-tree.h:422 [inline]
> > idr_for_each+0x206/0x220 lib/idr.c:202
> > io_destroy_buffers fs/io_uring.c:8275 [inline]
>
> Matthew, can you shed any light on the link between the use of idr
> routines and the UAF reported?

I presume it's some misuse of IDR by io_uring. I'd rather io_uring
didn't use the IDR at all. This compiles; I promise no more than that.

diff --git a/fs/io_uring.c b/fs/io_uring.c
index ef3cd7fe4416..2fcf196bb3c3 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -344,7 +344,7 @@ struct io_ring_ctx {
struct socket *ring_sock;
#endif

- struct idr io_buffer_idr;
+ struct xarray io_buffers;

struct idr personality_idr;

@@ -1298,7 +1298,7 @@ static struct io_ring_ctx *io_ring_ctx_alloc(struct io_uring_params *p)
INIT_LIST_HEAD(&ctx->cq_overflow_list);
init_completion(&ctx->ref_comp);
init_completion(&ctx->sq_thread_comp);
- idr_init(&ctx->io_buffer_idr);
+ xa_init(&ctx->io_buffers);
idr_init(&ctx->personality_idr);
mutex_init(&ctx->uring_lock);
init_waitqueue_head(&ctx->wait);
@@ -3042,7 +3042,7 @@ static struct io_buffer *io_buffer_select(struct io_kiocb *req, size_t *len,

lockdep_assert_held(&req->ctx->uring_lock);

- head = idr_find(&req->ctx->io_buffer_idr, bgid);
+ head = xa_load(&req->ctx->io_buffers, bgid);
if (head) {
if (!list_empty(&head->list)) {
kbuf = list_last_entry(&head->list, struct io_buffer,
@@ -3050,7 +3050,7 @@ static struct io_buffer *io_buffer_select(struct io_kiocb *req, size_t *len,
list_del(&kbuf->list);
} else {
kbuf = head;
- idr_remove(&req->ctx->io_buffer_idr, bgid);
+ xa_erase(&req->ctx->io_buffers, bgid);
}
if (*len > kbuf->len)
*len = kbuf->len;
@@ -4130,7 +4130,8 @@ static int __io_remove_buffers(struct io_ring_ctx *ctx, struct io_buffer *buf,
}
i++;
kfree(buf);
- idr_remove(&ctx->io_buffer_idr, bgid);
+ if (nbufs != -1U)
+ xa_erase(&ctx->io_buffers, bgid);

return i;
}
@@ -4148,7 +4149,7 @@ static int io_remove_buffers(struct io_kiocb *req, bool force_nonblock,
lockdep_assert_held(&ctx->uring_lock);

ret = -ENOENT;
- head = idr_find(&ctx->io_buffer_idr, p->bgid);
+ head = xa_load(&ctx->io_buffers, p->bgid);
if (head)
ret = __io_remove_buffers(ctx, head, p->bgid, p->nbufs);

@@ -4225,15 +4226,15 @@ static int io_provide_buffers(struct io_kiocb *req, bool force_nonblock,

lockdep_assert_held(&ctx->uring_lock);

- list = head = idr_find(&ctx->io_buffer_idr, p->bgid);
+ list = head = xa_load(&ctx->io_buffers, p->bgid);

ret = io_add_buffers(p, &head);
if (ret < 0)
goto out;

if (!list) {
- ret = idr_alloc(&ctx->io_buffer_idr, head, p->bgid, p->bgid + 1,
- GFP_KERNEL);
+ ret = xa_err(xa_store(&ctx->io_buffers, p->bgid, head,
+ GFP_KERNEL));
if (ret < 0) {
__io_remove_buffers(ctx, head, p->bgid, -1U);
goto out;
@@ -8468,19 +8469,15 @@ static int io_eventfd_unregister(struct io_ring_ctx *ctx)

return -ENXIO;
}

-static int __io_destroy_buffers(int id, void *p, void *data)

-{

- struct io_ring_ctx *ctx = data;
- struct io_buffer *buf = p;

-

- __io_remove_buffers(ctx, buf, id, -1U);
- return 0;
-}

-

static void io_destroy_buffers(struct io_ring_ctx *ctx)
{

- idr_for_each(&ctx->io_buffer_idr, __io_destroy_buffers, ctx);
- idr_destroy(&ctx->io_buffer_idr);
+ unsigned long pgid;

+ struct io_buffer *buf;
+

+ xa_for_each(&ctx->io_buffers, pgid, buf) {
+ xa_erase(&ctx->io_buffers, pgid);
+ __io_remove_buffers(ctx, buf, pgid, -1U);
+ }
}

static void io_ring_ctx_free(struct io_ring_ctx *ctx)

[Hillf Danton]

On Sun, 29 Nov 2020 12:26:00 +0000 Matthew Wilcox wrote:
>On Sun, Nov 29, 2020 at 07:34:29PM +0800, Hillf Danton wrote:
>> > radix_tree_next_slot include/linux/radix-tree.h:422 [inline]
>> > idr_for_each+0x206/0x220 lib/idr.c:202
>> > io_destroy_buffers fs/io_uring.c:8275 [inline]
>>
>> Matthew, can you shed any light on the link between the use of idr
>> routines and the UAF reported?
>
>I presume it's some misuse of IDR by io_uring. I'd rather io_uring
>didn't use the IDR at all. This compiles; I promise no more than that.

Thanks so much for your xa lesson :)

[Jens Axboe]

On 11/29/20 5:26 AM, Matthew Wilcox wrote:
> On Sun, Nov 29, 2020 at 07:34:29PM +0800, Hillf Danton wrote:
>>> radix_tree_next_slot include/linux/radix-tree.h:422 [inline]
>>> idr_for_each+0x206/0x220 lib/idr.c:202
>>> io_destroy_buffers fs/io_uring.c:8275 [inline]
>>
>> Matthew, can you shed any light on the link between the use of idr
>> routines and the UAF reported?
>
> I presume it's some misuse of IDR by io_uring. I'd rather io_uring
> didn't use the IDR at all. This compiles; I promise no more than that.

Looks reasonable to me. Care to send as an actual patch?

This would just leave the personality idr as the last idr use case in
io_uring, hint hint :-)

Would be nice to fully understand why this issue exists with idr, I
don't immediately see anything wrong. But as I cannot even reproduce, I
can't verify that the xa version is sane wrt fixing it either...

--
Jens Axboe

[Pavel Begunkov]

#syz test: git://git.kernel.dk/linux-block dfea9fce29fda6f2f91161677e0e0d9b671bc099

--
Pavel Begunkov

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
KASAN: use-after-free Read in idr_for_each

==================================================================
BUG: KASAN: use-after-free in radix_tree_next_slot include/linux/radix-tree.h:422 [inline]
BUG: KASAN: use-after-free in idr_for_each+0x206/0x220 lib/idr.c:202

Read of size 8 at addr ffff888042e76040 by task kworker/u4:5/3340

CPU: 0 PID: 3340 Comm: kworker/u4:5 Not tainted 5.10.0-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound io_ring_exit_work
Call Trace:

__dump_stack lib/dump_stack.c:79 [inline]
dump_stack+0x107/0x163 lib/dump_stack.c:120

print_address_description.constprop.0.cold+0xae/0x4c8 mm/kasan/report.c:385
__kasan_report mm/kasan/report.c:545 [inline]
kasan_report.cold+0x1f/0x37 mm/kasan/report.c:562
radix_tree_next_slot include/linux/radix-tree.h:422 [inline]
idr_for_each+0x206/0x220 lib/idr.c:202

io_destroy_buffers fs/io_uring.c:8541 [inline]
io_ring_ctx_free fs/io_uring.c:8564 [inline]
io_ring_exit_work+0x394/0x730 fs/io_uring.c:8639
process_one_work+0x98d/0x1630 kernel/workqueue.c:2275
worker_thread+0x64c/0x1120 kernel/workqueue.c:2421

kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296

Allocated by task 28625:

kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_set_track mm/kasan/common.c:56 [inline]
__kasan_kmalloc.constprop.0+0xc2/0xd0 mm/kasan/common.c:461

slab_post_alloc_hook mm/slab.h:512 [inline]
slab_alloc_node mm/slub.c:2889 [inline]
slab_alloc mm/slub.c:2897 [inline]
kmem_cache_alloc+0x145/0x350 mm/slub.c:2902
radix_tree_node_alloc.constprop.0+0x7c/0x350 lib/radix-tree.c:274
idr_get_free+0x554/0xa60 lib/radix-tree.c:1504

idr_alloc_u32+0x170/0x2d0 lib/idr.c:46
idr_alloc+0xc2/0x130 lib/idr.c:87

io_provide_buffers fs/io_uring.c:4230 [inline]
io_issue_sqe+0x3681/0x44e0 fs/io_uring.c:6264
__io_queue_sqe+0x228/0x1120 fs/io_uring.c:6477
io_queue_sqe+0x631/0x10f0 fs/io_uring.c:6543
io_submit_sqe fs/io_uring.c:6616 [inline]
io_submit_sqes+0x135a/0x2530 fs/io_uring.c:6864
__do_sys_io_uring_enter+0x591/0x1c00 fs/io_uring.c:9174
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9

Freed by task 8890:

kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_set_track+0x1c/0x30 mm/kasan/common.c:56

kasan_set_free_info+0x1b/0x30 mm/kasan/generic.c:352

__kasan_slab_free+0x102/0x140 mm/kasan/common.c:422
slab_free_hook mm/slub.c:1544 [inline]
slab_free_freelist_hook+0x5d/0x150 mm/slub.c:1577

slab_free mm/slub.c:3140 [inline]
kmem_cache_free+0x82/0x360 mm/slub.c:3156
rcu_do_batch kernel/rcu/tree.c:2489 [inline]
rcu_core+0x75d/0xf80 kernel/rcu/tree.c:2723
__do_softirq+0x2bc/0xa77 kernel/softirq.c:343

Last potentially related work creation:
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_record_aux_stack+0xc0/0xf0 mm/kasan/generic.c:343
__call_rcu kernel/rcu/tree.c:2965 [inline]
call_rcu+0xbb/0x710 kernel/rcu/tree.c:3038

radix_tree_node_free lib/radix-tree.c:308 [inline]
delete_node+0x591/0x8c0 lib/radix-tree.c:571
__radix_tree_delete+0x190/0x370 lib/radix-tree.c:1377
radix_tree_delete_item+0xe7/0x230 lib/radix-tree.c:1428

__io_remove_buffers fs/io_uring.c:4122 [inline]
__io_remove_buffers fs/io_uring.c:4101 [inline]
__io_destroy_buffers+0x161/0x200 fs/io_uring.c:8535
idr_for_each+0x113/0x220 lib/idr.c:208
io_destroy_buffers fs/io_uring.c:8541 [inline]
io_ring_ctx_free fs/io_uring.c:8564 [inline]
io_ring_exit_work+0x394/0x730 fs/io_uring.c:8639
process_one_work+0x98d/0x1630 kernel/workqueue.c:2275
worker_thread+0x64c/0x1120 kernel/workqueue.c:2421

kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296

Second to last potentially related work creation:
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:48
kasan_record_aux_stack+0xc0/0xf0 mm/kasan/generic.c:343
__call_rcu kernel/rcu/tree.c:2965 [inline]
call_rcu+0xbb/0x710 kernel/rcu/tree.c:3038
xa_node_free lib/xarray.c:258 [inline]
xas_delete_node lib/xarray.c:494 [inline]
update_node lib/xarray.c:756 [inline]
xas_store+0xbeb/0x1c10 lib/xarray.c:841
__xa_erase lib/xarray.c:1489 [inline]
xa_erase+0xb0/0x170 lib/xarray.c:1510
io_uring_del_task_file fs/io_uring.c:8889 [inline]
__io_uring_files_cancel+0xdbf/0x1550 fs/io_uring.c:8925
io_uring_files_cancel include/linux/io_uring.h:51 [inline]
exit_files+0xe4/0x170 fs/file.c:431
do_exit+0xb4f/0x2a00 kernel/exit.c:818
do_group_exit+0x125/0x310 kernel/exit.c:920
get_signal+0x3e9/0x2160 kernel/signal.c:2770
arch_do_signal_or_restart+0x2a8/0x1eb0 arch/x86/kernel/signal.c:811
handle_signal_work kernel/entry/common.c:147 [inline]
exit_to_user_mode_loop kernel/entry/common.c:171 [inline]
exit_to_user_mode_prepare+0x124/0x200 kernel/entry/common.c:201
__syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline]
syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302
entry_SYSCALL_64_after_hwframe+0x44/0xa9

The buggy address belongs to the object at ffff888042e76000

which belongs to the cache radix_tree_node of size 576
The buggy address is located 64 bytes inside of

576-byte region [ffff888042e76000, ffff888042e76240)

The buggy address belongs to the page:

page:0000000090e8be83 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x42e76
head:0000000090e8be83 order:1 compound_mapcount:0
flags: 0xfff00000010200(slab|head)
raw: 00fff00000010200 dead000000000100 dead000000000122 ffff88801084db40
raw: ffff888042e76580 00000000800b000a 00000001ffffffff 0000000000000000

page dumped because: kasan: bad access detected

Memory state around the buggy address:

ffff888042e75f00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
ffff888042e75f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff888042e76000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff888042e76080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff888042e76100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================


Tested on:

commit: dfea9fce io_uring: close a small race gap for files cancel
git tree: git://git.kernel.dk/linux-block
console output: https://syzkaller.appspot.com/x/log.txt?x=1263a46b500000
kernel config: https://syzkaller.appspot.com/x/.config?x=4db50a97037d9f3e

[Pavel Begunkov]

On 18/12/2020 16:44, syzbot wrote:
> Hello,
>
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> KASAN: use-after-free Read in idr_for_each

#syz test: git://git.kernel.dk/linux-block io_uring-5.12

--
Pavel Begunkov

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+12056a...@syzkaller.appspotmail.com

Tested on:

commit: ece5fae7 io_uring: don't leak creds on SQO attach error
git tree: git://git.kernel.dk/linux-block io_uring-5.12
kernel config: https://syzkaller.appspot.com/x/.config?x=28f8268e740d48dd

dashboard link: https://syzkaller.appspot.com/bug?extid=12056a09a0311d758e60
compiler:

Note: testing is done by a robot and is best-effort only.

[syzbot]

syzbot suspects this issue was fixed by commit:

commit 61cf93700fe6359552848ed5e3becba6cd760efa
Author: Matthew Wilcox (Oracle) <wi...@infradead.org>
Date: Mon Mar 8 14:16:16 2021 +0000

io_uring: Convert personality_idr to XArray

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=16f91b9ad00000
start commit: dd86e7fa Merge tag 'pci-v5.11-fixes-2' of git://git.kernel..
git tree: upstream
kernel config: https://syzkaller.appspot.com/x/.config?x=e83e68d0a6aba5f6
dashboard link: https://syzkaller.appspot.com/bug?extid=12056a09a0311d758e60
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=174b80ef500000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=165522d4d00000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: io_uring: Convert personality_idr to XArray

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Pavel Begunkov]

--
Pavel Begunkov

[FJKONG]

Hi all,

I tested on 5.10.41 using syszkaller, It still can be reproduced.

Should we back port patch for this version?

Great Thanks

Kong