net/kcm: use-after-free in kcm_wq

[Dmitry Vyukov]

Hello,

I am getting the following use-after-free reports while running
syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. Seems
that work items are not properly cancelled.

==================================================================
BUG: KASAN: use-after-free in worker_thread+0x190f/0x1990
kernel/workqueue.c:2228 at addr ffff880063de5a20
Read of size 8 by task kworker/u8:0/5
CPU: 1 PID: 5 Comm: kworker/u8:0 Not tainted 4.10.0+ #260
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:15 [inline]
dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
print_address_description mm/kasan/report.c:204 [inline]
kasan_report_error mm/kasan/report.c:288 [inline]
kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
kasan_report mm/kasan/report.c:331 [inline]
__asan_report_load8_noabort+0x29/0x30 mm/kasan/report.c:331
worker_thread+0x190f/0x1990 kernel/workqueue.c:2228
kthread+0x326/0x3f0 kernel/kthread.c:227
ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
Object at ffff880063de5900, in cache kcm_psock_cache size: 616
Allocated:
PID = 11085
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
save_stack+0x43/0xd0 mm/kasan/kasan.c:502
set_track mm/kasan/kasan.c:514 [inline]
kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
kmem_cache_zalloc include/linux/slab.h:653 [inline]
kcm_attach net/kcm/kcmsock.c:1384 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1455 [inline]
kcm_ioctl+0x303/0x1800 net/kcm/kcmsock.c:1690
sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993
vfs_ioctl fs/ioctl.c:43 [inline]
do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
SYSC_ioctl fs/ioctl.c:698 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 11095
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
save_stack+0x43/0xd0 mm/kasan/kasan.c:502
set_track mm/kasan/kasan.c:514 [inline]
kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
__cache_free mm/slab.c:3513 [inline]
kmem_cache_free+0x71/0x240 mm/slab.c:3773
kcm_unattach+0xee7/0x1520 net/kcm/kcmsock.c:1558
kcm_unattach_ioctl net/kcm/kcmsock.c:1603 [inline]
kcm_ioctl+0xfae/0x1800 net/kcm/kcmsock.c:1700
sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993
vfs_ioctl fs/ioctl.c:43 [inline]
do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
SYSC_ioctl fs/ioctl.c:698 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
entry_SYSCALL_64_fastpath+0x1f/0xc2
Memory state around the buggy address:
ffff880063de5900: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff880063de5980: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff880063de5a00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff880063de5a80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff880063de5b00: fb fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc
==================================================================


==================================================================
BUG: KASAN: use-after-free in worker_thread+0x190f/0x1990
kernel/workqueue.c:2228 at addr ffff88005378ed20
Read of size 8 by task kworker/u8:2/6942
CPU: 3 PID: 6942 Comm: kworker/u8:2 Not tainted 4.10.0+ #260
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:15 [inline]
dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
print_address_description mm/kasan/report.c:204 [inline]
kasan_report_error mm/kasan/report.c:288 [inline]
kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
kasan_report mm/kasan/report.c:331 [inline]
__asan_report_load8_noabort+0x29/0x30 mm/kasan/report.c:331
worker_thread+0x190f/0x1990 kernel/workqueue.c:2228
kthread+0x326/0x3f0 kernel/kthread.c:227
ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
Object at ffff88005378ec00, in cache kcm_psock_cache size: 616
Allocated:
PID = 3023
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
save_stack+0x43/0xd0 mm/kasan/kasan.c:502
set_track mm/kasan/kasan.c:514 [inline]
kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
kmem_cache_zalloc include/linux/slab.h:653 [inline]
kcm_attach net/kcm/kcmsock.c:1384 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1455 [inline]
kcm_ioctl+0x303/0x1800 net/kcm/kcmsock.c:1690
sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993
vfs_ioctl fs/ioctl.c:43 [inline]
do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
SYSC_ioctl fs/ioctl.c:698 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 6942
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
save_stack+0x43/0xd0 mm/kasan/kasan.c:502
set_track mm/kasan/kasan.c:514 [inline]
kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
__cache_free mm/slab.c:3513 [inline]
kmem_cache_free+0x71/0x240 mm/slab.c:3773
unreserve_psock+0x585/0x760 net/kcm/kcmsock.c:545
kcm_write_msgs+0xbba/0x1b90 net/kcm/kcmsock.c:588
kcm_tx_work+0x2e/0x190 net/kcm/kcmsock.c:729
process_one_work+0xbd0/0x1c10 kernel/workqueue.c:2096
worker_thread+0x223/0x1990 kernel/workqueue.c:2230
kthread+0x326/0x3f0 kernel/kthread.c:227
ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
Memory state around the buggy address:
ffff88005378ec00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff88005378ec80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff88005378ed00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff88005378ed80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff88005378ee00: fb fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc
==================================================================

[Dmitry Vyukov]

Also like this one:

==================================================================
BUG: KASAN: use-after-free in atomic_long_read
include/linux/compiler.h:254 [inline] at addr ffff8800538aba60
BUG: KASAN: use-after-free in get_work_pool+0x2f2/0x340
kernel/workqueue.c:709 at addr ffff8800538aba60
Read of size 8 by task syz-executor6/7965
CPU: 2 PID: 7965 Comm: syz-executor6 Not tainted 4.10.0+ #248

Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:15 [inline]
dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
print_address_description mm/kasan/report.c:204 [inline]
kasan_report_error mm/kasan/report.c:288 [inline]
kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
kasan_report mm/kasan/report.c:331 [inline]
__asan_report_load8_noabort+0x29/0x30 mm/kasan/report.c:331

atomic_long_read include/linux/compiler.h:254 [inline]
get_work_pool+0x2f2/0x340 kernel/workqueue.c:709
__queue_work+0x2b3/0x1210 kernel/workqueue.c:1401
queue_work_on+0x2e9/0x330 kernel/workqueue.c:1486
queue_work include/linux/workqueue.h:487 [inline]
strp_check_rcv+0x25/0x30 net/strparser/strparser.c:494
kcm_attach net/kcm/kcmsock.c:1434 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1455 [inline]
kcm_ioctl+0x8bb/0x1800 net/kcm/kcmsock.c:1690

sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993
vfs_ioctl fs/ioctl.c:43 [inline]
do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
SYSC_ioctl fs/ioctl.c:698 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
entry_SYSCALL_64_fastpath+0x1f/0xc2

RIP: 0033:0x4458d9
RSP: 002b:00007f1dce9d1b58 EFLAGS: 00000286 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000024 RCX: 00000000004458d9
RDX: 0000000020b68000 RSI: 00000000000089e0 RDI: 0000000000000024
RBP: 00000000006e0220 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000286 R12: 00000000007080a8
R13: 0000000000000000 R14: 00007f1dce9d29c0 R15: 00007f1dce9d2700
Object at ffff8800538ab940, in cache kcm_psock_cache size: 616
Allocated:
PID = 7965

save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
save_stack+0x43/0xd0 mm/kasan/kasan.c:502
set_track mm/kasan/kasan.c:514 [inline]
kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
kmem_cache_zalloc include/linux/slab.h:653 [inline]
kcm_attach net/kcm/kcmsock.c:1384 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1455 [inline]
kcm_ioctl+0x303/0x1800 net/kcm/kcmsock.c:1690
sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993
vfs_ioctl fs/ioctl.c:43 [inline]
do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
SYSC_ioctl fs/ioctl.c:698 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:

PID = 7982

save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
save_stack+0x43/0xd0 mm/kasan/kasan.c:502
set_track mm/kasan/kasan.c:514 [inline]
kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
__cache_free mm/slab.c:3513 [inline]
kmem_cache_free+0x71/0x240 mm/slab.c:3773
kcm_unattach+0xee7/0x1520 net/kcm/kcmsock.c:1558
kcm_unattach_ioctl net/kcm/kcmsock.c:1603 [inline]
kcm_ioctl+0xfae/0x1800 net/kcm/kcmsock.c:1700
sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993
vfs_ioctl fs/ioctl.c:43 [inline]
do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
SYSC_ioctl fs/ioctl.c:698 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
entry_SYSCALL_64_fastpath+0x1f/0xc2
Memory state around the buggy address:

ffff8800538ab900: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
ffff8800538ab980: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff8800538aba00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff8800538aba80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8800538abb00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================

[Dmitry Vyukov]

FTR, this is reproducible by running the following syzkaller program:

mmap(&(0x7f0000000000/0xfbb000)=nil, (0xfbb000), 0x3, 0x32,
0xffffffffffffffff, 0x0)
r0 = socket(0x0, 0x0, 0x0)
r1 = socket(0xa, 0x1, 0x0)
setsockopt$NETROM_N2(r1, 0x103, 0x3, &(0x7f000072a000)=0x7, 0x4)
r2 = bpf$PROG_LOAD(0x5, &(0x7f0000b50000-0x30)={0x1, 0x2,
&(0x7f0000b4f000)=[@generic={0xffffffffffffffb7, 0x0, 0x0, 0x0},
@generic={0x9595, 0x0, 0x0, 0x0}], &(0x7f000015a000)="008b", 0x9,
0x80, &(0x7f0000b4f000)="0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
0x0}, 0x30)
pread64(r1, &(0x7f0000fb8000)="", 0x0, 0x38)
mmap(&(0x7f0000fbb000/0x1000)=nil, (0x1000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
setsockopt$SCTP_AUTO_ASCONF(r1, 0x84, 0x1e, &(0x7f0000fbc000-0x4)=0x7, 0x4)
r3 = dup3(r0, r1, 0x0)
ioctl$UFFDIO_API(r2, 0xc018aa3f, &(0x7f000073d000)={0xaa, 0x0, 0x0})
recvmsg$kcm(r3, &(0x7f0000fb9000)={0x0, 0x0,
&(0x7f0000fba000-0x40)=[{&(0x7f00001c0000)="", 0x0},
{&(0x7f0000fba000-0x43)="", 0x0}, {&(0x7f00009ec000)="", 0x0},
{&(0x7f0000fb9000)="", 0x0}], 0x4, &(0x7f0000fba000-0xd6)="", 0x0,
0x7}, 0x2)
bpf$PROG_LOAD(0x5, &(0x7f0000fb8000)={0x80, 0x1,
&(0x7f0000302000-0x28)=[@generic={0x0, 0xc00000, 0x0, 0x0}],
&(0x7f0000a85000)="008b", 0x0, 0x0, &(0x7f0000fb5000)="", 0x9}, 0x30)
r4 = socket$kcm(0x29, 0x5, 0x0)
write(r4, &(0x7f0000fb3000-0xe9)="", 0x0)
ioctl$SIOCSIFHWADDR(r3, 0x8924,
&(0x7f00008d8000)={"4682049c0228afcdfa4945e37a244a3a",
@data="ecefff20cec4768429383ab454036dad"})
ioctl$TUNSETPERSIST(r3, 0x400454cb, &(0x7f0000fbb000-0x4)=0x1)
mmap(&(0x7f0000fbc000/0x1000)=nil, (0x1000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
mmap(&(0x7f0000fbc000/0x1000)=nil, (0x1000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
getsockopt$dccp6_int(r3, 0x21, 0x1, &(0x7f0000fbc000)=0x0,
&(0x7f0000fbd000-0x4)=0x4)
ioctl$SIOCKCMATTACH(r4, 0x89e0, &(0x7f0000b5c000)={r1, r2})
ioctl$SIOCKCMUNATTACH(r4, 0x89e1, &(0x7f0000fb5000)={r1})

[Cong Wang]

It is not kcm_wq, it is strp_wq, and the work struct is strp->rx_work
which lives in struct kcm_psock. The work is cancelled by strp_done(),
it seems get queued again after strp_done()...

[Dmitry Vyukov]

on 093b995e3b55a0ae0670226ddfcb05bfbf0099ae:

==================================================================
BUG: KASAN: use-after-free in worker_thread+0x1024/0x1340
kernel/workqueue.c:2229 at addr ffff88006d164ae0
Read of size 8 by task kworker/u8:3/25139
CPU: 2 PID: 25139 Comm: kworker/u8:3 Not tainted 4.11.0-rc3+ #364

Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:

__dump_stack lib/dump_stack.c:16 [inline]
dump_stack+0x1b8/0x28d lib/dump_stack.c:52
kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
print_address_description mm/kasan/report.c:210 [inline]
kasan_report_error mm/kasan/report.c:294 [inline]
kasan_report.part.2+0x1be/0x480 mm/kasan/report.c:316
kasan_report mm/kasan/report.c:337 [inline]
__asan_report_load8_noabort+0x29/0x30 mm/kasan/report.c:337
worker_thread+0x1024/0x1340 kernel/workqueue.c:2229
kthread+0x359/0x420 kernel/kthread.c:229
ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
Object at ffff88006d1649c0, in cache kcm_psock_cache size: 616
Allocated:
PID = 25123
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x43/0xd0 mm/kasan/kasan.c:517
set_track mm/kasan/kasan.c:529 [inline]
kasan_kmalloc+0xbc/0xf0 mm/kasan/kasan.c:620
kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:559
kmem_cache_alloc+0x110/0x720 mm/slab.c:3572
kmem_cache_zalloc include/linux/slab.h:653 [inline]
kcm_attach net/kcm/kcmsock.c:1386 [inline]
kcm_attach_ioctl net/kcm/kcmsock.c:1457 [inline]
kcm_ioctl+0x2bc/0x17e0 net/kcm/kcmsock.c:1692

sock_do_ioctl+0x65/0xb0 net/socket.c:895
sock_ioctl+0x2c2/0x440 net/socket.c:993

vfs_ioctl fs/ioctl.c:45 [inline]
do_vfs_ioctl+0x1af/0x16d0 fs/ioctl.c:685
SYSC_ioctl fs/ioctl.c:700 [inline]
SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 25139
save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
save_stack+0x43/0xd0 mm/kasan/kasan.c:517
set_track mm/kasan/kasan.c:529 [inline]
kasan_slab_free+0x81/0xc0 mm/kasan/kasan.c:593
__cache_free mm/slab.c:3514 [inline]
kmem_cache_free+0x71/0x240 mm/slab.c:3774
unreserve_psock+0x5d4/0x7b0 net/kcm/kcmsock.c:547
kcm_write_msgs+0xba6/0x1ba0 net/kcm/kcmsock.c:590
kcm_tx_work+0x32/0x1f0 net/kcm/kcmsock.c:731
process_one_work+0xb20/0x1b40 kernel/workqueue.c:2097
worker_thread+0x1b4/0x1340 kernel/workqueue.c:2231
kthread+0x359/0x420 kernel/kthread.c:229
ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430

Memory state around the buggy address:

ffff88006d164980: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
ffff88006d164a00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff88006d164a80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff88006d164b00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff88006d164b80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================