WARNING: kmalloc bug in xdp_umem_create

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: 7d3bf613e99a Merge tag 'libnvdimm-for-4.18' of git://git.k..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1073f68f800000
kernel config: https://syzkaller.appspot.com/x/.config?x=f04d8d0a2afb789a
dashboard link: https://syzkaller.appspot.com/bug?extid=4abadc5d69117b346506
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syzkaller repro:https://syzkaller.appspot.com/x/repro.syz?x=13c9756f800000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16366f9f800000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+4abadc...@syzkaller.appspotmail.com

random: sshd: uninitialized urandom read (32 bytes read)
random: sshd: uninitialized urandom read (32 bytes read)
random: sshd: uninitialized urandom read (32 bytes read)
random: sshd: uninitialized urandom read (32 bytes read)
random: sshd: uninitialized urandom read (32 bytes read)
WARNING: CPU: 1 PID: 4537 at mm/slab_common.c:996 kmalloc_slab+0x56/0x70
mm/slab_common.c:996
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 4537 Comm: syz-executor849 Not tainted 4.17.0+ #92
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1b9/0x294 lib/dump_stack.c:113
panic+0x22f/0x4de kernel/panic.c:184
__warn.cold.8+0x163/0x1b3 kernel/panic.c:536
report_bug+0x252/0x2d0 lib/bug.c:186
fixup_bug arch/x86/kernel/traps.c:178 [inline]
do_error_trap+0x1fc/0x4d0 arch/x86/kernel/traps.c:296
do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:316
invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:992
RIP: 0010:kmalloc_slab+0x56/0x70 mm/slab_common.c:996
Code: c5 c0 ca d0 88 5d c3 b8 10 00 00 00 48 85 ff 74 f4 83 ef 01 c1 ef 03
0f b6 87 e0 c9 d0 88 eb d8 31 c0 81 e6 00 02 00 00 75 db <0f> 0b 5d c3 48
8b 04 c5 00 ca d0 88 5d c3 66 90 66 2e 0f 1f 84 00
RSP: 0018:ffff8801acc67998 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffffffff877abea2
RDX: 1ffff10035e17ce3 RSI: 0000000000000000 RDI: 0000000001000010
RBP: ffff8801acc67998 R08: ffff8801d91d82c0 R09: ffffed0035e17cd9
R10: ffffed0035e17cd9 R11: ffff8801af0be6cb R12: dffffc0000000000
R13: 0000000020000000 R14: ffff8801af0be6b0 R15: 00000000006080c0
__do_kmalloc mm/slab.c:3713 [inline]
__kmalloc+0x25/0x760 mm/slab.c:3727
kmalloc_array include/linux/slab.h:634 [inline]
kcalloc include/linux/slab.h:645 [inline]
xdp_umem_pin_pages net/xdp/xdp_umem.c:205 [inline]
xdp_umem_reg net/xdp/xdp_umem.c:318 [inline]
xdp_umem_create+0x5c9/0x10f0 net/xdp/xdp_umem.c:349
xsk_setsockopt+0x443/0x550 net/xdp/xsk.c:531
__sys_setsockopt+0x1bd/0x390 net/socket.c:1935
__do_sys_setsockopt net/socket.c:1946 [inline]
__se_sys_setsockopt net/socket.c:1943 [inline]
__x64_sys_setsockopt+0xbe/0x150 net/socket.c:1943
do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x43fce9
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 6b 45 00 00 c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffedcafaac8 EFLAGS: 00000213 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 00000000004002c8 RCX: 000000000043fce9
RDX: 0000000000000004 RSI: 000000000000011b RDI: 0000000000000003
RBP: 00000000006ca018 R08: 0000000000000018 R09: 00000000004002c8
R10: 0000000020000040 R11: 0000000000000213 R12: 0000000000401610
R13: 00000000004016a0 R14: 0000000000000000 R15: 0000000000000000
Dumping ftrace buffer:
(ftrace buffer empty)
Kernel Offset: disabled
Rebooting in 86400 seconds..


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches

[Tetsuo Handa]

On 2018/06/10 7:47, syzbot wrote:
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit:    7d3bf613e99a Merge tag 'libnvdimm-for-4.18' of git://git.k..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=1073f68f800000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=f04d8d0a2afb789a
> dashboard link: https://syzkaller.appspot.com/bug?extid=4abadc5d69117b346506
> compiler:       gcc (GCC) 8.0.1 20180413 (experimental)
> syzkaller repro:https://syzkaller.appspot.com/x/repro.syz?x=13c9756f800000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=16366f9f800000
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+4abadc...@syzkaller.appspotmail.com
>
> random: sshd: uninitialized urandom read (32 bytes read)
> random: sshd: uninitialized urandom read (32 bytes read)
> random: sshd: uninitialized urandom read (32 bytes read)
> random: sshd: uninitialized urandom read (32 bytes read)
> random: sshd: uninitialized urandom read (32 bytes read)
> WARNING: CPU: 1 PID: 4537 at mm/slab_common.c:996 kmalloc_slab+0x56/0x70 mm/slab_common.c:996
> Kernel panic - not syncing: panic_on_warn set ...

syzbot gave up upon kmalloc(), but actually error handling path has
NULL pointer dereference bug.

----------
#include <sys/socket.h>
#include <unistd.h>
#define PF_XDP 44
#define SOL_XDP 283
#define XDP_UMEM_REG 4

int main(int argc, char *argv[])
{
int fd = socket(PF_XDP, SOCK_RAW, 0);
struct xdp_umem_reg {
unsigned long long addr;
unsigned long long len;
unsigned int chunk_size;
unsigned int headroom;
} arg = {
0x20000000,
0x200002000,
0x800,
2
};
setsockopt(fd, SOL_XDP, XDP_UMEM_REG, &arg, sizeof(arg));
return 0;
}
----------

[ 95.172962] WARNING: CPU: 3 PID: 2891 at mm/page_alloc.c:4065 __alloc_pages_nodemask+0x283/0xdf0
[ 95.175179] Modules linked in: pcspkr sg vmw_vmci i2c_piix4 sd_mod ata_generic pata_acpi ahci libahci vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi i2c_core mptscsih e1000 mptbase libata serio_raw
[ 95.180614] CPU: 3 PID: 2891 Comm: a.out Kdump: loaded Not tainted 4.17.0+ #421
[ 95.182351] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
[ 95.184909] RIP: 0010:__alloc_pages_nodemask+0x283/0xdf0
[ 95.186319] Code: 00 00 04 00 41 0f 44 c6 48 3b 5c 24 78 c6 84 24 90 00 00 00 00 0f 85 50 0b 00 00 41 83 fd 0a 76 1d f6 c4 02 0f 85 3b ff ff ff <0f> 0b e9 34 ff ff ff 0f 0b 0f 1f 40 00 e9 10 fe ff ff 0f 0b 89 c2
[ 95.190997] RSP: 0018:ffffc900008efd20 EFLAGS: 00010246
[ 95.192257] RAX: 000000000060c0c0 RBX: 0000000000000000 RCX: ffff88013f7fe920
[ 95.194005] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000000
[ 95.195697] RBP: 000000000060c0c0 R08: 0000000000000001 R09: ffffffffffffef81
[ 95.197393] R10: 000000000000000d R11: 0000000000000e8c R12: 0000000000000001
[ 95.199084] R13: 000000000000000d R14: 000000000060c0c0 R15: 0000000000000000
[ 95.200735] FS: 00007f8387e61740(0000) GS:ffff88013f4c0000(0000) knlGS:0000000000000000
[ 95.203441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 95.205726] CR2: 0000000020000040 CR3: 0000000133e2c006 CR4: 00000000001606e0
[ 95.207743] Call Trace:
[ 95.208427] ? __lock_acquire+0x22a/0x1830
[ 95.209391] ? kmalloc_order+0x15/0x60
[ 95.210266] ? __kmalloc+0x20a/0x210
[ 95.211104] ? xdp_umem_create+0x16e/0x3c0
[ 95.212095] ? xsk_setsockopt+0x153/0x1a0
[ 95.213143] ? __sys_setsockopt+0x67/0xb0
[ 95.214058] ? __x64_sys_setsockopt+0x1b/0x20
[ 95.215040] ? do_syscall_64+0x4f/0x1f0
[ 95.215890] ? entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 95.217079] irq event stamp: 5296
[ 95.217785] hardirqs last enabled at (5295): [<ffffffff810b2a77>] __raw_spin_lock_init+0x17/0x50
[ 95.220381] hardirqs last disabled at (5296): [<ffffffff81800f33>] error_entry+0x73/0xc0
[ 95.222447] softirqs last enabled at (5284): [<ffffffff81a00183>] __do_softirq+0x183/0x204
[ 95.224328] softirqs last disabled at (5277): [<ffffffff81061bcd>] irq_exit+0xcd/0xf0
[ 95.226065] ---[ end trace 75b6f67917663997 ]---
[ 95.227250] BUG: unable to handle kernel NULL pointer dereference at 0000000000000060
[ 95.229101] PGD 1342eb067 P4D 1342eb067 PUD 1314a2067 PMD 0
[ 95.230398] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[ 95.231418] CPU: 3 PID: 2891 Comm: a.out Kdump: loaded Tainted: G W 4.17.0+ #421
[ 95.233474] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
[ 95.236636] RIP: 0010:xdp_umem_create+0x228/0x3c0
[ 95.237867] Code: f4 ff ff ff e8 b9 f9 ff ff 48 8b bb 90 00 00 00 e8 3d d9 a7 ff 48 c7 83 90 00 00 00 00 00 00 00 48 8b 43 30 8b 93 98 00 00 00 <f0> 48 29 50 60 48 8b 7b 30 49 63 ec e8 57 10 92 ff 48 8b 7b 38 e8
[ 95.241945] RSP: 0018:ffffc900008efe88 EFLAGS: 00010246
[ 95.243236] RAX: 0000000000000000 RBX: ffff880133401288 RCX: 000000000060c0c0
[ 95.244789] RDX: 0000000000200002 RSI: 0000000001000010 RDI: 0000000000000000
[ 95.247382] RBP: 0000000000200002 R08: 0000000000000001 R09: ffffffffffffef81
[ 95.249735] R10: 000000000000000d R11: 0000000000000e8c R12: 00000000fffffff4
[ 95.252391] R13: 0000000000000040 R14: 0000000020000000 R15: 00000000000007c0
[ 95.255280] FS: 00007f8387e61740(0000) GS:ffff88013f4c0000(0000) knlGS:0000000000000000
[ 95.257918] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 95.260068] CR2: 0000000000000060 CR3: 0000000133e2c006 CR4: 00000000001606e0
[ 95.262535] Call Trace:
[ 95.263900] ? xsk_setsockopt+0x153/0x1a0
[ 95.265495] ? __sys_setsockopt+0x67/0xb0
[ 95.267108] ? __x64_sys_setsockopt+0x1b/0x20
[ 95.269532] ? do_syscall_64+0x4f/0x1f0
[ 95.271474] ? entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 95.273292] Modules linked in: pcspkr sg vmw_vmci i2c_piix4 sd_mod ata_generic pata_acpi ahci libahci vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi i2c_core mptscsih e1000 mptbase libata serio_raw
[ 95.279548] CR2: 0000000000000060
[ 95.281044] ---[ end trace 75b6f67917663998 ]---
[ 95.283132] RIP: 0010:xdp_umem_create+0x228/0x3c0
[ 95.285257] Code: f4 ff ff ff e8 b9 f9 ff ff 48 8b bb 90 00 00 00 e8 3d d9 a7 ff 48 c7 83 90 00 00 00 00 00 00 00 48 8b 43 30 8b 93 98 00 00 00 <f0> 48 29 50 60 48 8b 7b 30 49 63 ec e8 57 10 92 ff 48 8b 7b 38 e8
[ 95.291487] RSP: 0018:ffffc900008efe88 EFLAGS: 00010246
[ 95.293429] RAX: 0000000000000000 RBX: ffff880133401288 RCX: 000000000060c0c0
[ 95.295761] RDX: 0000000000200002 RSI: 0000000001000010 RDI: 0000000000000000
[ 95.298072] RBP: 0000000000200002 R08: 0000000000000001 R09: ffffffffffffef81
[ 95.300403] R10: 000000000000000d R11: 0000000000000e8c R12: 00000000fffffff4
[ 95.303699] R13: 0000000000000040 R14: 0000000020000000 R15: 00000000000007c0
[ 95.306178] FS: 00007f8387e61740(0000) GS:ffff88013f4c0000(0000) knlGS:0000000000000000
[ 95.308645] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 95.310782] CR2: 0000000000000060 CR3: 0000000133e2c006 CR4: 00000000001606e0

xdp_umem_create+0x228/0x3c0:
arch_atomic64_sub at arch/x86/include/asm/atomic64_64.h:60
(inlined by) atomic64_sub at include/asm-generic/atomic-instrumented.h:145
(inlined by) atomic_long_sub at include/asm-generic/atomic-long.h:199
(inlined by) xdp_umem_unaccount_pages at net/xdp/xdp_umem.c:135
(inlined by) xdp_umem_reg at net/xdp/xdp_umem.c:334
(inlined by) xdp_umem_create at net/xdp/xdp_umem.c:349

[Björn Töpel]

Den sön 10 juni 2018 kl 04:53 skrev Tetsuo Handa
<penguin...@i-love.sakura.ne.jp>:

>
> On 2018/06/10 7:47, syzbot wrote:
> > Hello,
> >
> > syzbot found the following crash on:
> >
> > HEAD commit: 7d3bf613e99a Merge tag 'libnvdimm-for-4.18' of git://git.k..
> > git tree: upstream
> > console output: https://syzkaller.appspot.com/x/log.txt?x=1073f68f800000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=f04d8d0a2afb789a
> > dashboard link: https://syzkaller.appspot.com/bug?extid=4abadc5d69117b346506
> > compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> > syzkaller repro:https://syzkaller.appspot.com/x/repro.syz?x=13c9756f800000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16366f9f800000
> >
> > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > Reported-by: syzbot+4abadc...@syzkaller.appspotmail.com
> >
> > random: sshd: uninitialized urandom read (32 bytes read)
> > random: sshd: uninitialized urandom read (32 bytes read)
> > random: sshd: uninitialized urandom read (32 bytes read)
> > random: sshd: uninitialized urandom read (32 bytes read)
> > random: sshd: uninitialized urandom read (32 bytes read)
> > WARNING: CPU: 1 PID: 4537 at mm/slab_common.c:996 kmalloc_slab+0x56/0x70 mm/slab_common.c:996
> > Kernel panic - not syncing: panic_on_warn set ...
>
> syzbot gave up upon kmalloc(), but actually error handling path has
> NULL pointer dereference bug.
>

Thanks Tetsuo! This crash has been fixed by Daniel Borkmann in commit
c09290c56376 ("bpf, xdp: fix crash in xdp_umem_unaccount_pages").


Björn

[Dmitry Vyukov]

Let's tell syzbot about this:

#syz fix: bpf, xdp: fix crash in xdp_umem_unaccount_pages

[Tetsuo Handa]

Excuse me, but that patch fixes NULL pointer dereference which occurs after kmalloc()'s

"WARNING: CPU: 1 PID: 4537 at mm/slab_common.c:996 kmalloc_slab+0x56/0x70 mm/slab_common.c:996"

message. That is, "Too large memory allocation" itself is not yet fixed.

[Dmitry Vyukov]

You are right! I fixed it up. Thanks

[Björn Töpel]

Den sön 10 juni 2018 kl 14:53 skrev Tetsuo Handa

The code relies on that the sl{u,a,o}b layer says no, and the
setsockopt bails out. The warning could be opted out using
__GFP_NOWARN. Is there another preferred way? Two get_user_pages
calls, where the first call would set pages to NULL just to fault the
region? Walk the process' VMAs? Something else?


Björn

[Dmitry Vyukov]

Hi Björn,

Yes, either __GFP_NOWARN for allocations with user-controllable size
or stricter custom limit (if we don't want current sla/u/ob
implementation details to be part of public kernel interface).

[Daniel Borkmann]

(Now resolved as well.)

#syz fix: xsk: silence warning on memory allocation failure