BUG: using __this_cpu_read() in preemptible code in ip6_finish_output

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: e1427237 macsec: add noinline tag to avoid a frame size wa..
git tree: net-next
console output: https://syzkaller.appspot.com/x/log.txt?x=15594553200000
kernel config: https://syzkaller.appspot.com/x/.config?x=8e9bc94c16d346a6
dashboard link: https://syzkaller.appspot.com/bug?extid=51471b4aae195285a4a3
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=156ffb07200000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14412673200000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+51471b...@syzkaller.appspotmail.com

BUG: using __this_cpu_read() in preemptible [00000000] code:
syz-executor222/7596
caller is dev_recursion_level include/linux/netdevice.h:3052 [inline]
caller is ip6_skb_dst_mtu include/net/ip6_route.h:245 [inline]
caller is ip6_finish_output+0x335/0xdc0 net/ipv6/ip6_output.c:149
CPU: 1 PID: 7596 Comm: syz-executor222 Not tainted 5.1.0-rc2+ #118
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x172/0x1f0 lib/dump_stack.c:113
__this_cpu_preempt_check+0x246/0x270 lib/smp_processor_id.c:47
dev_recursion_level include/linux/netdevice.h:3052 [inline]
ip6_skb_dst_mtu include/net/ip6_route.h:245 [inline]
ip6_finish_output+0x335/0xdc0 net/ipv6/ip6_output.c:149
NF_HOOK_COND include/linux/netfilter.h:278 [inline]
ip6_output+0x235/0x7f0 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:433 [inline]
NF_HOOK include/linux/netfilter.h:289 [inline]
NF_HOOK include/linux/netfilter.h:283 [inline]
ip6_xmit+0xe41/0x20c0 net/ipv6/ip6_output.c:275
inet6_csk_xmit+0x2fb/0x5d0 net/ipv6/inet6_connection_sock.c:139
__tcp_transmit_skb+0x1a32/0x3750 net/ipv4/tcp_output.c:1155
tcp_transmit_skb net/ipv4/tcp_output.c:1171 [inline]
tcp_send_syn_data net/ipv4/tcp_output.c:3494 [inline]
tcp_connect+0x1e47/0x4280 net/ipv4/tcp_output.c:3533
tcp_v6_connect+0x150b/0x20a0 net/ipv6/tcp_ipv6.c:331
__inet_stream_connect+0x83f/0xea0 net/ipv4/af_inet.c:659
tcp_sendmsg_fastopen net/ipv4/tcp.c:1155 [inline]
tcp_sendmsg_locked+0x231f/0x37f0 net/ipv4/tcp.c:1197
tcp_sendmsg+0x30/0x50 net/ipv4/tcp.c:1431
inet_sendmsg+0x147/0x5e0 net/ipv4/af_inet.c:802
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xdd/0x130 net/socket.c:661
__sys_sendto+0x262/0x380 net/socket.c:1932
__do_sys_sendto net/socket.c:1944 [inline]
__se_sys_sendto net/socket.c:1940 [inline]
__x64_sys_sendto+0xe1/0x1a0 net/socket.c:1940
do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x440189
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 fb 13 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffed7abd1a8 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00000000004002c8 RCX: 0000000000440189
RDX: 0000000000000000 RSI: 0000000000000000 RD


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches

[syzbot]

syzbot has bisected this bug to:

commit 97cdcf37b57e3f204be3000b9eab9686f38b4356
Author: Florian Westphal <f...@strlen.de>
Date: Mon Apr 1 14:42:13 2019 +0000

net: place xmit recursion in softnet data

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11c04b0f200000
start commit: e1427237 macsec: add noinline tag to avoid a frame size wa..
git tree: net-next
final crash: https://syzkaller.appspot.com/x/report.txt?x=13c04b0f200000
console output: https://syzkaller.appspot.com/x/log.txt?x=15c04b0f200000

kernel config: https://syzkaller.appspot.com/x/.config?x=8e9bc94c16d346a6
dashboard link: https://syzkaller.appspot.com/bug?extid=51471b4aae195285a4a3

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=156ffb07200000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14412673200000

Reported-by: syzbot+51471b...@syzkaller.appspotmail.com
Fixes: 97cdcf37b57e ("net: place xmit recursion in softnet data")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Krzysztof Kozlowski]

On Wed, 3 Apr 2019 at 03:14, syzbot
<syzbot+51471b...@syzkaller.appspotmail.com> wrote:
>
> syzbot has bisected this bug to:
>
> commit 97cdcf37b57e3f204be3000b9eab9686f38b4356
> Author: Florian Westphal <f...@strlen.de>
> Date: Mon Apr 1 14:42:13 2019 +0000
>
> net: place xmit recursion in softnet data

I am seeing this as well on ARMv7 board booted from NFS root (exynos_defconfig):

[ 30.221238] BUG: using __this_cpu_read() in preemptible [00000000]
code: systemd-network/236
[ 30.228576] caller is ip6_output+0x68/0x3e8
[ 30.232578] CPU: 1 PID: 236 Comm: systemd-network Not tainted
5.1.0-rc3-next-20190405 #2
[ 30.240657] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
[ 30.246719] [<c011238c>] (unwind_backtrace) from [<c010df50>]
(show_stack+0x10/0x14)
[ 30.254447] [<c010df50>] (show_stack) from [<c0a87270>] (dump_stack+0x98/0xc4)
[ 30.261638] [<c0a87270>] (dump_stack) from [<c0494a94>]
(__this_cpu_preempt_check+0x124/0x128)
[ 30.270238] [<c0494a94>] (__this_cpu_preempt_check) from [<c08e486c>]
(ip6_output+0x68/0x3e8)
[ 30.278730] [<c08e486c>] (ip6_output) from [<c08e52ac>]
(ip6_send_skb+0x30/0x1d0)
[ 30.286180] [<c08e52ac>] (ip6_send_skb) from [<c0912798>]
(rawv6_sendmsg+0x824/0x9c0)
[ 30.294005] [<c0912798>] (rawv6_sendmsg) from [<c07d8ff8>]
(sock_sendmsg+0x14/0x24)
[ 30.301628] [<c07d8ff8>] (sock_sendmsg) from [<c07d97cc>]
(___sys_sendmsg+0x230/0x244)
[ 30.309514] [<c07d97cc>] (___sys_sendmsg) from [<c07da9f8>]
(__sys_sendmsg+0x50/0x8c)
[ 30.317316] [<c07da9f8>] (__sys_sendmsg) from [<c01011ac>]
(__sys_trace_return+0x0/0x14)
[ 30.325376] Exception stack(0xe79f9fa8 to 0xe79f9ff0)
[ 30.330377] 9fa0: 00000000 b6f3da58 0000000b beeb2a6c 00000000 00000000
[ 30.338560] 9fc0: 00000000 b6f3da58 004015ce 00000128 004b10c8
00000000 020d28f9 00000000
[ 30.346708] 9fe0: 004aeee0 beeb2a40 0047cbdc b6d27684

Full log:
https://krzk.eu/#/builders/22/builds/1055/steps/13/logs/serial0

Best regards,
Krzysztof Kozlowski

[Florian Westphal]

Krzysztof Kozlowski <kr...@kernel.org> wrote:
> On Wed, 3 Apr 2019 at 03:14, syzbot
> <syzbot+51471b...@syzkaller.appspotmail.com> wrote:
> >
> > syzbot has bisected this bug to:
> >
> > commit 97cdcf37b57e3f204be3000b9eab9686f38b4356
> > Author: Florian Westphal <f...@strlen.de>
> > Date: Mon Apr 1 14:42:13 2019 +0000
> >
> > net: place xmit recursion in softnet data
>
> I am seeing this as well on ARMv7 board booted from NFS root (exynos_defconfig):

Sorry about this, the fix is now in net-next though:
28b05b92886871bdd ("net: use correct this_cpu primitive in dev_recursion_level")

[Dmitry Vyukov]

Krzysztof, just in case, you can see the current bug status on
dashboard as well. E.g. for this one it says:

Fix commit: net: use correct this_cpu primitive in dev_recursion_level