WARNING: suspicious RCU usage in ovs_flow_tbl_destroy

11 views

Skip to first unread message

syzbot

unread,

Aug 4, 2020, 11:35:31 AM8/4/20

to da...@davemloft.net, d...@openvswitch.org, ku...@kernel.org, linux-...@vger.kernel.org, net...@vger.kernel.org, psh...@ovn.org, syzkall...@googlegroups.com

Hello,

syzbot found the following issue on:

HEAD commit: 2f631133 net: Pass NULL to skb_network_protocol() when we ..
git tree: net-next
console output: https://syzkaller.appspot.com/x/log.txt?x=12daae2a900000
kernel config: https://syzkaller.appspot.com/x/.config?x=91a13b78c7dc258d
dashboard link: https://syzkaller.appspot.com/bug?extid=c0eb9e7cdde04e4eb4be
compiler: gcc (GCC) 10.1.0-syz 20200507
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=172cd3d4900000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=11fbd342900000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+c0eb9e...@syzkaller.appspotmail.com

netlink: 'syz-executor399': attribute type 2 has an invalid length.
=============================
WARNING: suspicious RCU usage
5.8.0-rc7-syzkaller #0 Not tainted
-----------------------------
net/openvswitch/flow_table.c:521 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1
1 lock held by syz-executor399/6802:
#0: ffffffff8a8319b0 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 net/netlink/genetlink.c:741

stack backtrace:
CPU: 0 PID: 6802 Comm: syz-executor399 Not tainted 5.8.0-rc7-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x18f/0x20d lib/dump_stack.c:118
ovs_flow_tbl_destroy+0x1d6/0x210 net/openvswitch/flow_table.c:521
ovs_dp_cmd_new+0x8ca/0xea0 net/openvswitch/datapath.c:1747
genl_family_rcv_msg_doit net/netlink/genetlink.c:669 [inline]
genl_family_rcv_msg net/netlink/genetlink.c:714 [inline]
genl_rcv_msg+0x61d/0x980 net/netlink/genetlink.c:731
netlink_rcv_skb+0x15a/0x430 net/netlink/af_netlink.c:2470
genl_rcv+0x24/0x40 net/netlink/genetlink.c:742
netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:671
____sys_sendmsg+0x6e8/0x810 net/socket.c:2359
___sys_sendmsg+0xf3/0x170 net/socket.c:2413
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2446
do_syscall_64+0x60/0xe0 arch/x86/entry/common.c:384
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x4402d9
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b 13 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffdfda722f8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00000000004002c8 RCX: 00000000004402d9
RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000003
RBP: 00000000006ca018 R08: 0000000000000000 R09: 00000000004002c8
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000401ae0
R13: 0000000000401b70 R14: 0000000000000000 R15: 0000000000000000

=============================
WARNING: suspicious RCU usage
5.8.0-rc7-syzkaller #0 Not tainted
-----------------------------
net/openvswitch/flow_table.c:522 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1
1 lock held by syz-executor399/6802:
#0: ffffffff8a8319b0 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 net/netlink/genetlink.c:741

stack backtrace:
CPU: 0 PID: 6802 Comm: syz-executor399 Not tainted 5.8.0-rc7-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x18f/0x20d lib/dump_stack.c:118
ovs_flow_tbl_destroy+0x190/0x210 net/openvswitch/flow_table.c:522
ovs_dp_cmd_new+0x8ca/0xea0 net/openvswitch/datapath.c:1747
genl_family_rcv_msg_doit net/netlink/genetlink.c:669 [inline]
genl_family_rcv_msg net/netlink/genetlink.c:714 [inline]
genl_rcv_msg+0x61d/0x980 net/netlink/genetlink.c:731
netlink_rcv_skb+0x15a/0x430 net/netlink/af_netlink.c:2470
genl_rcv+0x24/0x40 net/netlink/genetlink.c:742
netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:671
____sys_sendmsg+0x6e8/0x810 net/socket.c:2359
___sys_sendmsg+0xf3/0x170 net/socket.c:2413
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2446
do_syscall_64+0x60/0xe0 arch/x86/entry/common.c:384
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x4402d9
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b 13 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffdfda722f8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00000000004002c8 RCX: 00000000004402d9
RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000003
RBP: 00000000006ca018 R08: 0000000000000000 R09: 0000000000400

---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

Paolo Abeni

unread,

Aug 4, 2020, 12:26:10 PM8/4/20

to syzbot, syzkall...@googlegroups.com

On Tue, 2020-08-04 at 08:35 -0700, syzbot wrote:
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: 2f631133 net: Pass NULL to skb_network_protocol() when we ..
> git tree: net-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=12daae2a900000
> kernel config: https://syzkaller.appspot.com/x/.config?x=91a13b78c7dc258d
> dashboard link: https://syzkaller.appspot.com/bug?extid=c0eb9e7cdde04e4eb4be
> compiler: gcc (GCC) 10.1.0-syz 20200507
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=172cd3d4900000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=11fbd342900000
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+c0eb9e...@syzkaller.appspotmail.com
>
> netlink: 'syz-executor399': attribute type 2 has an invalid length.
> =============================
> WARNING: suspicious RCU usage
> 5.8.0-rc7-syzkaller #0 Not tainted
> -----------------------------
> net/openvswitch/flow_table.c:521 suspicious rcu_dereference_check() usage!
>
> other info that might help us debug this:

#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git master

---
diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c
index 6527d84c3ea6..cb2e8438f8a4 100644
--- a/net/openvswitch/flow_table.c
+++ b/net/openvswitch/flow_table.c
@@ -518,7 +518,7 @@ void ovs_flow_tbl_destroy(struct flow_table *table)
{
struct table_instance *ti = rcu_dereference_raw(table->ti);
struct table_instance *ufid_ti = rcu_dereference_raw(table->ufid_ti);
- struct mask_cache *mc = rcu_dereference(table->mask_cache);
+ struct mask_cache *mc = rcu_dereference_ovsl(table->mask_cache);
struct mask_array *ma = rcu_dereference_ovsl(table->mask_array);

call_rcu(&mc->rcu, mask_cache_rcu_cb);
@@ -937,7 +937,7 @@ int ovs_flow_tbl_num_masks(const struct flow_table *table)

u32 ovs_flow_tbl_masks_cache_size(const struct flow_table *table)
{
- struct mask_cache *mc = rcu_dereference(table->mask_cache);
+ struct mask_cache *mc = rcu_dereference_ovsl(table->mask_cache);

return READ_ONCE(mc->cache_size);
}

syzbot

unread,

Aug 4, 2020, 3:10:08 PM8/4/20

to da...@davemloft.net, d...@openvswitch.org, echa...@redhat.com, ku...@kernel.org, linux-...@vger.kernel.org, net...@vger.kernel.org, pab...@redhat.com, psh...@ovn.org, syzkall...@googlegroups.com, xiangxi...@gmail.com

syzbot has bisected this issue to:

commit 9bf24f594c6acf676fb8c229f152c21bfb915ddb
Author: Eelco Chaudron <echa...@redhat.com>
Date: Fri Jul 31 12:21:34 2020 +0000

net: openvswitch: make masks cache size configurable

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=17461dea900000
start commit: 2f631133 net: Pass NULL to skb_network_protocol() when we ..
git tree: net-next
final oops: https://syzkaller.appspot.com/x/report.txt?x=14c61dea900000
console output: https://syzkaller.appspot.com/x/log.txt?x=10c61dea900000

kernel config: https://syzkaller.appspot.com/x/.config?x=91a13b78c7dc258d
dashboard link: https://syzkaller.appspot.com/bug?extid=c0eb9e7cdde04e4eb4be

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=172cd3d4900000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=11fbd342900000

Reported-by: syzbot+c0eb9e...@syzkaller.appspotmail.com
Fixes: 9bf24f594c6a ("net: openvswitch: make masks cache size configurable")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

syzbot

unread,

Aug 4, 2020, 3:23:09 PM8/4/20

to pab...@redhat.com, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
WARNING: suspicious RCU usage in ovs_flow_tbl_destroy

netlink: 'syz-executor.3': attribute type 2 has an invalid length.

=============================
WARNING: suspicious RCU usage
5.8.0-rc7-syzkaller #0 Not tainted
-----------------------------
net/openvswitch/flow_table.c:521 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1

1 lock held by syz-executor.3/8221:
#0: ffffffff8a832390 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 net/netlink/genetlink.c:741

stack backtrace:
CPU: 1 PID: 8221 Comm: syz-executor.3 Not tainted 5.8.0-rc7-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x18f/0x20d lib/dump_stack.c:118

ovs_flow_tbl_destroy+0x1f3/0x230 net/openvswitch/flow_table.c:521

ovs_dp_cmd_new+0x8ca/0xea0 net/openvswitch/datapath.c:1747
genl_family_rcv_msg_doit net/netlink/genetlink.c:669 [inline]
genl_family_rcv_msg net/netlink/genetlink.c:714 [inline]
genl_rcv_msg+0x61d/0x980 net/netlink/genetlink.c:731
netlink_rcv_skb+0x15a/0x430 net/netlink/af_netlink.c:2470
genl_rcv+0x24/0x40 net/netlink/genetlink.c:742
netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:671
____sys_sendmsg+0x6e8/0x810 net/socket.c:2359
___sys_sendmsg+0xf3/0x170 net/socket.c:2413
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2446
do_syscall_64+0x60/0xe0 arch/x86/entry/common.c:384
entry_SYSCALL_64_after_hwframe+0x44/0xa9

RIP: 0033:0x45cce9
Code: 2d b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 fb b5 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f36dc868c78 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 000000000002bec0 RCX: 000000000045cce9

RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000003

RBP: 000000000078bf40 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000078bf0c
R13: 00007ffee22683af R14: 00007f36dc8699c0 R15: 000000000078bf0c

=============================
WARNING: suspicious RCU usage
5.8.0-rc7-syzkaller #0 Not tainted
-----------------------------

net/openvswitch/flow_table.c:522 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x18f/0x20d lib/dump_stack.c:118

ovs_flow_tbl_destroy+0x190/0x230 net/openvswitch/flow_table.c:522

RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000003

RBP: 000000000078bf40 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000078bf0c
R13: 00007ffee22683af R14: 00007f36dc8699c0 R15: 000000000078bf0c

Tested on:

commit: da795540 sfc: Fix build with CONFIG_RFS_ACCEL disabled.
git tree: net-next
console output: https://syzkaller.appspot.com/x/log.txt?x=14ffdfd4900000

kernel config: https://syzkaller.appspot.com/x/.config?x=91a13b78c7dc258d
dashboard link: https://syzkaller.appspot.com/bug?extid=c0eb9e7cdde04e4eb4be
compiler: gcc (GCC) 10.1.0-syz 20200507