possible deadlock in nf_tables_netdev_event

17 views
Skip to first unread message

syzbot

unread,
Feb 26, 2020, 11:50:14 AM2/26/20
to syzkaller...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 98db2bf2 Linux 4.14.171
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=123ffbe9e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=49cf7d1ab0dd7451
dashboard link: https://syzkaller.appspot.com/bug?extid=73d30634b8778fc45911
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14e687d9e00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=122e80e3e00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+73d306...@syzkaller.appspotmail.com

audit: type=1400 audit(1582735613.098:37): avc: denied { create } for pid=7762 comm="syz-executor699" scontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 tcontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 tclass=netlink_netfilter_socket permissive=1
audit: type=1400 audit(1582735613.098:38): avc: denied { write } for pid=7762 comm="syz-executor699" scontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 tcontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 tclass=netlink_netfilter_socket permissive=1
ip_tables: iptables: counters copy to user failed while replacing table
======================================================
WARNING: possible circular locking dependency detected
4.14.171-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor699/7770 is trying to acquire lock:
(&table[i].mutex){+.+.}, at: [<ffffffff852a415d>] nf_tables_netdev_event+0x10d/0x4d0 net/netfilter/nf_tables_netdev.c:122

but task is already holding lock:
(rtnl_mutex){+.+.}, at: [<ffffffff850246dd>] rtnl_lock net/core/rtnetlink.c:72 [inline]
(rtnl_mutex){+.+.}, at: [<ffffffff850246dd>] rtnetlink_rcv_msg+0x31d/0xb10 net/core/rtnetlink.c:4310

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (rtnl_mutex){+.+.}:
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
unregister_netdevice_notifier+0x5e/0x2b0 net/core/dev.c:1634
tee_tg_destroy+0x5c/0xb0 net/netfilter/xt_TEE.c:123
cleanup_entry+0x169/0x220 net/ipv4/netfilter/ip_tables.c:666
__do_replace+0x38d/0x570 net/ipv4/netfilter/ip_tables.c:1086
do_replace net/ipv4/netfilter/ip_tables.c:1142 [inline]
do_ipt_set_ctl+0x255/0x39d net/ipv4/netfilter/ip_tables.c:1674
nf_sockopt net/netfilter/nf_sockopt.c:106 [inline]
nf_setsockopt+0x5f/0xb0 net/netfilter/nf_sockopt.c:115
ip_setsockopt net/ipv4/ip_sockglue.c:1255 [inline]
ip_setsockopt+0x94/0xb0 net/ipv4/ip_sockglue.c:1240
udp_setsockopt+0x45/0x80 net/ipv4/udp.c:2451
SYSC_setsockopt net/socket.c:1865 [inline]
SyS_setsockopt+0x110/0x1e0 net/socket.c:1844
do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #1 (&xt[i].mutex){+.+.}:
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
xt_find_revision+0x89/0x200 net/netfilter/x_tables.c:373
nfnl_compat_get+0x1f7/0x8a0 net/netfilter/nft_compat.c:678
nfnetlink_rcv_msg+0xa08/0xc00 net/netfilter/nfnetlink.c:214
netlink_rcv_skb+0x127/0x370 net/netlink/af_netlink.c:2432
nfnetlink_rcv+0x1ab/0x1650 net/netfilter/nfnetlink.c:515
netlink_unicast_kernel net/netlink/af_netlink.c:1286 [inline]
netlink_unicast+0x437/0x620 net/netlink/af_netlink.c:1312
netlink_sendmsg+0x733/0xbe0 net/netlink/af_netlink.c:1877
sock_sendmsg_nosec net/socket.c:646 [inline]
sock_sendmsg+0xc5/0x100 net/socket.c:656
___sys_sendmsg+0x70a/0x840 net/socket.c:2062
__sys_sendmsg+0xa3/0x120 net/socket.c:2096
SYSC_sendmsg net/socket.c:2107 [inline]
SyS_sendmsg+0x27/0x40 net/socket.c:2103
do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #0 (&table[i].mutex){+.+.}:
lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
nf_tables_netdev_event+0x10d/0x4d0 net/netfilter/nf_tables_netdev.c:122
notifier_call_chain+0x107/0x1a0 kernel/notifier.c:93
call_netdevice_notifiers net/core/dev.c:1687 [inline]
rollback_registered_many+0x694/0xa70 net/core/dev.c:7205
rollback_registered+0xd6/0x180 net/core/dev.c:7247
unregister_netdevice_queue net/core/dev.c:8259 [inline]
unregister_netdevice_queue+0x1a1/0x220 net/core/dev.c:8252
unregister_netdevice include/linux/netdevice.h:2442 [inline]
veth_newlink+0x53e/0x8a0 drivers/net/veth.c:463
rtnl_newlink+0xecb/0x1720 net/core/rtnetlink.c:2728
rtnetlink_rcv_msg+0x3be/0xb10 net/core/rtnetlink.c:4315
netlink_rcv_skb+0x127/0x370 net/netlink/af_netlink.c:2432
netlink_unicast_kernel net/netlink/af_netlink.c:1286 [inline]
netlink_unicast+0x437/0x620 net/netlink/af_netlink.c:1312
netlink_sendmsg+0x733/0xbe0 net/netlink/af_netlink.c:1877
sock_sendmsg_nosec net/socket.c:646 [inline]
sock_sendmsg+0xc5/0x100 net/socket.c:656
___sys_sendmsg+0x70a/0x840 net/socket.c:2062
__sys_sendmsg+0xa3/0x120 net/socket.c:2096
SYSC_sendmsg net/socket.c:2107 [inline]
SyS_sendmsg+0x27/0x40 net/socket.c:2103
do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Chain exists of:
&table[i].mutex --> &xt[i].mutex --> rtnl_mutex

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(rtnl_mutex);
lock(&xt[i].mutex);
lock(rtnl_mutex);
lock(&table[i].mutex);

*** DEADLOCK ***

1 lock held by syz-executor699/7770:
#0: (rtnl_mutex){+.+.}, at: [<ffffffff850246dd>] rtnl_lock net/core/rtnetlink.c:72 [inline]
#0: (rtnl_mutex){+.+.}, at: [<ffffffff850246dd>] rtnetlink_rcv_msg+0x31d/0xb10 net/core/rtnetlink.c:4310

stack backtrace:
CPU: 1 PID: 7770 Comm: syz-executor699 Not tainted 4.14.171-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x13e/0x194 lib/dump_stack.c:58
print_circular_bug.isra.0.cold+0x1c4/0x282 kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
nf_tables_netdev_event+0x10d/0x4d0 net/netfilter/nf_tables_netdev.c:122
notifier_call_chain+0x107/0x1a0 kernel/notifier.c:93
call_netdevice_notifiers net/core/dev.c:1687 [inline]
rollback_registered_many+0x694/0xa70 net/core/dev.c:7205
rollback_registered+0xd6/0x180 net/core/dev.c:7247
unregister_netdevice_queue net/core/dev.c:8259 [inline]
unregister_netdevice_queue+0x1a1/0x220 net/core/dev.c:8252
unregister_netdevice include/linux/netdevice.h:2442 [inline]
veth_newlink+0x53e/0x8a0 drivers/net/veth.c:463
rtnl_newlink+0xecb/0x1720 net/core/rtnetlink.c:2728
rtnetlink_rcv_msg+0x3be/0xb10 net/core/rtnetlink.c:4315
netlink_rcv_skb+0x127/0x370 net/netlink/af_netlink.c:2432
netlink_unicast_kernel net/netlink/af_netlink.c:1286 [inline]
netlink_unicast+0x437/0x620 net/netlink/af_netlink.c:1312
netlink_sendmsg+0x733/0xbe0 net/netlink/af_netlink.c:1877
sock_sendmsg_nosec net/socket.c:646 [inline]
sock_sendmsg+0xc5/0x100 net/socket.c:656
___sys_sendmsg+0x70a/0x840 net/socket.c:2062
__sys_sendmsg+0xa3/0x120 net/socket.c:2096
SYSC_sendmsg net/socket.c:2107 [inline]
SyS_sendmsg+0x27/0x40 net/socket.c:2103
do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x441d79
RSP: 002b:00007ffcd8706bb8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000441d79
RDX: 0000000000000000 RSI: 0000000020000040 RDI: 0000000000000004
RBP: 000000000000c957 R08: 00000000004002c8 R09: 00000000004002c8
R10: 00000000004002c8 R11: 0000000000000246 R12: 0000000000402ba0
R13: 0000000000402c30 R14: 0000000000000000 R15: 0000000000000000
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
net_ratelimit: 26 callbacks suppressed
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table
ip_tables: iptables: counters copy to user failed while replacing table


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches
Reply all
Reply to author
Forward
0 new messages