possible deadlock in __dev_queue_xmit
19 views
Skip to first unread message
syzbot
Jul 31, 2020, 3:05:21 AM7/31/20
to syzkaller...@googlegroups.com
Hello,
syzbot found the following issue on:
HEAD commit: 205a42ce Linux 4.19.135
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=11ce4c98900000
kernel config: https://syzkaller.appspot.com/x/.config?x=155b0b328eb5e700
dashboard link: https://syzkaller.appspot.com/bug?extid=ea0ea1495c431e6d12a3
compiler: gcc (GCC) 10.1.0-syz 20200507
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=17066782900000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=170382a2900000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+ea0ea1...@syzkaller.appspotmail.com
batman_adv: batadv0: Interface activated: batadv_slave_1
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
============================================
WARNING: possible recursive locking detected
4.19.135-syzkaller #0 Not tainted
--------------------------------------------
syz-executor916/6447 is trying to acquire lock:
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: __dev_queue_xmit+0x2592/0x2e00 net/core/dev.c:3834
but task is already holding lock:
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf50 net/sched/sch_generic.c:330
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(_xmit_ETHER#2);
lock(_xmit_ETHER#2);
*** DEADLOCK ***
May be due to missing lock nesting notation
11 locks held by syz-executor916/6447:
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: l3mdev_l3_out include/net/l3mdev.h:172 [inline]
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: l3mdev_ip6_out include/net/l3mdev.h:193 [inline]
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: rawv6_send_hdrinc net/ipv6/raw.c:682 [inline]
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: rawv6_sendmsg+0x1e3d/0x36a0 net/ipv6/raw.c:947
#1: 00000000fac33b54 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
#1: 00000000fac33b54 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1f2/0x2290 net/ipv6/ip6_output.c:106
#2: 00000000fac33b54 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x278b/0x2e00 net/core/dev.c:3807
#4: 00000000ce3642cd (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
#5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
#5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
#5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf50 net/sched/sch_generic.c:330
#6: 0000000041be4ca5 (rcu_read_lock){....}, at: icmpv6_send+0x0/0x210 net/ipv6/ip6_icmp.c:31
#7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
#7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: icmpv6_xmit_lock net/ipv6/icmp.c:119 [inline]
#7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: icmp6_send+0xfea/0x2230 net/ipv6/icmp.c:532
#8: 0000000041be4ca5 (rcu_read_lock){....}, at: icmp6_send+0x1685/0x2230 net/ipv6/icmp.c:565
#9: 00000000fac33b54 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
#9: 00000000fac33b54 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1f2/0x2290 net/ipv6/ip6_output.c:106
#10: 00000000fac33b54 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
stack backtrace:
CPU: 0 PID: 6447 Comm: syz-executor916 Not tainted 4.19.135-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1fc/0x2fe lib/dump_stack.c:118
print_deadlock_bug kernel/locking/lockdep.c:1763 [inline]
check_deadlock kernel/locking/lockdep.c:1807 [inline]
validate_chain kernel/locking/lockdep.c:2403 [inline]
__lock_acquire.cold+0x121/0x57e kernel/locking/lockdep.c:3415
lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3907
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
__netif_tx_lock include/linux/netdevice.h:3828 [inline]
__dev_queue_xmit+0x2592/0x2e00 net/core/dev.c:3834
neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
neigh_output include/net/neighbour.h:501 [inline]
ip6_finish_output2+0x113d/0x2290 net/ipv6/ip6_output.c:120
ip6_finish_output+0x7eb/0xc10 net/ipv6/ip6_output.c:154
NF_HOOK_COND include/linux/netfilter.h:278 [inline]
ip6_output+0x205/0x770 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:455 [inline]
ip6_local_out+0xaf/0x170 net/ipv6/output_core.c:178
ip6_send_skb+0xb3/0x300 net/ipv6/ip6_output.c:1699
ip6_push_pending_frames+0xbd/0xe0 net/ipv6/ip6_output.c:1719
icmpv6_push_pending_frames+0x294/0x470 net/ipv6/icmp.c:288
icmp6_send+0x1b5f/0x2230 net/ipv6/icmp.c:584
icmpv6_send+0xde/0x210 net/ipv6/ip6_icmp.c:43
ip6_link_failure+0x26/0x4e0 net/ipv6/route.c:2297
dst_link_failure include/net/dst.h:438 [inline]
ip_tunnel_xmit+0x195f/0x37f0 net/ipv4/ip_tunnel.c:789
erspan_xmit+0x1837/0x2790 net/ipv4/ip_gre.c:756
__netdev_start_xmit include/linux/netdevice.h:4333 [inline]
netdev_start_xmit include/linux/netdevice.h:4347 [inline]
xmit_one net/core/dev.c:3256 [inline]
dev_hard_start_xmit+0x1a8/0x920 net/core/dev.c:3272
sch_direct_xmit+0x2d6/0xf50 net/sched/sch_generic.c:332
qdisc_restart net/sched/sch_generic.c:395 [inline]
__qdisc_run+0x4d0/0x1630 net/sched/sch_generic.c:403
qdisc_run include/net/pkt_sched.h:120 [inline]
__dev_xmit_skb net/core/dev.c:3451 [inline]
__dev_queue_xmit+0x2102/0x2e00 net/core/dev.c:3807
neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
neigh_output include/net/neighbour.h:501 [inline]
ip6_finish_output2+0x113d/0x2290 net/ipv6/ip6_output.c:120
ip6_finish_output+0x7eb/0xc10 net/ipv6/ip6_output.c:154
NF_HOOK_COND include/linux/netfilter.h:278 [inline]
ip6_output+0x205/0x770 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:455 [inline]
NF_HOOK include/linux/netfilter.h:289 [inline]
rawv6_send_hdrinc net/ipv6/raw.c:692 [inline]
rawv6_sendmsg+0x202c/0x36a0 net/ipv6/raw.c:947
inet_sendmsg+0x132/0x5a0 net/ipv4/af_inet.c:798
sock_sendmsg_nosec net/socket.c:622 [inline]
sock_sendmsg+0xc3/0x120 net/socket.c:632
sock_write_iter+0x287/0x3c0 net/socket.c:901
call_write_iter include/linux/fs.h:1821 [inline]
new_sync_write fs/read_write.c:474 [inline]
__vfs_write+0x51b/0x770 fs/read_write.c:487
vfs_write+0x1f3/0x540 fs/read_write.c:549
ksys_write+0x12b/0x2a0 fs/read_write.c:599
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x449ee9
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b 0e fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fffc4bee918 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000000449ee9
RDX: 0000000000000028 RSI: 0000000020000140 RDI: 0000000000000005
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fffc4bee990
R13: 0000000000000003 R14: 0000000000959914 R15: 0000000000000013
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches
syzbot found the following issue on:
HEAD commit: 205a42ce Linux 4.19.135
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=11ce4c98900000
kernel config: https://syzkaller.appspot.com/x/.config?x=155b0b328eb5e700
dashboard link: https://syzkaller.appspot.com/bug?extid=ea0ea1495c431e6d12a3
compiler: gcc (GCC) 10.1.0-syz 20200507
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=17066782900000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=170382a2900000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+ea0ea1...@syzkaller.appspotmail.com
batman_adv: batadv0: Interface activated: batadv_slave_1
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
============================================
WARNING: possible recursive locking detected
4.19.135-syzkaller #0 Not tainted
--------------------------------------------
syz-executor916/6447 is trying to acquire lock:
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: __dev_queue_xmit+0x2592/0x2e00 net/core/dev.c:3834
but task is already holding lock:
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf50 net/sched/sch_generic.c:330
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(_xmit_ETHER#2);
lock(_xmit_ETHER#2);
*** DEADLOCK ***
May be due to missing lock nesting notation
11 locks held by syz-executor916/6447:
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: l3mdev_l3_out include/net/l3mdev.h:172 [inline]
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: l3mdev_ip6_out include/net/l3mdev.h:193 [inline]
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: rawv6_send_hdrinc net/ipv6/raw.c:682 [inline]
#0: 0000000041be4ca5 (rcu_read_lock){....}, at: rawv6_sendmsg+0x1e3d/0x36a0 net/ipv6/raw.c:947
#1: 00000000fac33b54 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
#1: 00000000fac33b54 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1f2/0x2290 net/ipv6/ip6_output.c:106
#2: 00000000fac33b54 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline]
#3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x278b/0x2e00 net/core/dev.c:3807
#4: 00000000ce3642cd (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
#5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
#5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
#5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf50 net/sched/sch_generic.c:330
#6: 0000000041be4ca5 (rcu_read_lock){....}, at: icmpv6_send+0x0/0x210 net/ipv6/ip6_icmp.c:31
#7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
#7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: icmpv6_xmit_lock net/ipv6/icmp.c:119 [inline]
#7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: icmp6_send+0xfea/0x2230 net/ipv6/icmp.c:532
#8: 0000000041be4ca5 (rcu_read_lock){....}, at: icmp6_send+0x1685/0x2230 net/ipv6/icmp.c:565
#9: 00000000fac33b54 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
#9: 00000000fac33b54 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1f2/0x2290 net/ipv6/ip6_output.c:106
#10: 00000000fac33b54 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
stack backtrace:
CPU: 0 PID: 6447 Comm: syz-executor916 Not tainted 4.19.135-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1fc/0x2fe lib/dump_stack.c:118
print_deadlock_bug kernel/locking/lockdep.c:1763 [inline]
check_deadlock kernel/locking/lockdep.c:1807 [inline]
validate_chain kernel/locking/lockdep.c:2403 [inline]
__lock_acquire.cold+0x121/0x57e kernel/locking/lockdep.c:3415
lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3907
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
__netif_tx_lock include/linux/netdevice.h:3828 [inline]
__dev_queue_xmit+0x2592/0x2e00 net/core/dev.c:3834
neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
neigh_output include/net/neighbour.h:501 [inline]
ip6_finish_output2+0x113d/0x2290 net/ipv6/ip6_output.c:120
ip6_finish_output+0x7eb/0xc10 net/ipv6/ip6_output.c:154
NF_HOOK_COND include/linux/netfilter.h:278 [inline]
ip6_output+0x205/0x770 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:455 [inline]
ip6_local_out+0xaf/0x170 net/ipv6/output_core.c:178
ip6_send_skb+0xb3/0x300 net/ipv6/ip6_output.c:1699
ip6_push_pending_frames+0xbd/0xe0 net/ipv6/ip6_output.c:1719
icmpv6_push_pending_frames+0x294/0x470 net/ipv6/icmp.c:288
icmp6_send+0x1b5f/0x2230 net/ipv6/icmp.c:584
icmpv6_send+0xde/0x210 net/ipv6/ip6_icmp.c:43
ip6_link_failure+0x26/0x4e0 net/ipv6/route.c:2297
dst_link_failure include/net/dst.h:438 [inline]
ip_tunnel_xmit+0x195f/0x37f0 net/ipv4/ip_tunnel.c:789
erspan_xmit+0x1837/0x2790 net/ipv4/ip_gre.c:756
__netdev_start_xmit include/linux/netdevice.h:4333 [inline]
netdev_start_xmit include/linux/netdevice.h:4347 [inline]
xmit_one net/core/dev.c:3256 [inline]
dev_hard_start_xmit+0x1a8/0x920 net/core/dev.c:3272
sch_direct_xmit+0x2d6/0xf50 net/sched/sch_generic.c:332
qdisc_restart net/sched/sch_generic.c:395 [inline]
__qdisc_run+0x4d0/0x1630 net/sched/sch_generic.c:403
qdisc_run include/net/pkt_sched.h:120 [inline]
__dev_xmit_skb net/core/dev.c:3451 [inline]
__dev_queue_xmit+0x2102/0x2e00 net/core/dev.c:3807
neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
neigh_output include/net/neighbour.h:501 [inline]
ip6_finish_output2+0x113d/0x2290 net/ipv6/ip6_output.c:120
ip6_finish_output+0x7eb/0xc10 net/ipv6/ip6_output.c:154
NF_HOOK_COND include/linux/netfilter.h:278 [inline]
ip6_output+0x205/0x770 net/ipv6/ip6_output.c:171
dst_output include/net/dst.h:455 [inline]
NF_HOOK include/linux/netfilter.h:289 [inline]
rawv6_send_hdrinc net/ipv6/raw.c:692 [inline]
rawv6_sendmsg+0x202c/0x36a0 net/ipv6/raw.c:947
inet_sendmsg+0x132/0x5a0 net/ipv4/af_inet.c:798
sock_sendmsg_nosec net/socket.c:622 [inline]
sock_sendmsg+0xc3/0x120 net/socket.c:632
sock_write_iter+0x287/0x3c0 net/socket.c:901
call_write_iter include/linux/fs.h:1821 [inline]
new_sync_write fs/read_write.c:474 [inline]
__vfs_write+0x51b/0x770 fs/read_write.c:487
vfs_write+0x1f3/0x540 fs/read_write.c:549
ksys_write+0x12b/0x2a0 fs/read_write.c:599
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x449ee9
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b 0e fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fffc4bee918 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000000449ee9
RDX: 0000000000000028 RSI: 0000000020000140 RDI: 0000000000000005
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fffc4bee990
R13: 0000000000000003 R14: 0000000000959914 R15: 0000000000000013
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches
Reply all
Reply to author
Forward
0 new messages