possible deadlock in sch_direct_xmit

6 views
Skip to first unread message

syzbot

unread,
Apr 11, 2019, 4:44:41 AM4/11/19
to syzkaller-a...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 07c01385 Merge 4.4.138 into android-4.4
git tree: android-4.4
console output: https://syzkaller.appspot.com/x/log.txt?x=10714aff800000
kernel config: https://syzkaller.appspot.com/x/.config?x=9015d1e2403e29b6
dashboard link: https://syzkaller.appspot.com/bug?extid=ecda1e218cc594d9cd61
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=131e8bd8400000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=151a2720400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+ecda1e...@syzkaller.appspotmail.com

random: sshd: uninitialized urandom read (32 bytes read, 39 bits of entropy
available)
random: nonblocking pool is initialized
IPVS: Creating netns size=2552 id=1

======================================================
[ INFO: possible circular locking dependency detected ]
4.4.138-g07c0138 #62 Not tainted
-------------------------------------------------------
syz-executor011/3847 is trying to acquire lock:
(_xmit_NETROM){+.-...}, at: [<ffffffff8301197c>] spin_lock
include/linux/spinlock.h:302 [inline]
(_xmit_NETROM){+.-...}, at: [<ffffffff8301197c>] __netif_tx_lock
include/linux/netdevice.h:3299 [inline]
(_xmit_NETROM){+.-...}, at: [<ffffffff8301197c>]
sch_direct_xmit+0x23c/0x6e0 net/sched/sch_generic.c:163

but task is already holding lock:
(&(&q->lock)->rlock){+.-...}, at: [<ffffffff834bf795>] spin_lock
include/linux/spinlock.h:302 [inline]
(&(&q->lock)->rlock){+.-...}, at: [<ffffffff834bf795>]
ipv6_frag_rcv+0x605/0x4fd0 net/ipv6/reassembly.c:560

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

[<ffffffff81235a7e>] lock_acquire+0x15e/0x450
kernel/locking/lockdep.c:3592
[<ffffffff838c235e>] __raw_spin_lock_irqsave
include/linux/spinlock_api_smp.h:112 [inline]
[<ffffffff838c235e>] _raw_spin_lock_irqsave+0x4e/0x70
kernel/locking/spinlock.c:159
[<ffffffff8128e6b5>] lock_timer_base+0xd5/0x170
kernel/time/timer.c:779
[<ffffffff81299247>] __mod_timer kernel/time/timer.c:799 [inline]
[<ffffffff81299247>] mod_timer+0x1b7/0xa80 kernel/time/timer.c:925
[<ffffffff833390aa>] inet_frag_intern net/ipv4/inet_fragment.c:350
[inline]
[<ffffffff833390aa>] inet_frag_create net/ipv4/inet_fragment.c:397
[inline]
[<ffffffff833390aa>] inet_frag_find+0x71a/0x9c0
net/ipv4/inet_fragment.c:426
[<ffffffff8320682d>] ip_find net/ipv4/ip_fragment.c:275 [inline]
[<ffffffff8320682d>] ip_defrag+0x2ed/0x3fe0
net/ipv4/ip_fragment.c:676
[<ffffffff8320a8e8>] ip_check_defrag+0x3c8/0x7e0
net/ipv4/ip_fragment.c:724
[<ffffffff8357cbea>] packet_rcv_fanout+0x52a/0x5e0
net/packet/af_packet.c:1458
[<ffffffff82f89664>] dev_queue_xmit_nit net/core/dev.c:1913 [inline]
[<ffffffff82f89664>] xmit_one net/core/dev.c:2755 [inline]
[<ffffffff82f89664>] dev_hard_start_xmit+0x644/0x11c0
net/core/dev.c:2775
[<ffffffff83011a01>] sch_direct_xmit+0x2c1/0x6e0
net/sched/sch_generic.c:165
[<ffffffff82f8b393>] __dev_xmit_skb net/core/dev.c:2957 [inline]
[<ffffffff82f8b393>] __dev_queue_xmit+0xef3/0x1c80
net/core/dev.c:3175
[<ffffffff82f8c137>] dev_queue_xmit+0x17/0x20 net/core/dev.c:3241
[<ffffffff82fa8c07>] neigh_resolve_output+0x637/0x790
net/core/neighbour.c:1326
[<ffffffff8321612b>] dst_neigh_output include/net/dst.h:461 [inline]
[<ffffffff8321612b>] ip_finish_output2+0x6ab/0x1110
net/ipv4/ip_output.c:213
[<ffffffff8321855c>] ip_do_fragment+0x19cc/0x2190
net/ipv4/ip_output.c:633
[<ffffffff83218e63>] ip_fragment.constprop.51+0x143/0x200
net/ipv4/ip_output.c:503
[<ffffffff832193aa>] ip_finish_output+0x48a/0xc00
net/ipv4/ip_output.c:286
[<ffffffff8321c7d3>] NF_HOOK_COND include/linux/netfilter.h:240
[inline]
[<ffffffff8321c7d3>] ip_mc_output+0x233/0x980
net/ipv4/ip_output.c:347
[<ffffffff83219ffb>] dst_output include/net/dst.h:498 [inline]
[<ffffffff83219ffb>] ip_local_out+0x9b/0x180 net/ipv4/ip_output.c:119
[<ffffffff8321fd7c>] ip_send_skb+0x3c/0xc0 net/ipv4/ip_output.c:1451
[<ffffffff832c8603>] udp_send_skb+0x5c3/0xc60 net/ipv4/udp.c:842
[<ffffffff832d11b9>] udp_sendmsg+0x16c9/0x1c70 net/ipv4/udp.c:1072
[<ffffffff83300dd3>] inet_sendmsg+0x203/0x4d0 net/ipv4/af_inet.c:755
[<ffffffff82f1e22c>] sock_sendmsg_nosec net/socket.c:625 [inline]
[<ffffffff82f1e22c>] sock_sendmsg+0xcc/0x110 net/socket.c:635
[<ffffffff82f1ef0c>] SYSC_sendto+0x21c/0x370 net/socket.c:1665
[<ffffffff82f21590>] SyS_sendto+0x40/0x50 net/socket.c:1633
[<ffffffff838c2725>] entry_SYSCALL_64_fastpath+0x22/0x9e

[<ffffffff81232932>] check_prev_add kernel/locking/lockdep.c:1853
[inline]
[<ffffffff81232932>] check_prevs_add kernel/locking/lockdep.c:1958
[inline]
[<ffffffff81232932>] validate_chain kernel/locking/lockdep.c:2144
[inline]
[<ffffffff81232932>] __lock_acquire+0x3902/0x5270
kernel/locking/lockdep.c:3213
[<ffffffff81235a7e>] lock_acquire+0x15e/0x450
kernel/locking/lockdep.c:3592
[<ffffffff838c1846>] __raw_spin_lock
include/linux/spinlock_api_smp.h:144 [inline]
[<ffffffff838c1846>] _raw_spin_lock+0x36/0x50
kernel/locking/spinlock.c:151
[<ffffffff8301197c>] spin_lock include/linux/spinlock.h:302 [inline]
[<ffffffff8301197c>] __netif_tx_lock include/linux/netdevice.h:3299
[inline]
[<ffffffff8301197c>] sch_direct_xmit+0x23c/0x6e0
net/sched/sch_generic.c:163
[<ffffffff82f8b393>] __dev_xmit_skb net/core/dev.c:2957 [inline]
[<ffffffff82f8b393>] __dev_queue_xmit+0xef3/0x1c80
net/core/dev.c:3175
[<ffffffff82f8c137>] dev_queue_xmit+0x17/0x20 net/core/dev.c:3241
[<ffffffff82fa8c07>] neigh_resolve_output+0x637/0x790
net/core/neighbour.c:1326
[<ffffffff83429c29>] dst_neigh_output include/net/dst.h:461 [inline]
[<ffffffff83429c29>] ip6_finish_output2+0x929/0x1ca0
net/ipv6/ip6_output.c:113
[<ffffffff83433268>] ip6_finish_output+0x3b8/0x760
net/ipv6/ip6_output.c:131
[<ffffffff834337c8>] NF_HOOK_COND include/linux/netfilter.h:240
[inline]
[<ffffffff834337c8>] ip6_output+0x1b8/0x520 net/ipv6/ip6_output.c:145
[<ffffffff834858cf>] dst_output include/net/dst.h:498 [inline]
[<ffffffff834858cf>] NF_HOOK_THRESH.constprop.29+0x11f/0x310
include/linux/netfilter.h:226
[<ffffffff834866a7>] NF_HOOK include/linux/netfilter.h:249 [inline]
[<ffffffff834866a7>] ndisc_send_skb+0x7e7/0xf20 net/ipv6/ndisc.c:471
[<ffffffff8348a411>] ndisc_send_ns+0x501/0x700 net/ipv6/ndisc.c:595
[<ffffffff8348a8b2>] ndisc_solicit+0x2a2/0x430 net/ipv6/ndisc.c:686
[<ffffffff82f9d41a>] neigh_probe+0xca/0x100 net/core/neighbour.c:871
[<ffffffff82fa2df0>] __neigh_event_send+0x2a0/0xc50
net/core/neighbour.c:1027
[<ffffffff82fa8abb>] neigh_event_send include/net/neighbour.h:431
[inline]
[<ffffffff82fa8abb>] neigh_resolve_output+0x4eb/0x790
net/core/neighbour.c:1310
[<ffffffff83429c29>] dst_neigh_output include/net/dst.h:461 [inline]
[<ffffffff83429c29>] ip6_finish_output2+0x929/0x1ca0
net/ipv6/ip6_output.c:113
[<ffffffff83433268>] ip6_finish_output+0x3b8/0x760
net/ipv6/ip6_output.c:131
[<ffffffff834337c8>] NF_HOOK_COND include/linux/netfilter.h:240
[inline]
[<ffffffff834337c8>] ip6_output+0x1b8/0x520 net/ipv6/ip6_output.c:145
[<ffffffff8355ca9b>] dst_output include/net/dst.h:498 [inline]
[<ffffffff8355ca9b>] ip6_local_out+0x9b/0x180
net/ipv6/output_core.c:169
[<ffffffff83435801>] ip6_send_skb+0xa1/0x340
net/ipv6/ip6_output.c:1725
[<ffffffff83435b53>] ip6_push_pending_frames+0xb3/0xe0
net/ipv6/ip6_output.c:1745
[<ffffffff834a4abc>] icmpv6_push_pending_frames+0x33c/0x530
net/ipv6/icmp.c:276
[<ffffffff834a627d>] icmp6_send+0x15cd/0x1b80 net/ipv6/icmp.c:537
[<ffffffff834a7c39>] icmpv6_param_prob+0x29/0x40 net/ipv6/icmp.c:551
[<ffffffff834c3124>] ip6_frag_queue net/ipv6/reassembly.c:263
[inline]
[<ffffffff834c3124>] ipv6_frag_rcv+0x3f94/0x4fd0
net/ipv6/reassembly.c:562
[<ffffffff834368ee>] ip6_input_finish+0x32e/0x1550
net/ipv6/ip6_input.c:248
[<ffffffff834398d6>] NF_HOOK_THRESH include/linux/netfilter.h:226
[inline]
[<ffffffff834398d6>] NF_HOOK include/linux/netfilter.h:249 [inline]
[<ffffffff834398d6>] ip6_input+0xf6/0x200 net/ipv6/ip6_input.c:280
[<ffffffff834360bd>] dst_input include/net/dst.h:504 [inline]
[<ffffffff834360bd>] ip6_rcv_finish+0x13d/0x640
net/ipv6/ip6_input.c:62
[<ffffffff83438bdb>] NF_HOOK_THRESH include/linux/netfilter.h:226
[inline]
[<ffffffff83438bdb>] NF_HOOK include/linux/netfilter.h:249 [inline]
[<ffffffff83438bdb>] ipv6_rcv+0x10cb/0x1cd0 net/ipv6/ip6_input.c:186
[<ffffffff82f7eb06>] __netif_receive_skb_core+0x12d6/0x2940
net/core/dev.c:4019
[<ffffffff82f801cb>] __netif_receive_skb+0x5b/0x1b0
net/core/dev.c:4054
[<ffffffff82f84756>] process_backlog+0x216/0x6a0 net/core/dev.c:4647
[<ffffffff82f81592>] napi_poll net/core/dev.c:4885 [inline]
[<ffffffff82f81592>] net_rx_action+0x3a2/0xdb0 net/core/dev.c:4950
[<ffffffff838c5bec>] __do_softirq+0x22c/0xa1a kernel/softirq.c:273
[<ffffffff838c399c>] do_softirq_own_stack+0x1c/0x30
arch/x86/entry/entry_64.S:929
[<ffffffff8113d9d4>] do_softirq.part.16+0x54/0x60
kernel/softirq.c:317
[<ffffffff8113f6c9>] do_softirq+0x19/0x20 kernel/softirq.c:320
[<ffffffff82f7caac>] netif_rx_ni+0xec/0x3a0 net/core/dev.c:3653
[<ffffffff82753187>] tun_get_user+0xbe7/0x2410 drivers/net/tun.c:1264
[<ffffffff82754bc5>] tun_chr_write_iter+0xd5/0x190
drivers/net/tun.c:1283
[<ffffffff8151d8fc>] do_iter_readv_writev+0x13c/0x1e0
fs/read_write.c:664
[<ffffffff8151f020>] do_readv_writev+0x2e0/0x6e0 fs/read_write.c:808
[<ffffffff8151f54b>] vfs_writev+0x7b/0xb0 fs/read_write.c:847
[<ffffffff81521a19>] SYSC_writev fs/read_write.c:880 [inline]
[<ffffffff81521a19>] SyS_writev+0xd9/0x250 fs/read_write.c:872
[<ffffffff838c2725>] entry_SYSCALL_64_fastpath+0x22/0x9e

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&(&q->lock)->rlock);
lock(_xmit_NETROM);
lock(&(&q->lock)->rlock);
lock(_xmit_NETROM);

*** DEADLOCK ***

9 locks held by syz-executor011/3847:
#0: (rcu_read_lock){......}, at: [<ffffffff82f846f2>] __skb_unlink
include/linux/skbuff.h:1643 [inline]
#0: (rcu_read_lock){......}, at: [<ffffffff82f846f2>] __skb_dequeue
include/linux/skbuff.h:1659 [inline]
#0: (rcu_read_lock){......}, at: [<ffffffff82f846f2>]
process_backlog+0x1b2/0x6a0 net/core/dev.c:4644
#1: (rcu_read_lock){......}, at: [<ffffffff834365c0>]
ip6_input_finish+0x0/0x1550 include/linux/compiler.h:218
#2: (&(&q->lock)->rlock){+.-...}, at: [<ffffffff834bf795>] spin_lock
include/linux/spinlock.h:302 [inline]
#2: (&(&q->lock)->rlock){+.-...}, at: [<ffffffff834bf795>]
ipv6_frag_rcv+0x605/0x4fd0 net/ipv6/reassembly.c:560
#3: (slock-AF_INET6){+.....}, at: [<ffffffff834a5498>] spin_trylock
include/linux/spinlock.h:312 [inline]
#3: (slock-AF_INET6){+.....}, at: [<ffffffff834a5498>] icmpv6_xmit_lock
net/ipv6/icmp.c:120 [inline]
#3: (slock-AF_INET6){+.....}, at: [<ffffffff834a5498>]
icmp6_send+0x7e8/0x1b80 net/ipv6/icmp.c:485
#4: (rcu_read_lock){......}, at: [<ffffffff834a5cdb>]
icmp6_send+0x102b/0x1b80 net/ipv6/icmp.c:517
#5: (rcu_read_lock_bh){......}, at: [<ffffffff834294d5>]
ip6_finish_output2+0x1d5/0x1ca0 net/ipv6/ip6_output.c:71
#6: (rcu_read_lock){......}, at: [<ffffffff834864f7>] ip6_nd_hdr
net/ipv6/ndisc.c:427 [inline]
#6: (rcu_read_lock){......}, at: [<ffffffff834864f7>]
ndisc_send_skb+0x637/0xf20 net/ipv6/ndisc.c:465
#7: (rcu_read_lock_bh){......}, at: [<ffffffff834294d5>]
ip6_finish_output2+0x1d5/0x1ca0 net/ipv6/ip6_output.c:71
#8: (rcu_read_lock_bh){......}, at: [<ffffffff82f8a677>]
__dev_queue_xmit+0x1d7/0x1c80 net/core/dev.c:3139

stack backtrace:
CPU: 0 PID: 3847 Comm: syz-executor011 Not tainted 4.4.138-g07c0138 #62
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
0000000000000000 2ae3f3d68d4df4d2 ffff8801db2062d8 ffffffff81e0ed0d
ffffffff853eba40 ffffffff853ec2b0 ffffffff853eba40 ffff8801d8c15150
ffff8801d8c14800 ffff8801db206320 ffffffff8140e6ab 0000000000000003
Call Trace:
<IRQ> [<ffffffff81e0ed0d>] __dump_stack lib/dump_stack.c:15 [inline]
<IRQ> [<ffffffff81e0ed0d>] dump_stack+0xc1/0x124 lib/dump_stack.c:51
[<ffffffff8140e6ab>] print_circular_bug.cold.50+0x1bd/0x27d
kernel/locking/lockdep.c:1226
[<ffffffff81232932>] check_prev_add kernel/locking/lockdep.c:1853 [inline]
[<ffffffff81232932>] check_prevs_add kernel/locking/lockdep.c:1958 [inline]
[<ffffffff81232932>] validate_chain kernel/locking/lockdep.c:2144 [inline]
[<ffffffff81232932>] __lock_acquire+0x3902/0x5270
kernel/locking/lockdep.c:3213
[<ffffffff81235a7e>] lock_acquire+0x15e/0x450 kernel/locking/lockdep.c:3592
[<ffffffff838c1846>] __raw_spin_lock include/linux/spinlock_api_smp.h:144
[inline]
[<ffffffff838c1846>] _raw_spin_lock+0x36/0x50 kernel/locking/spinlock.c:151
[<ffffffff8301197c>] spin_lock include/linux/spinlock.h:302 [inline]
[<ffffffff8301197c>] __netif_tx_lock include/linux/netdevice.h:3299
[inline]
[<ffffffff8301197c>] sch_direct_xmit+0x23c/0x6e0
net/sched/sch_generic.c:163
[<ffffffff82f8b393>] __dev_xmit_skb net/core/dev.c:2957 [inline]
[<ffffffff82f8b393>] __dev_queue_xmit+0xef3/0x1c80 net/core/dev.c:3175
[<ffffffff82f8c137>] dev_queue_xmit+0x17/0x20 net/core/dev.c:3241
[<ffffffff82fa8c07>] neigh_resolve_output+0x637/0x790
net/core/neighbour.c:1326
[<ffffffff83429c29>] dst_neigh_output include/net/dst.h:461 [inline]
[<ffffffff83429c29>] ip6_finish_output2+0x929/0x1ca0
net/ipv6/ip6_output.c:113
[<ffffffff83433268>] ip6_finish_output+0x3b8/0x760
net/ipv6/ip6_output.c:131
[<ffffffff834337c8>] NF_HOOK_COND include/linux/netfilter.h:240 [inline]
[<ffffffff834337c8>] ip6_output+0x1b8/0x520 net/ipv6/ip6_output.c:145
[<ffffffff834858cf>] dst_output include/net/dst.h:498 [inline]
[<ffffffff834858cf>] NF_HOOK_THRESH.constprop.29+0x11f/0x310
include/linux/netfilter.h:226
[<ffffffff834866a7>] NF_HOOK include/linux/netfilter.h:249 [inline]
[<ffffffff834866a7>] ndisc_send_skb+0x7e7/0xf20 net/ipv6/ndisc.c:471
[<ffffffff8348a411>] ndisc_send_ns+0x501/0x700 net/ipv6/ndisc.c:595
[<ffffffff8348a8b2>] ndisc_solicit+0x2a2/0x430 net/ipv6/ndisc.c:686
[<ffffffff82f9d41a>] neigh_probe+0xca/0x100 net/core/neighbour.c:871
[<ffffffff82fa2df0>] __neigh_event_send+0x2a0/0xc50
net/core/neighbour.c:1027
[<ffffffff82fa8abb>] neigh_event_send include/net/neighbour.h:431 [inline]
[<ffffffff82fa8abb>] neigh_resolve_output+0x4eb/0x790
net/core/neighbour.c:1310
[<ffffffff83429c29>] dst_neigh_output include/net/dst.h:461 [inline]
[<ffffffff83429c29>] ip6_finish_output2+0x929/0x1ca0
net/ipv6/ip6_output.c:113
[<ffffffff83433268>] ip6_finish_output+0x3b8/0x760
net/ipv6/ip6_output.c:131
[<ffffffff834337c8>] NF_HOOK_COND include/linux/netfilter.h:240 [inline]
[<ffffffff834337c8>] ip6_output+0x1b8/0x520 net/ipv6/ip6_output.c:145
[<ffffffff8355ca9b>] dst_output include/net/dst.h:498 [inline]
[<ffffffff8355ca9b>] ip6_local_out+0x9b/0x180 net/ipv6/output_core.c:169
[<ffffffff83435801>] ip6_send_skb+0xa1/0x340 net/ipv6/ip6_output.c:1725
[<ffffffff83435b53>] ip6_push_pending_frames+0xb3/0xe0
net/ipv6/ip6_output.c:1745
[<ffffffff834a4abc>] icmpv6_push_pending_frames+0x33c/0x530
net/ipv6/icmp.c:276
[<ffffffff834a627d>] icmp6_send+0x15cd/0x1b80 net/ipv6/icmp.c:537
[<ffffffff834a7c39>] icmpv6_param_prob+0x29/0x40 net/ipv6/icmp.c:551
[<ffffffff834c3124>] ip6_frag_queue net/ipv6/reassembly.c:263 [inline]
[<ffffffff834c3124>] ipv6_frag_rcv+0x3f94/0x4fd0 net/ipv6/reassembly.c:562
[<ffffffff834368ee>] ip6_input_finish+0x32e/0x1550 net/ipv6/ip6_input.c:248
[<ffffffff834398d6>] NF_HOOK_THRESH include/linux/netfilter.h:226 [inline]
[<ffffffff834398d6>] NF_HOOK include/linux/netfilter.h:249 [inline]
[<ffffffff834398d6>] ip6_input+0xf6/0x200 net/ipv6/ip6_input.c:280
[<ffffffff834360bd>] dst_input include/net/dst.h:504 [inline]
[<ffffffff834360bd>] ip6_rcv_finish+0x13d/0x640 net/ipv6/ip6_input.c:62
[<ffffffff83438bdb>] NF_HOOK_THRESH include/linux/netfilter.h:226 [inline]
[<ffffffff83438bdb>] NF_HOOK include/linux/netfilter.h:249 [inline]
[<ffffffff83438bdb>] ipv6_rcv+0x10cb/0x1cd0 net/ipv6/ip6_input.c:186
[<ffffffff82f7eb06>] __netif_receive_skb_core+0x12d6/0x2940
net/core/dev.c:4019
[<ffffffff82f7d830>] ? dev_cpu_callback+


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches
Reply all
Reply to author
Forward
0 new messages