[v5.15] INFO: task hung in switchdev_deferred_process_work (2)

0 views
Skip to first unread message

syzbot

unread,
Jan 11, 2024, 1:09:26 PMJan 11
to syzkaller...@googlegroups.com
Hello,

syzbot found the following issue on:

HEAD commit: 26c690eff0a5 Linux 5.15.146
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=138124a5e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=dc70d39e176dd118
dashboard link: https://syzkaller.appspot.com/bug?extid=09993a9559cdf0051fa8
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/e52ed2e53c97/disk-26c690ef.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/0baec14a2844/vmlinux-26c690ef.xz
kernel image: https://storage.googleapis.com/syzbot-assets/d33fddfbaf83/bzImage-26c690ef.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+09993a...@syzkaller.appspotmail.com

INFO: task kworker/1:5:3569 blocked for more than 143 seconds.
Not tainted 5.15.146-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/1:5 state:D stack:23640 pid: 3569 ppid: 2 flags:0x00004000
Workqueue: events switchdev_deferred_process_work
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5030 [inline]
__schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
schedule+0x11b/0x1f0 kernel/sched/core.c:6459
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6518
__mutex_lock_common+0xe34/0x25a0 kernel/locking/mutex.c:669
__mutex_lock kernel/locking/mutex.c:729 [inline]
mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
switchdev_deferred_process_work+0xa/0x20 net/switchdev/switchdev.c:74
process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310
worker_thread+0xaca/0x1280 kernel/workqueue.c:2457
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/27:
#0: ffffffff8c91f220 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x30
5 locks held by kworker/u4:3/155:
#0: ffff888011dcd138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
#1: ffffc90001ee7d20 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
#2: ffffffff8d9ce7d0 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0xf1/0xb60 net/core/net_namespace.c:558
#3: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock_unregistering net/core/dev.c:11576 [inline]
#3: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: default_device_exit_batch+0x1ac/0x3f0 net/core/dev.c:11614
#4: ffffffff8c9237e8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:290 [inline]
#4: ffffffff8c9237e8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x280/0x740 kernel/rcu/tree_exp.h:845
2 locks held by getty/3264:
#0: ffff888024d17098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:252
#1: ffffc9000209b2e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6af/0x1db0 drivers/tty/n_tty.c:2158
3 locks held by kworker/1:5/3569:
#0: ffff888011c70d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
#1: ffffc90003137d20 (deferred_process_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
#2: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: switchdev_deferred_process_work+0xa/0x20 net/switchdev/switchdev.c:74
3 locks held by kworker/u4:6/8962:
2 locks held by kworker/u4:7/9228:
3 locks held by kworker/0:0/21544:
#0: ffff888011c70d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
#1: ffffc90003527d20 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
#2: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: linkwatch_event+0xa/0x50 net/core/link_watch.c:251
3 locks held by kworker/0:2/22070:
#0: ffff88814ac23d38 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
#1: ffffc90002f67d20 ((addr_chk_work).work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
#2: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: addrconf_verify_work+0xa/0x20 net/ipv6/addrconf.c:4640
3 locks held by kworker/1:10/25915:
#0: ffff888011c71d38 ((wq_completion)events_power_efficient){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
#1: ffffc90003817d20 ((reg_check_chans).work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
#2: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: reg_check_chans_work+0x7f/0xb70 net/wireless/reg.c:2436
2 locks held by kworker/1:16/25918:
#0: ffff888011c72538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
#1: ffffc90003867d20 ((work_completion)(&rew.rew_work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
3 locks held by syz-executor.1/25986:
#0: ffff888023482460 (sb_writers#5){.+.+}-{0:0}, at: do_coredump+0x1825/0x31e0 fs/coredump.c:825
#1: ffff88805701a1d8 (&sb->s_type->i_mutex_key#9){++++}-{3:3}, at: inode_lock include/linux/fs.h:787 [inline]
#1: ffff88805701a1d8 (&sb->s_type->i_mutex_key#9){++++}-{3:3}, at: ext4_buffered_write_iter+0xa0/0x360 fs/ext4/file.c:262
#2: ffff8880234820e0 (&type->s_umount_key#32){++++}-{3:3}, at: try_to_writeback_inodes_sb+0x1d/0xb0 fs/fs-writeback.c:2729
1 lock held by syz-executor.4/25995:
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: tun_detach drivers/net/tun.c:698 [inline]
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: tun_chr_close+0x3a/0x1b0 drivers/net/tun.c:3434
1 lock held by syz-executor.2/26225:
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: tun_detach drivers/net/tun.c:698 [inline]
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: tun_chr_close+0x3a/0x1b0 drivers/net/tun.c:3434
1 lock held by syz-executor.0/26517:
1 lock held by syz-executor.3/26524:
2 locks held by syz-executor.5/26531:
1 lock held by syz-executor.0/26534:
#0: ffff888023484bd8 (&sbi->s_writepages_rwsem){.+.+}-{0:0}, at: ext4_writepages+0x1f6/0x3d10 fs/ext4/inode.c:2677
1 lock held by syz-executor.3/26537:
2 locks held by syz-executor.5/26542:
1 lock held by syz-executor.1/26550:
1 lock held by syz-executor.0/26552:
1 lock held by syz-executor.3/26554:
#0: ffff888023484bd8 (&sbi->s_writepages_rwsem){.+.+}-{0:0}, at: ext4_writepages+0x1f6/0x3d10 fs/ext4/inode.c:2677
2 locks held by syz-executor.5/26562:
#0: ffff888023484bd8 (&sbi->s_writepages_rwsem){.+.+}-{0:0}, at: ext4_writepages+0x1f6/0x3d10 fs/ext4/inode.c:2677
#1: ffff888023486990 (jbd2_handle){++++}-{0:0}, at: start_this_handle+0x12b9/0x1570 fs/jbd2/transaction.c:462
2 locks held by syz-executor.1/26567:
1 lock held by syz-executor.0/26569:
2 locks held by syz-executor.1/26576:
3 locks held by syz-executor.0/26580:
1 lock held by syz-executor.3/26583:
1 lock held by syz-executor.5/26588:
1 lock held by syz-executor.4/26598:
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by syz-executor.2/26600:
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by syz-executor.1/26608:
1 lock held by syz-executor.3/26610:
1 lock held by syz-executor.0/26614:
2 locks held by syz-executor.5/26616:
1 lock held by syz-executor.1/26618:
1 lock held by syz-executor.3/26627:
1 lock held by syz-executor.5/26629:
1 lock held by syz-executor.0/26630:
1 lock held by syz-executor.5/26640:
1 lock held by syz-executor.3/26644:
1 lock held by syz-executor.5/26664:
#0: ffff888023484bd8 (&sbi->s_writepages_rwsem){.+.+}-{0:0}, at: ext4_writepages+0x1f6/0x3d10 fs/ext4/inode.c:2677
1 lock held by syz-executor.0/26671:
2 locks held by syz-executor.5/26694:
1 lock held by syz-executor.3/26696:
#0: ffff888023484bd8 (&sbi->s_writepages_rwsem){.+.+}-{0:0}, at: ext4_writepages+0x1f6/0x3d10 fs/ext4/inode.c:2677
2 locks held by syz-executor.0/26700:
1 lock held by syz-executor.1/26704:
1 lock held by syz-executor.5/26713:
1 lock held by syz-executor.2/26782:
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by syz-executor.4/26785:
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
#0: ffffffff8d9da4c8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 27 Comm: khungtaskd Not tainted 5.15.146-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
watchdog+0xe72/0xeb0 kernel/hung_task.c:295
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 20986 Comm: kworker/u4:8 Not tainted 5.15.146-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Workqueue: phy50 ieee80211_iface_work
RIP: 0010:cfg80211_put_bss+0x0/0x230 net/wireless/scan.c:2581
Code: c4 d0 e3 8d 80 e1 07 80 c1 03 38 c1 0f 8c ca fe ff ff 48 c7 c7 c4 d0 e3 8d e8 cc 50 3a f8 e9 b9 fe ff ff e8 22 9e f0 f7 0f 0b <55> 41 57 41 56 41 55 41 54 53 49 89 f7 48 89 fb e8 0b 9e f0 f7 48
RSP: 0018:ffffc900071479b8 EFLAGS: 00000283
RAX: 1ffff11003d541bc RBX: ffff88805a8cd000 RCX: dffffc0000000000
RDX: 0000000000000000 RSI: ffff888019f05868 RDI: ffff88801eaa05c0
RBP: ffffc90007147bd8 R08: ffffffff89aafb74 R09: ffffc90007147910
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888036780c80
R13: ffff888019f058c8 R14: ffff888036781ae6 R15: dffffc0000000000
FS: 0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd48ed43a8 CR3: 000000007507b000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
ieee80211_rx_bss_info net/mac80211/ibss.c:1195 [inline]
ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1614 [inline]
ieee80211_ibss_rx_queued_mgmt+0x1923/0x2af0 net/mac80211/ibss.c:1643
ieee80211_iface_process_skb net/mac80211/iface.c:1441 [inline]
ieee80211_iface_work+0x78f/0xcc0 net/mac80211/iface.c:1495
process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310
worker_thread+0xaca/0x1280 kernel/workqueue.c:2457
kthread+0x3f6/0x4f0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup
Reply all
Reply to author
Forward
0 new messages