INFO: task hung in nfnetlink_net_exit_batch

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: ce5b127b172e Merge branch 's390-qeth-next'
git tree: net-next
console output: https://syzkaller.appspot.com/x/log.txt?x=10e27111400000
kernel config: https://syzkaller.appspot.com/x/.config?x=4b80e2f7c5d2b2c3
dashboard link: https://syzkaller.appspot.com/bug?extid=80a2b6a3afb5e783e4c1
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
CC: [core...@netfilter.org da...@davemloft.net f...@strlen.de
kad...@blackhole.kfki.hu linux-...@vger.kernel.org
net...@vger.kernel.org netfilt...@vger.kernel.org pa...@netfilter.org]

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+80a2b6...@syzkaller.appspotmail.com

INFO: task kworker/u4:2:30 blocked for more than 140 seconds.
Not tainted 4.19.0-rc3+ #219
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u4:2 D15048 30 2 0x80000000
Workqueue: netns cleanup_net
Call Trace:
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x86c/0x1ed0 kernel/sched/core.c:3473
schedule+0xfe/0x460 kernel/sched/core.c:3517
exp_funnel_lock kernel/rcu/tree_exp.h:320 [inline]
_synchronize_rcu_expedited+0xc68/0xfd0 kernel/rcu/tree_exp.h:667
synchronize_rcu_expedited+0x35/0xb0 kernel/rcu/tree_exp.h:795
synchronize_net+0x3b/0x60 net/core/dev.c:9013
nfnetlink_net_exit_batch+0x10b/0x200 net/netfilter/nfnetlink.c:608
ops_exit_list.isra.7+0x105/0x160 net/core/net_namespace.c:156
cleanup_net+0x555/0xb10 net/core/net_namespace.c:551
process_one_work+0xc90/0x1b90 kernel/workqueue.c:2153
worker_thread+0x17f/0x1390 kernel/workqueue.c:2296
kthread+0x35a/0x420 kernel/kthread.c:246
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:413
INFO: task syz-executor3:12268 blocked for more than 140 seconds.
Not tainted 4.19.0-rc3+ #219
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syz-executor3 D25272 12268 5360 0x00000004
Call Trace:
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x86c/0x1ed0 kernel/sched/core.c:3473
schedule+0xfe/0x460 kernel/sched/core.c:3517
__rwsem_down_write_failed_common+0xbb9/0x1670
kernel/locking/rwsem-xadd.c:566
rwsem_down_write_failed+0xe/0x10 kernel/locking/rwsem-xadd.c:595
call_rwsem_down_write_failed+0x17/0x30 arch/x86/lib/rwsem.S:117
__down_write arch/x86/include/asm/rwsem.h:142 [inline]
down_write+0xa5/0x130 kernel/locking/rwsem.c:72
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
raw_init+0x2a3/0x330 net/can/raw.c:343
can_create+0x28a/0x4b0 net/can/af_can.c:183
__sock_create+0x536/0x930 net/socket.c:1275
sock_create net/socket.c:1315 [inline]
__sys_socket+0x106/0x260 net/socket.c:1345
__do_sys_socket net/socket.c:1354 [inline]
__se_sys_socket net/socket.c:1352 [inline]
__x64_sys_socket+0x73/0xb0 net/socket.c:1352
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4572d9
Code: 42 08 48 8b 40 08 48 89 04 24 e8 02 d3 fc ff 48 8b 6c 24 08 48 83 c4
10 c3 e8 e3 2b 00 00 eb c1 cc 64 48 8b 0c 25 f8 ff ff ff <48> 3b 61 10 76
75 48 83 ec 38 48 89 6c 24 30 48 8d 6c 24 30 48 8b
RSP: 002b:00007f967300ec78 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 00007f967300f6d4 RCX: 00000000004572d9
RDX: 0000000000000001 RSI: 0000000000000003 RDI: 000000000000001d
RBP: 00000000009300a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004d6c48 R14: 00000000004c9f97 R15: 0000000000000000
INFO: task syz-executor3:12277 blocked for more than 140 seconds.
Not tainted 4.19.0-rc3+ #219
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syz-executor3 D25272 12277 5360 0x00000004
Call Trace:
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x86c/0x1ed0 kernel/sched/core.c:3473
schedule+0xfe/0x460 kernel/sched/core.c:3517
__rwsem_down_write_failed_common+0xbb9/0x1670
kernel/locking/rwsem-xadd.c:566
rwsem_down_write_failed+0xe/0x10 kernel/locking/rwsem-xadd.c:595
call_rwsem_down_write_failed+0x17/0x30 arch/x86/lib/rwsem.S:117
__down_write arch/x86/include/asm/rwsem.h:142 [inline]
down_write+0xa5/0x130 kernel/locking/rwsem.c:72
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
raw_init+0x2a3/0x330 net/can/raw.c:343
can_create+0x28a/0x4b0 net/can/af_can.c:183
__sock_create+0x536/0x930 net/socket.c:1275
sock_create net/socket.c:1315 [inline]
__sys_socket+0x106/0x260 net/socket.c:1345
__do_sys_socket net/socket.c:1354 [inline]
__se_sys_socket net/socket.c:1352 [inline]
__x64_sys_socket+0x73/0xb0 net/socket.c:1352
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4572d9
Code: 42 08 48 8b 40 08 48 89 04 24 e8 02 d3 fc ff 48 8b 6c 24 08 48 83 c4
10 c3 e8 e3 2b 00 00 eb c1 cc 64 48 8b 0c 25 f8 ff ff ff <48> 3b 61 10 76
75 48 83 ec 38 48 89 6c 24 30 48 8d 6c 24 30 48 8b
RSP: 002b:00007f9672fedc78 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 00007f9672fee6d4 RCX: 00000000004572d9
RDX: 0000000000000001 RSI: 0000000000000003 RDI: 000000000000001d
RBP: 0000000000930140 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004d6c48 R14: 00000000004c9f97 R15: 0000000000000001
INFO: task syz-executor3:12283 blocked for more than 140 seconds.
Not tainted 4.19.0-rc3+ #219
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syz-executor3 D25400 12283 5360 0x00000004
Call Trace:
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x86c/0x1ed0 kernel/sched/core.c:3473
schedule+0xfe/0x460 kernel/sched/core.c:3517
__rwsem_down_write_failed_common+0xbb9/0x1670
kernel/locking/rwsem-xadd.c:566
rwsem_down_write_failed+0xe/0x10 kernel/locking/rwsem-xadd.c:595
call_rwsem_down_write_failed+0x17/0x30 arch/x86/lib/rwsem.S:117
__down_write arch/x86/include/asm/rwsem.h:142 [inline]
down_write+0xa5/0x130 kernel/locking/rwsem.c:72
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
raw_init+0x2a3/0x330 net/can/raw.c:343
can_create+0x28a/0x4b0 net/can/af_can.c:183
__sock_create+0x536/0x930 net/socket.c:1275
sock_create net/socket.c:1315 [inline]
__sys_socket+0x106/0x260 net/socket.c:1345
__do_sys_socket net/socket.c:1354 [inline]
__se_sys_socket net/socket.c:1352 [inline]
__x64_sys_socket+0x73/0xb0 net/socket.c:1352
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4572d9
Code: 42 08 48 8b 40 08 48 89 04 24 e8 02 d3 fc ff 48 8b 6c 24 08 48 83 c4
10 c3 e8 e3 2b 00 00 eb c1 cc 64 48 8b 0c 25 f8 ff ff ff <48> 3b 61 10 76
75 48 83 ec 38 48 89 6c 24 30 48 8d 6c 24 30 48 8b
RSP: 002b:00007f9672fccc78 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 00007f9672fcd6d4 RCX: 00000000004572d9
RDX: 0000000000000001 RSI: 0000000000000003 RDI: 000000000000001d
RBP: 00000000009301e0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004d6c48 R14: 00000000004c9f97 R15: 0000000000000002
INFO: task syz-executor3:12327 blocked for more than 140 seconds.
Not tainted 4.19.0-rc3+ #219
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syz-executor3 D25272 12327 5360 0x00000004
Call Trace:
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x86c/0x1ed0 kernel/sched/core.c:3473
schedule+0xfe/0x460 kernel/sched/core.c:3517
__rwsem_down_write_failed_common+0xbb9/0x1670
kernel/locking/rwsem-xadd.c:566
rwsem_down_write_failed+0xe/0x10 kernel/locking/rwsem-xadd.c:595
call_rwsem_down_write_failed+0x17/0x30 arch/x86/lib/rwsem.S:117
__down_write arch/x86/include/asm/rwsem.h:142 [inline]
down_write+0xa5/0x130 kernel/locking/rwsem.c:72
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
raw_init+0x2a3/0x330 net/can/raw.c:343
can_create+0x28a/0x4b0 net/can/af_can.c:183
__sock_create+0x536/0x930 net/socket.c:1275
sock_create net/socket.c:1315 [inline]
__sys_socket+0x106/0x260 net/socket.c:1345
__do_sys_socket net/socket.c:1354 [inline]
__se_sys_socket net/socket.c:1352 [inline]
__x64_sys_socket+0x73/0xb0 net/socket.c:1352
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4572d9
Code: 42 08 48 8b 40 08 48 89 04 24 e8 02 d3 fc ff 48 8b 6c 24 08 48 83 c4
10 c3 e8 e3 2b 00 00 eb c1 cc 64 48 8b 0c 25 f8 ff ff ff <48> 3b 61 10 76
75 48 83 ec 38 48 89 6c 24 30 48 8d 6c 24 30 48 8b
RSP: 002b:00007f9672f8ac78 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 00007f9672f8b6d4 RCX: 00000000004572d9
RDX: 0000000000000001 RSI: 0000000000000003 RDI: 000000000000001d
RBP: 0000000000930320 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004d6c48 R14: 00000000004c9f97 R15: 0000000000000004

Showing all locks held in the system:
2 locks held by kworker/1:1/24:
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
__write_once_size include/linux/compiler.h:215 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at: atomic64_set
include/asm-generic/atomic-instrumented.h:40 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at: atomic_long_set
include/asm-generic/atomic-long.h:59 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at: set_work_data
kernel/workqueue.c:617 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
process_one_work+0xb43/0x1b90 kernel/workqueue.c:2124
#1: 000000002b95467a ((work_completion)(&smc->tcp_listen_work)){+.+.}, at:
process_one_work+0xb9a/0x1b90 kernel/workqueue.c:2128
3 locks held by kworker/u4:2/30:
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at:
__write_once_size include/linux/compiler.h:215 [inline]
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at:
arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at: atomic64_set
include/asm-generic/atomic-instrumented.h:40 [inline]
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at:
atomic_long_set include/asm-generic/atomic-long.h:59 [inline]
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at: set_work_data
kernel/workqueue.c:617 [inline]
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at:
set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline]
#0: 00000000b9edb4fd ((wq_completion)"%s""netns"){+.+.}, at:
process_one_work+0xb43/0x1b90 kernel/workqueue.c:2124
#1: 000000005355ac7b (net_cleanup_work){+.+.}, at:
process_one_work+0xb9a/0x1b90 kernel/workqueue.c:2128
#2: 000000004c2fc324 (pernet_ops_rwsem){++++}, at: cleanup_net+0x13f/0xb10
net/core/net_namespace.c:518
1 lock held by khungtaskd/984:
#0: 000000000318e4c4 (rcu_read_lock){....}, at:
debug_show_all_locks+0xd0/0x424 kernel/locking/lockdep.c:4436
1 lock held by rsyslogd/5223:
#0: 0000000032585ac7 (&f->f_pos_lock){+.+.}, at: __fdget_pos+0x1bb/0x200
fs/file.c:766
2 locks held by getty/5313:
#0: 00000000e7719689 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000c0b110d2 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5314:
#0: 00000000c88a30cb (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000d54e9f28 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5315:
#0: 00000000814d47cb (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000eda76c52 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5316:
#0: 00000000e790a1fd (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 000000000f5d6aa4 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5317:
#0: 0000000080144687 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000d3b684e4 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5318:
#0: 000000007174db53 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 00000000ca8338a3 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/5319:
#0: 00000000399eb149 (&tty->ldisc_sem){++++}, at:
ldsem_down_read+0x32/0x40 drivers/tty/tty_ldsem.c:353
#1: 000000003e069785 (&ldata->atomic_read_lock){+.+.}, at:
n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by kworker/1:3/5770:
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
__write_once_size include/linux/compiler.h:215 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at: atomic64_set
include/asm-generic/atomic-instrumented.h:40 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at: atomic_long_set
include/asm-generic/atomic-long.h:59 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at: set_work_data
kernel/workqueue.c:617 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline]
#0: 00000000dde524d0 ((wq_completion)"events"){+.+.}, at:
process_one_work+0xb43/0x1b90 kernel/workqueue.c:2124
#1: 000000006eacc246 ((work_completion)(&smc->tcp_listen_work)){+.+.}, at:
process_one_work+0xb9a/0x1b90 kernel/workqueue.c:2128
1 lock held by syz-executor3/12268:
#0: 000000004c2fc324 (pernet_ops_rwsem){++++}, at:
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
1 lock held by syz-executor3/12277:
#0: 000000004c2fc324 (pernet_ops_rwsem){++++}, at:
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
1 lock held by syz-executor3/12283:
#0: 000000004c2fc324 (pernet_ops_rwsem){++++}, at:
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628
1 lock held by syz-executor3/12327:
#0: 000000004c2fc324 (pernet_ops_rwsem){++++}, at:
register_netdevice_notifier+0xbc/0x910 net/core/dev.c:1628

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 984 Comm: khungtaskd Not tainted 4.19.0-rc3+ #219
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
nmi_cpu_backtrace.cold.3+0x63/0xa2 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x1b3/0x1ed lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_all_cpu_backtrace include/linux/nmi.h:144 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:204 [inline]
watchdog+0xb3e/0x1050 kernel/hung_task.c:265
kthread+0x35a/0x420 kernel/kthread.c:246
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:413
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x6/0x10
arch/x86/include/asm/irqflags.h:57


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

[syzbot]

Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.