[syzbot] INFO: task hung in add_early_randomness (2)

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 4b0986a3613c Linux 5.18
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=14d795e5f00000
kernel config: https://syzkaller.appspot.com/x/.config?x=facb2be252153c68
dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+5b59d6...@syzkaller.appspotmail.com

INFO: task kworker/0:5:29071 blocked for more than 143 seconds.
Not tainted 5.18.0-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/0:5 state:D stack:25080 pid:29071 ppid: 2 flags:0x00004000
Workqueue: usb_hub_wq hub_event
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5073 [inline]
__schedule+0xa9a/0x4cc0 kernel/sched/core.c:6388
schedule+0xd2/0x1f0 kernel/sched/core.c:6460
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6519
__mutex_lock_common kernel/locking/mutex.c:673 [inline]
__mutex_lock+0xa32/0x12f0 kernel/locking/mutex.c:733
add_early_randomness+0x1a/0x170 drivers/char/hw_random/core.c:69
hwrng_register+0x399/0x510 drivers/char/hw_random/core.c:599
chaoskey_probe+0x7b5/0xc40 drivers/usb/misc/chaoskey.c:205
usb_probe_interface+0x315/0x7f0 drivers/usb/core/driver.c:396
call_driver_probe drivers/base/dd.c:542 [inline]
really_probe+0x23e/0xb20 drivers/base/dd.c:621
__driver_probe_device+0x338/0x4d0 drivers/base/dd.c:752
driver_probe_device+0x4c/0x1a0 drivers/base/dd.c:782
__device_attach_driver+0x20b/0x2f0 drivers/base/dd.c:899
bus_for_each_drv+0x15f/0x1e0 drivers/base/bus.c:427
__device_attach+0x228/0x4a0 drivers/base/dd.c:970
bus_probe_device+0x1e4/0x290 drivers/base/bus.c:487
device_add+0xb83/0x1e20 drivers/base/core.c:3405
usb_set_configuration+0x101e/0x1900 drivers/usb/core/message.c:2170
usb_generic_driver_probe+0xba/0x100 drivers/usb/core/generic.c:238
usb_probe_device+0xd9/0x2c0 drivers/usb/core/driver.c:293
call_driver_probe drivers/base/dd.c:542 [inline]
really_probe+0x23e/0xb20 drivers/base/dd.c:621
__driver_probe_device+0x338/0x4d0 drivers/base/dd.c:752
driver_probe_device+0x4c/0x1a0 drivers/base/dd.c:782
__device_attach_driver+0x20b/0x2f0 drivers/base/dd.c:899
bus_for_each_drv+0x15f/0x1e0 drivers/base/bus.c:427
__device_attach+0x228/0x4a0 drivers/base/dd.c:970
bus_probe_device+0x1e4/0x290 drivers/base/bus.c:487
device_add+0xb83/0x1e20 drivers/base/core.c:3405
usb_new_device.cold+0x641/0x1091 drivers/usb/core/hub.c:2566
hub_port_connect drivers/usb/core/hub.c:5363 [inline]
hub_port_connect_change drivers/usb/core/hub.c:5507 [inline]
port_event drivers/usb/core/hub.c:5665 [inline]
hub_event+0x25c6/0x4680 drivers/usb/core/hub.c:5747
process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/27:
#0: ffffffff8bd820e0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6467
1 lock held by hwrng/755:
#0: ffffffff8c81b268 (reading_mutex){+.+.}-{3:3}, at: hwrng_fillfn+0x141/0x370 drivers/char/hw_random/core.c:503
2 locks held by getty/3282:
#0: ffff888148d8a098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x22/0x80 drivers/tty/tty_ldisc.c:244
#1: ffffc90002ce62e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xcea/0x1230 drivers/tty/n_tty.c:2075
3 locks held by udevd/19534:
#0: ffff88807b230088 (&of->mutex){+.+.}-{3:3}, at: kernfs_file_read_iter fs/kernfs/file.c:198 [inline]
#0: ffff88807b230088 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_read_iter+0x189/0x6f0 fs/kernfs/file.c:237
#1: ffff88801cf57da0 (kn->active#86){++++}-{0:0}, at: kernfs_file_read_iter fs/kernfs/file.c:199 [inline]
#1: ffff88801cf57da0 (kn->active#86){++++}-{0:0}, at: kernfs_fop_read_iter+0x1ac/0x6f0 fs/kernfs/file.c:237
#2: ffff8880450c0220 (&dev->mutex){....}-{3:3}, at: device_lock_interruptible include/linux/device.h:772 [inline]
#2: ffff8880450c0220 (&dev->mutex){....}-{3:3}, at: read_descriptors+0x3c/0x2c0 drivers/usb/core/sysfs.c:873
6 locks held by kworker/0:5/29071:
#0: ffff8880167d4938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff8880167d4938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
#0: ffff8880167d4938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1280 [inline]
#0: ffff8880167d4938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:636 [inline]
#0: ffff8880167d4938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:663 [inline]
#0: ffff8880167d4938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: process_one_work+0x87a/0x1610 kernel/workqueue.c:2260
#1: ffffc90005507da8 ((work_completion)(&hub->events)){+.+.}-{0:0}, at: process_one_work+0x8ae/0x1610 kernel/workqueue.c:2264
#2: ffff88801e5d3220 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:767 [inline]
#2: ffff88801e5d3220 (&dev->mutex){....}-{3:3}, at: hub_event+0x1c1/0x4680 drivers/usb/core/hub.c:5693
#3: ffff8880450c0220 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:767 [inline]
#3: ffff8880450c0220 (&dev->mutex){....}-{3:3}, at: __device_attach+0x7a/0x4a0 drivers/base/dd.c:945
#4: ffff88801b1351a8 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:767 [inline]
#4: ffff88801b1351a8 (&dev->mutex){....}-{3:3}, at: __device_attach+0x7a/0x4a0 drivers/base/dd.c:945
#5: ffffffff8c81b268 (reading_mutex){+.+.}-{3:3}, at: add_early_randomness+0x1a/0x170 drivers/char/hw_random/core.c:69

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 27 Comm: khungtaskd Not tainted 5.18.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:212 [inline]
watchdog+0xc1d/0xf50 kernel/hung_task.c:369
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5271 Comm: kworker/0:6 Not tainted 5.18.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events kfree_rcu_work
RIP: 0010:deref_stack_reg arch/x86/kernel/unwind_orc.c:355 [inline]
RIP: 0010:unwind_next_frame+0xf70/0x1cc0 arch/x86/kernel/unwind_orc.c:600
Code: 8d 78 20 e8 02 ea ff ff 48 89 c5 49 8d 7d 40 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 c4 0a 00 00 <49> 89 6d 40 48 b8 00 00 00 00 00 fc ff df 4c 89 ea 48 c1 ea 03 0f
RSP: 0018:ffffc900142b7730 EFLAGS: 00000246
RAX: dffffc0000000000 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 1ffff92002856efc RSI: ffffc900142b7ce8 RDI: ffffc900142b77e0
RBP: ffff8880b9c28118 R08: ffffffff8e39a02a R09: ffffc900142b77d4
R10: fffff52002856eff R11: ffffc900142b7d10 R12: ffffc900142b77d5
R13: ffffc900142b77a0 R14: ffffc900142b7ce8 R15: ffffffff8e39a02e
FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f29cddfe718 CR3: 000000007f592000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 000000000000000f DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
arch_stack_walk+0x7d/0xe0 arch/x86/kernel/stacktrace.c:25
stack_trace_save+0x8c/0xc0 kernel/stacktrace.c:122
kasan_save_stack+0x1e/0x40 mm/kasan/common.c:38
kasan_set_track+0x21/0x30 mm/kasan/common.c:45
kasan_set_free_info+0x20/0x30 mm/kasan/generic.c:370
____kasan_slab_free mm/kasan/common.c:366 [inline]
____kasan_slab_free+0x166/0x1a0 mm/kasan/common.c:328
kasan_slab_free include/linux/kasan.h:200 [inline]
slab_free_hook mm/slub.c:1728 [inline]
slab_free_freelist_hook+0x8b/0x1c0 mm/slub.c:1754
slab_free mm/slub.c:3510 [inline]
kmem_cache_free_bulk mm/slub.c:3657 [inline]
kmem_cache_free_bulk+0x2c0/0xb60 mm/slub.c:3644
kfree_bulk include/linux/slab.h:437 [inline]
kfree_rcu_work+0x51c/0xa10 kernel/rcu/tree.c:3318
process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
</TASK>
----------------
Code disassembly (best guess):
0: 8d 78 20 lea 0x20(%rax),%edi
3: e8 02 ea ff ff callq 0xffffea0a
8: 48 89 c5 mov %rax,%rbp
b: 49 8d 7d 40 lea 0x40(%r13),%rdi
f: 48 b8 00 00 00 00 00 movabs $0xdffffc0000000000,%rax
16: fc ff df
19: 48 89 fa mov %rdi,%rdx
1c: 48 c1 ea 03 shr $0x3,%rdx
20: 80 3c 02 00 cmpb $0x0,(%rdx,%rax,1)
24: 0f 85 c4 0a 00 00 jne 0xaee
* 2a: 49 89 6d 40 mov %rbp,0x40(%r13) <-- trapping instruction
2e: 48 b8 00 00 00 00 00 movabs $0xdffffc0000000000,%rax
35: fc ff df
38: 4c 89 ea mov %r13,%rdx
3b: 48 c1 ea 03 shr $0x3,%rdx
3f: 0f .byte 0xf


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: f2906aa86338 Linux 5.19-rc1
git tree: upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=15d62393f00000
kernel config: https://syzkaller.appspot.com/x/.config?x=cbd131cc02ee620e

dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=104f4d4ff00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14d6782df00000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+5b59d6...@syzkaller.appspotmail.com

INFO: task kworker/1:4:2941 blocked for more than 143 seconds.
Not tainted 5.19.0-rc1-syzkaller #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:kworker/1:4 state:D stack:23968 pid: 2941 ppid: 2 flags:0x00004000

Workqueue: usb_hub_wq hub_event
Call Trace:
<TASK>

context_switch kernel/sched/core.c:5116 [inline]
__schedule+0xa00/0x4b30 kernel/sched/core.c:6428
schedule+0xd2/0x1f0 kernel/sched/core.c:6500
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6559
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0xa70/0x1350 kernel/locking/mutex.c:747

add_early_randomness+0x1a/0x170 drivers/char/hw_random/core.c:69
hwrng_register+0x399/0x510 drivers/char/hw_random/core.c:599
chaoskey_probe+0x7b5/0xc40 drivers/usb/misc/chaoskey.c:205
usb_probe_interface+0x315/0x7f0 drivers/usb/core/driver.c:396

call_driver_probe drivers/base/dd.c:555 [inline]
really_probe+0x23e/0xb90 drivers/base/dd.c:634
__driver_probe_device+0x338/0x4d0 drivers/base/dd.c:764
driver_probe_device+0x4c/0x1a0 drivers/base/dd.c:794
__device_attach_driver+0x20b/0x2f0 drivers/base/dd.c:917
bus_for_each_drv+0x15f/0x1e0 drivers/base/bus.c:427
__device_attach+0x1e4/0x530 drivers/base/dd.c:989
bus_probe_device+0x1e4/0x290 drivers/base/bus.c:487
device_add+0xbda/0x1ea0 drivers/base/core.c:3417

usb_set_configuration+0x101e/0x1900 drivers/usb/core/message.c:2170
usb_generic_driver_probe+0xba/0x100 drivers/usb/core/generic.c:238
usb_probe_device+0xd9/0x2c0 drivers/usb/core/driver.c:293

call_driver_probe drivers/base/dd.c:555 [inline]
really_probe+0x23e/0xb90 drivers/base/dd.c:634
__driver_probe_device+0x338/0x4d0 drivers/base/dd.c:764
driver_probe_device+0x4c/0x1a0 drivers/base/dd.c:794
__device_attach_driver+0x20b/0x2f0 drivers/base/dd.c:917
bus_for_each_drv+0x15f/0x1e0 drivers/base/bus.c:427
__device_attach+0x1e4/0x530 drivers/base/dd.c:989
bus_probe_device+0x1e4/0x290 drivers/base/bus.c:487
device_add+0xbda/0x1ea0 drivers/base/core.c:3417

usb_new_device.cold+0x641/0x1091 drivers/usb/core/hub.c:2566
hub_port_connect drivers/usb/core/hub.c:5363 [inline]
hub_port_connect_change drivers/usb/core/hub.c:5507 [inline]

port_event drivers/usb/core/hub.c:5663 [inline]
hub_event+0x25d5/0x4690 drivers/usb/core/hub.c:5745

process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376

ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302

</TASK>

Showing all locks held in the system:

2 locks held by pr/ttyS0/16:
1 lock held by khungtaskd/29:
#0: ffffffff8bd83be0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6491

1 lock held by hwrng/755:

#0: ffffffff8c829c28 (reading_mutex){+.+.}-{3:3}, at: hwrng_fillfn+0x141/0x370 drivers/char/hw_random/core.c:503
6 locks held by kworker/1:4/2941:
#0: ffff888017505538 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff888017505538 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
#0: ffff888017505538 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1280 [inline]
#0: ffff888017505538 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:636 [inline]
#0: ffff888017505538 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:663 [inline]
#0: ffff888017505538 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: process_one_work+0x87a/0x1610 kernel/workqueue.c:2260
#1: ffffc9000306fda8 ((work_completion)(&hub->events)){+.+.}-{0:0}, at: process_one_work+0x8ae/0x1610 kernel/workqueue.c:2264
#2: ffff8881476d6190 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:835 [inline]
#2: ffff8881476d6190 (&dev->mutex){....}-{3:3}, at: hub_event+0x1c1/0x4690 drivers/usb/core/hub.c:5691
#3: ffff888017679190 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:835 [inline]
#3: ffff888017679190 (&dev->mutex){....}-{3:3}, at: __device_attach+0x76/0x530 drivers/base/dd.c:964
#4: ffff888017670118 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:835 [inline]
#4: ffff888017670118 (&dev->mutex){....}-{3:3}, at: __device_attach+0x76/0x530 drivers/base/dd.c:964
#5: ffffffff8c829c28 (reading_mutex){+.+.}-{3:3}, at: add_early_randomness+0x1a/0x170 drivers/char/hw_random/core.c:69
2 locks held by getty/3293:
#0: ffff88802858a098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x22/0x80 drivers/tty/tty_ldisc.c:244
#1: ffffc90001c182e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xe50/0x13c0 drivers/tty/n_tty.c:2124

=============================================

NMI backtrace for cpu 1

CPU: 1 PID: 29 Comm: khungtaskd Not tainted 5.19.0-rc1-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]

check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
watchdog+0xc22/0xf90 kernel/hung_task.c:378
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302

</TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0

CPU: 0 PID: 16 Comm: pr/ttyS0 Not tainted 5.19.0-rc1-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

RIP: 0010:io_serial_in+0x83/0xa0 drivers/tty/serial/8250/8250_port.c:461
Code: 29 1a fd 48 8d 7d 40 44 89 e1 48 b8 00 00 00 00 00 fc ff df 48 89 fa d3 e3 48 c1 ea 03 80 3c 02 00 75 16 66 03 5d 40 89 da ec <5b> 0f b6 c0 5d 41 5c c3 e8 10 4d 66 fd eb a6 e8 39 4d 66 fd eb e3
RSP: 0018:ffffc9000036fb10 EFLAGS: 00000002
RAX: dffffc0000000000 RBX: 00000000000003fd RCX: 0000000000000000
RDX: 00000000000003fd RSI: ffffffff845f7e1c RDI: ffffffff91221a80
RBP: ffffffff91221a40 R08: 0000000000000001 R09: 000000000000001f
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
R13: fffffbfff224439c R14: fffffbfff2244352 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 000055f845ac6200 CR3: 000000007dccd000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
serial_in drivers/tty/serial/8250/8250.h:118 [inline]
wait_for_lsr+0x96/0xf0 drivers/tty/serial/8250/8250_port.c:2085
serial8250_console_fifo_write drivers/tty/serial/8250/8250_port.c:3354 [inline]
serial8250_console_write+0xc87/0xfe0 drivers/tty/serial/8250/8250_port.c:3436
call_console_driver kernel/printk/printk.c:2075 [inline]
__console_emit_next_record+0x896/0xa60 kernel/printk/printk.c:2916
console_emit_next_record kernel/printk/printk.c:3721 [inline]
printk_kthread_func.cold+0x702/0x73d kernel/printk/printk.c:3837
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302

</TASK>
----------------
Code disassembly (best guess):

0: 29 1a sub %ebx,(%rdx)
2: fd std
3: 48 8d 7d 40 lea 0x40(%rbp),%rdi
7: 44 89 e1 mov %r12d,%ecx
a: 48 b8 00 00 00 00 00 movabs $0xdffffc0000000000,%rax
11: fc ff df
14: 48 89 fa mov %rdi,%rdx
17: d3 e3 shl %cl,%ebx
19: 48 c1 ea 03 shr $0x3,%rdx
1d: 80 3c 02 00 cmpb $0x0,(%rdx,%rax,1)
21: 75 16 jne 0x39
23: 66 03 5d 40 add 0x40(%rbp),%bx
27: 89 da mov %ebx,%edx
29: ec in (%dx),%al
* 2a: 5b pop %rbx <-- trapping instruction
2b: 0f b6 c0 movzbl %al,%eax
2e: 5d pop %rbp
2f: 41 5c pop %r12
31: c3 retq
32: e8 10 4d 66 fd callq 0xfd664d47
37: eb a6 jmp 0xffffffdf
39: e8 39 4d 66 fd callq 0xfd664d77
3e: eb e3 jmp 0x23

[syzbot]

syzbot has bisected this issue to:

commit 8b4ec69d7e098a7ddf832e1e7840de53ed474c77
Author: Jason Wang <jaso...@redhat.com>
Date: Fri May 27 06:01:19 2022 +0000

virtio: harden vring IRQ

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1175c3c7f00000
start commit: f2906aa86338 Linux 5.19-rc1
git tree: upstream
final oops: https://syzkaller.appspot.com/x/report.txt?x=1375c3c7f00000
console output: https://syzkaller.appspot.com/x/log.txt?x=1575c3c7f00000

kernel config: https://syzkaller.appspot.com/x/.config?x=cbd131cc02ee620e
dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=104f4d4ff00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14d6782df00000

Reported-by: syzbot+5b59d6...@syzkaller.appspotmail.com
Fixes: 8b4ec69d7e09 ("virtio: harden vring IRQ")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Jason Wang]

I wonder if it's related to shared IRQ.

Want to know if the attached patch works.

Thanks

[Michael S. Tsirkin]

syzbot will test it for you if you ask it nicely.
See e.g.
https://lore.kernel.org/r/20220221054115.1270-1-hdanton%40sina.com

for an example of how to do it.

--
MST

[Michael S. Tsirkin]

On Tue, Jun 07, 2022 at 05:05:41PM +0800, Jason Wang wrote:

Please, post patches inline.
In this case I don't see the printk in the console.

[Jason Wang]

Ok, let's try this:

#syz test
git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 13a7348cedff..1af55e576505 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2137,7 +2137,7 @@ irqreturn_t vring_interrupt(int irq, void *_vq)
if (unlikely(vq->broken)) {
dev_warn_once(&vq->vq.vdev->dev,
"virtio vring IRQ raised before DRIVER_OK");
- return IRQ_NONE;
+ return IRQ_HANDLED;
}

/* Just a hint for performance: so it's ok that this can be racy! */

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in add_early_randomness

INFO: task kworker/0:0:6 blocked for more than 143 seconds.
Not tainted 5.18.0-syzkaller-11503-gbd8bb9aed56b-dirty #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:kworker/0:0 state:D stack:24728 pid: 6 ppid: 2 flags:0x00004000

Workqueue: usb_hub_wq hub_event
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5116 [inline]

__schedule+0xa00/0x4b30 kernel/sched/core.c:6431
schedule+0xd2/0x1f0 kernel/sched/core.c:6503
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6562

__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0xa70/0x1350 kernel/locking/mutex.c:747
add_early_randomness+0x1a/0x170 drivers/char/hw_random/core.c:69
hwrng_register+0x399/0x510 drivers/char/hw_random/core.c:599
chaoskey_probe+0x7b5/0xc40 drivers/usb/misc/chaoskey.c:205
usb_probe_interface+0x315/0x7f0 drivers/usb/core/driver.c:396

call_driver_probe drivers/base/dd.c:542 [inline]
really_probe+0x23e/0xb20 drivers/base/dd.c:621
__driver_probe_device+0x338/0x4d0 drivers/base/dd.c:752
driver_probe_device+0x4c/0x1a0 drivers/base/dd.c:782
__device_attach_driver+0x20b/0x2f0 drivers/base/dd.c:899
bus_for_each_drv+0x15f/0x1e0 drivers/base/bus.c:427
__device_attach+0x228/0x4a0 drivers/base/dd.c:970
bus_probe_device+0x1e4/0x290 drivers/base/bus.c:487

device_add+0xb83/0x1e20 drivers/base/core.c:3402

usb_set_configuration+0x101e/0x1900 drivers/usb/core/message.c:2170
usb_generic_driver_probe+0xba/0x100 drivers/usb/core/generic.c:238
usb_probe_device+0xd9/0x2c0 drivers/usb/core/driver.c:293

call_driver_probe drivers/base/dd.c:542 [inline]
really_probe+0x23e/0xb20 drivers/base/dd.c:621
__driver_probe_device+0x338/0x4d0 drivers/base/dd.c:752
driver_probe_device+0x4c/0x1a0 drivers/base/dd.c:782
__device_attach_driver+0x20b/0x2f0 drivers/base/dd.c:899
bus_for_each_drv+0x15f/0x1e0 drivers/base/bus.c:427
__device_attach+0x228/0x4a0 drivers/base/dd.c:970
bus_probe_device+0x1e4/0x290 drivers/base/bus.c:487

device_add+0xb83/0x1e20 drivers/base/core.c:3402

usb_new_device.cold+0x641/0x1091 drivers/usb/core/hub.c:2566
hub_port_connect drivers/usb/core/hub.c:5363 [inline]
hub_port_connect_change drivers/usb/core/hub.c:5507 [inline]

port_event drivers/usb/core/hub.c:5665 [inline]
hub_event+0x25c6/0x4680 drivers/usb/core/hub.c:5747

process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302
</TASK>

Showing all locks held in the system:

6 locks held by kworker/0:0/6:
#0: ffff88801756a938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff88801756a938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
#0: ffff88801756a938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1280 [inline]
#0: ffff88801756a938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:636 [inline]
#0: ffff88801756a938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:663 [inline]
#0: ffff88801756a938 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: process_one_work+0x87a/0x1610 kernel/workqueue.c:2260
#1: ffffc900002cfda8 ((work_completion)(&hub->events)){+.+.}-{0:0}, at: process_one_work+0x8ae/0x1610 kernel/workqueue.c:2264
#2: ffff888147718190 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:762 [inline]
#2: ffff888147718190 (&dev->mutex){....}-{3:3}, at: hub_event+0x1c1/0x4680 drivers/usb/core/hub.c:5693
#3: ffff88806a2f0190 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:762 [inline]
#3: ffff88806a2f0190 (&dev->mutex){....}-{3:3}, at: __device_attach+0x7a/0x4a0 drivers/base/dd.c:945
#4: ffff88806a270118 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:762 [inline]
#4: ffff88806a270118 (&dev->mutex){....}-{3:3}, at: __device_attach+0x7a/0x4a0 drivers/base/dd.c:945
#5: ffffffff8c828728 (reading_mutex){+.+.}-{3:3}, at: add_early_randomness+0x1a/0x170 drivers/char/hw_random/core.c:69

1 lock held by khungtaskd/29:

#0: ffffffff8bd840e0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6491
2 locks held by kworker/u4:5/58:
#0: ffff8880b9a39f98 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x2b/0x120 kernel/sched/core.c:544
#1: ffff8880b9a277c8 (&per_cpu_ptr(group->pcpu, cpu)->seq){-.-.}-{0:0}, at: psi_task_switch+0x176/0x4e0 kernel/sched/psi.c:880
1 lock held by hwrng/756:
#0: ffffffff8c828728 (reading_mutex){+.+.}-{3:3}, at: hwrng_fillfn+0x141/0x370 drivers/char/hw_random/core.c:503
2 locks held by getty/3281:
#0: ffff888022b5a098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x22/0x80 drivers/tty/tty_ldisc.c:244
#1: ffffc90001c382e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xcea/0x1230 drivers/tty/n_tty.c:2075

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 5.18.0-syzkaller-11503-gbd8bb9aed56b-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
watchdog+0xc22/0xf90 kernel/hung_task.c:378
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302
</TASK>

Sending NMI from CPU 0 to CPUs 1:

NMI backtrace for cpu 1

CPU: 1 PID: 1389 Comm: kworker/u4:7 Not tainted 5.18.0-syzkaller-11503-gbd8bb9aed56b-dirty #0
CPU: 1 PID: 1389 Comm: kworker/u4:7 Not tainted 5.18.0-syzkaller-11503-gbd8bb9aed56b-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

Workqueue: phy5 ieee80211_iface_work
RIP: 0010:unwind_next_frame+0x8c5/0x1cc0 arch/x86/kernel/unwind_orc.c:466
Code: ff 80 3d a8 c0 70 0c 00 0f 85 ee fa ff ff e9 22 54 fb 07 48 b8 00 00 00 00 00 fc ff df 48 8b 54 24 08 48 c1 ea 03 80 3c 02 00 <0f> 85 73 10 00 00 4c 89 c0 4d 8b 75 38 48 ba 00 00 00 00 00 fc ff
RSP: 0018:ffffc900059e7528 EFLAGS: 00000246
RAX: dffffc0000000000 RBX: 0000000000000001 RCX: ffffffff8e3c9c1f
RDX: 1ffff92000b3ceba RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffffc900059e75e0 R08: ffffffff8e3c9c1a R09: ffffc900059e75cc
R10: fffff52000b3cebe R11: 000000000008a07a R12: ffffc900059e75cd
R13: ffffc900059e7598 R14: ffffffff81689fbb R15: ffffffff8e3c9c1e
FS: 0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007f00c3e11110 CR3: 000000000ba8e000 CR4: 00000000003506e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>

arch_stack_walk+0x7d/0xe0 arch/x86/kernel/stacktrace.c:25
stack_trace_save+0x8c/0xc0 kernel/stacktrace.c:122
kasan_save_stack+0x1e/0x40 mm/kasan/common.c:38

kasan_set_track mm/kasan/common.c:45 [inline]
set_alloc_info mm/kasan/common.c:436 [inline]
____kasan_kmalloc mm/kasan/common.c:515 [inline]
____kasan_kmalloc mm/kasan/common.c:474 [inline]
__kasan_kmalloc+0xa6/0xd0 mm/kasan/common.c:524
kasan_kmalloc include/linux/kasan.h:234 [inline]
kmem_cache_alloc_trace+0x1ea/0x4a0 mm/slab.c:3569
kmalloc include/linux/slab.h:600 [inline]
kzalloc include/linux/slab.h:733 [inline]
ieee802_11_parse_elems_crc+0xd5/0x1060 net/mac80211/util.c:1502
ieee802_11_parse_elems net/mac80211/ieee80211_i.h:2257 [inline]
ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1605 [inline]
ieee80211_ibss_rx_queued_mgmt+0xda5/0x33f0 net/mac80211/ibss.c:1639
ieee80211_iface_process_skb net/mac80211/iface.c:1527 [inline]
ieee80211_iface_work+0xa78/0xd10 net/mac80211/iface.c:1581

process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302
</TASK>

----------------
Code disassembly (best guess):

0: ff 80 3d a8 c0 70 incl 0x70c0a83d(%rax)
6: 0c 00 or $0x0,%al
8: 0f 85 ee fa ff ff jne 0xfffffafc
e: e9 22 54 fb 07 jmpq 0x7fb5435
13: 48 b8 00 00 00 00 00 movabs $0xdffffc0000000000,%rax
1a: fc ff df
1d: 48 8b 54 24 08 mov 0x8(%rsp),%rdx
22: 48 c1 ea 03 shr $0x3,%rdx
26: 80 3c 02 00 cmpb $0x0,(%rdx,%rax,1)
* 2a: 0f 85 73 10 00 00 jne 0x10a3 <-- trapping instruction
30: 4c 89 c0 mov %r8,%rax
33: 4d 8b 75 38 mov 0x38(%r13),%r14
37: 48 rex.W
38: ba 00 00 00 00 mov $0x0,%edx
3d: 00 fc add %bh,%ah
3f: ff .byte 0xff


Tested on:

commit: bd8bb9ae vdpa: ifcvf: set pci driver data in probe
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=162120bff00000
kernel config: https://syzkaller.appspot.com/x/.config?x=8d4cbb773055caee

dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=141d9e6ff00000

[Jason Wang]

On Wed, Jun 8, 2022 at 9:37 AM syzbot
<syzbot+5b59d6...@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> INFO: task hung in add_early_randomness
>

Ok, looking at virtio-rng.c, probe_common() tries to kick the
virtqueue before DRIVER_OK which is kind of violates the spec.

Please test the following patch:

#syz test
git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next

diff --git a/drivers/char/hw_random/virtio-rng.c
b/drivers/char/hw_random/virtio-rng.c
index e856df7e285c..a6f3a8a2aca6 100644
--- a/drivers/char/hw_random/virtio-rng.c
+++ b/drivers/char/hw_random/virtio-rng.c
@@ -159,6 +159,8 @@ static int probe_common(struct virtio_device *vdev)
goto err_find;
}

+ virtio_device_ready(vdev);
+
/* we always have a pending entropy request */
request_entropy(vi);

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to apply patch:
checking file drivers/char/hw_random/virtio-rng.c
patch: **** unexpected end of file in patch

Tested on:

commit: bd8bb9ae vdpa: ifcvf: set pci driver data in probe
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next

dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54
compiler:

patch: https://syzkaller.appspot.com/x/patch.diff?x=15f814cff00000

[Jason Wang]

On Wed, Jun 8, 2022 at 10:37 AM syzbot
<syzbot+5b59d6...@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot tried to test the proposed patch but the build/boot failed:
>
> failed to apply patch:
> checking file drivers/char/hw_random/virtio-rng.c
> patch: **** unexpected end of file in patch
>

Copy-paste error :(

Let's try this:

#syz test
git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next

diff --git a/drivers/char/hw_random/virtio-rng.c
b/drivers/char/hw_random/virtio-rng.c
index e856df7e285c..a6f3a8a2aca6 100644
--- a/drivers/char/hw_random/virtio-rng.c
+++ b/drivers/char/hw_random/virtio-rng.c
@@ -159,6 +159,8 @@ static int probe_common(struct virtio_device *vdev)
goto err_find;
}

+ virtio_device_ready(vdev);
+
/* we always have a pending entropy request */
request_entropy(vi);

--

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to apply patch:
checking file drivers/char/hw_random/virtio-rng.c
patch: **** unexpected end of file in patch

Tested on:

commit: bd8bb9ae vdpa: ifcvf: set pci driver data in probe
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54
compiler:

patch: https://syzkaller.appspot.com/x/patch.diff?x=165a0febf00000

[Jason Wang]

On Wed, Jun 8, 2022 at 10:51 AM syzbot
<syzbot+5b59d6...@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot tried to test the proposed patch but the build/boot failed:
>
> failed to apply patch:
> checking file drivers/char/hw_random/virtio-rng.c
> patch: **** unexpected end of file in patch
>
>

It looks like something is wrong in my email client.

Let's try with attachment.

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+5b59d6...@syzkaller.appspotmail.com

Tested on:

commit: bd8bb9ae vdpa: ifcvf: set pci driver data in probe
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next

kernel config: https://syzkaller.appspot.com/x/.config?x=8d4cbb773055caee
dashboard link: https://syzkaller.appspot.com/bug?extid=5b59d6d459306a556f54

compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=1731eaf7f00000

Note: testing is done by a robot and is best-effort only.

[Jason Wang]

Cool, let me post a formal patch.

Thanks