[syzbot] INFO: task hung in hci_req_sync

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: c9194f32bfd9 Merge tag 'ext4_for_linus_stable' of git://gi..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1488f59e300000
kernel config: https://syzkaller.appspot.com/x/.config?x=343fd21f6f4da2d6
dashboard link: https://syzkaller.appspot.com/bug?extid=be2baed593ea56c6a84c
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15b5afc6300000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15fcd192300000

Bisection is inconclusive: the issue happens on the oldest tested release.

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=17dce4fa300000
final oops: https://syzkaller.appspot.com/x/report.txt?x=143ce4fa300000
console output: https://syzkaller.appspot.com/x/log.txt?x=103ce4fa300000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+be2bae...@syzkaller.appspotmail.com

INFO: task syz-executor446:8489 blocked for more than 143 seconds.
Not tainted 5.14.0-rc4-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor446 state:D stack:28712 pid: 8489 ppid: 8452 flags:0x00000000
Call Trace:
context_switch kernel/sched/core.c:4683 [inline]
__schedule+0x93a/0x26f0 kernel/sched/core.c:5940
schedule+0xd3/0x270 kernel/sched/core.c:6019
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6078
__mutex_lock_common kernel/locking/mutex.c:1036 [inline]
__mutex_lock+0x7b6/0x10a0 kernel/locking/mutex.c:1104
hci_req_sync+0x33/0xd0 net/bluetooth/hci_request.c:276
hci_inquiry+0x6f4/0x9e0 net/bluetooth/hci_core.c:1357
hci_sock_ioctl+0x1a7/0x910 net/bluetooth/hci_sock.c:1060
sock_do_ioctl+0xcb/0x2d0 net/socket.c:1094
sock_ioctl+0x477/0x6a0 net/socket.c:1221
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:1069 [inline]
__se_sys_ioctl fs/ioctl.c:1055 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:1055
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x446449
RSP: 002b:00007f36ab8342e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00000000004cb400 RCX: 0000000000446449
RDX: 00000000200000c0 RSI: 00000000800448f0 RDI: 0000000000000004
RBP: 00000000004cb40c R08: 0000000000000000 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000003
R13: 0000000000000004 R14: 00007f36ab8346b8 R15: 00000000004cb408
INFO: task syz-executor446:8491 blocked for more than 143 seconds.
Not tainted 5.14.0-rc4-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor446 state:D stack:28176 pid: 8491 ppid: 8452 flags:0x00000004
Call Trace:
context_switch kernel/sched/core.c:4683 [inline]
__schedule+0x93a/0x26f0 kernel/sched/core.c:5940
schedule+0xd3/0x270 kernel/sched/core.c:6019
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6078
__mutex_lock_common kernel/locking/mutex.c:1036 [inline]
__mutex_lock+0x7b6/0x10a0 kernel/locking/mutex.c:1104
hci_req_sync+0x33/0xd0 net/bluetooth/hci_request.c:276
hci_inquiry+0x6f4/0x9e0 net/bluetooth/hci_core.c:1357
hci_sock_ioctl+0x1a7/0x910 net/bluetooth/hci_sock.c:1060
sock_do_ioctl+0xcb/0x2d0 net/socket.c:1094
sock_ioctl+0x477/0x6a0 net/socket.c:1221
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:1069 [inline]
__se_sys_ioctl fs/ioctl.c:1055 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:1055
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x446449
RSP: 002b:00007f36ab8342e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00000000004cb400 RCX: 0000000000446449
RDX: 00000000200000c0 RSI: 00000000800448f0 RDI: 0000000000000004
RBP: 00000000004cb40c R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000003
R13: 0000000000000004 R14: 00007f36ab8346b8 R15: 00000000004cb408

Showing all locks held in the system:
6 locks held by kworker/u4:0/8:
1 lock held by khungtaskd/1635:
#0: ffffffff8b97c180 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446
1 lock held by in:imklog/8352:
#0: ffff888033e1d4f0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:974
1 lock held by syz-executor446/8486:
#0: ffff8880349c4ff0 (&hdev->req_lock){+.+.}-{3:3}, at: hci_req_sync+0x33/0xd0 net/bluetooth/hci_request.c:276
1 lock held by syz-executor446/8489:
#0: ffff8880349c4ff0 (&hdev->req_lock){+.+.}-{3:3}, at: hci_req_sync+0x33/0xd0 net/bluetooth/hci_request.c:276
1 lock held by syz-executor446/8491:
#0: ffff8880349c4ff0 (&hdev->req_lock){+.+.}-{3:3}, at: hci_req_sync+0x33/0xd0 net/bluetooth/hci_request.c:276

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 1635 Comm: khungtaskd Not tainted 5.14.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
watchdog+0xd0a/0xfc0 kernel/hung_task.c:295
kthread+0x3e5/0x4d0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 8 Comm: kworker/u4:0 Not tainted 5.14.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:440 [inline]
RIP: 0010:smp_call_function_many_cond+0x452/0xc20 kernel/smp.c:967
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 f4 4c 89 f5 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 d0 47 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 33 06 00 00 8b 43 08 31
RSP: 0018:ffffc90000cd7a00 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9d570c0 RCX: 0000000000000000
RDX: ffff88813fe6d4c0 RSI: ffffffff816a6400 RDI: 0000000000000003
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff816a6426 R11: 0000000000000000 R12: ffffed10173aae19
R13: 0000000000000001 R14: ffff8880b9d570c8 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f56e8f43000 CR3: 000000000b68e000 CR4: 0000000000350ef0
Call Trace:
on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1133
on_each_cpu include/linux/smp.h:71 [inline]
text_poke_sync arch/x86/kernel/alternative.c:929 [inline]
text_poke_bp_batch+0x47d/0x560 arch/x86/kernel/alternative.c:1183
text_poke_flush arch/x86/kernel/alternative.c:1268 [inline]
text_poke_flush arch/x86/kernel/alternative.c:1265 [inline]
text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1275
arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
jump_label_update+0x1d5/0x430 kernel/jump_label.c:830
static_key_enable_cpuslocked+0x1b1/0x260 kernel/jump_label.c:177
static_key_enable+0x16/0x20 kernel/jump_label.c:190
toggle_allocation_gate mm/kfence/core.c:623 [inline]
toggle_allocation_gate+0x100/0x390 mm/kfence/core.c:615
process_one_work+0x98d/0x1630 kernel/workqueue.c:2276
worker_thread+0x658/0x11f0 kernel/workqueue.c:2422
kthread+0x3e5/0x4d0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

[Pavel Skripkin]

Looks like too big timeout is passed from ioctl:


C repro:

*(uint16_t*)0x200000c0 = 0;
*(uint16_t*)0x200000c2 = 0;
memcpy((void*)0x200000c4, "\xf0\x08\xa7", 3);
*(uint8_t*)0x200000c7 = 0x81; <- ir.length
*(uint8_t*)0x200000c8 = 0;
syscall(__NR_ioctl, r[0], 0x800448f0, 0x200000c0ul);


Then ir.length * msecs_to_jiffies(2000) timeout is passed to
hci_req_sync(). Task will stuck here

err = wait_event_interruptible_timeout(hdev->req_wait_q,
hdev->req_status != HCI_REQ_PEND, timeout);

for 258 seconds (I guess, it's because of test environment, but, maybe,
we should add sanity check for timeout value)



With regards,
Pavel Skripkin

[Hillf Danton]

On Wed, 11 Aug 2021 23:13:21 -0700

Need to bisect again with CONFIG_CSD_LOCK_WAIT_DEBUG enabled if this falls in
the class of csd_lock timeout[1].

[1] https://lore.kernel.org/lkml/5881c48a-0429-199c...@suse.com/

[Marcel Holtmann]

Hi Pavel,

I agree. Feel free to send a patch.

Regards

Marcel

[Pavel Skripkin]

On 8/16/21 6:56 PM, Marcel Holtmann wrote:
> Hi Pavel,
>

[snip]

> I agree. Feel free to send a patch.
>

Thank you, Marcel! I will send a patch if it will pass syzbot testing.

I believe, 60 seconds will be more than enough for inquiry request. I've
searched for examples on the internet and maximum ir.length I found was
8. Maybe, we have users, which need more than 60 seconds, idk...



#syz test
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master




With regards,
Pavel Skripkin

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

net/bluetooth/hci_core.c:1346:18: error: 'HCI_MAX_TIMEOUT' undeclared (first use in this function); did you mean 'HCI_CMD_TIMEOUT'?


Tested on:

commit: a2824f19 Merge tag 'mtd/fixes-for-5.14-rc7' of git://g..
git tree: upstream

dashboard link: https://syzkaller.appspot.com/bug?extid=be2baed593ea56c6a84c
compiler:

patch: https://syzkaller.appspot.com/x/patch.diff?x=145874a6300000

[Pavel Skripkin]

Woooooops, I forgot to build-test after define rename.

[kernel test robot]

Hi Pavel,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on bluetooth/master]
[also build test ERROR on bluetooth-next/master net-next/master net/master sparc-next/master v5.14-rc6 next-20210816]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url: https://github.com/0day-ci/linux/commits/Pavel-Skripkin/Bluetooth-add-timeout-sanity-check-to-hci_inquiry/20210817-040113
base: https://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth.git master
config: arc-randconfig-r043-20210816 (attached as .config)
compiler: arc-elf-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/cb175bf2ea0de6152c66ce30cd1d3d665fda338b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Pavel-Skripkin/Bluetooth-add-timeout-sanity-check-to-hci_inquiry/20210817-040113
git checkout cb175bf2ea0de6152c66ce30cd1d3d665fda338b
# save the attached .config to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arc SHELL=/bin/bash net/bluetooth/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <l...@intel.com>

All errors (new ones prefixed by >>):

net/bluetooth/hci_core.c: In function 'hci_inquiry':
>> net/bluetooth/hci_core.c:1346:25: error: 'HCI_MAX_TIMEOUT' undeclared (first use in this function); did you mean 'HCI_CMD_TIMEOUT'?
1346 | if (ir.length > HCI_MAX_TIMEOUT) {
| ^~~~~~~~~~~~~~~
| HCI_CMD_TIMEOUT
net/bluetooth/hci_core.c:1346:25: note: each undeclared identifier is reported only once for each function it appears in


vim +1346 net/bluetooth/hci_core.c

1309
1310 int hci_inquiry(void __user *arg)
1311 {
1312 __u8 __user *ptr = arg;
1313 struct hci_inquiry_req ir;
1314 struct hci_dev *hdev;
1315 int err = 0, do_inquiry = 0, max_rsp;
1316 long timeo;
1317 __u8 *buf;
1318
1319 if (copy_from_user(&ir, ptr, sizeof(ir)))
1320 return -EFAULT;
1321
1322 hdev = hci_dev_get(ir.dev_id);
1323 if (!hdev)
1324 return -ENODEV;
1325
1326 if (hci_dev_test_flag(hdev, HCI_USER_CHANNEL)) {
1327 err = -EBUSY;
1328 goto done;
1329 }
1330
1331 if (hci_dev_test_flag(hdev, HCI_UNCONFIGURED)) {
1332 err = -EOPNOTSUPP;
1333 goto done;
1334 }
1335
1336 if (hdev->dev_type != HCI_PRIMARY) {
1337 err = -EOPNOTSUPP;
1338 goto done;
1339 }
1340
1341 if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED)) {
1342 err = -EOPNOTSUPP;
1343 goto done;
1344 }
1345
> 1346 if (ir.length > HCI_MAX_TIMEOUT) {
1347 err = -EINVAL;
1348 goto done;
1349 }
1350
1351 hci_dev_lock(hdev);
1352 if (inquiry_cache_age(hdev) > INQUIRY_CACHE_AGE_MAX ||
1353 inquiry_cache_empty(hdev) || ir.flags & IREQ_CACHE_FLUSH) {
1354 hci_inquiry_cache_flush(hdev);
1355 do_inquiry = 1;
1356 }
1357 hci_dev_unlock(hdev);
1358
1359 timeo = ir.length * msecs_to_jiffies(2000);
1360
1361 if (do_inquiry) {
1362 err = hci_req_sync(hdev, hci_inq_req, (unsigned long) &ir,
1363 timeo, NULL);
1364 if (err < 0)
1365 goto done;
1366
1367 /* Wait until Inquiry procedure finishes (HCI_INQUIRY flag is
1368 * cleared). If it is interrupted by a signal, return -EINTR.
1369 */
1370 if (wait_on_bit(&hdev->flags, HCI_INQUIRY,
1371 TASK_INTERRUPTIBLE)) {
1372 err = -EINTR;
1373 goto done;
1374 }
1375 }
1376
1377 /* for unlimited number of responses we will use buffer with
1378 * 255 entries
1379 */
1380 max_rsp = (ir.num_rsp == 0) ? 255 : ir.num_rsp;
1381
1382 /* cache_dump can't sleep. Therefore we allocate temp buffer and then
1383 * copy it to the user space.
1384 */
1385 buf = kmalloc_array(max_rsp, sizeof(struct inquiry_info), GFP_KERNEL);
1386 if (!buf) {
1387 err = -ENOMEM;
1388 goto done;
1389 }
1390
1391 hci_dev_lock(hdev);
1392 ir.num_rsp = inquiry_cache_dump(hdev, max_rsp, buf);
1393 hci_dev_unlock(hdev);
1394
1395 BT_DBG("num_rsp %d", ir.num_rsp);
1396
1397 if (!copy_to_user(ptr, &ir, sizeof(ir))) {
1398 ptr += sizeof(ir);
1399 if (copy_to_user(ptr, buf, sizeof(struct inquiry_info) *
1400 ir.num_rsp))
1401 err = -EFAULT;
1402 } else
1403 err = -EFAULT;
1404
1405 kfree(buf);
1406
1407 done:
1408 hci_dev_put(hdev);
1409 return err;
1410 }
1411

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuil...@lists.01.org

[kernel test robot]

Hi Pavel,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on bluetooth/master]
[also build test ERROR on bluetooth-next/master net-next/master net/master sparc-next/master v5.14-rc6 next-20210816]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url: https://github.com/0day-ci/linux/commits/Pavel-Skripkin/Bluetooth-add-timeout-sanity-check-to-hci_inquiry/20210817-040113
base: https://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth.git master

config: hexagon-randconfig-r022-20210816 (attached as .config)
compiler: clang version 12.0.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/cb175bf2ea0de6152c66ce30cd1d3d665fda338b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Pavel-Skripkin/Bluetooth-add-timeout-sanity-check-to-hci_inquiry/20210817-040113
git checkout cb175bf2ea0de6152c66ce30cd1d3d665fda338b
# save the attached .config to linux build tree
mkdir build_dir

COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross O=build_dir ARCH=hexagon SHELL=/bin/bash net/bluetooth/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <l...@intel.com>

All errors (new ones prefixed by >>):

>> net/bluetooth/hci_core.c:1346:18: error: use of undeclared identifier 'HCI_MAX_TIMEOUT'
if (ir.length > HCI_MAX_TIMEOUT) {
^
1 error generated.


vim +/HCI_MAX_TIMEOUT +1346 net/bluetooth/hci_core.c

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+be2bae...@syzkaller.appspotmail.com

Tested on:

commit: 794c7931 Merge branch 'linus' of git://git.kernel.org/..
git tree: upstream
kernel config: https://syzkaller.appspot.com/x/.config?x=96f0602203250753
dashboard link: https://syzkaller.appspot.com/bug?extid=be2baed593ea56c6a84c

compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1

patch: https://syzkaller.appspot.com/x/patch.diff?x=12c85965300000

Note: testing is done by a robot and is best-effort only.