[PATCH] Test for 4652b8e4f3ff

[Edward Adam Davis]

please test warning in blk_mq_start_request

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 4652b8e4f3ff


diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 22a3cf7f32e2..397bdfad6d27 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -1724,6 +1724,8 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx,
cmd->fake_timeout = should_timeout_request(rq) ||
blk_should_fake_timeout(rq->q);

+ printk("%d, %s\n", READ_ONCE(rq->state), __func__);
+ WRITE_ONCE(rq->state, MQ_RQ_IDLE);
blk_mq_start_request(rq);

if (should_requeue_request(rq)) {

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+fcc47b...@syzkaller.appspotmail.com

Tested on:

commit: 4652b8e4 Merge tag '6.7-rc-ksmbd-server-fixes' of git:..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=1629a00f680000
kernel config: https://syzkaller.appspot.com/x/.config?x=5316ad879647e3c5
dashboard link: https://syzkaller.appspot.com/bug?extid=fcc47ba2476570cbbeb0
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
patch: https://syzkaller.appspot.com/x/patch.diff?x=107e3ac0e80000

Note: testing is done by a robot and is best-effort only.

[Edward Adam Davis]

please test warning in blk_mq_start_request

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 4652b8e4f3ff

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c

index 22a3cf7f32e2..0726534a5a24 100644

--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -1724,6 +1724,8 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx,
cmd->fake_timeout = should_timeout_request(rq) ||
blk_should_fake_timeout(rq->q);

+ if (READ_ONCE(rq->state))

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+fcc47b...@syzkaller.appspotmail.com

Tested on:

commit: 4652b8e4 Merge tag '6.7-rc-ksmbd-server-fixes' of git:..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

console output: https://syzkaller.appspot.com/x/log.txt?x=1400e647680000

kernel config: https://syzkaller.appspot.com/x/.config?x=5316ad879647e3c5
dashboard link: https://syzkaller.appspot.com/bug?extid=fcc47ba2476570cbbeb0
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=12a5d81f680000

[Edward Adam Davis]

please test warning in blk_mq_start_request

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 4652b8e4f3ff

diff --git a/block/blk-mq.c b/block/blk-mq.c
index e2d11183f62e..26f2921cbc50 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2575,6 +2575,8 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx,
* Any other error (busy), just add it to our list as we
* previously would have done.
*/

+ if (READ_ONCE(rq->state))
+ WRITE_ONCE(rq->state, MQ_RQ_IDLE);

ret = q->mq_ops->queue_rq(hctx, &bd);
switch (ret) {
case BLK_STS_OK:

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+fcc47b...@syzkaller.appspotmail.com

Tested on:

commit: 4652b8e4 Merge tag '6.7-rc-ksmbd-server-fixes' of git:..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

console output: https://syzkaller.appspot.com/x/log.txt?x=14ca647b680000

kernel config: https://syzkaller.appspot.com/x/.config?x=5316ad879647e3c5
dashboard link: https://syzkaller.appspot.com/bug?extid=fcc47ba2476570cbbeb0
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=126f8204e80000

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 4652b8e4f3ff Merge tag '6.7-rc-ksmbd-server-fixes' of git:..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=117a6ebb680000

kernel config: https://syzkaller.appspot.com/x/.config?x=5316ad879647e3c5
dashboard link: https://syzkaller.appspot.com/bug?extid=fcc47ba2476570cbbeb0
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13489af3680000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=145c7d07680000

Downloadable assets:
disk image (non-bootable): https://storage.googleapis.com/syzbot-assets/7bc7510fe41f/non_bootable_disk-4652b8e4.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/9c5be65b1bc8/vmlinux-4652b8e4.xz
kernel image: https://storage.googleapis.com/syzbot-assets/8799a4da0be5/bzImage-4652b8e4.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+fcc47b...@syzkaller.appspotmail.com

R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffff489c59c
R13: 00007ffff489c5b0 R14: 00007ffff489c5f0 R15: 0000000000000003
</TASK>
------------[ cut here ]------------
WARNING: CPU: 3 PID: 5178 at block/blk-mq.c:1258 blk_mq_start_request+0x643/0x780 block/blk-mq.c:1258
Modules linked in:
CPU: 3 PID: 5178 Comm: syz-executor268 Not tainted 6.6.0-syzkaller-10396-g4652b8e4f3ff #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
RIP: 0010:blk_mq_start_request+0x643/0x780 block/blk-mq.c:1258
Code: 00 00 fc ff df 48 8d 7d 10 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 2f 01 00 00 48 89 df ff 55 10 e9 64 fc ff ff e8 1d 9c 44 fd <0f> 0b e9 f6 fa ff ff e8 11 9c 44 fd 0f 0b e9 06 fa ff ff e8 05 9c
RSP: 0018:ffffc900035ef298 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffff888018539e00 RCX: ffffffff8442db37
RDX: ffff888030114f00 RSI: ffffffff8442e043 RDI: 0000000000000005
RBP: ffff888018539e94 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffff888106a33350
R13: 0000000000000001 R14: ffff888018539f1d R15: ffff888018539f20
FS: 0000555555f00480(0000) GS:ffff88806b900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555555f00788 CR3: 000000002238a000 CR4: 0000000000350ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
null_queue_rq+0x2e2/0x7a0 drivers/block/null_blk/main.c:1727
__blk_mq_issue_directly+0xe3/0x270 block/blk-mq.c:2578
blk_mq_request_issue_directly+0x120/0x190 block/blk-mq.c:2663
blk_mq_plug_issue_direct+0x19e/0x670 block/blk-mq.c:2684
blk_mq_flush_plug_list.part.0+0x16b6/0x1e90 block/blk-mq.c:2793
blk_mq_flush_plug_list+0x62/0x80 block/blk-mq.c:2770
__blk_flush_plug+0x2c0/0x430 block/blk-core.c:1142
blk_finish_plug block/blk-core.c:1166 [inline]
blk_finish_plug+0x54/0xa0 block/blk-core.c:1163
read_pages+0x69b/0xdb0 mm/readahead.c:183
page_cache_ra_unbounded+0x457/0x5e0 mm/readahead.c:269
do_page_cache_ra mm/readahead.c:299 [inline]
page_cache_ra_order+0x72b/0xa80 mm/readahead.c:546
ondemand_readahead+0x493/0x1130 mm/readahead.c:668
page_cache_sync_ra+0x174/0x1d0 mm/readahead.c:695
page_cache_sync_readahead.constprop.0+0xb2/0xf0 include/linux/pagemap.h:1293
cramfs_blkdev_read fs/cramfs/inode.c:218 [inline]
cramfs_read+0x33f/0xae0 fs/cramfs/inode.c:278
cramfs_read_super+0xbd/0x9b0 fs/cramfs/inode.c:522
cramfs_blkdev_fill_super+0x116/0x2f0 fs/cramfs/inode.c:622
get_tree_bdev+0x3b5/0x650 fs/super.c:1595
cramfs_get_tree fs/cramfs/inode.c:958 [inline]
cramfs_get_tree+0x40/0x50 fs/cramfs/inode.c:948
vfs_get_tree+0x8c/0x370 fs/super.c:1768
do_new_mount fs/namespace.c:3337 [inline]
path_mount+0x1492/0x1ed0 fs/namespace.c:3664
do_mount fs/namespace.c:3677 [inline]
__do_sys_mount fs/namespace.c:3886 [inline]
__se_sys_mount fs/namespace.c:3863 [inline]
__x64_sys_mount+0x293/0x310 fs/namespace.c:3863
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3f/0x110 arch/x86/entry/common.c:82
entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7feca35df9f9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 a1 1a 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffff489c548 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
RAX: ffffffffffffffda RBX: 00007ffff489c550 RCX: 00007feca35df9f9
RDX: 0000000020000040 RSI: 00000000200000c0 RDI: 0000000020000000
RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000003134
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffff489c59c
R13: 00007ffff489c5b0 R14: 00007ffff489c5f0 R15: 0000000000000003
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[syzbot]

syzbot has bisected this issue to:

commit d78bfa1346ab1fe04d20aa45a0678d1fc866f37c
Author: Chengming Zhou <zhouch...@bytedance.com>
Date: Wed Sep 13 15:16:16 2023 +0000

block/null_blk: add queue_rqs() support

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=106414a8e80000
start commit: 13d88ac54ddd Merge tag 'vfs-6.7.fsid' of git://git.kernel...
git tree: upstream
final oops: https://syzkaller.appspot.com/x/report.txt?x=126414a8e80000
console output: https://syzkaller.appspot.com/x/log.txt?x=146414a8e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=beb32a598fd79db9
dashboard link: https://syzkaller.appspot.com/bug?extid=fcc47ba2476570cbbeb0
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1465bb08e80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13e7881f680000

Reported-by: syzbot+fcc47b...@syzkaller.appspotmail.com
Fixes: d78bfa1346ab ("block/null_blk: add queue_rqs() support")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Chengming Zhou]

CONFIG_BLK_DEV_NULL_BLK_FAULT_INJECTION is enabled in the kernel config,
so null_queue_rq() will return BLK_STS_RESOURCE or BLK_STS_DEV_RESOURCE
for some requests, which have been marked as IN_FLIGHT status.

Then null_queue_rqs() put these requests in the rqlist and return back,
blk-mq will try to queue them individually once again, caused the warning
"WARN_ON_ONCE(blk_mq_rq_state(rq) != MQ_RQ_IDLE)" in blk_mq_start_request().

So handling of return value of null_queue_rq() in null_queue_rqs() is wrong,
maybe we should __blk_mq_requeue_request() for these requests, before
adding them in the rqlist?

Thanks!

[Bart Van Assche]

On 11/8/23 17:27, Chengming Zhou wrote:
> CONFIG_BLK_DEV_NULL_BLK_FAULT_INJECTION is enabled in the kernel config,
> so null_queue_rq() will return BLK_STS_RESOURCE or BLK_STS_DEV_RESOURCE
> for some requests, which have been marked as IN_FLIGHT status.
>
> Then null_queue_rqs() put these requests in the rqlist and return back,
> blk-mq will try to queue them individually once again, caused the warning
> "WARN_ON_ONCE(blk_mq_rq_state(rq) != MQ_RQ_IDLE)" in blk_mq_start_request().
>
> So handling of return value of null_queue_rq() in null_queue_rqs() is wrong,
> maybe we should __blk_mq_requeue_request() for these requests, before
> adding them in the rqlist?

Please follow the example of virtio_queue_rqs() and send any requests
that need to be requeued back to the block layer core instead of
handling these directly in null_queue_rqs().

Thanks,

Bart.

[Chengming Zhou]

Ok, I reviewed the code of virtio_queue_rqs(), found the main difference
is that request won't fail after blk_mq_start_request().

But in null_blk case, the request will fail after blk_mq_start_request(),
return BLK_STS_RESOURCE or BLK_STS_DEV_RESOURCE. If we return these rqs
back to the block layer core, they will be queued individually once again.
So caused the warning.

Thanks!

[Bart Van Assche]

On 11/13/23 07:05, Chengming Zhou wrote:
> Ok, I reviewed the code of virtio_queue_rqs(), found the main difference
> is that request won't fail after blk_mq_start_request().
>
> But in null_blk case, the request will fail after blk_mq_start_request(),
> return BLK_STS_RESOURCE or BLK_STS_DEV_RESOURCE. If we return these rqs
> back to the block layer core, they will be queued individually once again.
> So caused the warning.

I think it is safe to move the blk_mq_start_request() call under the if-block
that decides whether or not to requeue a request in null_queue_rq()

Thanks,

Bart.

[Chengming Zhou]

Right! And null_handle_throttled() in null_handle_cmd() may also return the
BLK_STS_DEV_RESOURCE, it's also needed to put in null_queue_rq() and before
the blk_mq_start_request().

Then request must return BLK_STS_OK after blk_mq_start_request().

Thanks!