iSCSI initiator setting max_sectors_kb=4 when target optimal_io_size=4096

[Alexis Oosterhoff]

Hi,

Looking into whether this is a bug, or an expect behavior with kernel 4.18+

RHEL 8.4 on AWS r5.xlarge hardware type, attaching nvme disks, observing the nvme device is configuring optimal_io_size to 4KB

i.e.

/sys/devices/pci0000:00/0000:00:1c.0/nvme/nvme4/nvme4n1/queue/optimal_io_size 4096

When attaching this device remotely using Linux-IO, the initiator device is using the target's 'optimal_io_size' to set the max_sectors_kb. 

i.e.

/sys/devices/platform/host1/session8/target1:0:0/1:0:0:0/block/sdb/queue/max_sectors_kb 4

This does not seem to be correct behavior. optimal_io_size and max_sectors_kb should not be directly related.  Do not observe this behavior with RHEL7.

target:

 - RHEL 8.4, 4.18.0-305.12.1.el8_4.x86_64

initiator: 

 - RHEL 8.4, 4.18.0-305.12.1.el8_4.x86_64

 - iscsi-initiator-utils-iscsiuio-6.2.1.2-1.gita8fcb37.el8.x86_64

 - iscsi-initiator-utils-6.2.1.2-1.gita8fcb37.el8.x86_64

Thanks,

Alexis.

[The Lee-Man]

Perhaps someone from RedHat can comment? I suspect this is a kernel change, though.

[Chris Leech]

Hi Alexis, I was just taking a look at the issue you opened with Red
Hat about this.

It appears to me that the sd driver for all SCSI disks is setting the
max_sectors limit for the queue to the optimal transfer length when
sanely reported. So the SCSI disk driver is limiting to the optimal
length at all times, while the NVMe code is reporting different
optimal and maximum limits to the block layer. The SCSI target code
is just passing the block layer limits through to the initiator, so
when exposing an NVMe namespace as an SCSI disk the smaller optimal
transfer length gets used as a maximum limit on the SCSI initiator.

I'm going to ask around and see if I can't get a better understanding
as to why the sd driver works that way and if optimal transfer length
when reported by SCSI disks are expected to be as small as this NVMe
device is reporting.

I don't think this is an iSCSI specific issue in this case.

- Chris

> --
> You received this message because you are subscribed to the Google Groups "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/open-iscsi/1b70c5d5-bb56-4f35-948b-1a2cbde569ban%40googlegroups.com.