introducing new transport layer abstraction and iSCSI Extensions for RDMA implementation

[roy.sh...@gmail.com]

Hi all,

I'm happy to introduce a new transport option in Libiscsi library,

This new transport called iSER (iSCSI Extensions for Remote Direct Memory Access (RDMA) implementation) and can be chosen by adding ?iser option in ISCSI URI.

For now iSER is available only on linux distros but I hope I will make a compatible version for BSD pretty soon.

iSER is a RDMA capable application (RCaP) that supports RDMA hardware, in order to run iSER one should have a Network Interface Card that supports RDMA (ConnectX4 for example). 

For those who aren't familiar with RDMA hardware technology, Remote Direct Memory Access hardware enables data transfer between two servers or between server and a storage node with minimal involvement of CPUs across the data path. RDMA relies on that each host will export the exact place in memory (address and offset) he wants other host to place the required data. RDMA gives one the possibility to over-head the operating system in data transferring operations, by that reducing the latency and improving the general performances.

The new transport abstraction includes a new structure of function pointers for each of the transport options, in order to use one of the transport dependent functions one should

use iscsi->t->{name_of_the_function}.

Initial results versus tgt user-space iSCSI target shows excellent performances when running Flexible I/O benchmark with libaio engine:

IO Depth	IOPS
1	13270
2	24907
4	49982
8	83195
16	113928
32	136641
64	151519
128	158232

  

Later on I will publish more performance results.

I would like to thank Ronnie for all the support and help with merging iSER into Libiscsi,

Hope to hear comments,

Roy Shterman

[Sagi Grimberg]

> Hi all,

Hey Roy,

> I'm happy to introduce a new transport option in Libiscsi library,

Great to finally see iSER adopted in libiscsi.

> This new transport called iSER (iSCSI Extensions for Remote Direct
> Memory Access (RDMA) implementation) and can be chosen by adding ?iser
> option in ISCSI URI.
> For now iSER is available only on linux distros but I hope I will make a
> compatible version for BSD pretty soon.

What about the Qemu changes needed to support iSER? are those included
as well? where can I find the reference?

Also, what is the proper location to document the rest of the
remaining TODO list (assuming that there are TODO items)?

> Initial results versus tgt user-space iSCSI target shows excellent
> performances when running Flexible I/O benchmark with libaio engine:
>
>
> IO Depth IOPS
> 1 13270
> 2 24907
> 4 49982
> 8 83195
> 16 113928
> 32 136641
> 64 151519
> 128 158232

Nice scaling, did you have a chance to test against other iscsi/iser
targets (in-kernel LIO or SCST)?

> I would like to thank Ronnie for all the support and help with merging
> iSER into Libiscsi,

Indeed.