iSCSI and Ceph RBD

[Bobby]

Hi,

I have some questions regarding iSCSI and Ceph RBD. If I have understood correctly, the RBD backstore module 

on target side can translate SCSI IO into Ceph OSD requests. The iSCSI target driver with rbd.ko can expose Ceph cluster

on iSCSI protocol. If correct, then that all is happening on target side.  

My confusion is what is  happening on client side?

Meaning, does linux mainline kernel code called "rbd" has any role with  Open-iSCSI initiator on client side? To put it more simple, 

is there any common ground for both protocols (iSCSI and rbd) in the linux kernel  of the client side? 

Thanks :-)

[Donald Williams]

Hello, 

 I am not an expert in CEPH. 

 

However,  iSCSI is the transport protocols to connect an initiator to a target.  On the client side, iSCSI traffic coming from target is broken down and the SCSI commands are handed to the client.   When writing data, the iSCSI initiator encoded the command and data and transports it to the iSCSI target where the target breaks down the iSCSI portion to get the client SCSI command,  I.e. a WRITE10, and processes that request. 

 So things like CEPH are above the iSCSI layer.  This is why CEPH can work with Fibre Channel, SAS, SCSI, iSCSI, etc...  

https://docs.ceph.com/docs/mimic/glossary/#term-ceph-osd-daemon 

OSD

A physical or logical storage unit (e.g., LUN). Sometimes, Ceph users use the term “OSD” to refer to Ceph OSD Daemon, though the proper term is “Ceph OSD”.

So iSCSI will provide the physical LUN that CEPH will use. This is good otherwise for every protocol of storage you want to use, you would have to have a specific driver for CEPH. 

This is a pretty good intro to iSCSI 

https://blog.calsoftinc.com/2017/03/iscsi-introduction-steps-configure-iscsi-initiator-target.html 

 

Regards, 

Don 

 

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-iscsi/dc5e17db-5e78-49ff-be38-a17706428655%40googlegroups.com.

[The Lee-Man]

Did Donald answer your question(s)?

[Gorka Eguileor]

> --
> You received this message because you are subscribed to the Google Groups "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/open-iscsi/dc5e17db-5e78-49ff-be38-a17706428655%40googlegroups.com.

Hi,

Are you referring to the Ceph iSCSI gateway?

If you are, then the best explanation is on their docs [1]:

Each iSCSI gateway runs the Linux IO target kernel subsystem (LIO) to
provide the iSCSI protocol support. LIO utilizes a userspace
passthrough (TCMU) to interact with Ceph’s librbd library and expose
RBD images to iSCSI clients. With Ceph’s iSCSI gateway you can
effectively run a fully integrated block-storage infrastructure with
all the features and benefits of a conventional Storage Area Network
(SAN).

So basically LIO takes care of the iSCSI part and librbd takes care of
talking to the OSDs.

On the initiator side, there is no knowledge about the storage actually
being on a Ceph cluster, since it is connecting to a normal iSCSI
target.

Hope this is what you were looking for.

Cheers,
Gorka.

[1]: https://docs.ceph.com/docs/master/rbd/iscsi-overview/

[The Lee-Man]

In addition to the document Gorka pointed you at, I wrote up a presentation a few years ago on how SUSE is using Ceph and iSCSI and rbd, as well as tcmu-runner.

See http://gonzoleeman.net/documents/susecon2015_dandl_final_201.pdf