iscsi transport : add sgio pass-thru support
The group you are posting to is a
Usenet group . Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
From:
James Smart <James.Sm... @Emulex.Com>
Date: Fri, 30 Oct 2009 09:49:17 -0400
Local: Fri, Oct 30 2009 9:49 am
Subject: [RFC] iscsi transport : add sgio pass-thru support
[reposting to include open-iscsi list] This patch implements the same infrastructure as found in the FC transport for sgio request/response handling.
The patch creates (and exports to userland) a new header - scsi_bsg_iscsi.h
-- james s
Signed-off-by: James Smart <james.sm... @emulex.com>
---
drivers/scsi/scsi_transport_iscsi.c | 414 +++++++++++++++++++++++++++++++++++- include/scsi/Kbuild | 1 include/scsi/scsi_bsg_iscsi.h | 111 +++++++++ include/scsi/scsi_transport_iscsi.h | 45 +++ 4 files changed, 570 insertions(+), 1 deletion(-)
diff -upNr a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c --- a/drivers/scsi/scsi_transport_iscsi.c 2009-10-26 12:58:17.000000000 -0400 +++ b/drivers/scsi/scsi_transport_iscsi.c 2009-10-29 15:00:57.000000000 -0400 @@ -29,6 +29,8 @@ #include <scsi/scsi_transport.h> #include <scsi/scsi_transport_iscsi.h> #include <scsi/iscsi_if.h> +#include <scsi/scsi_bsg_iscsi.h> +#include <scsi/scsi_cmnd.h>
#define ISCSI_SESSION_ATTRS 21 #define ISCSI_CONN_ATTRS 13 @@ -268,6 +270,402 @@ struct iscsi_endpoint *iscsi_lookup_endp } EXPORT_SYMBOL_GPL(iscsi_lookup_endpoint);
+ +/* + * BSG support + */ + +/** + * iscsi_destroy_bsgjob - routine to teardown/delete a iscsi bsg job + * @job: iscsi_bsg_job that is to be torn down + */ +static void +iscsi_destroy_bsgjob(struct iscsi_bsg_job *job) +{ + unsigned long flags; + + spin_lock_irqsave(&job->job_lock, flags); + if (job->ref_cnt) { + spin_unlock_irqrestore(&job->job_lock, flags); + return; + } + spin_unlock_irqrestore(&job->job_lock, flags); + + put_device(job->dev); /* release reference for the request */ + + kfree(job->request_payload.sg_list); + kfree(job->reply_payload.sg_list); + kfree(job); +} + +/** + * iscsi_bsg_jobdone - completion routine for bsg requests that the LLD has + * completed + * @job: iscsi_bsg_job that is complete + */ +static void +iscsi_bsg_jobdone(struct iscsi_bsg_job *job) +{ + struct request *req = job->req; + struct request *rsp = req->next_rq; + int err; + + err = job->req->errors = job->reply->result; + + if (err < 0) + /* we're only returning the result field in the reply */ + job->req->sense_len = sizeof(uint32_t); + else + job->req->sense_len = job->reply_len; + + /* we assume all request payload was transferred, residual == 0 */ + req->resid_len = 0; + + if (rsp) { + WARN_ON(job->reply->reply_payload_rcv_len > rsp->resid_len); + + /* set reply (bidi) residual */ + rsp->resid_len -= min(job->reply->reply_payload_rcv_len, + rsp->resid_len); + } + blk_complete_request(req); +} + +/** + * iscsi_bsg_softirq_done - softirq done routine for destroying the bsg requests + * @rq: BSG request that holds the job to be destroyed + */ +static void +iscsi_bsg_softirq_done(struct request *rq) +{ + struct iscsi_bsg_job *job = rq->special; + unsigned long flags; + + spin_lock_irqsave(&job->job_lock, flags); + job->state_flags |= ISCSI_RQST_STATE_DONE; + job->ref_cnt--; + spin_unlock_irqrestore(&job->job_lock, flags); + + blk_end_request_all(rq, rq->errors); + iscsi_destroy_bsgjob(job); +} + +/** + * iscsi_bsg_job_timeout - handler for when a bsg request timesout + * @req: request that timed out + */ +static enum blk_eh_timer_return +iscsi_bsg_job_timeout(struct request *req) +{ + struct iscsi_bsg_job *job = (void *) req->special; + struct Scsi_Host *shost = job->shost; + struct iscsi_internal *i = to_iscsi_internal(shost->transportt); + unsigned long flags; + int err = 0, done = 0; + + spin_lock_irqsave(&job->job_lock, flags); + if (job->state_flags & ISCSI_RQST_STATE_DONE) + done = 1; + else + job->ref_cnt++; + spin_unlock_irqrestore(&job->job_lock, flags); + + if (!done && i->iscsi_transport->bsg_timeout) { + /* call LLDD to abort the i/o as it has timed out */ + err = i->iscsi_transport->bsg_timeout(job); + if (err) + printk(KERN_ERR "ERROR: iSCSI BSG request timeout - " + "LLD abort failed with status %d\n", err); + } + + /* the blk_end_sync_io() doesn't check the error */ + if (done) + return BLK_EH_NOT_HANDLED; + else + return BLK_EH_HANDLED; +} + +static int +iscsi_bsg_map_buffer(struct iscsi_bsg_buffer *buf, struct request *req) +{ + size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments); + + BUG_ON(!req->nr_phys_segments); + + buf->sg_list = kzalloc(sz, GFP_KERNEL); + if (!buf->sg_list) + return -ENOMEM; + sg_init_table(buf->sg_list, req->nr_phys_segments); + buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list); + buf->payload_len = blk_rq_bytes(req); + return 0; +} + +/** + * iscsi_req_to_bsgjob - Allocate/create the iscsi_bsg_job structure for the + * bsg request + * @req: BSG request that needs a job structure + * @shost: SCSI Host corresponding to the bsg object + */ +static int +iscsi_req_to_bsgjob(struct request *req, struct Scsi_Host *shost) +{ + struct iscsi_internal *i = to_iscsi_internal(shost->transportt); + struct request *rsp = req->next_rq; + struct iscsi_bsg_job *job; + int ret; + + BUG_ON(req->special); + + job = kzalloc(sizeof(struct iscsi_bsg_job) + + i->iscsi_transport->dd_bsg_size, GFP_KERNEL); + if (!job) + return -ENOMEM; + + /* + * Note: this is a bit silly. + * The request gets formatted as a SGIO v4 ioctl request, which + * then gets reformatted as a blk request, which then gets + * reformatted as a iscsi bsg request. And on completion, we have + * to wrap return results such that SGIO v4 thinks it was a scsi + * status. I hope this was all worth it. + */ + + req->special = job; + job->shost = shost; + job->req = req; + if (i->iscsi_transport->dd_bsg_size) + job->dd_data = (void *)&job[1]; + spin_lock_init(&job->job_lock); + job->request = (struct iscsi_bsg_request *)req->cmd; + job->request_len = req->cmd_len; + job->reply = req->sense; + job->reply_len = SCSI_SENSE_BUFFERSIZE; /* Size of sense buffer + * allocated */ + if (req->bio) { + ret = iscsi_bsg_map_buffer(&job->request_payload, req); + if (ret) + goto failjob_rls_job; + } + if (rsp && rsp->bio) { + ret = iscsi_bsg_map_buffer(&job->reply_payload, rsp); + if (ret) + goto failjob_rls_rqst_payload; + } + job->job_done = iscsi_bsg_jobdone; + job->dev = &shost->shost_gendev; + get_device(job->dev); /* take a reference for the request */ + + job->ref_cnt = 1; + + return 0; + + +failjob_rls_rqst_payload: + kfree(job->request_payload.sg_list); +failjob_rls_job: + kfree(job); + return -ENOMEM; +} + + +enum iscsi_dispatch_result { + DISPATCH_BREAK, /* on return, q is locked, break from q loop */ + DISPATCH_LOCKED, /* on return, q is locked, continue on */ + DISPATCH_UNLOCKED, /* on return, q is unlocked, continue on */ +}; + +/** + * iscsi_bsg_host_dispatch - process iscsi host bsg requests and + * dispatch to LLDD + * @q: iscsi host request queue + * @shost: scsi host request queue attached to + * @job: bsg job to be processed + */ +static enum iscsi_dispatch_result +iscsi_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost, + struct iscsi_bsg_job *job) +{ + struct iscsi_internal *i = to_iscsi_internal(shost->transportt); + int cmdlen = sizeof(uint32_t); /* start with length of msgcode */ + int ret; + + /* Validate the host command */ + switch (job->request->msgcode) { + case ISCSI_BSG_HST_VENDOR: + cmdlen += sizeof(struct iscsi_bsg_host_vendor); + if ((shost->hostt->vendor_id == 0L) || + (job->request->rqst_data.h_vendor.vendor_id != + shost->hostt->vendor_id)) { + ret = -ESRCH; + goto fail_host_msg; + } + break; + + default: + ret = -EBADR; + goto fail_host_msg; + } + + /* check if we really have all the request data needed */ + if (job->request_len < cmdlen) { + ret = -ENOMSG; + goto fail_host_msg; + } + + ret = i->iscsi_transport->bsg_request(job); + if (!ret) + return DISPATCH_UNLOCKED; + +fail_host_msg: + /* return the errno failure code as the only status */ + BUG_ON(job->reply_len < sizeof(uint32_t)); + job->reply->result = ret; + job->reply_len = sizeof(uint32_t); + iscsi_bsg_jobdone(job); + return DISPATCH_UNLOCKED; +} + +/** + * iscsi_bsg_request_handler - generic handler for bsg requests + * @q: request queue to manage + * @shost: Scsi_Host related to the bsg object + * @dev: device structure for bsg object + * + * NOTE: only shost messages are expected + */ +static void +iscsi_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost, + struct device *dev) +{ + struct request *req; + struct iscsi_bsg_job *job; + enum iscsi_dispatch_result ret; + + if (!get_device(dev)) + return; + + while (!blk_queue_plugged(q)) { + + req = blk_fetch_request(q); + if (!req) + break; + + spin_unlock_irq(q->queue_lock); + + ret = iscsi_req_to_bsgjob(req, shost); + if (ret) { + req->errors = ret; + blk_end_request(req, ret, blk_rq_bytes(req)); + spin_lock_irq(q->queue_lock); + continue; + } + + job = req->special; + + /* check if we have the msgcode value at least */ + if (job->request_len < sizeof(uint32_t)) { + BUG_ON(job->reply_len < sizeof(uint32_t)); + job->reply->result = -ENOMSG; + job->reply_len = sizeof(uint32_t); + iscsi_bsg_jobdone(job); + spin_lock_irq(q->queue_lock); + continue; + } + + /* the dispatch routines will unlock the queue_lock */ + iscsi_bsg_host_dispatch(q, shost, job); + + /* did dispatcher hit state that can't process any more */ + if (ret == DISPATCH_BREAK) + break; + + /* did dispatcher had released the lock */ + if (ret == DISPATCH_UNLOCKED) + spin_lock_irq(q->queue_lock); + } + + spin_unlock_irq(q->queue_lock); + put_device(dev); + spin_lock_irq(q->queue_lock); +} + +/** + * iscsi_bsg_host_handler - handler for bsg requests for a iscsi host + * @q: iscsi host request queue + */ +static void +iscsi_bsg_host_handler(struct request_queue *q) +{ + struct Scsi_Host *shost = q->queuedata; + + iscsi_bsg_request_handler(q, shost, &shost->shost_gendev); +} + +/** + * iscsi_bsg_hostadd - Create and add the bsg hooks so we
...
read more »
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
James Smart <James.Sm... @Emulex.Com>
Date: Tue, 3 Nov 2009 09:37:03 -0500
Local: Tues, Nov 3 2009 9:37 am
Subject: Re: [RFC] iscsi transport : add sgio pass-thru support
I actually started the patch with this in mind - making a common layer. I was able to commonize: xx_bsg_destroy_job(), xx_bsg_jobdone(), xx_softirq_done(), a helper for the timeout function (chkjobdone()), xx_bsg_map_buffer(), xx_req_to_bsgjob(), and xx_bsg_goose_queue(). However, what I was finding was I was jumping through hoops with the data structures (whose header where, structures within structures, nested private areas, etc). Additionally, I kept finding chunks of the code flow, which had parallels to the items in the common routines, that had to be left within the transport (e.g. rx path in transport, tx in common; or vice versa) - e.g. if I can't encapsulate both sides of the code flow within the common code I lose many of the advantages - I ended up abandoning it under the guise of "complexity==bad"
I can post some of the work to see if you have the same conclusion. Yes, I don't like the replication either.
-- james s
Mike Christie wrote:
> James Smart wrote:
>> This patch implements the same infrastructure as found in the FC transport
>> for sgio request/response handling.
>> The patch creates (and exports to userland) a new header - scsi_bsg_iscsi.h
> Sorry for the late reply. I am trying to sell my house and move.
> Based on your experience with fc bsg support, do you think there is some > common code? It looks like a lot of this is generic. I just started > looking at the fc bsg stuff again, so I am not sure ATM.
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
Mike Christie <micha... @cs.wisc.edu>
Date: Wed, 04 Nov 2009 09:52:34 -0600
Local: Wed, Nov 4 2009 10:52 am
Subject: Re: [RFC] iscsi transport : add sgio pass-thru support
James Smart wrote:
> I actually started the patch with this in mind - making a common layer.
> I was able to commonize: xx_bsg_destroy_job(), xx_bsg_jobdone(),
> xx_softirq_done(), a helper for the timeout function (chkjobdone()),
> xx_bsg_map_buffer(), xx_req_to_bsgjob(), and xx_bsg_goose_queue().
> However, what I was finding was I was jumping through hoops with the > data structures (whose header where, structures within structures, > nested private areas, etc). Additionally, I kept finding chunks of the > code flow, which had parallels to the items in the common routines, that > had to be left within the transport (e.g. rx path in transport, tx in > common; or vice versa) - e.g. if I can't encapsulate both sides of the > code flow within the common code I lose many of the advantages - I ended > up abandoning it under the guise of "complexity==bad"
> I can post some of the work to see if you have the same conclusion. Yes, > I don't like the replication either.
Do not worry about it. I am looking at it in more depth now.
You must
Sign in before you can post messages.
You do not have the permission required to post.