[PATCH-v3 0/5] iscsi-target: Add support for login multi-plexing support

[Nicholas A. Bellinger]

From: Nicholas Bellinger <n...@linux-iscsi.org>

This updated -v3 series for v3.12-rc1 adds support for login multi-plexing,
that allows subsequent login request/request PDUs beyond the initial
exchange to be pushed off to workqueue process context, so that other
incoming login requests can be serviced in parallel.

This addresses a long-standing issue with login latency with many (100's)
of parallel login requests to the same network portal being shared
across many (100's) of TargetName+TargetPortalGroup endpoints.

Note that login negotiation to the same TargetName+TargetPortalGroup
endpoint is still sychronized in order to enforce session reinstatement
state machines.

The changes for -v3 include:

- Convert iscsi_portal_group->np_login_lock to ->np_login_sem
- Add new LOGIN_FLAGS_* definitions
- Convert iscsi_target_sk_data_ready() lock access to
write[lock,unlock]_bh()
- Only clear LOGIN_FLAGS_READ_ACTIVE when iscsi_target_do_login()
returns zero
- Add LOGIN_FLAGS_READY + LOGIN_FLAGS_CLOSED bit checks to
iscsi_target_sk_data_ready()
- Make INIT_DELAYED_WORK() + iscsi_target_set_sock_callbacks() setup
happen earlier by moving from iscsi_target_start_negotiation() into
iscsi_target_locate_portal()
- Set LOGIN_FLAGS_READY bit in iscsi_target_start_negotiation()
after iscsi_target_do_login() returns zero.
- Add sk->sk_state_change to cleanup after TCP failure
- Add explicit sk_state_check to iscsi_target_do_login()
- Use sk->sk_sndtimeo -> sk->sk_rcvtimeo settings during login
- Drop unnecessary LOGIN_FLAGS_READ_ACTIVE bit set in
isert_rx_login_req()

This is the final series ahead of the initial v3.12-rc1 merge.

Thank you,

--nab

Nicholas Bellinger (5):
iscsi-target: Prepare login code for multi-plexing support
iscsi-target: Add login negotiation multi-plexing support
iscsi-target: Add sk->sk_state_change to cleanup after TCP failure
iscsi-target: Remove left-over iscsi_target_do_login_io
iser-target: Updates for login negotiation multi-plexing support

drivers/infiniband/ulp/isert/ib_isert.c | 14 ++-
drivers/target/iscsi/iscsi_target.c | 55 +++---
drivers/target/iscsi/iscsi_target.h | 6 +-
drivers/target/iscsi/iscsi_target_core.h | 18 ++-
drivers/target/iscsi/iscsi_target_login.c | 162 ++++++++------
drivers/target/iscsi/iscsi_target_login.h | 3 +
drivers/target/iscsi/iscsi_target_nego.c | 335 +++++++++++++++++++++++++++-
drivers/target/iscsi/iscsi_target_tpg.c | 25 ++-
drivers/target/iscsi/iscsi_target_tpg.h | 4 +-
9 files changed, 500 insertions(+), 122 deletions(-)

--
1.7.2.5

[Nicholas A. Bellinger]

From: Nicholas Bellinger <n...@linux-iscsi.org>

This patch prepares the iscsi-target login code for multi-plexing
support. This includes:

- Adding iscsi_tpg_np->tpg_np_kref + iscsit_login_kref_put() for
handling callback of iscsi_tpg_np->tpg_np_comp
- Adding kref_put() in iscsit_deaccess_np()
- Adding kref_put() and wait_for_completion() in
iscsit_reset_np_thread()
- Refactor login failure path release logic into
iscsi_target_login_sess_out()
- Update __iscsi_target_login_thread() to handle
iscsi_post_login_handler() asynchronous completion
- Add shutdown parameter for iscsit_clear_tpg_np_login_thread*()

v3 changes:

- Convert iscsi_portal_group->np_login_lock to ->np_login_sem

- Add LOGIN_FLAGS definitions

v2 changes:
- Remove duplicate call to iscsi_post_login_handler() in
__iscsi_target_login_thread()
- Drop unused iscsi_np->np_login_tpg

Signed-off-by: Nicholas Bellinger <n...@linux-iscsi.org>
---
drivers/target/iscsi/iscsi_target.c | 55 +++++-----
drivers/target/iscsi/iscsi_target.h | 6 +-
drivers/target/iscsi/iscsi_target_core.h | 15 +++-
drivers/target/iscsi/iscsi_target_login.c | 162 +++++++++++++++++------------
drivers/target/iscsi/iscsi_target_login.h | 3 +
drivers/target/iscsi/iscsi_target_tpg.c | 18 ++--
drivers/target/iscsi/iscsi_target_tpg.h | 2 +-
7 files changed, 154 insertions(+), 107 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index c4aeac3..1dff3e0 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -220,11 +220,6 @@ int iscsit_access_np(struct iscsi_np *np, struct iscsi_portal_group *tpg)
spin_unlock_bh(&np->np_thread_lock);
return -1;
}
- if (np->np_login_tpg) {
- pr_err("np->np_login_tpg() is not NULL!\n");
- spin_unlock_bh(&np->np_thread_lock);
- return -1;
- }
spin_unlock_bh(&np->np_thread_lock);
/*
* Determine if the portal group is accepting storage traffic.
@@ -239,26 +234,38 @@ int iscsit_access_np(struct iscsi_np *np, struct iscsi_portal_group *tpg)
/*
* Here we serialize access across the TIQN+TPG Tuple.
*/
- ret = mutex_lock_interruptible(&tpg->np_login_lock);
+ ret = down_interruptible(&tpg->np_login_sem);
if ((ret != 0) || signal_pending(current))
return -1;

- spin_lock_bh(&np->np_thread_lock);
- np->np_login_tpg = tpg;
- spin_unlock_bh(&np->np_thread_lock);
+ spin_lock_bh(&tpg->tpg_state_lock);
+ if (tpg->tpg_state != TPG_STATE_ACTIVE) {
+ spin_unlock_bh(&tpg->tpg_state_lock);
+ up(&tpg->np_login_sem);
+ return -1;
+ }
+ spin_unlock_bh(&tpg->tpg_state_lock);

return 0;
}

-int iscsit_deaccess_np(struct iscsi_np *np, struct iscsi_portal_group *tpg)
+void iscsit_login_kref_put(struct kref *kref)
+{
+ struct iscsi_tpg_np *tpg_np = container_of(kref,
+ struct iscsi_tpg_np, tpg_np_kref);
+
+ complete(&tpg_np->tpg_np_comp);
+}
+
+int iscsit_deaccess_np(struct iscsi_np *np, struct iscsi_portal_group *tpg,
+ struct iscsi_tpg_np *tpg_np)
{
struct iscsi_tiqn *tiqn = tpg->tpg_tiqn;

- spin_lock_bh(&np->np_thread_lock);
- np->np_login_tpg = NULL;
- spin_unlock_bh(&np->np_thread_lock);
+ up(&tpg->np_login_sem);

- mutex_unlock(&tpg->np_login_lock);
+ if (tpg_np)
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);

if (tiqn)
iscsit_put_tiqn_for_login(tiqn);
@@ -410,20 +417,10 @@ struct iscsi_np *iscsit_add_np(
int iscsit_reset_np_thread(
struct iscsi_np *np,
struct iscsi_tpg_np *tpg_np,
- struct iscsi_portal_group *tpg)
+ struct iscsi_portal_group *tpg,
+ bool shutdown)
{
spin_lock_bh(&np->np_thread_lock);
- if (tpg && tpg_np) {
- /*
- * The reset operation need only be performed when the
- * passed struct iscsi_portal_group has a login in progress
- * to one of the network portals.
- */
- if (tpg_np->tpg_np->np_login_tpg != tpg) {
- spin_unlock_bh(&np->np_thread_lock);
- return 0;
- }
- }
if (np->np_thread_state == ISCSI_NP_THREAD_INACTIVE) {
spin_unlock_bh(&np->np_thread_lock);
return 0;
@@ -438,6 +435,12 @@ int iscsit_reset_np_thread(
}
spin_unlock_bh(&np->np_thread_lock);

+ if (tpg_np && shutdown) {
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
+
+ wait_for_completion(&tpg_np->tpg_np_comp);
+ }
+
return 0;
}

diff --git a/drivers/target/iscsi/iscsi_target.h b/drivers/target/iscsi/iscsi_target.h
index 2c437cb..f82f627 100644
--- a/drivers/target/iscsi/iscsi_target.h
+++ b/drivers/target/iscsi/iscsi_target.h
@@ -7,13 +7,15 @@ extern void iscsit_put_tiqn_for_login(struct iscsi_tiqn *);
extern struct iscsi_tiqn *iscsit_add_tiqn(unsigned char *);
extern void iscsit_del_tiqn(struct iscsi_tiqn *);
extern int iscsit_access_np(struct iscsi_np *, struct iscsi_portal_group *);
-extern int iscsit_deaccess_np(struct iscsi_np *, struct iscsi_portal_group *);
+extern void iscsit_login_kref_put(struct kref *);
+extern int iscsit_deaccess_np(struct iscsi_np *, struct iscsi_portal_group *,
+ struct iscsi_tpg_np *);
extern bool iscsit_check_np_match(struct __kernel_sockaddr_storage *,
struct iscsi_np *, int);
extern struct iscsi_np *iscsit_add_np(struct __kernel_sockaddr_storage *,
char *, int);
extern int iscsit_reset_np_thread(struct iscsi_np *, struct iscsi_tpg_np *,
- struct iscsi_portal_group *);
+ struct iscsi_portal_group *, bool);
extern int iscsit_del_np(struct iscsi_np *);
extern int iscsit_reject_cmd(struct iscsi_cmd *cmd, u8, unsigned char *);
extern void iscsit_set_unsoliticed_dataout(struct iscsi_cmd *);
diff --git a/drivers/target/iscsi/iscsi_target_core.h b/drivers/target/iscsi/iscsi_target_core.h
index 4f77a78..4622846 100644
--- a/drivers/target/iscsi/iscsi_target_core.h
+++ b/drivers/target/iscsi/iscsi_target_core.h
@@ -554,6 +554,13 @@ struct iscsi_conn {
struct completion rx_half_close_comp;
/* socket used by this connection */
struct socket *sock;
+ void (*orig_data_ready)(struct sock *, int);
+#define LOGIN_FLAGS_READ_ACTIVE 1
+#define LOGIN_FLAGS_CLOSED 2
+#define LOGIN_FLAGS_READY 4
+ unsigned long login_flags;
+ struct delayed_work login_work;
+ struct iscsi_login *login;
struct timer_list nopin_timer;
struct timer_list nopin_response_timer;
struct timer_list transport_timer;
@@ -584,6 +591,7 @@ struct iscsi_conn {
void *context;
struct iscsi_login_thread_s *login_thread;
struct iscsi_portal_group *tpg;
+ struct iscsi_tpg_np *tpg_np;
/* Pointer to parent session */
struct iscsi_session *sess;
/* Pointer to thread_set in use for this conn's threads */
@@ -682,6 +690,7 @@ struct iscsi_login {
u8 version_max;
u8 login_complete;
u8 login_failed;
+ bool zero_tsih;
char isid[6];
u32 cmd_sn;
itt_t init_task_tag;
@@ -694,6 +703,7 @@ struct iscsi_login {
char *req_buf;
char *rsp_buf;
struct iscsi_conn *conn;
+ struct iscsi_np *np;
} ____cacheline_aligned;

struct iscsi_node_attrib {
@@ -773,7 +783,6 @@ struct iscsi_np {
struct __kernel_sockaddr_storage np_sockaddr;
struct task_struct *np_thread;
struct timer_list np_login_timer;
- struct iscsi_portal_group *np_login_tpg;
void *np_context;
struct iscsit_transport *np_transport;
struct list_head np_list;
@@ -788,6 +797,8 @@ struct iscsi_tpg_np {
struct list_head tpg_np_parent_list;
struct se_tpg_np se_tpg_np;
spinlock_t tpg_np_parent_lock;
+ struct completion tpg_np_comp;
+ struct kref tpg_np_kref;
};

struct iscsi_portal_group {
@@ -809,7 +820,7 @@ struct iscsi_portal_group {
spinlock_t tpg_state_lock;
struct se_portal_group tpg_se_tpg;
struct mutex tpg_access_lock;
- struct mutex np_login_lock;
+ struct semaphore np_login_sem;
struct iscsi_tpg_attrib tpg_attrib;
struct iscsi_node_auth tpg_demo_auth;
/* Pointer to default list of iSCSI parameters for TPG */
diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c
index 0e85238..d903c0c 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -50,6 +50,7 @@ static struct iscsi_login *iscsi_login_init_conn(struct iscsi_conn *conn)
pr_err("Unable to allocate memory for struct iscsi_login.\n");
return NULL;
}
+ conn->login = login;
login->conn = conn;
login->first_request = 1;

@@ -684,7 +685,7 @@ static void iscsi_post_login_start_timers(struct iscsi_conn *conn)
iscsit_start_nopin_timer(conn);
}

-static int iscsi_post_login_handler(
+int iscsi_post_login_handler(
struct iscsi_np *np,
struct iscsi_conn *conn,
u8 zero_tsih)
@@ -1124,6 +1125,77 @@ iscsit_conn_set_transport(struct iscsi_conn *conn, struct iscsit_transport *t)
return 0;
}

+void iscsi_target_login_sess_out(struct iscsi_conn *conn,
+ struct iscsi_np *np, bool zero_tsih, bool new_sess)
+{
+ if (new_sess == false)
+ goto old_sess_out;
+
+ pr_err("iSCSI Login negotiation failed.\n");
+ iscsit_collect_login_stats(conn, ISCSI_STATUS_CLS_INITIATOR_ERR,
+ ISCSI_LOGIN_STATUS_INIT_ERR);
+ if (!zero_tsih || !conn->sess)
+ goto old_sess_out;
+ if (conn->sess->se_sess)
+ transport_free_session(conn->sess->se_sess);
+ if (conn->sess->session_index != 0) {
+ spin_lock_bh(&sess_idr_lock);
+ idr_remove(&sess_idr, conn->sess->session_index);
+ spin_unlock_bh(&sess_idr_lock);
+ }
+ kfree(conn->sess->sess_ops);
+ kfree(conn->sess);
+
+old_sess_out:
+ iscsi_stop_login_thread_timer(np);
+ /*
+ * If login negotiation fails check if the Time2Retain timer
+ * needs to be restarted.
+ */
+ if (!zero_tsih && conn->sess) {
+ spin_lock_bh(&conn->sess->conn_lock);
+ if (conn->sess->session_state == TARG_SESS_STATE_FAILED) {
+ struct se_portal_group *se_tpg =
+ &ISCSI_TPG_C(conn)->tpg_se_tpg;
+
+ atomic_set(&conn->sess->session_continuation, 0);
+ spin_unlock_bh(&conn->sess->conn_lock);
+ spin_lock_bh(&se_tpg->session_lock);
+ iscsit_start_time2retain_handler(conn->sess);
+ spin_unlock_bh(&se_tpg->session_lock);
+ } else
+ spin_unlock_bh(&conn->sess->conn_lock);
+ iscsit_dec_session_usage_count(conn->sess);
+ }
+
+ if (!IS_ERR(conn->conn_rx_hash.tfm))
+ crypto_free_hash(conn->conn_rx_hash.tfm);
+ if (!IS_ERR(conn->conn_tx_hash.tfm))
+ crypto_free_hash(conn->conn_tx_hash.tfm);
+
+ if (conn->conn_cpumask)
+ free_cpumask_var(conn->conn_cpumask);
+
+ kfree(conn->conn_ops);
+
+ if (conn->param_list) {
+ iscsi_release_param_list(conn->param_list);
+ conn->param_list = NULL;
+ }
+ iscsi_target_nego_release(conn);
+
+ if (conn->sock) {
+ sock_release(conn->sock);
+ conn->sock = NULL;
+ }
+
+ if (conn->conn_transport->iscsit_free_conn)
+ conn->conn_transport->iscsit_free_conn(conn);
+
+ iscsit_put_transport(conn->conn_transport);
+ kfree(conn);
+}
+
static int __iscsi_target_login_thread(struct iscsi_np *np)
{
u8 *buffer, zero_tsih = 0;
@@ -1132,6 +1204,8 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
struct iscsi_login *login;
struct iscsi_portal_group *tpg = NULL;
struct iscsi_login_req *pdu;
+ struct iscsi_tpg_np *tpg_np;
+ bool new_sess = false;

flush_signals(current);

@@ -1273,6 +1347,7 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
tpg = conn->tpg;
goto new_sess_out;
}
+ login->zero_tsih = zero_tsih;

tpg = conn->tpg;
if (!tpg) {
@@ -1288,7 +1363,8 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
goto old_sess_out;
}

- if (iscsi_target_start_negotiation(login, conn) < 0)
+ ret = iscsi_target_start_negotiation(login, conn);
+ if (ret < 0)
goto new_sess_out;

if (!conn->sess) {
@@ -1301,84 +1377,32 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
if (signal_pending(current))
goto new_sess_out;

- ret = iscsi_post_login_handler(np, conn, zero_tsih);
+ if (ret == 1) {
+ tpg_np = conn->tpg_np;

- if (ret < 0)
- goto new_sess_out;
+ ret = iscsi_post_login_handler(np, conn, zero_tsih);
+ if (ret < 0)
+ goto new_sess_out;
+
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ }

- iscsit_deaccess_np(np, tpg);
tpg = NULL;
+ tpg_np = NULL;
/* Get another socket */
return 1;

new_sess_out:
- pr_err("iSCSI Login negotiation failed.\n");
- iscsit_collect_login_stats(conn, ISCSI_STATUS_CLS_INITIATOR_ERR,
- ISCSI_LOGIN_STATUS_INIT_ERR);
- if (!zero_tsih || !conn->sess)
- goto old_sess_out;
- if (conn->sess->se_sess)
- transport_free_session(conn->sess->se_sess);
- if (conn->sess->session_index != 0) {
- spin_lock_bh(&sess_idr_lock);
- idr_remove(&sess_idr, conn->sess->session_index);
- spin_unlock_bh(&sess_idr_lock);
- }
- kfree(conn->sess->sess_ops);
- kfree(conn->sess);
+ new_sess = true;
old_sess_out:
- iscsi_stop_login_thread_timer(np);
- /*
- * If login negotiation fails check if the Time2Retain timer
- * needs to be restarted.
- */
- if (!zero_tsih && conn->sess) {
- spin_lock_bh(&conn->sess->conn_lock);
- if (conn->sess->session_state == TARG_SESS_STATE_FAILED) {
- struct se_portal_group *se_tpg =
- &ISCSI_TPG_C(conn)->tpg_se_tpg;
-
- atomic_set(&conn->sess->session_continuation, 0);
- spin_unlock_bh(&conn->sess->conn_lock);
- spin_lock_bh(&se_tpg->session_lock);
- iscsit_start_time2retain_handler(conn->sess);
- spin_unlock_bh(&se_tpg->session_lock);
- } else
- spin_unlock_bh(&conn->sess->conn_lock);
- iscsit_dec_session_usage_count(conn->sess);
- }
-
- if (!IS_ERR(conn->conn_rx_hash.tfm))
- crypto_free_hash(conn->conn_rx_hash.tfm);
- if (!IS_ERR(conn->conn_tx_hash.tfm))
- crypto_free_hash(conn->conn_tx_hash.tfm);
-
- if (conn->conn_cpumask)
- free_cpumask_var(conn->conn_cpumask);
-
- kfree(conn->conn_ops);
-
- if (conn->param_list) {
- iscsi_release_param_list(conn->param_list);
- conn->param_list = NULL;
- }
- iscsi_target_nego_release(conn);
-
- if (conn->sock) {
- sock_release(conn->sock);
- conn->sock = NULL;
- }
-
- if (conn->conn_transport->iscsit_free_conn)
- conn->conn_transport->iscsit_free_conn(conn);
-
- iscsit_put_transport(conn->conn_transport);
-
- kfree(conn);
+ tpg_np = conn->tpg_np;
+ iscsi_target_login_sess_out(conn, np, zero_tsih, new_sess);
+ new_sess = false;

if (tpg) {
- iscsit_deaccess_np(np, tpg);
+ iscsit_deaccess_np(np, tpg, tpg_np);
tpg = NULL;
+ tpg_np = NULL;
}

out:
diff --git a/drivers/target/iscsi/iscsi_target_login.h b/drivers/target/iscsi/iscsi_target_login.h
index 63efd28..29d0983 100644
--- a/drivers/target/iscsi/iscsi_target_login.h
+++ b/drivers/target/iscsi/iscsi_target_login.h
@@ -12,6 +12,9 @@ extern int iscsit_accept_np(struct iscsi_np *, struct iscsi_conn *);
extern int iscsit_get_login_rx(struct iscsi_conn *, struct iscsi_login *);
extern int iscsit_put_login_tx(struct iscsi_conn *, struct iscsi_login *, u32);
extern void iscsit_free_conn(struct iscsi_np *, struct iscsi_conn *);
+extern int iscsi_post_login_handler(struct iscsi_np *, struct iscsi_conn *, u8);
+extern void iscsi_target_login_sess_out(struct iscsi_conn *, struct iscsi_np *,
+ bool, bool);
extern int iscsi_target_login_thread(void *);
extern int iscsi_login_disable_FIM_keys(struct iscsi_param_list *, struct iscsi_conn *);

diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index 439260b..4bf8913 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -49,7 +49,7 @@ struct iscsi_portal_group *iscsit_alloc_portal_group(struct iscsi_tiqn *tiqn, u1
INIT_LIST_HEAD(&tpg->tpg_gnp_list);
INIT_LIST_HEAD(&tpg->tpg_list);
mutex_init(&tpg->tpg_access_lock);
- mutex_init(&tpg->np_login_lock);
+ sema_init(&tpg->np_login_sem, 1);
spin_lock_init(&tpg->tpg_state_lock);
spin_lock_init(&tpg->tpg_np_lock);

@@ -175,18 +175,20 @@ void iscsit_put_tpg(struct iscsi_portal_group *tpg)

static void iscsit_clear_tpg_np_login_thread(
struct iscsi_tpg_np *tpg_np,
- struct iscsi_portal_group *tpg)
+ struct iscsi_portal_group *tpg,
+ bool shutdown)
{
if (!tpg_np->tpg_np) {
pr_err("struct iscsi_tpg_np->tpg_np is NULL!\n");
return;
}

- iscsit_reset_np_thread(tpg_np->tpg_np, tpg_np, tpg);
+ iscsit_reset_np_thread(tpg_np->tpg_np, tpg_np, tpg, shutdown);
}

void iscsit_clear_tpg_np_login_threads(
- struct iscsi_portal_group *tpg)
+ struct iscsi_portal_group *tpg,
+ bool shutdown)
{
struct iscsi_tpg_np *tpg_np;

@@ -197,7 +199,7 @@ void iscsit_clear_tpg_np_login_threads(
continue;
}
spin_unlock(&tpg->tpg_np_lock);
- iscsit_clear_tpg_np_login_thread(tpg_np, tpg);
+ iscsit_clear_tpg_np_login_thread(tpg_np, tpg, shutdown);
spin_lock(&tpg->tpg_np_lock);
}
spin_unlock(&tpg->tpg_np_lock);
@@ -268,6 +270,8 @@ int iscsit_tpg_del_portal_group(
tpg->tpg_state = TPG_STATE_INACTIVE;
spin_unlock(&tpg->tpg_state_lock);

+ iscsit_clear_tpg_np_login_threads(tpg, true);
+
if (iscsit_release_sessions_for_tpg(tpg, force) < 0) {
pr_err("Unable to delete iSCSI Target Portal Group:"
" %hu while active sessions exist, and force=0\n",
@@ -368,7 +372,7 @@ int iscsit_tpg_disable_portal_group(struct iscsi_portal_group *tpg, int force)
tpg->tpg_state = TPG_STATE_INACTIVE;
spin_unlock(&tpg->tpg_state_lock);

- iscsit_clear_tpg_np_login_threads(tpg);
+ iscsit_clear_tpg_np_login_threads(tpg, false);

if (iscsit_release_sessions_for_tpg(tpg, force) < 0) {
spin_lock(&tpg->tpg_state_lock);
@@ -520,7 +524,7 @@ static int iscsit_tpg_release_np(
struct iscsi_portal_group *tpg,
struct iscsi_np *np)
{
- iscsit_clear_tpg_np_login_thread(tpg_np, tpg);
+ iscsit_clear_tpg_np_login_thread(tpg_np, tpg, true);

pr_debug("CORE[%s] - Removed Network Portal: %s:%hu,%hu on %s\n",
tpg->tpg_tiqn->tiqn, np->np_ip, np->np_port, tpg->tpgt,
diff --git a/drivers/target/iscsi/iscsi_target_tpg.h b/drivers/target/iscsi/iscsi_target_tpg.h
index dda48c1..60ef4a9 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.h
+++ b/drivers/target/iscsi/iscsi_target_tpg.h
@@ -8,7 +8,7 @@ extern struct iscsi_portal_group *iscsit_get_tpg_from_np(struct iscsi_tiqn *,
struct iscsi_np *);
extern int iscsit_get_tpg(struct iscsi_portal_group *);
extern void iscsit_put_tpg(struct iscsi_portal_group *);
-extern void iscsit_clear_tpg_np_login_threads(struct iscsi_portal_group *);
+extern void iscsit_clear_tpg_np_login_threads(struct iscsi_portal_group *, bool);
extern void iscsit_tpg_dump_params(struct iscsi_portal_group *);
extern int iscsit_tpg_add_portal_group(struct iscsi_tiqn *, struct iscsi_portal_group *);
extern int iscsit_tpg_del_portal_group(struct iscsi_tiqn *, struct iscsi_portal_group *,
--
1.7.2.5

[Nicholas A. Bellinger]

From: Nicholas Bellinger <n...@linux-iscsi.org>

This patch adds support for login negotiation multi-plexing in
iscsi-target code.

This involves handling the first login request PDU + payload and
login response PDU + payload within __iscsi_target_login_thread()
process context, and then changing struct sock->sk_data_ready()
so that all subsequent exchanges are handled by workqueue process
context, to allow other incoming login requests to be received
in parallel by __iscsi_target_login_thread().

Upon login negotiation completion (or failure), ->sk_data_ready()
is replaced with the original kernel sockets handler saved in
iscsi_conn->orig_data_ready.

v3 changes:

- Convert iscsi_target_sk_data_ready() lock access to
write[lock,unlock]_bh()
- Only clear LOGIN_FLAGS_READ_ACTIVE when iscsi_target_do_login()
returns zero
- Add LOGIN_FLAGS_READY + LOGIN_FLAGS_CLOSED bit checks to
iscsi_target_sk_data_ready()
- Make INIT_DELAYED_WORK() + iscsi_target_set_sock_callbacks() setup
happen earlier by moving from iscsi_target_start_negotiation() into
iscsi_target_locate_portal()
- Set LOGIN_FLAGS_READY bit in iscsi_target_start_negotiation()
after iscsi_target_do_login() returns zero.

v2 changes:
- Add login_timer in iscsi_target_do_login_rx() to avoid
possible endless sleep with MSG_WAITALL for traditional
iscsi-target in certain network configurations.
- Convert lprintk() -> pr_debug()
- Remove forward declarations of iscsi_target_set_sock_callbacks(),
iscsi_target_restore_sock_callbacks() and iscsi_target_sk_data_ready()
- Make iscsi_target_set_sock_callbacks + iscsi_target_restore_sock_callbacks()
static (Fengguang)
- Make iscsi_target_do_login_rx() safe for iser-target w/o conn->sock

Signed-off-by: Nicholas Bellinger <n...@linux-iscsi.org>
---

drivers/target/iscsi/iscsi_target_core.h | 1 +
drivers/target/iscsi/iscsi_target_nego.c | 234 +++++++++++++++++++++++++++++-
drivers/target/iscsi/iscsi_target_tpg.c | 7 +-
drivers/target/iscsi/iscsi_target_tpg.h | 2 +-
4 files changed, 234 insertions(+), 10 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target_core.h b/drivers/target/iscsi/iscsi_target_core.h
index 4622846..43228dc 100644
--- a/drivers/target/iscsi/iscsi_target_core.h
+++ b/drivers/target/iscsi/iscsi_target_core.h
@@ -564,6 +564,7 @@ struct iscsi_conn {

struct timer_list nopin_timer;
struct timer_list nopin_response_timer;
struct timer_list transport_timer;

+ struct task_struct *login_kworker;
/* Spinlock used for add/deleting cmd's from conn_cmd_list */
spinlock_t cmd_lock;
spinlock_t conn_usage_lock;
diff --git a/drivers/target/iscsi/iscsi_target_nego.c b/drivers/target/iscsi/iscsi_target_nego.c
index c4675b4..9defee0f 100644
--- a/drivers/target/iscsi/iscsi_target_nego.c
+++ b/drivers/target/iscsi/iscsi_target_nego.c
@@ -377,14 +377,207 @@ static int iscsi_target_do_tx_login_io(struct iscsi_conn *conn, struct iscsi_log
return 0;
}

+static void iscsi_target_sk_data_ready(struct sock *sk, int count)
+{
+ struct iscsi_conn *conn = sk->sk_user_data;
+ bool rc;
+
+ pr_debug("Entering iscsi_target_sk_data_ready: conn: %p\n", conn);
+
+ write_lock_bh(&sk->sk_callback_lock);
+ if (!sk->sk_user_data) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ return;
+ }
+ if (!test_bit(LOGIN_FLAGS_READY, &conn->login_flags)) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ pr_debug("Got LOGIN_FLAGS_READY=0, conn: %p >>>>\n", conn);
+ return;
+ }
+ if (test_bit(LOGIN_FLAGS_CLOSED, &conn->login_flags)) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ pr_debug("Got LOGIN_FLAGS_CLOSED=1, conn: %p >>>>\n", conn);
+ return;
+ }
+ if (test_and_set_bit(LOGIN_FLAGS_READ_ACTIVE, &conn->login_flags)) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ pr_debug("Got LOGIN_FLAGS_READ_ACTIVE=1, conn: %p >>>>\n", conn);
+ return;
+ }
+
+ rc = schedule_delayed_work(&conn->login_work, 0);
+ if (rc == false) {
+ pr_debug("iscsi_target_sk_data_ready, schedule_delayed_work"
+ " got false\n");
+ }
+ write_unlock_bh(&sk->sk_callback_lock);
+}
+
+static void iscsi_target_set_sock_callbacks(struct iscsi_conn *conn)
+{
+ struct sock *sk;
+
+ if (!conn->sock)
+ return;
+
+ sk = conn->sock->sk;
+ pr_debug("Entering iscsi_target_set_sock_callbacks: conn: %p\n", conn);
+
+ write_lock_bh(&sk->sk_callback_lock);
+ sk->sk_user_data = conn;
+ conn->orig_data_ready = sk->sk_data_ready;
+ sk->sk_data_ready = iscsi_target_sk_data_ready;
+ write_unlock_bh(&sk->sk_callback_lock);
+}
+
+static void iscsi_target_restore_sock_callbacks(struct iscsi_conn *conn)
+{
+ struct sock *sk;
+
+ if (!conn->sock)
+ return;
+
+ sk = conn->sock->sk;
+ pr_debug("Entering iscsi_target_restore_sock_callbacks: conn: %p\n", conn);
+
+ write_lock_bh(&sk->sk_callback_lock);
+ if (!sk->sk_user_data) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ return;
+ }
+ sk->sk_user_data = NULL;
+ sk->sk_data_ready = conn->orig_data_ready;
+ write_unlock_bh(&sk->sk_callback_lock);
+}
+
+static int iscsi_target_do_login(struct iscsi_conn *, struct iscsi_login *);
+
+static bool iscsi_target_sk_state_check(struct sock *sk)
+{
+ if (sk->sk_state == TCP_CLOSE_WAIT || sk->sk_state == TCP_CLOSE) {
+ pr_debug("iscsi_target_sk_state_check: TCP_CLOSE_WAIT|TCP_CLOSE,"
+ "returning FALSE\n");
+ return false;
+ }
+ return true;
+}
+
+static void iscsi_target_login_drop(struct iscsi_conn *conn, struct iscsi_login *login)
+{
+ struct iscsi_np *np = login->np;
+ bool zero_tsih = login->zero_tsih;
+
+ iscsi_remove_failed_auth_entry(conn);
+ iscsi_target_nego_release(conn);
+ iscsi_target_login_sess_out(conn, np, zero_tsih, true);
+}
+
+static void iscsi_target_login_timeout(unsigned long data)
+{
+ struct iscsi_conn *conn = (struct iscsi_conn *)data;
+
+ pr_debug("Entering iscsi_target_login_timeout >>>>>>>>>>>>>>>>>>>\n");
+
+ if (conn->login_kworker) {
+ pr_debug("Sending SIGINT to conn->login_kworker %s/%d\n",
+ conn->login_kworker->comm, conn->login_kworker->pid);
+ send_sig(SIGINT, conn->login_kworker, 1);
+ }
+}
+
+static void iscsi_target_do_login_rx(struct work_struct *work)
+{
+ struct iscsi_conn *conn = container_of(work,
+ struct iscsi_conn, login_work.work);
+ struct iscsi_login *login = conn->login;
+ struct iscsi_np *np = login->np;
+ struct iscsi_portal_group *tpg = conn->tpg;
+ struct iscsi_tpg_np *tpg_np = conn->tpg_np;
+ struct timer_list login_timer;
+ int rc, zero_tsih = login->zero_tsih;
+ bool state;
+
+ pr_debug("entering iscsi_target_do_login_rx, conn: %p, %s:%d\n",
+ conn, current->comm, current->pid);
+
+ spin_lock(&tpg->tpg_state_lock);
+ state = (tpg->tpg_state == TPG_STATE_ACTIVE);
+ spin_unlock(&tpg->tpg_state_lock);
+
+ if (state == false) {
+ pr_debug("iscsi_target_do_login_rx: tpg_state != TPG_STATE_ACTIVE\n");
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ return;
+ }
+
+ if (conn->sock) {
+ struct sock *sk = conn->sock->sk;
+
+ read_lock_bh(&sk->sk_callback_lock);
+ state = iscsi_target_sk_state_check(sk);
+ read_unlock_bh(&sk->sk_callback_lock);
+
+ if (state == false) {
+ pr_debug("iscsi_target_do_login_rx, TCP state CLOSE\n");
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ return;
+ }
+ }
+
+ conn->login_kworker = current;
+ allow_signal(SIGINT);
+
+ init_timer(&login_timer);
+ login_timer.expires = (get_jiffies_64() + TA_LOGIN_TIMEOUT * HZ);
+ login_timer.data = (unsigned long)conn;
+ login_timer.function = iscsi_target_login_timeout;
+ add_timer(&login_timer);
+ pr_debug("Starting login_timer for %s/%d\n", current->comm, current->pid);
+
+ rc = conn->conn_transport->iscsit_get_login_rx(conn, login);
+ del_timer_sync(&login_timer);
+ flush_signals(current);
+ conn->login_kworker = NULL;
+
+ if (rc < 0) {
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ return;
+ }
+
+ pr_debug("iscsi_target_do_login_rx after rx_login_io, %p, %s:%d\n",
+ conn, current->comm, current->pid);
+
+ rc = iscsi_target_do_login(conn, login);
+ if (rc < 0) {
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ } else if (!rc) {
+ if (conn->sock) {
+ struct sock *sk = conn->sock->sk;
+
+ write_lock_bh(&sk->sk_callback_lock);
+ clear_bit(LOGIN_FLAGS_READ_ACTIVE, &conn->login_flags);
+ write_unlock_bh(&sk->sk_callback_lock);
+ }
+ } else if (rc == 1) {
+ iscsi_target_nego_release(conn);
+ iscsi_post_login_handler(np, conn, zero_tsih);

+ iscsit_deaccess_np(np, tpg, tpg_np);
+ }

+}
+
static int iscsi_target_do_login_io(struct iscsi_conn *conn, struct iscsi_login *login)
{
if (iscsi_target_do_tx_login_io(conn, login) < 0)
return -1;

- if (conn->conn_transport->iscsit_get_login_rx(conn, login) < 0)
- return -1;
-
return 0;
}

@@ -643,10 +836,11 @@ static int iscsi_target_do_login(struct iscsi_conn *conn, struct iscsi_login *lo
if (login_rsp->flags & ISCSI_FLAG_LOGIN_TRANSIT) {
login->tsih = conn->sess->tsih;
login->login_complete = 1;
+ iscsi_target_restore_sock_callbacks(conn);
if (iscsi_target_do_tx_login_io(conn,
login) < 0)
return -1;
- return 0;
+ return 1;
}
break;
default:
@@ -663,6 +857,7 @@ static int iscsi_target_do_login(struct iscsi_conn *conn, struct iscsi_login *lo
login_rsp->flags &= ~ISCSI_FLAG_LOGIN_TRANSIT;
login_rsp->flags &= ~ISCSI_FLAG_LOGIN_NEXT_STAGE_MASK;
}
+ break;
}

return 0;
@@ -695,10 +890,16 @@ int iscsi_target_locate_portal(
char *tmpbuf, *start = NULL, *end = NULL, *key, *value;
struct iscsi_session *sess = conn->sess;
struct iscsi_tiqn *tiqn;
+ struct iscsi_tpg_np *tpg_np = NULL;
struct iscsi_login_req *login_req;
u32 payload_length;
int sessiontype = 0, ret = 0;

+ INIT_DELAYED_WORK(&conn->login_work, iscsi_target_do_login_rx);
+ iscsi_target_set_sock_callbacks(conn);
+
+ login->np = np;
+
login_req = (struct iscsi_login_req *) login->req;
payload_length = ntoh24(login_req->dlength);

@@ -822,7 +1023,7 @@ get_target:
/*
* Locate Target Portal Group from Storage Node.
*/
- conn->tpg = iscsit_get_tpg_from_np(tiqn, np);
+ conn->tpg = iscsit_get_tpg_from_np(tiqn, np, &tpg_np);
if (!conn->tpg) {
pr_err("Unable to locate Target Portal Group"
" on %s\n", tiqn->tiqn);
@@ -832,12 +1033,16 @@ get_target:
ret = -1;
goto out;
}
+ conn->tpg_np = tpg_np;
pr_debug("Located Portal Group Object: %hu\n", conn->tpg->tpgt);
/*
* Setup crc32c modules from libcrypto
*/
if (iscsi_login_setup_crypto(conn) < 0) {
pr_err("iscsi_login_setup_crypto() failed\n");
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
+ iscsit_put_tiqn_for_login(tiqn);
+ conn->tpg = NULL;
ret = -1;
goto out;
}
@@ -846,11 +1051,12 @@ get_target:
* process login attempt.
*/
if (iscsit_access_np(np, conn->tpg) < 0) {
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
iscsit_put_tiqn_for_login(tiqn);
iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
ISCSI_LOGIN_STATUS_SVC_UNAVAILABLE);
- ret = -1;
conn->tpg = NULL;
+ ret = -1;
goto out;
}

@@ -897,10 +1103,22 @@ int iscsi_target_start_negotiation(
int ret;

ret = iscsi_target_do_login(conn, login);
- if (ret != 0)
+ if (!ret) {
+ if (conn->sock) {
+ struct sock *sk = conn->sock->sk;
+
+ write_lock_bh(&sk->sk_callback_lock);
+ set_bit(LOGIN_FLAGS_READY, &conn->login_flags);
+ write_unlock_bh(&sk->sk_callback_lock);
+ }
+ } else if (ret < 0) {
+ cancel_delayed_work_sync(&conn->login_work);
+ iscsi_target_restore_sock_callbacks(conn);
iscsi_remove_failed_auth_entry(conn);
+ }
+ if (ret != 0)
+ iscsi_target_nego_release(conn);

- iscsi_target_nego_release(conn);
return ret;
}

diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index 4bf8913..e070ce2 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -129,7 +129,8 @@ void iscsit_release_discovery_tpg(void)

struct iscsi_portal_group *iscsit_get_tpg_from_np(
struct iscsi_tiqn *tiqn,
- struct iscsi_np *np)
+ struct iscsi_np *np,
+ struct iscsi_tpg_np **tpg_np_out)
{

struct iscsi_portal_group *tpg = NULL;

struct iscsi_tpg_np *tpg_np;
@@ -147,6 +148,8 @@ struct iscsi_portal_group *iscsit_get_tpg_from_np(
spin_lock(&tpg->tpg_np_lock);
list_for_each_entry(tpg_np, &tpg->tpg_gnp_list, tpg_np_list) {
if (tpg_np->tpg_np == np) {
+ *tpg_np_out = tpg_np;
+ kref_get(&tpg_np->tpg_np_kref);
spin_unlock(&tpg->tpg_np_lock);
spin_unlock(&tiqn->tiqn_tpg_lock);
return tpg;
@@ -494,6 +497,8 @@ struct iscsi_tpg_np *iscsit_tpg_add_network_portal(
INIT_LIST_HEAD(&tpg_np->tpg_np_child_list);
INIT_LIST_HEAD(&tpg_np->tpg_np_parent_list);
spin_lock_init(&tpg_np->tpg_np_parent_lock);
+ init_completion(&tpg_np->tpg_np_comp);
+ kref_init(&tpg_np->tpg_np_kref);
tpg_np->tpg_np = np;
tpg_np->tpg = tpg;

diff --git a/drivers/target/iscsi/iscsi_target_tpg.h b/drivers/target/iscsi/iscsi_target_tpg.h
index 60ef4a9..b77693e 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.h
+++ b/drivers/target/iscsi/iscsi_target_tpg.h
@@ -5,7 +5,7 @@ extern struct iscsi_portal_group *iscsit_alloc_portal_group(struct iscsi_tiqn *,
extern int iscsit_load_discovery_tpg(void);
extern void iscsit_release_discovery_tpg(void);

extern struct iscsi_portal_group *iscsit_get_tpg_from_np(struct iscsi_tiqn *,

- struct iscsi_np *);
+ struct iscsi_np *, struct iscsi_tpg_np **);

extern int iscsit_get_tpg(struct iscsi_portal_group *);
extern void iscsit_put_tpg(struct iscsi_portal_group *);

extern void iscsit_clear_tpg_np_login_threads(struct iscsi_portal_group *, bool);

--
1.7.2.5

[Nicholas A. Bellinger]

From: Nicholas Bellinger <n...@linux-iscsi.org>

This patch adds a sock->sk_state_change() -> iscsi_target_sk_state_change()
callback in order to handle transient TCP failures during the login process,
where sock->sk_data_ready() -> iscsi_target_sk_data_ready() may not be
called to release connection resources, and relinquish tpg->np_login_lock
via iscsit_deaccess_np()

It performs the sk->sk_state check using iscsi_target_sk_state_check() to
look for TCP_CLOSE_WAIT + TCP_CLOSE, and invokes schedule_delayed_work() ->
iscsi_target_do_cleanup() to perform the remaining cleanup from process
context.

It adds an explicit sk_state_check to iscsi_target_do_login() in order
to determine a state failure when iscsi_target_sk_state_change() may
not be able to proceed before LOGIN_FLAGS_READY=1 is set.

Also use sk->sk_sndtimeo -> sk->sk_rcvtimeo settings during login to
iscsi_target_set_sock_callbacks(), and revert back post login to use
MAX_SCHEDULE_TIMEOUT in iscsi_target_restore_sock_callbacks().

Signed-off-by: Nicholas Bellinger <n...@linux-iscsi.org>
---

drivers/target/iscsi/iscsi_target_core.h | 2 +
drivers/target/iscsi/iscsi_target_nego.c | 101 ++++++++++++++++++++++++++++++
2 files changed, 103 insertions(+), 0 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target_core.h b/drivers/target/iscsi/iscsi_target_core.h
index 43228dc..089a0e9 100644
--- a/drivers/target/iscsi/iscsi_target_core.h
+++ b/drivers/target/iscsi/iscsi_target_core.h
@@ -555,11 +555,13 @@ struct iscsi_conn {

/* socket used by this connection */
struct socket *sock;

void (*orig_data_ready)(struct sock *, int);

+ void (*orig_state_change)(struct sock *);
#define LOGIN_FLAGS_READ_ACTIVE 1
#define LOGIN_FLAGS_CLOSED 2
#define LOGIN_FLAGS_READY 4
unsigned long login_flags;
struct delayed_work login_work;
+ struct delayed_work login_cleanup_work;
struct iscsi_login *login;

struct timer_list nopin_timer;
struct timer_list nopin_response_timer;

diff --git a/drivers/target/iscsi/iscsi_target_nego.c b/drivers/target/iscsi/iscsi_target_nego.c
index 9defee0f..9d0c910 100644
--- a/drivers/target/iscsi/iscsi_target_nego.c
+++ b/drivers/target/iscsi/iscsi_target_nego.c
@@ -413,6 +413,8 @@ static void iscsi_target_sk_data_ready(struct sock *sk, int count)
write_unlock_bh(&sk->sk_callback_lock);
}

+static void iscsi_target_sk_state_change(struct sock *);
+
static void iscsi_target_set_sock_callbacks(struct iscsi_conn *conn)
{
struct sock *sk;
@@ -426,8 +428,13 @@ static void iscsi_target_set_sock_callbacks(struct iscsi_conn *conn)
write_lock_bh(&sk->sk_callback_lock);
sk->sk_user_data = conn;

conn->orig_data_ready = sk->sk_data_ready;

+ conn->orig_state_change = sk->sk_state_change;
sk->sk_data_ready = iscsi_target_sk_data_ready;
+ sk->sk_state_change = iscsi_target_sk_state_change;
write_unlock_bh(&sk->sk_callback_lock);
+
+ sk->sk_sndtimeo = TA_LOGIN_TIMEOUT * HZ;
+ sk->sk_rcvtimeo = TA_LOGIN_TIMEOUT * HZ;
}

static void iscsi_target_restore_sock_callbacks(struct iscsi_conn *conn)
@@ -447,7 +454,11 @@ static void iscsi_target_restore_sock_callbacks(struct iscsi_conn *conn)
}
sk->sk_user_data = NULL;

sk->sk_data_ready = conn->orig_data_ready;

+ sk->sk_state_change = conn->orig_state_change;
write_unlock_bh(&sk->sk_callback_lock);
+
+ sk->sk_sndtimeo = MAX_SCHEDULE_TIMEOUT;
+ sk->sk_rcvtimeo = MAX_SCHEDULE_TIMEOUT;

}

static int iscsi_target_do_login(struct iscsi_conn *, struct iscsi_login *);

@@ -573,6 +584,79 @@ static void iscsi_target_do_login_rx(struct work_struct *work)
}
}

+static void iscsi_target_do_cleanup(struct work_struct *work)

+{
+ struct iscsi_conn *conn = container_of(work,

+ struct iscsi_conn, login_cleanup_work.work);

+ struct sock *sk = conn->sock->sk;

+ struct iscsi_login *login = conn->login;
+ struct iscsi_np *np = login->np;
+ struct iscsi_portal_group *tpg = conn->tpg;
+ struct iscsi_tpg_np *tpg_np = conn->tpg_np;
+

+ pr_debug("Entering iscsi_target_do_cleanup\n");
+
+ cancel_delayed_work_sync(&conn->login_work);
+ conn->orig_state_change(sk);
+

+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+

+ pr_debug("iscsi_target_do_cleanup done()\n");
+}
+
+static void iscsi_target_sk_state_change(struct sock *sk)
+{
+ struct iscsi_conn *conn;
+ void (*orig_state_change)(struct sock *);
+ bool state;
+
+ pr_debug("Entering iscsi_target_sk_state_change\n");
+
+ write_lock_bh(&sk->sk_callback_lock);
+ conn = sk->sk_user_data;
+ if (!conn) {

+ write_unlock_bh(&sk->sk_callback_lock);
+ return;
+ }

+ orig_state_change = conn->orig_state_change;

+
+ if (!test_bit(LOGIN_FLAGS_READY, &conn->login_flags)) {

+ pr_debug("Got LOGIN_FLAGS_READY=0 sk_state_change conn: %p\n",
+ conn);
+ write_unlock_bh(&sk->sk_callback_lock);
+ orig_state_change(sk);
+ return;
+ }
+ if (test_bit(LOGIN_FLAGS_READ_ACTIVE, &conn->login_flags)) {
+ pr_debug("Got LOGIN_FLAGS_READ_ACTIVE=1 sk_state_change"
+ " conn: %p\n", conn);
+ write_unlock_bh(&sk->sk_callback_lock);
+ orig_state_change(sk);
+ return;
+ }
+ if (test_and_set_bit(LOGIN_FLAGS_CLOSED, &conn->login_flags)) {
+ pr_debug("Got LOGIN_FLAGS_CLOSED=1 sk_state_change conn: %p\n",
+ conn);
+ write_unlock_bh(&sk->sk_callback_lock);
+ orig_state_change(sk);
+ return;
+ }
+
+ state = iscsi_target_sk_state_check(sk);
+ write_unlock_bh(&sk->sk_callback_lock);
+
+ pr_debug("iscsi_target_sk_state_change: state: %d\n", state);
+
+ if (!state) {
+ pr_debug("iscsi_target_sk_state_change got failed state\n");
+ schedule_delayed_work(&conn->login_cleanup_work, 0);
+ return;
+ }
+ orig_state_change(sk);

+}
+
static int iscsi_target_do_login_io(struct iscsi_conn *conn, struct iscsi_login *login)
{
if (iscsi_target_do_tx_login_io(conn, login) < 0)

@@ -860,6 +944,21 @@ static int iscsi_target_do_login(struct iscsi_conn *conn, struct iscsi_login *lo
break;

}

+ if (conn->sock) {
+ struct sock *sk = conn->sock->sk;

+ bool state;

+
+ read_lock_bh(&sk->sk_callback_lock);
+ state = iscsi_target_sk_state_check(sk);
+ read_unlock_bh(&sk->sk_callback_lock);
+

+ if (!state) {
+ pr_debug("iscsi_target_do_login() failed state for"
+ " conn: %p\n", conn);
+ return -1;
+ }
+ }
+
return 0;
}

@@ -896,6 +995,7 @@ int iscsi_target_locate_portal(

int sessiontype = 0, ret = 0;

INIT_DELAYED_WORK(&conn->login_work, iscsi_target_do_login_rx);
+ INIT_DELAYED_WORK(&conn->login_cleanup_work, iscsi_target_do_cleanup);
iscsi_target_set_sock_callbacks(conn);

login->np = np;
@@ -1113,6 +1213,7 @@ int iscsi_target_start_negotiation(
}
} else if (ret < 0) {
cancel_delayed_work_sync(&conn->login_work);
+ cancel_delayed_work_sync(&conn->login_cleanup_work);
iscsi_target_restore_sock_callbacks(conn);
iscsi_remove_failed_auth_entry(conn);
}
--
1.7.2.5

[Nicholas A. Bellinger]

From: Nicholas Bellinger <n...@linux-iscsi.org>

There is no need for iscsi_target_do_login_io() anymore in modern code,
so go ahead and call iscsi_target_do_tx_login_io() directly within
iscsi_target_do_login().

Signed-off-by: Nicholas Bellinger <n...@linux-iscsi.org>
---

drivers/target/iscsi/iscsi_target_nego.c | 10 +---------
1 files changed, 1 insertions(+), 9 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target_nego.c b/drivers/target/iscsi/iscsi_target_nego.c
index 9d0c910..2cce636 100644
--- a/drivers/target/iscsi/iscsi_target_nego.c
+++ b/drivers/target/iscsi/iscsi_target_nego.c
@@ -657,14 +657,6 @@ static void iscsi_target_sk_state_change(struct sock *sk)
orig_state_change(sk);
}

-static int iscsi_target_do_login_io(struct iscsi_conn *conn, struct iscsi_login *login)
-{
- if (iscsi_target_do_tx_login_io(conn, login) < 0)
- return -1;
-

- return 0;
-}
-

/*
* NOTE: We check for existing sessions or connections AFTER the initiator
* has been successfully authenticated in order to protect against faked
@@ -934,7 +926,7 @@ static int iscsi_target_do_login(struct iscsi_conn *conn, struct iscsi_login *lo
break;
}

- if (iscsi_target_do_login_io(conn, login) < 0)
+ if (iscsi_target_do_tx_login_io(conn, login) < 0)
return -1;

if (login_rsp->flags & ISCSI_FLAG_LOGIN_TRANSIT) {

--
1.7.2.5

[Nicholas A. Bellinger]

From: Nicholas Bellinger <n...@linux-iscsi.org>

This patch updates iser-target code to support login negotiation
multi-plexing. This includes only using isert_conn->conn_login_comp
for the first login request PDU, pushing the subsequent processing
to iscsi_conn->login_work -> iscsi_target_do_login_rx(), and turning
isert_get_login_rx() into a NOP.

v3 changes:

- Drop unnecessary LOGIN_FLAGS_READ_ACTIVE bit set in
isert_rx_login_req()

Signed-off-by: Nicholas Bellinger <n...@linux-iscsi.org>
---

drivers/infiniband/ulp/isert/ib_isert.c | 14 +++++++++++++-
1 files changed, 13 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index 3f62041..c42748b 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -869,7 +869,11 @@ isert_rx_login_req(struct iser_rx_desc *rx_desc, int rx_buflen,
size, rx_buflen, MAX_KEY_VALUE_PAIRS);
memcpy(login->req_buf, &rx_desc->data[0], size);

- complete(&isert_conn->conn_login_comp);
+ if (login->first_request) {
+ complete(&isert_conn->conn_login_comp);
+ return;
+ }
+ schedule_delayed_work(&conn->login_work, 0);
}

static void
@@ -2224,6 +2228,14 @@ isert_get_login_rx(struct iscsi_conn *conn, struct iscsi_login *login)
int ret;

pr_debug("isert_get_login_rx before conn_login_comp conn: %p\n", conn);
+ /*
+ * For login requests after the first PDU, isert_rx_login_req() will
+ * kick schedule_delayed_work(&conn->login_work) as the packet is
+ * received, which turns this callback from iscsi_target_do_login_rx()
+ * into a NOP.
+ */
+ if (!login->first_request)
+ return 0;

ret = wait_for_completion_interruptible(&isert_conn->conn_login_comp);
if (ret)
--
1.7.2.5