information on the config option -- node.session.iscsi.FastAbort = No

[maguar887]

We are currently running open iscsi version 2.0-871 on RHEL 5.3
(2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
(firmware 4.3.5)

During some routine member volume operaions (moving volumes from one
member to another)
we had a strange error in /var/log/messages indicating that the
mounted volume would be mounted read only. This is strange, becasue
the array requested a logout/and re-logged in within 3 seconds, which
should be well below the iscsi settings to return an error to the SCSI
subsystem.

Our vendor (Dell/Equallogic) does not recommend adjusting any settings
in iscsid.conf so we are running all the defaults

I am interested in this configuration option.
"node.session.iscsi.FastAbort = No" which is disabled by default, but
it is mentioned that it can be beneficial for Equallogic.

Can anyone else using Equallogic comment if this option has increased
stability during volume operations such as moving volumes between
storage pools/arrays

From the default iscsid.conf:
#************
# Workarounds
#************

# Some targets like IET prefer after an initiator has sent a task
# management function like an ABORT TASK or LOGICAL UNIT RESET, that
# it does not respond to PDUs like R2Ts. To enable this behavior
uncomment
# the following line (The default behavior is Yes):
node.session.iscsi.FastAbort = Yes

# Some targets like Equalogic prefer that after an initiator has sent
# a task management function like an ABORT TASK or LOGICAL UNIT RESET,
that
# it continue to respond to R2Ts. To enable this uncomment this line
# node.session.iscsi.FastAbort = No




Logs below:
View from iscsi san:

INFO 4/18/10 11:18:11 AM SATA01 iSCSI session to target
'10.30.0.84:3260, iqn.2001-05.com.equallogic:
6-8a0900-540e73901-9410003608347990-disk2' from initiator
'10.30.0.225:16261, iqn.1994-05.com.redhat:ec43807e8964' was closed.
Logout request was received from the initiator.

INFO 4/18/10 11:18:15 AM SATA01 iSCSI login to target
'10.30.0.52:3260, iqn.2001-05.com.equallogic:
6-8a0900-540e73901-9410003608347990-disk2' from initiator
'10.30.0.225:51784, iqn.1994-05.com.redhat:ec43807e8964' successful,
using Jumbo Frame length.

View from linux var/log/messages

Apr 18 11:18:11 serv02 kernel: sd 24:0:0:0: SCSI error: return code =
0x00070000
Apr 18 11:18:11 serv02 kernel: end_request: I/O error, dev sdt, sector
27424
Apr 18 11:18:11 serv02 kernel: sd 24:0:0:0: SCSI error: return code =
0x00070000
Apr 18 11:18:11 serv02 kernel: end_request: I/O error, dev sdt, sector
27520
Apr 18 11:18:11 serv02 kernel: Buffer I/O error on device sdt1,
logical block 3436
Apr 18 11:18:11 serv02 kernel: lost page write due to I/O error on
sdt1
Apr 18 11:18:11 serv02 kernel: Aborting journal on device sdt1.
Apr 18 11:18:11 serv02 kernel: ext3_abort called.
Apr 18 11:18:11 serv02 kernel: EXT3-fs error (device sdt1):
ext3_journal_start_sb: Detected aborted journal
Apr 18 11:18:11 serv02 kernel: Remounting filesystem read-only
Apr 18 11:18:12 serv02 iscsid: Target requests logout within 3 seconds
for connection
Apr 18 11:18:16 serv02 iscsid: connection22:0 is operational after
recovery (1 attempts)

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To post to this group, send email to open-...@googlegroups.com.
To unsubscribe from this group, send email to open-iscsi+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.

[Mike Christie]

On 04/28/2010 10:40 AM, maguar887 wrote:
> We are currently running open iscsi version 2.0-871 on RHEL 5.3
> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
> (firmware 4.3.5)
>

You need to upgrade your kernel. It had a bug with eql targets in the
async logout path. It is fixed in 5.4 and 5.5 kernels.

>
> Our vendor (Dell/Equallogic) does not recommend adjusting any settings
> in iscsid.conf so we are running all the defaults
>
> I am interested in this configuration option.
> "node.session.iscsi.FastAbort = No" which is disabled by default, but
> it is mentioned that it can be beneficial for Equallogic.
>
> Can anyone else using Equallogic comment if this option has increased
> stability during volume operations such as moving volumes between
> storage pools/arrays

It has nothing to do with your problem. It only comes into play when a
scsi command times out. In that path you do not get 0x00070000 errors.

[Mike Christie]

On 04/28/2010 12:43 PM, Mike Christie wrote:
> On 04/28/2010 10:40 AM, maguar887 wrote:
>> We are currently running open iscsi version 2.0-871 on RHEL 5.3
>> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
>> (firmware 4.3.5)
>>
>
> You need to upgrade your kernel. It had a bug with eql targets in the
> async logout path. It is fixed in 5.4 and 5.5 kernels.
>

Upgrade your iscsi-initiator-utils too.

[Mike Christie]

On 04/28/2010 12:43 PM, Mike Christie wrote:

> On 04/28/2010 10:40 AM, maguar887 wrote:
>> We are currently running open iscsi version 2.0-871 on RHEL 5.3
>> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
>> (firmware 4.3.5)
>>
>
> You need to upgrade your kernel. It had a bug with eql targets in the
> async logout path. It is fixed in 5.4 and 5.5 kernels.
>

Actually it was fixed in 5.3 too.

The weird thing here is that we get this error *before* we get the async
logout request.

Do you have multiple sessions? Is sdt accessed through session 22 (run
iscsiadm -m session -P 3 to see) or a different session? If a different
session is there more log and can you send it?

[Mike Christie]

On 04/28/2010 12:56 PM, Mike Christie wrote:
> On 04/28/2010 12:43 PM, Mike Christie wrote:
>> On 04/28/2010 10:40 AM, maguar887 wrote:
>>> We are currently running open iscsi version 2.0-871 on RHEL 5.3
>>> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
>>> (firmware 4.3.5)
>>>
>>
>> You need to upgrade your kernel. It had a bug with eql targets in the
>> async logout path. It is fixed in 5.4 and 5.5 kernels.
>>
>
> Actually it was fixed in 5.3 too.
>
> The weird thing here is that we get this error *before* we get the async
> logout request.
>
> Do you have multiple sessions? Is sdt accessed through session 22 (run
> iscsiadm -m session -P 3 to see) or a different session? If a different
> session is there more log and can you send it?
>

Could you also take a ethereal trace?

Looking at the iscsi code for that kernel the only place to get
0x00070000 seems to be this:

if (rhdr->response != ISCSI_STATUS_CMD_COMPLETED) {
sc->result = DID_ERROR << 16;
goto out;
}

which means the target did not complete the command. The
initiator/scsi-layer would have retried this command up to 5 times
before failing.

[maguar887]

Mike,

Thanks for the info!

Do you know exactly which kernel it was patched in?
what is available to us is:
2.6.18-194.0.0.0.4.el5Matt

Matt

> For more options, visit this group athttp://groups.google.com/group/open-iscsi?hl=en.- Hide quoted text -
>
> - Show quoted text -

[Mike Christie]

On 04/28/2010 02:07 PM, maguar887 wrote:
> Mike,
>
> Thanks for the info!
>
> Do you know exactly which kernel it was patched in?
> what is available to us is:
> 2.6.18-194.0.0.0.4.el5Matt

Forget the upgrade your kernel comment. The bug I was thinking about was
fixed in the kernel you were using, 2.6.18-92.1.6.0.2.el5 (that is
actually the kernel for RHEL 5.2 but you said you were using 5.3 btw).


Could you get me a ethereal trace, so I can see if the target is failing
the IO on us?

[Joseph Salisbury]

On Wed, Apr 28, 2010 at 1:43 PM, Mike Christie <mich...@cs.wisc.edu> wrote:

On 04/28/2010 10:40 AM, maguar887 wrote:

We are currently running open iscsi version 2.0-871 on RHEL 5.3
(2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
(firmware 4.3.5)

You need to upgrade your kernel. It had a bug with eql targets in the async logout path. It is fixed in 5.4 and 5.5 kernels.

Mike, do you know if this fix made it's way into the upstream kernel?
 

[Mike Christie]

On 04/28/2010 12:52 PM, Joseph Salisbury wrote:
> On Wed, Apr 28, 2010 at 1:43 PM, Mike Christie<mich...@cs.wisc.edu> wrote:
>
>> On 04/28/2010 10:40 AM, maguar887 wrote:
>>
>>> We are currently running open iscsi version 2.0-871 on RHEL 5.3
>>> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
>>> (firmware 4.3.5)
>>>
>>>
>> You need to upgrade your kernel. It had a bug with eql targets in the async
>> logout path. It is fixed in 5.4 and 5.5 kernels.
>>
>
> Mike, do you know if this fix made it's way into the upstream kernel?
>

Yes. Everything goes upstream :)

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=9000bcd649b26aef4c35d2941f8a65f05bbb9ee1;hp=bca1088a8194f4fc850b65ea4f324c15d47d3cc3

[maguar887]

Sorry for the confusion, we had 2 seperate systems, and I pulled the
onfo from the wrong one.

The system with the issue turns out to be RHEL 5.2
with this kernel:
2.6.18-53.1.19.0.1.el5

I'll try and reproduce and get a packet capture

> >> For more options, visit this group athttp://groups.google.com/group/open-iscsi?hl=en.-Hide quoted text -

[Joseph Salisbury]

On Wed, Apr 28, 2010 at 3:35 PM, Mike Christie <mich...@cs.wisc.edu> wrote:

On 04/28/2010 12:52 PM, Joseph Salisbury wrote:

On Wed, Apr 28, 2010 at 1:43 PM, Mike Christie<mich...@cs.wisc.edu>  wrote:

On 04/28/2010 10:40 AM, maguar887 wrote:

We are currently running open iscsi version 2.0-871 on RHEL 5.3
(2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
(firmware 4.3.5)

You need to upgrade your kernel. It had a bug with eql targets in the async
logout path. It is fixed in 5.4 and 5.5 kernels.

Mike, do you know if this fix made it's way into the upstream kernel?

Yes. Everything goes upstream :)

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=9000bcd649b26aef4c35d2941f8a65f05bbb9ee1;hp=bca1088a8194f4fc850b65ea4f324c15d47d3cc3

Thanks!

[david elsen]

Mike,
 
How can I get the 5.4 and 5.5 kernel fixes and latest iscs-initiator-utils for RHEL 5.3 Kernel?
 
I would like to change the ISID from initiator while it is trying to connect to target. I would like to establish multiple iSCSI session between my Initiator and target. I do not see any option for this in the Linux initiator on my system.
 
I am using RHEL 5.3 with 2.6.18-128.el5 kernel.
 
Thanks,
David

 
> Date: Wed, 28 Apr 2010 12:48:44 -0500
> From: mich...@cs.wisc.edu
> To: open-...@googlegroups.com
> CC: mfor...@gmail.com
> Subject: Re: information on the config option -- node.session.iscsi.FastAbort = No

---

The New Busy is not the old busy. Search, chat and e-mail from your inbox. Get started.

[Mike Christie]

On 04/29/2010 12:48 PM, david elsen wrote:
>
> Mike,
>
>
>
> How can I get the 5.4 and 5.5 kernel fixes and latest iscs-initiator-utils for RHEL 5.3 Kernel?
>

I am not sure. Do you normally use Red Hat's rhn service for this?

>
>
> I would like to change the ISID from initiator while it is trying to connect to target. I would like to establish multiple iSCSI session between my Initiator and target. I do not see any option for this in the Linux initiator on my system.
>

You cannot control the ISID. The iscsi layer does thos for you. You can
control the initiator name, but if you just wanted to create multiple
sessions with different ISIDs and let the iscsi layer do it for you then
the tools in 5.3 should work. Do you see info in the
/usr/share/docs/iscsi-initiator-tools-$version/README about ifaces.

>
>
> I am using RHEL 5.3 with 2.6.18-128.el5 kernel.
>
>
>
> Thanks,
>
> David
>
>
>
>> Date: Wed, 28 Apr 2010 12:48:44 -0500
>> From: mich...@cs.wisc.edu
>> To: open-...@googlegroups.com
>> CC: mfor...@gmail.com
>> Subject: Re: information on the config option -- node.session.iscsi.FastAbort = No
>>
>> On 04/28/2010 12:43 PM, Mike Christie wrote:
>>> On 04/28/2010 10:40 AM, maguar887 wrote:
>>>> We are currently running open iscsi version 2.0-871 on RHEL 5.3
>>>> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
>>>> (firmware 4.3.5)
>>>>
>>>
>>> You need to upgrade your kernel. It had a bug with eql targets in the
>>> async logout path. It is fixed in 5.4 and 5.5 kernels.
>>>
>>
>> Upgrade your iscsi-initiator-utils too.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "open-iscsi" group.
>> To post to this group, send email to open-...@googlegroups.com.
>> To unsubscribe from this group, send email to open-iscsi+...@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
>>
>

> _________________________________________________________________

> The New Busy is not the old busy. Search, chat and e-mail from your inbox.

> http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3

[david elsen]

Mike,

 
> You cannot control the ISID. The iscsi layer does thos for you. You can
> control the initiator name, but if you just wanted to create multiple
> sessions with different ISIDs and let the iscsi layer do it for you then
> the tools in 5.3 should work. Do you see info in the
> /usr/share/docs/iscsi-initiator-tools-$version/README about ifaces.
 

Thanks, I will go through the README file and see if I can get it done.
Thanks,
David
 
> Date: Thu, 29 Apr 2010 13:50:37 -0500
> From: mich...@cs.wisc.edu
> To: elsen...@hotmail.com
> CC: open-...@googlegroups.com; mfor...@gmail.com

---

Hotmail is redefining busy with tools for the New Busy. Get more from your inbox. See how.

[Pasi Kärkkäinen]

On Thu, Apr 29, 2010 at 10:48:20AM -0700, david elsen wrote:
> Mike,
>
> How can I get the 5.4 and 5.5 kernel fixes and latest iscs-initiator-utils
> for RHEL 5.3 Kernel?
>
> I would like to change the ISID from initiator while it is trying to
> connect to target. I would like to establish multiple iSCSI session
> between my Initiator and target. I do not see any option for this in the
> Linux initiator on my system.
>
> I am using RHEL 5.3 with 2.6.18-128.el5 kernel.
>

"yum update" should do it.

-- Pasi

> Thanks,
> David
>
>
> > Date: Wed, 28 Apr 2010 12:48:44 -0500
> > From: mich...@cs.wisc.edu
> > To: open-...@googlegroups.com
> > CC: mfor...@gmail.com
> > Subject: Re: information on the config option --
> node.session.iscsi.FastAbort = No
> >
> > On 04/28/2010 12:43 PM, Mike Christie wrote:
> > > On 04/28/2010 10:40 AM, maguar887 wrote:
> > >> We are currently running open iscsi version 2.0-871 on RHEL 5.3
> > >> (2.6.18-92.1.6.0.2.el5) against a Dell Equallogic iScsi SAN group
> > >> (firmware 4.3.5)
> > >>
> > >
> > > You need to upgrade your kernel. It had a bug with eql targets in the
> > > async logout path. It is fixed in 5.4 and 5.5 kernels.
> > >
> >
> > Upgrade your iscsi-initiator-utils too.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
> > To post to this group, send email to open-...@googlegroups.com.
> > To unsubscribe from this group, send email to
> open-iscsi+...@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/open-iscsi?hl=en.
> >
>

> --------------------------------------------------------------------------

>
> The New Busy is not the old busy. Search, chat and e-mail from your inbox.

> [1]Get started.

>
> --
> You received this message because you are subscribed to the Google Groups
> "open-iscsi" group.
> To post to this group, send email to open-...@googlegroups.com.
> To unsubscribe from this group, send email to
> open-iscsi+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/open-iscsi?hl=en.
>

> References
>
> Visible links
> 1. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3