open-iscsi issue

[Farhan Ahmed]

Hi Forum,

I am having very weird open-iscsi issue. Could you guys help me to
find out the reason?

There are 4 network interfaces(eth2/eth3/eth5/eth6) for iSCSI traffic,
eth2/eth5 are patched into one HP switch and eth3/eth6 are patched to
second switch. In normal scenario all seem working fine.
We are testing all scenarios before getting the box into prod. I am
testing scenario in case switch fails but somehow iscsiadm cant
perform the discovery.

All interfaces are on same subnet and eth2 becomes the default for
that subnet.

If I disable eth2 from the HP switch I cant ping the array but I can
ping the array via eth3/eth5/eth6 but I cant perform iscsiadm
discovery through interfaces eth3/eth5/eth6

disable eth2 on switch

[root@nfs02 ~]# ping 192.168.42.190
PING 192.168.42.190 (192.168.42.190) 56(84) bytes of data.
From 192.168.42.70 icmp_seq=2 Destination Host Unreachable
From 192.168.42.70 icmp_seq=3 Destination Host Unreachable
From 192.168.42.70 icmp_seq=4 Destination Host Unreachable

[root@nfs02 ~]# ping -I eth3 192.168.42.190
PING 192.168.42.190 (192.168.42.190) from 192.168.42.71 eth3: 56(84)
bytes of data.
64 bytes from 192.168.42.190: icmp_seq=1 ttl=255 time=0.166 ms
^C
--- 192.168.42.190 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.166/0.166/0.166/0.000 ms
[root@nfs02 ~]# ping -I eth5 192.168.42.190
PING 192.168.42.190 (192.168.42.190) from 192.168.42.72 eth5: 56(84)
bytes of data.
64 bytes from 192.168.42.190: icmp_seq=1 ttl=255 time=0.365 ms
^C
--- 192.168.42.190 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.365/0.365/0.365/0.000 ms
[root@nfs02 ~]# ping -I eth6 192.168.42.190
PING 192.168.42.190 (192.168.42.190) from 192.168.42.73 eth6: 56(84)
bytes of data.
64 bytes from 192.168.42.190: icmp_seq=1 ttl=255 time=1.06 ms
^C
--- 192.168.42.190 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.068/1.068/1.068/0.000 ms

[root@nfs02 ~]# iscsiadm -m discovery -t st -p 192.168.42.190:3260 -I
eth3
iscsiadm: cannot make connection to 192.168.42.190:3260 (113)
iscsiadm: connection to discovery address 192.168.42.190 failed

[root@nfs02 ~]# iscsiadm -m discovery -t st -p 192.168.42.190:3260 -I
eth5
iscsiadm: cannot make connection to 192.168.42.190:3260 (113)
iscsiadm: connection to discovery address 192.168.42.190 failed

[root@nfs02 ~]# iscsiadm -m discovery -t st -p 192.168.42.190:3260 -I
eth6
iscsiadm: cannot make connection to 192.168.42.190:3260 (113)
iscsiadm: connection to discovery address 192.168.42.190 failed


Here is the configuration file

[root@nfs02 ~]# sysctl -a|grep arp_ignore
net.ipv4.conf.all.arp_ignore = 1


[root@nfs02 ~]# sysctl -a|grep arp_anno
net.ipv4.conf.all.arp_announce = 2

[root@nfs02 ~]# sysctl -a|grep net.ipv4.conf.default.rp_filter
net.ipv4.conf.default.rp_filter = 1


[root@nfs02 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref
Use Iface
0.0.0.0 163.189.222.33 0.0.0.0 UG 0 0
0 eth0
163.189.222.0 0.0.0.0 255.255.255.0 U 0 0
0 eth0
192.168.42.0 0.0.0.0 255.255.255.0 U 0 0
0 eth2
192.168.42.0 0.0.0.0 255.255.255.0 U 0 0
0 eth3
192.168.42.0 0.0.0.0 255.255.255.0 U 0 0
0 eth5
192.168.42.0 0.0.0.0 255.255.255.0 U 0 0
0 eth6



root@nfs02 ifaces]# more eth2
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth2
iface.net_ifacename = eth2
iface.hwaddress = 00:21:28:C0:BB:D2
iface.transport_name = tcp
# END RECORD

[root@nfs02 ifaces]# more eth3
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth3
iface.net_ifacename = eth3
iface.hwaddress = 00:21:28:C0:BB:D3
iface.transport_name = tcp
# END RECORD

[root@nfs02 ifaces]# more eth5
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth5
iface.net_ifacename = eth5
iface.hwaddress = 00:1B:21:9D:87:0F
iface.transport_name = tcp
# END RECORD

[root@nfs02 ifaces]# more eth6
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth6
iface.net_ifacename = eth6
iface.hwaddress = 00:1B:21:9D:87:0C
iface.transport_name = tcp
# END RECORD



[root@nfs02 ~]# ifconfig eth2
eth2 Link encap:Ethernet HWaddr 00:21:28:C0:BB:D2
inet addr:192.168.42.70 Bcast:192.168.42.255 Mask:
255.255.255.0

[root@nfs02 ~]# ifconfig eth3
eth3 Link encap:Ethernet HWaddr 00:21:28:C0:BB:D3
inet addr:192.168.42.71 Bcast:192.168.42.255 Mask:
255.255.255.0

[root@nfs02 ~]# ifconfig eth5
eth5 Link encap:Ethernet HWaddr 00:1B:21:9D:87:0F
inet addr:192.168.42.72 Bcast:192.168.42.255 Mask:
255.255.255.0

[root@nfs02 ~]# ifconfig eth6
eth6 Link encap:Ethernet HWaddr 00:1B:21:9D:87:0C
inet addr:192.168.42.73 Bcast:192.168.42.255 Mask:
255.255.255.0





[root@nfs02 ifaces]# uname -r
2.6.39.1

[root@nfs02 ifaces]# cat /etc/redhat-release
CentOS release 5.6 (Final)

[root@nfs02 ifaces]# iscsiadm --version
iscsiadm version 2.0-872

[root@nfs02 ifaces]# modinfo scsi_transport_iscsi
filename: /lib/modules/2.6.39.1/kernel/drivers/scsi/
scsi_transport_iscsi.ko
alias: net-pf-16-proto-8
version: 2.0-870
license: GPL
description: iSCSI Transport Interface
author: Mike Christie <mich...@cs.wisc.edu>, Dmitry Yusupov
<dmitr...@yahoo.com>, Alex Aizman <itn...@yahoo.com>
srcversion: 47617A77D87FB2745BA9CBD
depends: scsi_mod
vermagic: 2.6.39.1 SMP mod_unload modversions
parm: debug_session:Turn on debugging for sessions in
scsi_transport_iscsi module. Set to 1 to turn on, and zero to turn
off. Default is off. (int)
parm: debug_conn:Turn on debugging for connections in
scsi_transport_iscsi module. Set to 1 to turn on, and zero to turn
off. Default is off. (int)



Please guide me what is the issue?

Regards,

Farhan

[Farhan Ahmed]

[Mike Christie]

On 07/21/2011 11:28 PM, Farhan Ahmed wrote:
> Hi Forum,
>
> I am having very weird open-iscsi issue. Could you guys help me to
> find out the reason?
>
> There are 4 network interfaces(eth2/eth3/eth5/eth6) for iSCSI traffic,
> eth2/eth5 are patched into one HP switch and eth3/eth6 are patched to
> second switch. In normal scenario all seem working fine.
> We are testing all scenarios before getting the box into prod. I am
> testing scenario in case switch fails but somehow iscsiadm cant
> perform the discovery.
>
> All interfaces are on same subnet and eth2 becomes the default for
> that subnet.
>
> If I disable eth2 from the HP switch I cant ping the array but I can
> ping the array via eth3/eth5/eth6 but I cant perform iscsiadm
> discovery through interfaces eth3/eth5/eth6

The current version of open-iscsi uses the default routing for discovery
so if you cannot even do a normal ping like below then iscsiadm
discovery is not going to work. The code in git is able to use the iface
binding info to bind the discovery session to a interface.

When you disable eth2 shouldn't the network just figure out that the
other interface eth5 is on the same subnet and drop down to that? Do you
have to update the routing tables by hand then?

[Farhan Ahmed]

Thanks Mike for getting back to me. As I mentioned earlier if eth2 is
down we can still ping the array through other interfaces like

ping -I eth3 192.168.42.190

If I bring down the eth2 via "ifconfig eth2 down" then no issue and
routing table gets updated itself but I am disabling the eth2 by
disabling the port on switch so system routing doesnt change and If I
change the routing metric of eth2 then it uses other interfaces but I
dont think it is appropriate solution. My understanind is discovery
should do the job when we define the iface parmater in the command
line but follwoing command still try to connect array via eth2 which
is the first and default interface to connect array.

iscsiadm -m discovery -t st -p 192.168.42.190:3260 -I eth6

Should I try the git code

Farhan

> > 64 bytes from 192.168.42.190: icmp_seq=1 ttl=255 time=0.166 ms- Hide quoted text -
>
> - Show quoted text -- Hide quoted text -
>
> - Show quoted text -

[Mike Christie]

On 07/23/2011 02:49 AM, Farhan Ahmed wrote:
>
> If I bring down the eth2 via "ifconfig eth2 down" then no issue and
> routing table gets updated itself but I am disabling the eth2 by
> disabling the port on switch so system routing doesnt change and If I
> change the routing metric of eth2 then it uses other interfaces but I
> dont think it is appropriate solution. My understanind is discovery
> should do the job when we define the iface parmater in the command
> line but follwoing command still try to connect array via eth2 which
> is the first and default interface to connect array.
>
> iscsiadm -m discovery -t st -p 192.168.42.190:3260 -I eth6
>
> Should I try the git code
>

Yeah, that should do what you want. Let me know.

[Farhan Ahmed]

Much better I compiled git code and it is much better now even eth2 is
down discovery can run for other interfaces but now I run into
different type of issues and initiator cant login

I am getting different sort of errors

iscsi: invalid session 0.
scsi21 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.
scsi22 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.
scsi23 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.

iscsiadm -m discovery -t sendtargets -p 192.168.42.190 -I eth5

iscsiadm: could not find session info for session8
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session9
192.168.42.190:3260,1 iqn.2001-05.com.equallogic:
0-8a0906-8571ed402-7120012b81b4e002-nfs02
192.168.42.190:3260,1 iqn.2001-05.com.equallogic:0-8a0906-
f0a1ed402-1f40012bba54e28b-nfs02-bond


tail -100 /var/log/messages

Jul 23 20:11:53 nfs02 iscsid: could not read session targetname: 5
Jul 23 20:11:53 nfs02 iscsid: could not find session info for session8
Jul 23 20:11:53 nfs02 iscsid: could not read session targetname: 5
Jul 23 20:11:53 nfs02 iscsid: could not find session info for session9
Jul 23 20:11:53 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 20:11:53 nfs02 iscsid: Can't create connection.
Jul 23 20:11:53 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 20:11:53 nfs02 iscsid: can not safely destroy session 0
Jul 23 20:11:55 nfs02 udevd-event[4734]: wait_for_sysfs: waiting for '/
sys/devices/platform/host24/ioerr_cnt' failed
Jul 23 20:12:07 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 20:12:07 nfs02 iscsid: can't bind conn 0:0 to session 0,
retcode 1 (115)



[root@nfs02 iscsi]# /etc/init.d/open-iscsi stop
Stopping iSCSI initiator service: iscsiadm: could not read session
targetname: 5
iscsiadm: could not find session info for session1
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session10
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session11
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session12
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session13
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session14
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session15
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session16
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session17
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session2
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session3
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session4
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session5
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session6
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session7
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session8
iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session9
iscsiadm: No matching sessions found
Could not logout from all nodes, try again later

[root@nfs02 iscsi]# /etc/init.d/open-iscsi start
Starting iSCSI initiator service: [ OK ]
Setting up iSCSI targets: iscsiadm: No records found
[ OK ]

[root@nfs02 iscsi]# ps ax|grep iscsi
4155 ? S< 0:00 [iscsi_eh]
4223 ? S< 0:00 [iscsi_q_8]
4226 ? S< 0:00 [iscsi_q_9]
4230 ? S< 0:00 [iscsi_q_10]
4233 ? S< 0:00 [iscsi_q_11]
4350 ? S< 0:00 [iscsi_q_12]
4353 ? S< 0:00 [iscsi_q_13]
4357 ? S< 0:00 [iscsi_q_14]
4360 ? S< 0:00 [iscsi_q_15]
4470 ? S< 0:00 [iscsi_q_16]
4473 ? S< 0:00 [iscsi_q_17]
4477 ? S< 0:00 [iscsi_q_18]
4480 ? S< 0:00 [iscsi_q_19]
4563 ? S< 0:00 [iscsi_q_20]
4566 ? S< 0:00 [iscsi_q_21]
4570 ? S< 0:00 [iscsi_q_22]
4573 ? S< 0:00 [iscsi_q_23]
4660 ? Ss 0:00 /sbin/iscsid
4661 ? R<Ls 1:17 /sbin/iscsid
4732 ? S< 0:00 [iscsi_q_24]


Thanks,

Farhan

[Mike Christie]

On 07/23/2011 05:14 AM, Farhan Ahmed wrote:
> Much better I compiled git code and it is much better now even eth2 is
> down discovery can run for other interfaces but now I run into
> different type of issues and initiator cant login
>
> I am getting different sort of errors
>

Looks like sysfs compat issues.

Where did you get the tools you were using before? From your distro?

Where did you get your kernel from?

Could you send your kernel's .config so I can see your sysfs settings?

[Farhan Ahmed]

Yes rpm came with CentOS distro

[root@nfs02 ~]# rpm -qa|grep iscsi
iscsi-initiator-utils-6.2.0.872-6.el5


I removed rpm and compile git code

I had to compile the latest kernel , default kernel having issues with
USB devices and haging at start up

[root@nfs02 ~]# uname -r
2.6.39.1

I got the latest kernel from http://www.kernel.org

Thanks a lot mate for your help

I cant find a way to attach file , I am sending attacment directly to
your inbox

> >> Yeah, that should do what you want. Let me know.- Hide quoted text -

[Farhan Ahmed]

Here is the kernel .config file attached

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To post to this group, send email to open-...@googlegroups.com.
To unsubscribe from this group, send email to open-iscsi+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.

[Farhan Ahmed]

Not sure but I cant see any more wait_for_sysfs error by fixing the
following line

[root@nfs02 ~]# more /etc/udev/rules.d/05-udev-early.rules
ACTION=="add", SUBSYSTEM=="scsi", KERNEL=="[0-9]*:[0-9]*",
WAIT_FOR_SYSFS="ioerr_cnt"

But still iscsiadm cant perform discovery and giving error

Jul 23 22:37:21 nfs02 kernel: scsi8 : iSCSI Initiator over TCP/IP
Jul 23 22:37:21 nfs02 kernel: iscsi: invalid session 0.
Jul 23 22:37:21 nfs02 kernel: scsi9 : iSCSI Initiator over TCP/IP
Jul 23 22:37:21 nfs02 kernel: iscsi: invalid session 0.
Jul 23 22:37:21 nfs02 kernel: scsi10 : iSCSI Initiator over TCP/IP
Jul 23 22:37:21 nfs02 kernel: iscsi: invalid session 0.
Jul 23 22:37:21 nfs02 kernel: scsi11 : iSCSI Initiator over TCP/IP
Jul 23 22:37:21 nfs02 kernel: iscsi: invalid session 0.
Jul 23 22:37:21 nfs02 iscsid: Could not set session0 priority. READ/
WRITE throughout and latency could be affected.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: Can't create connection.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: can not safely destroy session 0
Jul 23 22:37:21 nfs02 iscsid: Could not set session0 priority. READ/
WRITE throughout and latency could be affected.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: Can't create connection.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: can not safely destroy session 0
Jul 23 22:37:21 nfs02 iscsid: Could not set session0 priority. READ/
WRITE throughout and latency could be affected.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: Can't create connection.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: can not safely destroy session 0
Jul 23 22:37:21 nfs02 iscsid: Could not set session0 priority. READ/
WRITE throughout and latency could be affected.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: Can't create connection.
Jul 23 22:37:21 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:21 nfs02 iscsid: can not safely destroy session 0
Jul 23 22:37:37 nfs02 iscsid: Received iferror -22: Invalid argument.
Jul 23 22:37:37 nfs02 iscsid: can't bind conn 0:0 to session 0,
retcode 1 (115)


dmesg

iscsi: registered transport (tcp)
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
warning: `dbus-daemon' uses 32-bit capabilities (legacy support in
use)
scsi8 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.
scsi9 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.
scsi10 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.
scsi11 : iSCSI Initiator over TCP/IP
iscsi: invalid session 0.

On Jul 23, 9:00 pm, Farhan Ahmed <ahmed.far...@gmail.com> wrote:
> Here is the kernel .config file attached
>

>  config
> 95KViewDownload

>
> On Sat, Jul 23, 2011 at 8:54 PM, Farhan Ahmed <ahmed.far...@gmail.com>wrote:
>
>
>
> > Yes rpm came with CentOS distro
>
> > [root@nfs02 ~]# rpm -qa|grep iscsi
> > iscsi-initiator-utils-6.2.0.872-6.el5
>
> > I removed rpm and compile git code
>
> > I had to compile the latest kernel , default kernel having issues with
> > USB devices and haging at start up
>
> > [root@nfs02 ~]# uname -r
> > 2.6.39.1
>

> > I got the latest kernel fromhttp://www.kernel.org

> >http://groups.google.com/group/open-iscsi?hl=en.- Hide quoted text -

[Mike Christie]

On 07/23/2011 05:54 AM, Farhan Ahmed wrote:
> Yes rpm came with CentOS distro
>

Is this centos 5 or 6? If it is 6 could you change

CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y

to

# CONFIG_SYSFS_DEPRECATED is not set

Also are you using 32 bit userspace for 64 bit? Make sure you compile as
64 bits like your kernel.

[Farhan Ahmed]

It is centos 5.6. I tried to compile kernel without DEPRECATED option
but kernel didnt boot and threw errors so I had to compile it again
with DEPRECATED support, I cant udnerstand why SYSFS had no issue with
the default version of open-iscsi. We can upgrade CENTOS from 5.6 to
6.0. May I ask when are you releasing new version of open-scsi that
fix this routing issue?

[Mike Christie]

On 07/24/2011 12:02 AM, Farhan Ahmed wrote:
> It is centos 5.6. I tried to compile kernel without DEPRECATED option
> but kernel didnt boot and threw errors so I had to compile it again
> with DEPRECATED support, I cant udnerstand why SYSFS had no issue with
> the default version of open-iscsi. We can upgrade CENTOS from 5.6 to
> 6.0. May I ask when are you releasing new version of open-scsi that
> fix this routing issue?
>

About Aug 15th.

[Farhan Ahmed]

Hi Mike,

I installed CentOS 6.0 and git code worked well. I can perform the
discovery now but iscsiadm cant login

eth2 is down from the Switch

[root@nfs02 ~]# ping 192.168.42.190
PING 192.168.42.190 (192.168.42.190) 56(84) bytes of data.

^C
--- 192.168.42.190 ping statistics ---

3 packets transmitted, 0 received, 100% packet loss, time 2930ms

[root@nfs02 ~]# ping -I eth3 192.168.42.190
PING 192.168.42.190 (192.168.42.190) from 192.168.42.71 eth3: 56(84)
bytes of data.

64 bytes from 192.168.42.190: icmp_seq=1 ttl=255 time=0.108 ms
64 bytes from 192.168.42.190: icmp_seq=2 ttl=255 time=0.107 ms

^C
--- 192.168.42.190 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1311ms
rtt min/avg/max/mdev = 0.107/0.107/0.108/0.010 ms

[root@nfs02 ~]# iscsiadm -m discovery -p 192.168.42.190 -t st -I eth3

192.168.42.190:3260,1 iqn.2001-05.com.equallogic:0-8a0906-
f0a1ed402-1f40012bba54e28b-nfs02-bond

[root@nfs02 ~]# iscsiadm -m node -T iqn.2001-05.com.equallogic:
0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond -I eth3 -l
Logging in to [iface: eth3, target: iqn.2001-05.com.equallogic:
0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond, portal:
192.168.42.190,3260] (multiple)
iscsiadm: Could not login to [iface: eth3, target: iqn.
2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond,
portal: 192.168.42.190,3260].
iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not log into all portals


Could you please advise? Secondly if we reboot our box while "eth2 is
down from switch" still iscsiadm cant login

Regards,

Farhan

[Farhan Ahmed]

Hi Mike,

Have you got chance to look into this issue?

Farhan

[Mike Christie]

On 07/27/2011 05:18 PM, Farhan Ahmed wrote:
> Hi Mike,
>
> Have you got chance to look into this issue?
>

I have not. You tried changing the rp_filter values right?

Run iscsid by hand in debug mode

/etc/init.d/iscsid stop
iscsid -d 8 -f all

iscsiadm -m node ..... -l

Send iscsid output.

Also send /var/log/messages output. Also run tcpdump to check that the
tcp ip SYN is going through the right interface.

> Farhan
>
> On Jul 26, 2011 12:54 PM, "Farhan Ahmed" <ahmed....@gmail.com

> <mailto:ahmed....@gmail.com>> wrote:
>> Hi Mike,
>>
>> I installed CentOS 6.0 and git code worked well. I can perform the
>> discovery now but iscsiadm cant login
>>
>> eth2 is down from the Switch
>>
>> [root@nfs02 ~]# ping 192.168.42.190
>> PING 192.168.42.190 (192.168.42.190) 56(84) bytes of data.
>> ^C
>> --- 192.168.42.190 ping statistics ---
>> 3 packets transmitted, 0 received, 100% packet loss, time 2930ms
>>
>> [root@nfs02 ~]# ping -I eth3 192.168.42.190
>> PING 192.168.42.190 (192.168.42.190) from 192.168.42.71 eth3: 56(84)
>> bytes of data.

>> 64 bytes from 192.168.42.190 <http://192.168.42.190>: icmp_seq=1
> ttl=255 time=0.108 ms
>> 64 bytes from 192.168.42.190 <http://192.168.42.190>: icmp_seq=2

> ttl=255 time=0.107 ms
>> ^C
>> --- 192.168.42.190 ping statistics ---
>> 2 packets transmitted, 2 received, 0% packet loss, time 1311ms
>> rtt min/avg/max/mdev = 0.107/0.107/0.108/0.010 ms
>>
>> [root@nfs02 ~]# iscsiadm -m discovery -p 192.168.42.190 -t st -I eth3

>> 192.168.42.190:3260 <http://192.168.42.190:3260>,1

> iqn.2001-05.com.equallogic:0-8a0906-
>> f0a1ed402-1f40012bba54e28b-nfs02-bond
>>
>>
>> [root@nfs02 ~]# iscsiadm -m node -T iqn.2001-05.com.equallogic:
>> 0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond -I eth3 -l
>> Logging in to [iface: eth3, target: iqn.2001-05.com.equallogic:
>> 0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond, portal:
>> 192.168.42.190,3260] (multiple)
>> iscsiadm: Could not login to [iface: eth3, target: iqn.
>> 2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond,
>> portal: 192.168.42.190,3260].
>> iscsiadm: initiator reported error (8 - connection timed out)
>> iscsiadm: Could not log into all portals
>>
>>
>> Could you please advise? Secondly if we reboot our box while "eth2 is
>> down from switch" still iscsiadm cant login
>>
>> Regards,
>>
>> Farhan
>>
>>
>>
>>
>>
>> On Jul 24, 6:20 pm, Mike Christie <micha...@cs.wisc.edu

> <mailto:micha...@cs.wisc.edu>> wrote:
>>> On 07/24/2011 12:02 AM, Farhan Ahmed wrote:
>>>
>>> > It is centos 5.6. I tried to compile kernel without DEPRECATED option
>>> > but kernel didnt boot and threw errors so I had to compile it again
>>> > with DEPRECATED support, I cant udnerstand why SYSFS had no issue with
>>> > the default version of open-iscsi. We can upgrade CENTOS from 5.6 to
>>> > 6.0. May I ask when are you releasing new version of open-scsi that
>>> > fix this routing issue?
>>>
>>> About Aug 15th.
>>
>> --
>> You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
>> To post to this group, send email to open-...@googlegroups.com

> <mailto:open-...@googlegroups.com>.

>> To unsubscribe from this group, send email to
> open-iscsi+...@googlegroups.com

> <mailto:open-iscsi%2Bunsu...@googlegroups.com>.

[Farhan Ahmed]

Thanks Mike, Yes I have set right kernel values of rp_filter, please find the attached file which includes debug output,tcpdump and command output. Discovery works fine for eth3 but iscsiadm cant login though. I can see the packets are coming in/out at right interface eth3

 

Regards,

 

Farhan

[Mike Christie]

On 07/27/2011 06:25 PM, Farhan Ahmed wrote:
> Thanks Mike, Yes I have set right kernel values of rp_filter, please
> find the attached file which includes debug output,tcpdump and command
> output. Discovery works fine for eth3 but iscsiadm cant login though. I
> can see the packets are coming in/out at right interface eth3
>

Looks like iscsid/iscsiadm did not try to bind the session so that is
why it failed.

Could you do

iscsiadm -m node -P 1

and send the output?

and then do

iscsiadm -m node -I eth3

and send the output?

Will try here too in a minute.

[Mike Christie]

On 07/27/2011 06:25 PM, Farhan Ahmed wrote:

> Thanks Mike, Yes I have set right kernel values of rp_filter, please find
> the attached file which includes debug output,tcpdump and command output.
> Discovery works fine for eth3 but iscsiadm cant login though. I can see the
> packets are coming in/out at right interface eth3
>

Are you seeing packets going through eth3 when you run the iscsiadm
login command or discovery?

I tested this here and confirmed it at least works for me. In the iscsid
-d 8 -f & log we want to see a line like:

iscsid: Binding session -1 to eth5

that which indicates we are doing binding instead of just using the
default routing.

(my iscsi iface is for eth5 you should see eth3). In the iscsiadm.txt
outpout I did not see that line so for some reason the normal session is
not getting bound to the proper iface.

So send the info I was asking about in the other mail and could you also
send the output of

iscsiadm -m iface -I eth3

For discovery if you did

iscsiadm -m discovery -t st -p ip -I eth3 -d 8

then you would see a similar "Binding session" line indicating we are
binding the session to a iface.

[Farhan Ahmed]

Here is the output

 

Hi Mike,

 

Here is the output

 

[root@nfs02 ~]# iscsiadm -m node -P 1
Target: iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond
        Portal: 192.168.42.190:3260,1
                Iface Name: eth3

[root@nfs02 ~]# iscsiadm -m node -l eth3
Logging in to [iface: eth3, target: iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond, portal: 1    92.168.42.190,3260] (multiple)
iscsiadm: Could not login to [iface: eth3, target: iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-    bond, portal: 192.168.42.190,3260].

iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not log into all portals

[Mike Christie]

On 07/28/2011 05:48 PM, Farhan Ahmed wrote:
> Here is the output
>
> Hi Mike,
>
> Here is the output
>
> [root@nfs02 ~]# iscsiadm -m node -P 1
> Target:
> iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond
> Portal: 192.168.42.190:3260,1
> Iface Name: eth3
>
> [root@nfs02 ~]# iscsiadm -m node -l eth3

Don't pass the login/-l parameter. Just do

iscsiadm -m node -I eth3

And send the output.

[Farhan Ahmed]

Hi Mike,

 

I issued command iscsiadm -m discovery -t st  -p ip -I eth3 -d 8

and I am getting this line

 

 

iscsiadm: Could not match iface[hw=00:21:28:C0:BB:D3,ip=,net_if=eth3,iscsi_if=eth3] to host.

 

Is it safe to ignore?

 

Farhan

[Farhan Ahmed]

Here is the output of iscsiadm -m node -I eth3

[root@nfs02 ~]# iscsiadm -m node -I eth3
# BEGIN RECORD 2.0-872
node.name = iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond
node.tpgt = 1
node.startup = automatic
node.leading_login = No

iface.hwaddress = 00:21:28:C0:BB:D3

iface.ipaddress = <empty>

iface.iscsi_ifacename = eth3
iface.net_ifacename = eth3

iface.transport_name = tcp
iface.initiatorname = <empty>
node.discovery_address = 192.168.42.190
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 2
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.nr_sessions = 1
node.session.auth.authmethod = None
node.session.auth.username = <empty>
node.session.auth.password = <empty>
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 0
node.session.err_timeo.abort_timeout = 10
node.session.err_timeo.lu_reset_timeout = 1
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = No
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 192.168.42.190
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 1
node.conn[0].timeo.login_timeout = 1
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 3
node.conn[0].timeo.noop_out_timeout = 2
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.DataDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD

[Mike Christie]

On 07/28/2011 05:56 PM, Farhan Ahmed wrote:
> Hi Mike,
>
> I issued command iscsiadm -m discovery -t st -p ip -I eth3 -d 8
> and I am getting this line
>
>
> iscsiadm: Could not match
> iface[hw=00:21:28:C0:BB:D3,ip=,net_if=eth3,iscsi_if=eth3] to host.
>
> Is it safe to ignore?

Did you pass "ip" or did you pass your discovery address
192.168.whatever.it.was?

[Farhan Ahmed]

No I passed my SAN discovery address

 

iscsiadm -m discovery -t st -p 192.168.42.190 -l eth3 -d 8

[Mike Christie]

On 07/28/2011 06:02 PM, Farhan Ahmed wrote:
> No I passed my SAN discovery address
>
> iscsiadm -m discovery -t st -p 192.168.42.190 -l eth3 -d 8
>

What is the command you have been running to do discovery? Just run that
same command but also pass it the -d 8 argument to get some debugging.

[Mike Christie]

On 07/28/2011 06:01 PM, Farhan Ahmed wrote:
> Here is the output of iscsiadm -m node -I eth3
>
> [root@nfs02 ~]# iscsiadm -m node -I eth3
> # BEGIN RECORD 2.0-872
> node.name =
> iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond
> node.tpgt = 1
> node.startup = automatic
> node.leading_login = No
> iface.hwaddress = 00:21:28:C0:BB:D3
> iface.ipaddress = <empty>
> iface.iscsi_ifacename = eth3
> iface.net_ifacename = eth3
> iface.transport_name = tcp

Could you run with the attached patch? Run iscsid as iscsid -d 8 -f &
again and send all the output when you run the login command like you
did before.

Also try just setting either iface.net_ifacename or iface.hwaddress. Do
not set both at the same time.

[Farhan Ahmed]

Thanks Mike for your help. Here is the attached debug without following patch. I will send another debug after the patch installed

I changed the iface file to remove the mac address now it looks like

[root@nfs02 ~]# more /etc/iscsi/ifaces/eth3
# BEGIN RECORD 2.0-872

iface.net_ifacename = eth3
iface.transport_name = tcp

# END RECORD

But iscsiadm is not trying to discover via eth3 and it is giving the following message you will find in the debug

Getting priority for eth2 returned -5

[Farhan Ahmed]

Hi Mike,

I applied the patch and here is the debug of all the commands.

Thanks a lot mate for your help.

Farhan

[Mike Christie]

On 07/28/2011 06:49 PM, Farhan Ahmed wrote:
> Thanks Mike for your help. Here is the attached debug without following
> patch. I will send another debug after the patch installed
>
> I changed the iface file to remove the mac address now it looks like
>
> [root@nfs02 ~]# more /etc/iscsi/ifaces/eth3
> # BEGIN RECORD 2.0-872
> iface.net_ifacename = eth3
> iface.transport_name = tcp
> # END RECORD
>
> But iscsiadm is not trying to discover via eth3 and it is giving the
> following message you will find in the debug
>
> Getting priority for eth2 returned -5
>

That error can be ignored. It is just debugging info.

In the log it shows that bounded to eth3 and eth6 ok and disocvered ok.

The command you are running is wrong

iscsiadm -m discovery -t st -p 192.168.42.190 -l eth3 -d 8

eth3 is just ignored here and we try to use all interfaces. You have to
pass in the iface with -I

iscsiadm -m discovery -t st -p 192.168.42.190 -I eth3 -d 8

Also just forget -l in the discovery command for now.

[Mike Christie]

On 07/29/2011 01:14 AM, Mike Christie wrote:
> eth3 is just ignored here and we try to use all interfaces. You have to
> pass in the iface with -I
>
> iscsiadm -m discovery -t st -p 192.168.42.190 -I eth3 -d 8
>
>

Ignore this. I see you figured it out in the other mail.

[Farhan Ahmed]

Thanks Please let me know if you need me to run any other command

[Mike Christie]

On 07/28/2011 07:07 PM, Farhan Ahmed wrote:
> Hi Mike,
>
> I applied the patch and here is the debug of all the commands.
>

Seems like it all worked there, right?

[Farhan Ahmed]

I wish it could work well but not , still iscsiadm cant login to initiator and it is still throwing connection time out

[Mike Christie]

On 07/29/2011 01:19 AM, Farhan Ahmed wrote:
> I wish it could work well but not , still iscsiadm cant login to
> initiator and it is still throwing connection time out
>

Could you take a tcpdump/wireshark trace?

Do not worry about discovery. Just do the normal session login.

I see us bind ok. I see us create a tcp socket ok. I see us send the
login command ok. But I do not see a response back from the target.

[Farhan Ahmed]

Hi Mike,

 

Did you get chance to look into this tcpdump outputs?

 

Farhan

On Fri, Jul 29, 2011 at 4:36 PM, Farhan Ahmed <ahmed....@gmail.com> wrote:

I ran three commands and their tpcdump output is attached

1) iscsiadm -m discovery -t st -p 192.168.42.190 -I eth3 -d 8

2) iscsiadm -m discovery -t st -p 192.168.42.190 -I eth3 -d 8   // -l login

3) iscsiadm -m node -T iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond -I eth3 -l

[Farhan Ahmed]

Thanks Mike

Here is the file attached

I noticed one more thing , If i disable eth2 I can ping the SAN array from Linux via eth3/eth5/eth6 interfaces but I cant ping any server eth3/eth5/eth6 interface from the equallogic array.

Regards,

Farhan

On Tue, Aug 2, 2011 at 1:52 PM, Mike Christie <mich...@cs.wisc.edu> wrote:

On 07/29/2011 01:36 AM, Farhan Ahmed wrote:
> I ran three commands and their tpcdump output is attached
>
> 1) iscsiadm -m discovery -t st -p 192.168.42.190 -I eth3 -d 8
>
> 2) iscsiadm -m discovery -t st -p 192.168.42.190 -I eth3 -d 8   // -l login
>

> 3) iscsiadm -m node -T

> iqn.2001-05.com.equallogic:0-8a0906-f0a1ed402-1f40012bba54e28b-nfs02-bond -I
> eth3 -l
>

You have to run tcpdump as

tcpdump -i eth3 -w mytcpdumpfile

then send me mytcpdumpfile

[Mike Christie]

Hi Shyam and Jim,

Do you know the answer to or can you redirect me to someone that can
answer the following:

The user below, Farhan Ahmed, is doing the following with a Equallogic
target.

On the initiator box he has multiple network interfaces on the same
subnet. He then has a iscsi iface for each one that is bound to the
iscsi target. So he has run

iscsiadm -m discovery -t st -p ip -I his_iface1

If he disables something in the network that affects the default route
then he can do:

ping -I net_interface $targets_ip

but when we try to login a normal session it will fail. See packet #97.

In this test he does not run something like route on the initiator box
to update the default route. Instead he is relying on the iscsi code's
interface binding to force the use of the interface.

We can see the iscsi login pdu get sent, but we do not see a iscsi
response. From what I can tell we just seem to see a tcp ack. Then later
in packet #104 we see the initiator give up on waiting for a login
response and disconnect the tcp/ip connection.

However, the weird thing is that before we try the normal session login
we see in packet #10 that a discovery session works ok in this test.

So command #1 works but command #3 does not and between that time he has
not done any changes to the iscsi config or network.

Is what Farhan is trying to do supported by EQL targets?

And is there a way to if the target is getting the iscsi login pdu
correctly?

[Jim_R...@dell.com]

Mike Christie wrote:
> We can see the iscsi login pdu get sent, but we do not see a iscsi response.
> From what I can tell we just seem to see a tcp ack. Then later in packet #104
> we see the initiator give up on waiting for a login response and disconnect
> the tcp/ip connection.
>
> However, the weird thing is that before we try the normal session login we
> see in packet #10 that a discovery session works ok in this test.
>
> So command #1 works but command #3 does not and between that time he
> has not done any changes to the iscsi config or network.

I noticed that after sending the login request (Packet #97) the initiator is closing the connection (Packet #104) after 1 second, which is quite soon - I would suggest adjusting the node.conn[0].timeo.login_timeout value back up to the default of 15 seconds and trying again.

For more help, I'd recommend contacting the Dell EqualLogic support organization. They're quite good, and have a lot of experience with Linux (including this tricky multiple-NICS-on-the-same-subnet setup):
http://support.dell.com/equallogic/

--
Jim Ramsay

[Farhan Ahmed]

Thanks Jim and Mike I increased time out from 1 second to 15 second
and finally array accepted the login request and initiator login. When
I tried to login from eth5 interface then it times out , anyway I have
increased timeout from 15 second to 45 second and now all good. Thanks
a lot. Yes I had already logged case with Equallogic and they are
still looking into it.

Last suggestion is should we use multipathing over bonded interface or
multipathing over simple ethernet interfaces? multipathiong over 2
bonded interface we are getting same throughput as we are getting
multipathing over 4 individual interfaces and array has 3 ports so my
understanding was 3 network interfaces will have more throughput as
compared to two bonded interface. But bonded interface are good in
regards of swicth failover or nic module failure , same scnerio which
I was tryin to test. Even the primary eth2 is down bonded interfcae
will have no impact.

What do you guys think about multipath over bonded interface or
multipathing over interfaces.

Thanks a lot for for your help guys.

Farhan

[Farhan Ahmed]

Hi Mike,

You mentioned new version is coming on 15th of this month. But it is
not released yet. When do you have plan to release?

Farhan

> > Jim Ramsay- Hide quoted text -
>
> - Show quoted text -

[Mike Christie]

On 08/15/2011 05:28 PM, Farhan Ahmed wrote:
> Hi Mike,
>
> You mentioned new version is coming on 15th of this month. But it is
> not released yet. When do you have plan to release?
>

You can grab my test version here if you want the rpm:
http://people.redhat.com/mchristi/iscsi/rhel6.2/iscsi-initiator-utils/
That is just a test version. It is not for production yet. It has all
the features I intended to add. It just needs more testing and some
smaller changes.

[BriK]

Hi !

My first post ever on this group but I ran into the same problem lately. Your problem is a routing problem I'd bet:

[root@nfs02 ~]# route -n 
Kernel IP routing table 
Destination     Gateway         Genmask         Flags Metric Ref 
Use Iface 
0.0.0.0         163.189.222.33  0.0.0.0         UG    0      0 
0 eth0 
163.189.222.0   0.0.0.0         255.255.255.0   U     0      0 
0 eth0 
192.168.42.0    0.0.0.0         255.255.255.0   U     0      0 
0 eth2 
192.168.42.0    0.0.0.0         255.255.255.0   U     0      0 
0 eth3 
192.168.42.0    0.0.0.0         255.255.255.0   U     0      0 
0 eth5 
192.168.42.0    0.0.0.0         255.255.255.0   U     0      0 
0 eth6 

You have multiple ethernet card on the same subnet plugged 2 by  2 on two switchs : You NEED multiple routing tables (one for each card in fact).

The way to do this is to use iproute2 to create additionnal routing tables : the route -n command show only the "main" table. You can add a rule to inspect a specific table for each of your interfaces when they are working.

How to do this :

Create the additionnal ip routing tables :

# Adding the route to table 20 (you can use a number up to 250 I think, I took 20 because of eth2)

ip route add 192.168.42.0/24 dev eth2 proto kernel scope link src 192.168.42.70 table 20

# Add the rule saying to use this table for packets émited from 192.168.42.70

ip rule add from 192.168.42.70 lookup 20

# If you want to remove the "wrong" route from the main table

ip route del 192.168.42.0/24 dev eth2 table main

# Refresh the route cache

ip route flush cache

You can check everything is OK with "ip rule ls", should display :

0: from all lookup local 

xxxxx: from 192.168.42.70 lookup 20 

xxxxx: from all lookup main 

xxxxx: from all lookup default 

You can check you table 20 with "ip route show table 20", should display :

192.168.42/24 dev eth2  proto kernel  scope link  src 192.168.42.70 

and do this for each of your interface.

To keep the routes between reboots/ifup/ifdown add a "POST_UP_SCRIPT" option to your ifcfg script (check man ifcfg).

I Hope this will help ;)

Regards,

BriK

[Farhan Ahmed]

Hi Mike,

Now I got run into other problem. I have installed the latest rpm from
the following URL and it seems it is having issues to add sessions
after 15, I can login to the targets till session 15 and after that I
am getting these errors. Any clue?, I have 4 network interfaces so If
I login to target it creates 4 session so I can't login to more than 4
targets as it is having issues with session 16.

[root@nfs01 ~]# iscsiadm -m node -T iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www -l -v automatic

iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session16

Logging in to [iface: eth2, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] (multiple)

iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session16

Logging in to [iface: eth3, target: iqn.2001-05.com.equallogic:

0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] (multiple)

iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session16

Logging in to [iface: eth5, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] (multiple)

iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session16

Logging in to [iface: eth6, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] (multiple)
Login to [iface: eth2, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] successful.
Login to [iface: eth3, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] successful.
Login to [iface: eth5, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] successful.
Login to [iface: eth6, target: iqn.2001-05.com.equallogic:
0-8a0906-899016d02-dbc0016ec1e4e657-nfs01-www, portal:
192.168.42.190,3260] successful.

On Aug 17, 2:51 pm, Mike Christie <micha...@cs.wisc.edu> wrote:

[Farhan Ahmed]

Hi Mike,

I tried to login target/portal one by one and it hanged when I was logging to number 16

[root@nfs01 ~]# iscsiadm -m node -T iqn.2001-05.com.equallogic:0-8a0906-7531ed402-a680012bfde4e55e-nfs01-users -I eth6 -p 192.168.42.190 -l

iscsiadm: could not read session targetname: 5
iscsiadm: could not find session info for session16

Sorry Here are the /var/log/messages

Sep  7 10:33:56 nfs01 iscsid: Received iferror -22: Invalid argument.
Sep  7 10:33:56 nfs01 iscsid: Received iferror -22: Invalid argument.
Sep  7 10:34:37 nfs01 kernel: connection17:0: detected conn error (1020)
Sep  7 10:35:27 nfs01 iscsid: Received iferror -22: Invalid argument.


Any idea?

On Wed, Sep 7, 2011 at 3:59 AM, Mike Christie <mich...@cs.wisc.edu> wrote:

On 09/06/2011 12:43 PM, Mike Christie wrote:

> On 09/06/2011 04:54 AM, Farhan Ahmed wrote:
>> Hi Mike,
>>

>> it is driving me nuts :) , I have installed the latest rpm from the

>> following URL and it seems it is having issues to add sessions after 15,

>> I can login to the targets till session and after that I am getting

>> these errors. Any clue?, I have 4 network interfaces so If I login to

>> target it creates 4 session so I can login to more than 4 targets as it

>> is having issues with session 16.
>>
>

> Looks like a regression added by dell when they added some leading login
> feature. There is basically a race going on during async login.
>
> If you login to each target/portal one by one does it work? So do
>
> iscsiadm -m node -T target -I iface -p ip -l
>
> for each target/portal combo.
>

Actually ignore this. I just tried it here. When you run the login
command in /var/log/messages do you see something like:

Sep  6 12:50:55 meanminna iscsid: Received iferror -22: Invalid argument.
Sep  6 12:50:55 meanminna iscsid: can't bind conn 16:0 to session 16,
retcode 1 (2)

?

In the future include that info. It will help me out :)

[Mike Christie]

On 09/06/2011 07:41 PM, Farhan Ahmed wrote:
> Hi Mike,
>

> I tried to login target/portal one by one and it hanged when I was logging
> to number 16
>
> [root@nfs01 ~]# iscsiadm -m node -T
> iqn.2001-05.com.equallogic:0-8a0906-7531ed402-a680012bfde4e55e-nfs01-users
> -I eth6 -p 192.168.42.190 -l
> iscsiadm: could not read session targetname: 5
> iscsiadm: could not find session info for session16
>
> Sorry Here are the /var/log/messages
>
> Sep 7 10:33:56 nfs01 iscsid: Received iferror -22: Invalid argument.
> Sep 7 10:33:56 nfs01 iscsid: Received iferror -22: Invalid argument.
> Sep 7 10:34:37 nfs01 kernel: connection17:0: detected conn error (1020)
> Sep 7 10:35:27 nfs01 iscsid: Received iferror -22: Invalid argument.
>
>
> Any idea?
>

I can replicate it here. It is caused by something that got brought in
with the qla4xxx management patches. I am working on it now.

It looks like userspace is sending down the correct info, but somewhere
between iscsid sending and scsi_transport_iscsi reading the data gets
corrupted and we hit this.

[Farhan Ahmed]

Thanks Mike, I hope it will fix soon, when can we expect the new stable release?

[Mike Christie]

On 09/06/2011 10:38 PM, Farhan Ahmed wrote:
> Thanks Mike, I hope it will fix soon, when can we expect the new stable
> release?
>

Found the problem. The new iscsi tools are sending the netlink request
in a way the iscsi class does not like. I should have a fix around Tues
(I am leaving on vacation after I send this).

A new upstream stable release should be done in about a month.

[Farhan Ahmed]

Thanks Mike, could you please let me know when it is fixed and new package released

[Farhan Ahmed]

Hi Mike,

Has the bug fixed yet? I am hoping You are going to release package today :) before you going to vacation.

Farhan

[Mike Christie]

On 09/13/2011 01:27 AM, Farhan Ahmed wrote:
> Hi Mike,
>
> Has the bug fixed yet? I am hoping You are going to release package today :)
> before you going to vacation.

I think I have a fix. Waiting to hear back from qlogic about their testing.