Re: iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3)
3,717 views
Skip to first unread message
parveen kumar
Aug 10, 2012, 5:34:32 AM8/10/12
to Michael Christie, open-...@googlegroups.com, Rajat Gupta
oki Mike.
--
On Thu, Aug 9, 2012 at 10:54 PM, Michael Christie <mich...@cs.wisc.edu> wrote:
In the future can you post these questions to the open-iscsi mailing list? This is the last time I will respond privately. It is best ti post on the list so others can see the answer (people searching in google for similar problems in the future).
It changes name because you probably still have some app/user with a reference to the device. When yourelogin the scsi layer will then give you a different name.
You should not be using the sdX types of names, because they are not persistent. You should use the udev names in /dev/disk/.
On Aug 9, 2012, at 3:17 AM, parveen kumar <coolbu...@gmail.com> wrote:
> Hi Mike Christie,
>
> ok , but what about /dev/sdb, first time its showing me /dev/sda then connection bracked and i had tried again and next its showing /dev/sdb
> means if network disruption again i have to edit /etc/fstab ?
>
> Regards,
> PARVEEN
>
> On Thu, Aug 9, 2012 at 11:26 AM, Michael Christie <mich...@cs.wisc.edu> wrote:
> There was a probably a network disruption that caused the initiator to lose the connection
>
> On Aug 9, 2012, at 12:11 AM, parveen kumar <coolbu...@gmail.com> wrote:
>
> > Hello Mike Christie,
> >
> > Few days back i mounted 2TB volume through ISCSI from SAN box in my server. After mounting the 2TB space i am bale to see partition /dev/sda of 2TB on my server.
> > But after 5-6 days m not able to write data on /dev/sda, then i see conncetion is lost and iscsi connection is braked, so i tried again to login to target with this command:
> > # iscsiadm -m node -T target_iqn_name -p ipaddress -l
> > after this, login is sucessfull, then i put command:
> > # fdisk -l
> > its showing me /dev/sdb of 2TB size
> > i m not able to understand what happend, first time its showing me /dev/sda then connection bracked and i had tried again and next its showing /dev/sdb
> > please help me to sort out this problem......
> >
> >
> > --
> > .
> >
>
>
>
>
> --
> .
>
.
Mike Christie
Aug 10, 2012, 1:33:41 PM8/10/12
to open-...@googlegroups.com, parveen kumar, Rajat Gupta
On 08/10/2012 04:34 AM, parveen kumar wrote:
> oki Mike.
>
Thanks.
One thing that is not clear to me, is why you have to do the logout then
relogin? Are you doing this because you have a app or FS using the iscsi
disk, and it gets IO errors when you get the conn errors?
Or, are you doing the relogin because the /dev/sdX is automatically
removed when you see the conn errors messages? If this is happening what
version of open-iscsi are you using and where did you get it
(open-iscsi.org or some distro)?
In /var/log/messages do you see
session recovery timed out after 120 secs
(the value for the secs may be different for you if you modified it in
iscsid.conf or with iscsiadm).
> On Thu, Aug 9, 2012 at 10:54 PM, Michael Christie <mich...@cs.wisc.edu
> <mailto:mich...@cs.wisc.edu>> wrote:
>
> In the future can you post these questions to the open-iscsi mailing
> list? This is the last time I will respond privately. It is best ti
> post on the list so others can see the answer (people searching in
> google for similar problems in the future).
>
> It changes name because you probably still have some app/user with a
> reference to the device. When yourelogin the scsi layer will then
> give you a different name.
>
> You should not be using the sdX types of names, because they are not
> persistent. You should use the udev names in /dev/disk/.
>
> On Aug 9, 2012, at 3:17 AM, parveen kumar <coolbu...@gmail.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
> To post to this group, send email to open-...@googlegroups.com.
> To unsubscribe from this group, send email to
> open-iscsi+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/open-iscsi?hl=en.
> oki Mike.
>
Thanks.
One thing that is not clear to me, is why you have to do the logout then
relogin? Are you doing this because you have a app or FS using the iscsi
disk, and it gets IO errors when you get the conn errors?
Or, are you doing the relogin because the /dev/sdX is automatically
removed when you see the conn errors messages? If this is happening what
version of open-iscsi are you using and where did you get it
(open-iscsi.org or some distro)?
In /var/log/messages do you see
session recovery timed out after 120 secs
(the value for the secs may be different for you if you modified it in
iscsid.conf or with iscsiadm).
> On Thu, Aug 9, 2012 at 10:54 PM, Michael Christie <mich...@cs.wisc.edu
> <mailto:mich...@cs.wisc.edu>> wrote:
>
> In the future can you post these questions to the open-iscsi mailing
> list? This is the last time I will respond privately. It is best ti
> post on the list so others can see the answer (people searching in
> google for similar problems in the future).
>
> It changes name because you probably still have some app/user with a
> reference to the device. When yourelogin the scsi layer will then
> give you a different name.
>
> You should not be using the sdX types of names, because they are not
> persistent. You should use the udev names in /dev/disk/.
>
> On Aug 9, 2012, at 3:17 AM, parveen kumar <coolbu...@gmail.com
> <mailto:coolbu...@gmail.com>> wrote:
>
> > Hi Mike Christie,
> >
> > ok , but what about /dev/sdb, first time its showing me /dev/sda
> then connection bracked and i had tried again and next its showing
> /dev/sdb
> > means if network disruption again i have to edit /etc/fstab ?
> >
> > Regards,
> > PARVEEN
> >
> > On Thu, Aug 9, 2012 at 11:26 AM, Michael Christie
> <mich...@cs.wisc.edu <mailto:mich...@cs.wisc.edu>> wrote:
> > There was a probably a network disruption that caused the
> initiator to lose the connection
> >
> > On Aug 9, 2012, at 12:11 AM, parveen kumar <coolbu...@gmail.com
>
> > Hi Mike Christie,
> >
> > ok , but what about /dev/sdb, first time its showing me /dev/sda
> then connection bracked and i had tried again and next its showing
> /dev/sdb
> > means if network disruption again i have to edit /etc/fstab ?
> >
> > Regards,
> > PARVEEN
> >
> > On Thu, Aug 9, 2012 at 11:26 AM, Michael Christie
> <mich...@cs.wisc.edu <mailto:mich...@cs.wisc.edu>> wrote:
> > There was a probably a network disruption that caused the
> initiator to lose the connection
> >
> > On Aug 9, 2012, at 12:11 AM, parveen kumar <coolbu...@gmail.com
> <mailto:coolbu...@gmail.com>> wrote:
> >
> > > Hello Mike Christie,
> > >
> > > Few days back i mounted 2TB volume through ISCSI from SAN box in
> my server. After mounting the 2TB space i am bale to see partition
> /dev/sda of 2TB on my server.
> > > But after 5-6 days m not able to write data on /dev/sda, then i
> see conncetion is lost and iscsi connection is braked, so i tried
> again to login to target with this command:
> > > # iscsiadm -m node -T target_iqn_name -p ipaddress -l
> > > after this, login is sucessfull, then i put command:
> > > # fdisk -l
> > > its showing me /dev/sdb of 2TB size
> > > i m not able to understand what happend, first time its showing
> me /dev/sda then connection bracked and i had tried again and next
> its showing /dev/sdb
> > > please help me to sort out this problem......
> > >
> > >
> > > --
> > > .
> > >
> >
> >
> >
> >
> > --
> > .
> >
>
>
>
>
> --
> *.*
> >
> > > Hello Mike Christie,
> > >
> > > Few days back i mounted 2TB volume through ISCSI from SAN box in
> my server. After mounting the 2TB space i am bale to see partition
> /dev/sda of 2TB on my server.
> > > But after 5-6 days m not able to write data on /dev/sda, then i
> see conncetion is lost and iscsi connection is braked, so i tried
> again to login to target with this command:
> > > # iscsiadm -m node -T target_iqn_name -p ipaddress -l
> > > after this, login is sucessfull, then i put command:
> > > # fdisk -l
> > > its showing me /dev/sdb of 2TB size
> > > i m not able to understand what happend, first time its showing
> me /dev/sda then connection bracked and i had tried again and next
> its showing /dev/sdb
> > > please help me to sort out this problem......
> > >
> > >
> > > --
> > > .
> > >
> >
> >
> >
> >
> > --
> > .
> >
>
>
>
>
> --
>
> --
> You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
> To post to this group, send email to open-...@googlegroups.com.
> To unsubscribe from this group, send email to
> open-iscsi+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/open-iscsi?hl=en.
parveen kumar
Aug 14, 2012, 2:20:17 AM8/14/12
to Mike Christie, open-...@googlegroups.com, Rajat Gupta
Dear Mike Christie,
I am using this disk for IO, basiclly this iscsi storage is mounted in CLUSTER(CentOS 5.3) and further shared with NFS from CLUSTER to all 50 nodes so every node runs a script after 5 or 10 mins. In nodes i had done entry in auto.master and auto.misc for NFS share at CLUSTER(i.e iscsi storage) automount with --timeout 10
-----------------------------------------------------------------------------------
My /var/log/messages file attached with this mail.M facing these error's
--------------------------------------------------------------------------------------------------------------------------------
Aug 6 12:13:09 master kernel: connection1:0: iscsi: detected conn error (1011)
Aug 6 12:13:09 master iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3)
Aug 6 12:13:33 master iscsid: connect failed (113)
Aug 6 12:13:39 master iscsid: connect failed (113)
Aug 6 12:13:42 master iscsid: received iferror -38
Aug 6 12:13:42 master iscsid: connection1:0 is operational after recovery (4 attempts)
Aug 6 12:57:57 master kernel: connection1:0: iscsi: detected conn error (1011)
Aug 6 12:57:58 master iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3)
Aug 6 12:58:25 master iscsid: received iferror -38
Aug 6 12:58:25 master iscsid: connection1:0 is operational after recovery (2 attempts)
Aug 6 18:24:09 master kernel: connection1:0: iscsi: detected conn error (1011)
Aug 6 18:24:10 master iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3)
Aug 6 18:24:39 master iscsid: connect failed (113)
Aug 6 18:25:09 master iscsid: connect failed (113)
Aug 6 18:26:09 master iscsid: connect failed (113)
Aug 6 18:26:09 master kernel: session1: iscsi: session recovery timed out after 120 secs
Aug 6 18:26:09 master kernel: iscsi: cmd 0x2a is not queued (8)
Aug 6 18:26:09 master kernel: iscsi: cmd 0x2a is not queued (8)
Aug 6 18:26:10 master kernel: iscsi: cmd 0x2a is not queued (8)
Aug 6 18:26:10 master kernel: iscsi: cmd 0x2a is not queued (8)
Aug 6 18:26:10 master kernel: iscsi: cmd 0x2a is not queued (8)
Aug 6 18:26:10 master kernel: iscsi: cmd 0x2a is not queued (8)
Aug 6 18:26:15 master iscsid: connect failed (113)
Aug 6 18:26:21 master iscsid: connect failed (113)
Aug 6 18:26:39 master iscsid: connect failed (113)
Aug 6 18:27:03 master iscsid: connect failed (113)
.......
Regards,
PARVEEN
9780933599
--
.
Mike Christie
Aug 14, 2012, 4:40:28 PM8/14/12
to open-...@googlegroups.com, parveen kumar, Rajat Gupta
On 08/14/2012 01:20 AM, parveen kumar wrote:
> Aug 6 12:13:09 master kernel: connection1:0: iscsi: detected conn
> error (1011)
> Aug 6 12:13:09 master iscsid: Kernel reported iSCSI connection 1:0
> error (1011) state (3)
1011 is a generic error. We do not really know what happened yet.
> Aug 6 12:13:09 master kernel: connection1:0: iscsi: detected conn
> error (1011)
> Aug 6 12:13:09 master iscsid: Kernel reported iSCSI connection 1:0
> error (1011) state (3)
> Aug 6 12:13:33 master iscsid: connect failed (113)
> Aug 6 12:13:39 master iscsid: connect failed (113)
> Aug 6 18:26:09 master kernel: session1: iscsi: session recovery timed
> out after 120 secs
for 2 minutes, but cannot even connect to it due to the network issue above.
parveen kumar
Aug 15, 2012, 2:12:24 PM8/15/12
to Mike Christie, open-...@googlegroups.com, Rajat Gupta
Dear Mike Christie,
:)
Oki tell me that what file to edit for Retry-Relogin from initiator to target when connection get lost in CentOS5.3.
Like: defaults is 4 tries and 120 sec how to increase this and what file to edit plz share some link that can i understand properly.
Best Regards,
PARVEEN
9780933599
--
.
Michael Christie
Aug 16, 2012, 12:23:12 PM8/16/12
to open-...@googlegroups.com, Rajat Gupta
On Aug 15, 2012, at 1:12 PM, parveen kumar <coolbu...@gmail.com> wrote:
Dear Mike Christie,:)Oki tell me that what file to edit for Retry-Relogin from initiator to target when connection get lost in CentOS5.3.Like: defaults is 4 tries and 120 sec how to increase this and what file to edit plz share some link that can i understand properly.
/usr/share/docs/iscsi-initiator-utils-VERSION/README:
6. Configuration ================ The default configuration file is /etc/iscsi/iscsid.conf. This file contains only configuration that could be overwritten by iSCSI Discovery, or manualy updated via iscsiadm utility. Its OK if this file does not exist in which case compiled-in default configuration will take place for newer discovered Target nodes.
You want to edit this setting:
8.1.2 replacement_timeout ------------------------- The next iSCSI timer that will need to be tweaked is: node.session.timeo.replacement_timeout = X Here X is in seconds. replacement_timeout will control how long to wait for session re-establishment before failing pending SCSI commands and commands that are being operated on by the SCSI layer's error handler up to a higher level like multipath or to an application if multipath is not being used.
If you set the iscsid.conf setting then rediscover your targets and then relogin.
But you really want to figure out why the network is out for so long or maybe use dm-multipath to handle really long failures like that.
Best Regards,PARVEEN9780933599On Wed, Aug 15, 2012 at 2:10 AM, Mike Christie <mich...@cs.wisc.edu> wrote:
On 08/14/2012 01:20 AM, parveen kumar wrote:1011 is a generic error. We do not really know what happened yet.
> Aug 6 12:13:09 master kernel: connection1:0: iscsi: detected conn
> error (1011)
> Aug 6 12:13:09 master iscsid: Kernel reported iSCSI connection 1:0
> error (1011) state (3)
It looks like we lose the network connection. 113 is "No route to host".
> Aug 6 12:13:33 master iscsid: connect failed (113)
> Aug 6 12:13:39 master iscsid: connect failed (113)
Looks like the problem is pretty severe. We try to relogin to the target
> Aug 6 18:26:09 master kernel: session1: iscsi: session recovery timed
> out after 120 secs
for 2 minutes, but cannot even connect to it due to the network issue above.
--.
Reply all
Reply to author
Forward
0 new messages