We using R1soft CDP tool to backup Linux files. on centos 5.2
The Backup works fine for normal disk, but for multipath devices its
giving error while taking backup
I am using 11 iSCSI partition on a SAN device configured in multipath.
I am using 8 path in multipath.
My multipath is using 2 Intel NIC which are connected to dedicated
managed switch(HP)
my mount point is
/dev/dm-0 on /home11 type ext3 (rw)
The moment i start backup for above mount point, i get this error
{code}
Jun 23 12:21:24 mss-us2 kernel: connection8:0: ping timeout of 5 secs
expired, last rx 7739207452, last ping 7739212452, now 7739217452
Jun 23 12:21:24 mss-us2 kernel: connection8:0: detected conn error
(1011)
Jun 23 12:21:24 mss-us2 kernel: sd 10:0:0:11: SCSI error: return code
= 0x00020000
Jun 23 12:21:24 mss-us2 kernel: end_request: I/O error, dev sdi,
sector 2362696
Jun 23 12:21:24 mss-us2 kernel: device-mapper: multipath: Failing path
8:128.
Jun 23 12:21:24 mss-us2 multipathd: sdab: tur checker reports path is
down
Jun 23 12:21:24 mss-us2 multipathd: checker failed path 65:176 in map
home13
Jun 23 12:21:24 mss-us2 kernel: device-mapper: multipath: Failing path
65:176.
Jun 23 12:21:24 mss-us2 multipathd: home13: remaining active paths: 7
Jun 23 12:21:25 mss-us2 iscsid: Kernel reported iSCSI connection 8:0
error (1011) state (3)
Jun 23 12:21:28 mss-us2 iscsid: connection8:0 is operational after
recovery (1 attempts)
Jun 23 12:22:00 mss-us2 kernel: connection2:0: ping timeout of 5 secs
expired, last rx 7739243726, last ping 7739248726, now 7739253726
Jun 23 12:22:00 mss-us2 kernel: connection2:0: detected conn error
(1011)
Jun 23 12:22:00 mss-us2 kernel: sd 4:0:0:11: SCSI error: return code =
0x00020000
Jun 23 12:22:00 mss-us2 kernel: end_request: I/O error, dev sdg,
sector 3146136
Jun 23 12:22:00 mss-us2 kernel: device-mapper: multipath: Failing path
8:96.
Jun 23 12:22:00 mss-us2 kernel: device-mapper: multipath: Failing path
8:208.
Jun 23 12:22:00 mss-us2 multipathd: sdn: tur checker reports path is
down
Jun 23 12:22:00 mss-us2 multipathd: checker failed path 8:208 in map
home12
Jun 23 12:22:00 mss-us2 multipathd: home12: remaining active paths: 7
Jun 23 12:22:01 mss-us2 iscsid: Kernel reported iSCSI connection 2:0
error (1011) state (3)
Jun 23 12:22:04 mss-us2 iscsid: connection2:0 is operational after
recovery (1 attempts)
Jun 23 12:22:08 mss-us2 multipathd: dm-0: add map (uevent)
Jun 23 12:22:08 mss-us2 multipathd: dm-0: devmap already registered
Jun 23 12:22:08 mss-us2 multipathd: 8:96: mark as failed
Jun 23 12:22:08 mss-us2 multipathd: home11: remaining active paths: 7
Jun 23 12:22:08 mss-us2 multipathd: 8:128: mark as failed
Jun 23 12:22:08 mss-us2 multipathd: home11: remaining active paths: 6
Jun 23 12:22:08 mss-us2 multipathd: dm-4: add map (uevent)
Jun 23 12:22:08 mss-us2 multipathd: dm-4: devmap already registered
Jun 23 12:22:08 mss-us2 multipathd: dm-0: add map (uevent)
Jun 23 12:22:08 mss-us2 multipathd: dm-0: devmap already registered
Jun 23 12:22:08 mss-us2 multipathd: dm-1: add map (uevent)
Jun 23 12:22:08 mss-us2 multipathd: dm-1: devmap already registered
Jun 23 12:22:13 mss-us2 multipathd: sdab: tur checker reports path is
up
Jun 23 12:22:13 mss-us2 multipathd: 65:176: reinstated
Jun 23 12:22:13 mss-us2 multipathd: home13: remaining active paths: 8
Jun 23 12:22:14 mss-us2 multipathd: sdg: tur checker reports path is
up
Jun 23 12:22:14 mss-us2 multipathd: 8:96: reinstated
Jun 23 12:22:14 mss-us2 multipathd: home11: remaining active paths: 7
Jun 23 12:22:14 mss-us2 multipathd: sdi: tur checker reports path is
up
Jun 23 12:22:14 mss-us2 multipathd: 8:128: reinstated
Jun 23 12:22:14 mss-us2 multipathd: home11: remaining active paths: 8
Jun 23 12:22:15 mss-us2 multipathd: sdn: tur checker reports path is
up
Jun 23 12:22:15 mss-us2 multipathd: 8:208: reinstated
{code}
R1Soft Support guys says there could be problems with the iscsi
device. It's possible that CDP agent may be overwhelming it with too
many requests. If that's the case, it may be resolved with a
configuration change to the iscsi driver.
Any suggestions/tips on what i might be doing wrong ?
> We using R1soft CDP tool to backup Linux files. on centos 5.2
> The Backup works fine for normal disk, but for multipath devices its > giving error while taking backup
> I am using 11 iSCSI partition on a SAN device configured in multipath. > I am using 8 path in multipath. > My multipath is using 2 Intel NIC which are connected to dedicated > managed switch(HP)
> my mount point is > /dev/dm-0 on /home11 type ext3 (rw)
> The moment i start backup for above mount point, i get this error > {code} > Jun 23 12:21:24 mss-us2 kernel: connection8:0: ping timeout of 5 secs > expired, last rx 7739207452, last ping 7739212452, now 7739217452 > Jun 23 12:21:24 mss-us2 kernel: connection8:0: detected conn error > (1011)
> > We using R1soft CDP tool to backup Linux files. on centos 5.2
> > The Backup works fine for normal disk, but for multipath devices its
> > giving error while taking backup
> > I am using 11 iSCSI partition on a SAN device configured in multipath.
> > I am using 8 path in multipath.
> > My multipath is using 2 Intel NIC which are connected to dedicated
> > managed switch(HP)
> > my mount point is
> > /dev/dm-0 on /home11 type ext3 (rw)
> > The moment i start backup for above mount point, i get this error
> > {code}
> > Jun 23 12:21:24 mss-us2 kernel: connection8:0: ping timeout of 5 secs
> > expired, last rx 7739207452, last ping 7739212452, now 7739217452
> > Jun 23 12:21:24 mss-us2 kernel: connection8:0: detected conn error
> > (1011)
If you set these to zero or set them higher does it help. You have to rerun the discovery command so the new iscsid.conf values are picked up or if you have the tools from cenots 5.3 then you can just run iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_interval -v 0 iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_timeout -v 0
Or set this to iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_interval -v 25 iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_timeout -v 25
> If you set these to zero or set them higher does it help. You have to
> rerun the discovery command so the new iscsid.conf values are picked up
> or if you have the tools from cenots 5.3 then you can just run
> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_interval -v 0
> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_timeout -v 0
> Or set this to
> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_interval -v 25
> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_timeout -v 25
i tried both the options, still im getting that same error :(
> On Jul 1, 8:10 am, Mike Christie<micha...@cs.wisc.edu> wrote: >> On 06/29/2009 06:25 AM, jeet2k123 wrote:
>>> node.conn[0].timeo.noop_out_interval = 5 >>> node.conn[0].timeo.noop_out_timeout = 5 >> If you set these to zero or set them higher does it help. You have to >> rerun the discovery command so the new iscsid.conf values are picked up >> or if you have the tools from cenots 5.3 then you can just run >> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_interval -v 0 >> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_timeout -v 0
>> Or set this to >> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_interval -v 25 >> iscsiadm -m node -o update -n node.conn[0].timeo.noop_out_timeout -v 25
> i tried both the options, still im getting that same error :(
If you set the values to 0 you should not get the ping timeout error message. When you use that value do you see:
Jun 23 12:21:24 mss-us2 kernel: connection8:0: ping timeout of 5 secs expired, last rx 7739207452, last ping 7739212452, now 7739217452
> Thanks for your help.
> node.conn[0].timeo.noop_out_* value to 0 made the backup worked
Wouldn't this cause issues with path failover's - cos multipath is
beiung used here ? Iam guessing setting the noop_timeout to 0 will
never fail the path.
> Wouldn't this cause issues with path failover's - cos multipath is
> beiung used here ? Iam guessing setting the noop_timeout to 0 will
> never fail the path.
setting node.conn[0].timeo.noop_out_interval and node.conn
[0].timeo.noop_out_timeout to 25 also works fine