Persistent connections across initiator reboot
HIMANSHU
performed login on 2) and 192.168.7.51(containing 5 targets,out of
which performed login on 3).I am using IET target and 'iscsi-initiator-
utils-6.2.0.868-0.7.el5' initiator.
So at present there are 5 connections on initiator.suddenly if
initiator reboots.
we are having following code in 'iscsi' service
iscsiadm -m node --logoutall=all(stop function)
iscsiadm -m node --loginall=automatic(start function)
But disadvantage is that it will login to all 15 targets from both
7.50 and 7.51 machines,and not 5 which already were there.This is
because previous discovery entry is still there in "/var/lib/iscsi/
nodes" and "/var/lib/iscsi/send_targets".
If I commented above 2 lines from 'iscsi' service,It works fine for
stop,start,restart service operations.i.e.initiator is restoring only
connections which were there previously.
But this doesn't work across reboot.I lost my 5 connections after
reboot though above 2 lines were commented.
Thus,what should I do for persistent connections across reboot?
Ulrich Windl
>
> I discovered 2 machine,192.168.7.50(containing 10 targets,out of which
> performed login on 2) and 192.168.7.51(containing 5 targets,out of
> which performed login on 3).I am using IET target and 'iscsi-initiator-
> utils-6.2.0.868-0.7.el5' initiator.
>
> So at present there are 5 connections on initiator.suddenly if
> initiator reboots.
>
> we are having following code in 'iscsi' service
> iscsiadm -m node --logoutall=all(stop function)
> iscsiadm -m node --loginall=automatic(start function)
Wouldn't your problem be solved if you set some of the "automatic" connections to
"manual"?
>
> But disadvantage is that it will login to all 15 targets from both
> 7.50 and 7.51 machines,and not 5 which already were there.This is
> because previous discovery entry is still there in "/var/lib/iscsi/
> nodes" and "/var/lib/iscsi/send_targets".
[...]
Ulrich
Mike Christie
> I discovered 2 machine,192.168.7.50(containing 10 targets,out of which
> performed login on 2) and 192.168.7.51(containing 5 targets,out of
> which performed login on 3).I am using IET target and 'iscsi-initiator-
> utils-6.2.0.868-0.7.el5' initiator.
>
> So at present there are 5 connections on initiator.suddenly if
> initiator reboots.
>
> we are having following code in 'iscsi' service
> iscsiadm -m node --logoutall=all(stop function)
> iscsiadm -m node --loginall=automatic(start function)
>
> But disadvantage is that it will login to all 15 targets from both
> 7.50 and 7.51 machines,and not 5 which already were there.This is
Do you have only the target records you want to log into automatically
marked as node.startup=automatic?
> because previous discovery entry is still there in "/var/lib/iscsi/
> nodes" and "/var/lib/iscsi/send_targets".
I didn't follow you when you meantioned the previous entry. Are you
running the iscsiadm discovery command at some point during startup?
>
> If I commented above 2 lines from 'iscsi' service,It works fine for
> stop,start,restart service operations.i.e.initiator is restoring only
> connections which were there previously.
Aare you running a custom script that logs into specific targets or how
do you log into targets if the init script does not run loginall=automatic?
>
> But this doesn't work across reboot.I lost my 5 connections after
> reboot though above 2 lines were commented.
>
> Thus,what should I do for persistent connections across reboot?
I think you just want to set the targets you want to log into with the
node.startup = automatic. When that loginall=automatic gets run, it will
only log into those targets.
iscsiadm -m node -T target -p ip:port -o update -n node.startup -v startup
I think with IET you can set up lists to control how can log into which
target. Does that work for you?
HIMANSHU
@Mike
On Apr 21, 5:08 am, Mike Christie <micha...@cs.wisc.edu> wrote:
> HIMANSHU wrote:
> > I discovered 2 machine,192.168.7.50(containing 10 targets,out of which
> > performed login on 2) and 192.168.7.51(containing 5 targets,out of
> > which performed login on 3).I am using IET target and 'iscsi-initiator-
> > utils-6.2.0.868-0.7.el5' initiator.
>
> > So at present there are 5 connections on initiator.suddenly if
> > initiator reboots.
>
> > we are having following code in 'iscsi' service
> > iscsiadm -m node --logoutall=all(stop function)
> > iscsiadm -m node --loginall=automatic(start function)
>
> > But disadvantage is that it will login to all 15 targets from both
> > 7.50 and 7.51 machines,and not 5 which already were there.This is
>
> Do you have only the target records you want to log into automatically
> marked as node.startup=automatic?
> > because previous discovery entry is still there in "/var/lib/iscsi/
> > nodes" and "/var/lib/iscsi/send_targets".
>
> I didn't follow you when you meantioned the previous entry. Are you
> running the iscsiadm discovery command at some point during startup?
> > If I commented above 2 lines from 'iscsi' service,It works fine for
> > stop,start,restart service operations.i.e.initiator is restoring only
> > connections which were there previously.
>
> do you log into targets if the init script does not run loginall=automatic?
followed by login command for particular target selected by user.so I
will fire 'iscsiadm update command to set node.startup as automatic'
for this
> > But this doesn't work across reboot.I lost my 5 connections after
> > reboot though above 2 lines were commented.
>
> > Thus,what should I do for persistent connections across reboot?
>
> I think you just want to set the targets you want to log into with the
> node.startup = automatic. When that loginall=automatic gets run, it will
> only log into those targets.
'iscsid.conf'.
so running update command to change it to 'automatic' only on login(or
I will give option for persistent connection across reboot).
But my problem these changes vanish after another discovery command.
I login to one target say TAR1, from 30.51 and fired update command to
change 'node.startup' to 'automatic'.
and then after some time,If I want to login to another target from
30.51,I need to fire discovery command,which will make 'node.startup'
of TAR1 changed to 'manual' again due to 'iscsid.conf'.
So what can be the solution for this?
How can I stop discovery command overwriting previous configuration?
or I need to again fire 'update command'(for all those logged on that
IP) after repeated discovery command?
Mike Christie
> Yeah.This worked for me.I specified 'node.startup=manual' in
> 'iscsid.conf'.
> so running update command to change it to 'automatic' only on login(or
> I will give option for persistent connection across reboot).
> But my problem these changes vanish after another discovery command.
>
> I login to one target say TAR1, from 30.51 and fired update command to
> change 'node.startup' to 'automatic'.
> and then after some time,If I want to login to another target from
> 30.51,I need to fire discovery command,which will make 'node.startup'
> of TAR1 changed to 'manual' again due to 'iscsid.conf'.
You can pass the discovery commands some flags to indicated how you want
it to manage the db.
iscsiadm -m discovery -t st -p ip -o new
will just add new target portals that are not in the db.
iscsiadm -m discovery -t st -p ip -o delete
will delete target portals that are in the db but no longer returned by
the target in sendtargets discvoery.
iscsiadm -m discovery -t st -p ip -o update
was supposed to update target portal records that are in the db and
returned by sendtargets using the iscsid.conf info. I checked the code
today and it actually updates existing records and will add new ones
using the iscsid.conf info.
Then you can pass in combos. I think you want
iscsiadm -m discovery -t st -p ip -o new -o delete
this will add new records for new target portals, and delete records for
targets no longer returned by the target, and for existing portals it
will not update the records so your existing record settings will not be
overwritten.
HIMANSHU
Steps
1. iscsiadm -m discovery -t sendtargets -p 192.168.30.12 -o new -o
delete
2. iscsiadm -m node -T iqn.2009-04.com.systems:Target1 -p
192.168.30.12:3260 -o update -n node.startup -v automatic
so '/var/lib/iscsi/nodes/iqn.2009-04.com.systems
\:Target1/192.168.30.12\,3260\,1/default' entry changed from
node.startup=manual(as specified in iscsid.conf) to
node.startup=automatic
3. iscsiadm -m node -T iqn.2009-04.com.systems:Target1 -p
192.168.30.12:3260 --login
4. Then if I again perform step 1(i.e. discovery),node.startup gets
overwritten as 'manual'.
I have specified 'node.startup' entry to be 'manual' and will update
it to 'automatic' only of those targets which are logged in.
What is going wrong.Am I performing the same operations as you told?
I am using iscsi-initiator-utils-6.2.0.868-0.7.el5 and it seems to
have discovery switches.
Do I need to upgrade it?
Mike Christie
> Thank you really.But I have not got the expected results.
>
> Steps
>
> 1. iscsiadm -m discovery -t sendtargets -p 192.168.30.12 -o new -o
> delete
>
> 2. iscsiadm -m node -T iqn.2009-04.com.systems:Target1 -p
> 192.168.30.12:3260 -o update -n node.startup -v automatic
>
> so '/var/lib/iscsi/nodes/iqn.2009-04.com.systems
> \:Target1/192.168.30.12\,3260\,1/default' entry changed from
> node.startup=manual(as specified in iscsid.conf) to
> node.startup=automatic
>
> 3. iscsiadm -m node -T iqn.2009-04.com.systems:Target1 -p
> 192.168.30.12:3260 --login
>
> 4. Then if I again perform step 1(i.e. discovery),node.startup gets
> overwritten as 'manual'.
>
> I have specified 'node.startup' entry to be 'manual' and will update
> it to 'automatic' only of those targets which are logged in.
>
> What is going wrong.Am I performing the same operations as you told?
> I am using iscsi-initiator-utils-6.2.0.868-0.7.el5 and it seems to
> have discovery switches.
> Do I need to upgrade it?
iscsi-initiator-utils-6.2.0.868-0.18.el5
HIMANSHU
performed the steps mentioned in my last post.
Still after when I discover using 'iscsiadm -m discovery -t
I am also not performing any target configuration changes on the
target side in this whole process.Just discovery followed by
'node.startup update to automatic command'.and again performing
discovery causes overwriting.
Am I going wrong somewhere in the steps?
Thanks and Regards,
Himanshu Padmanabhi.
HIMANSHU
overwriting problem?
Mike Christie
> I am now having 'iscsi-initiator-utils-6.2.0.868-0.18.el5' and
> performed the steps mentioned in my last post.
>
> Still after when I discover using 'iscsiadm -m discovery -t
> sendtargets -p 192.168.30.12 -o new -o
> delete',nodes and sendtargets entries gets overwritten.
>
Sorry, sorry, my fault. I was looking up the current devel version. In
RHEL 5.3/CentO35.3 and below there is no fix.
You have to use the upstream code. I can also give you a test src rpm I
am working on for RHEL 5.4/CentOs 5.4 if you want? It has the change needed.
HIMANSHU
Can you please give me the test src rpm for CentOS 5.4?
HIMANSHU
Suppose I login to 1st target from machine 30.12,it was having node
authentication.so I saved its credentials in iscsid.conf and then I
fired the discovery command followed by login command.It was
successful and those credentials also got stored in nodes and
send_targets.
Then if I want to login to 2nd target which is also having node
authentication from same machine,I am overwriting same iscsid.conf
file.So I am loosing my previous credentials from iscsid.conf.Also
after discovery,I am loosing previous target information from nodes
and send_targets.
So solution would be maintaining separate iscsid.conf files per logged
in target.Can I do that here?
ietd allows to read from different configuration file.right?
Mike Christie
> One more question analogues to this.
>
> Suppose I login to 1st target from machine 30.12,it was having node
> authentication.so I saved its credentials in iscsid.conf and then I
> fired the discovery command followed by login command.It was
> successful and those credentials also got stored in nodes and
> send_targets.
>
> Then if I want to login to 2nd target which is also having node
> authentication from same machine,I am overwriting same iscsid.conf
> file.So I am loosing my previous credentials from iscsid.conf.Also
> after discovery,I am loosing previous target information from nodes
> and send_targets.
-o new flag.
>
> So solution would be maintaining separate iscsid.conf files per logged
> in target.Can I do that here?
Ulrich Windl
>
> One more question analogues to this.
>
> Suppose I login to 1st target from machine 30.12,it was having node
> authentication.so I saved its credentials in iscsid.conf and then I
> fired the discovery command followed by login command.It was
> successful and those credentials also got stored in nodes and
> send_targets.
>
> Then if I want to login to 2nd target which is also having node
> authentication from same machine,I am overwriting same iscsid.conf
> file.So I am loosing my previous credentials from iscsid.conf.Also
> after discovery,I am loosing previous target information from nodes
> and send_targets.
Hi,
I'm no expert, but I think the credentials are stored per node/target in the
"iSCSI database" (like /etc/iscsi/send_targets/* and /etc/iscsi/nodes/*/*).
/etc/iscsi.conf just has the defaults. Probably it would be better to never touch
the iscsid.conf, but provide auth information when discovering targets or loggin
in to nodes/targets. However then the "secrets" would be on the command line (and
process list, etc).
[...]
Regards,
Ulrich
Mike Christie
> On 28 Apr 2009 at 7:10, HIMANSHU wrote:
>
>> One more question analogues to this.
>>
>> Suppose I login to 1st target from machine 30.12,it was having node
>> authentication.so I saved its credentials in iscsid.conf and then I
>> fired the discovery command followed by login command.It was
>> successful and those credentials also got stored in nodes and
>> send_targets.
>>
>> Then if I want to login to 2nd target which is also having node
>> authentication from same machine,I am overwriting same iscsid.conf
>> file.So I am loosing my previous credentials from iscsid.conf.Also
>> after discovery,I am loosing previous target information from nodes
>> and send_targets.
>
> Hi,
>
> I'm no expert, but I think the credentials are stored per node/target in the
> "iSCSI database" (like /etc/iscsi/send_targets/* and /etc/iscsi/nodes/*/*).
Yeah, that is correct. When you run the discovery command or manual
addition command, iscsiadm will read iscsid.conf and use those for the
initial defaults for what gets created in those dirs. You can then
change what is in those dirs using iscsiadm -m node -o update....
> /etc/iscsi.conf just has the defaults. Probably it would be better to never touch
> the iscsid.conf, but provide auth information when discovering targets or loggin
> in to nodes/targets. However then the "secrets" would be on the command line (and
> process list, etc).
>
I was thinking he has a issue where one target needs one set of CHAP
values for the discovery session, then they need another set of CHAP
values for another discovery session to another target. For this type of
setup, you have to edit iscsid.conf, run iscsiadm -m discovery ..., then
edit iscsid.conf again and then run iscsiadm -m discovery ... to the
other target.
Ulrich Windl
Hi,
That sounds like a workaround for some design deficit. Why not have a more
flexible approach like ~/.netrc (a file that stores authentication information for
several systems, keeping secrets away from the command line and the process list).
I mean an option for discovery like "--credentials-file=~/iscsi-credentials-for-
...". You get the idea?
Regards,
Ulrich
HIMANSHU
I am now having iscsiadm version 2.0-870,but still I am having a
problem in case of 2 targets having different NODE authentication.
My scenario
Suppose 2 targets (having different NODE AUTHENTICATION,assuming same
DISCOVERY AUTHENTICATION) from same target machine(say 30.12) are
logged in.I set up node.startup=automatic in both cases.
My observation
If I restart initiator machine,re-login can be possible in case of
last logged in target only,not both of them.
Steps performed
1. Discover machine using -o new -o delete->Enter Node credentials for
1st target from the list shown->Discovery again as iscsid.conf is now
having node credentials of target1->login to target1->making
node.startup=automatic
2. Discover machine using -o new -o delete->Enter different Node
credentials for 2nd target from the list shown->Discovery using -o
update as iscsid.conf is now having node credentials of target2->login
to target2->making node.startup=automatic
(using -o update overwrites my node.startup setting of target1,so I
have to again make node.startup of target1 to automatic.without -o
update,it wont allow me to login as I need to change iscsid.conf to
target2's credentials.)
My conclusion
Basically all this problem might be because of single iscsid.conf
file.It should be different for every logged in target.Then only I can
have all previous credentials stored.
On Apr 30, 12:31 pm, "Ulrich Windl" <ulrich.wi...@rz.uni-
Mike Christie
Could you just put it in the iscsid.conf file with the other discovery
settings? Instead of the single discovery address settings, you can
specific specific discovery addresses?
Someone just has to implement it. Put it on my TODO :)
Mike Christie
> Exactly Ulrich,there can be a better approach.
> I am now having iscsiadm version 2.0-870,but still I am having a
> problem in case of 2 targets having different NODE authentication.
>
> My scenario
>
> Suppose 2 targets (having different NODE AUTHENTICATION,assuming same
> DISCOVERY AUTHENTICATION) from same target machine(say 30.12) are
> logged in.I set up node.startup=automatic in both cases.
>
> My observation
>
> If I restart initiator machine,re-login can be possible in case of
> last logged in target only,not both of them.
>
> Steps performed
>
> 1. Discover machine using -o new -o delete->Enter Node credentials for
> 1st target from the list shown->Discovery again as iscsid.conf is now
> having node credentials of target1->login to target1->making
> node.startup=automatic
>
> 2. Discover machine using -o new -o delete->Enter different Node
> credentials for 2nd target from the list shown->Discovery using -o
> update as iscsid.conf is now having node credentials of target2->login
> to target2->making node.startup=automatic
> (using -o update overwrites my node.startup setting of target1,so I
> have to again make node.startup of target1 to automatic.without -o
> update,it wont allow me to login as I need to change iscsid.conf to
> target2's credentials.)
If you do this:
1. set DISCOVERY AUTHENTICATION in iscsid.conf.
2. run discovery iscsiadm -m discovery -t st -p ip
3. set NODE AUTHENTICATION for target1 found
iscsiadm -m node -T target1 -p ip:port -o update -n
node.session.auth.username -v user
iscsiadm -m node -T target1 -p ip:port -o update -n
node.session.auth.password -v pass
4.A If other target or portal or both is found through the same
discovery address and was just found then all you have to do is run
those iscsiadm -m node commands for the second target (you do not need
to run discovery again):
iscsiadm -m node -T target2 -p ip2:port -o update -n
node.session.auth.username -v user
iscsiadm -m node -T target2 -p ip2:port -o update -n
node.session.auth.password -v pass
4.B if the other target or portal or both is found through a different
discovery address but using the same DISCOVERY AUTHENTICATION then do
discovery to the other address:
iscsiadm -m discovery -t st -p ip2
If for some reason the other discovery address also gives you the
targets and portals from the other discovery address ip, then pass in
the -o new -o delete to the discovery command.
Then run iscsiadm -m node -o update .... to set the NODE AUTHENTICATION
for those targets.
4.C if the other target or portal or both is found through a different
discovery address but using a differenet DISCOVERY AUTHENTICATION then
you have to edit the discovery chap settings in iscsid.conf and then run
the discovery command
iscsiadm -m discovery -t st -p ip2
If for some reason the other discovery address also gives you the
targets and portals from the other discovery address ip, then pass in
the -o new -o delete to the discovery command.
Then just run iscsiadm -m node -o update .... to set the chap settings
for the targets found in there.
5. if then later on you want to discover new targets then run
iscsiadm -m discovery -t st -p ip -o new -o delete
And then run iscsiadm -m node -o update .... to set the chap settings
for what was found.
Mike Christie
It looks like there is a bug. If you wanted to run the discovery command
again but use what was in /etc/iscsi/send_targets/ip
This will not work.
I will fix this up.
Mike Christie
With the attached patch, if you run
iscsiadm -m discovery -p ip:port
(note no -t st is needed since the discovery type is in the db).
then we do discovery using what is in
/etc/iscsi/send_targets/ip,port/*config
So if you had different discovery auth settings stored in there then
they would just be used.
The problem is that if you did not pass in arguments like this:
iscsiadm -m discovery -p ip:port
it used to default to the same command as "iscsiadm -m discovery -p
ip:port -o show". So I am not sure if we can do this safely. Maybe need
to add some other code.
Mike Christie
command to create discovery records. So doing
iscsiadm -m discovery -t st -p ip -o new
Then you could run
iscsiadm -m discovery -t st -p ip -o update -n your_setting -v your_value
and then you can run the discovery command and it will use the settings
found in the db for that ip.
This would be like how in node mode you can create records under
/etc/iscsi/nodes.