controlling multiple expect sessions simultaneously
Dave P
I have read Don Libes book on Expect. In specific, Chapter 10 and 11 deal
with handling multiple processes, and more importantly, handling multiple
processes simultaneously. My goal is to login to about 50 Linux machines
simulatenously and run a set of commands (let's say reboot). I am trying
to accomplish this as quick as possible (logging into 50 sessions
consecutively takes multiple minutes). So far, this is the best I can
come up with:
(assuming I have a list of IPs)
set timeout 7
array set spawnarr {}
foreach ip $ipList {
spawn telnet $ip
set spawnarr($ip) $spawn_id
}
foreach ip $ipList {
expect {
-i $spawnarr($ip) "login:" {}
timeout {
puts "expect timed out on login for $ip"
exit 1
}
}
}
sleep .5
foreach ip $ipList {
send -i $spawnarr($ip) "root\r"
}
foreach ip $ipList {
expect {
-i $spawnarr($ip) "Password:" {}
timeout {
puts "expect timed out on password for $ip"
exit 1
}
}
}
sleep .5
foreach ip $ipList {
send -i $spawnarr($ip) "mypassword\r"
}
foreach ip $ipList {
expect {
-i $spawnarr($ip) ">" {}
timeout {
puts "expect timed out on prompt"
exit 1
}
}
}foreach ip $ipList {
send -i $spawnarr($ip) "$command\r"
}
foreach ip $ipList {
expect {
-i $spawnarr($ip) ">" {}
timeout {
puts "expect timed out on prompt"
exit 1
}
}
}
Is there a more efficient way to do this? Can I control multiple
processes simultaneously in a better way? Thanks.
-Dave
Bruce Hartweg
Dave P wrote:
> Greetings,
>
> I have read Don Libes book on Expect. In specific, Chapter 10 and 11 deal
> with handling multiple processes, and more importantly, handling multiple
> processes simultaneously. My goal is to login to about 50 Linux machines
> simulatenously and run a set of commands (let's say reboot). I am trying
> to accomplish this as quick as possible (logging into 50 sessions
> consecutively takes multiple minutes). So far, this is the best I can
> come up with:
>
> Is there a more efficient way to do this? Can I control multiple
> processes simultaneously in a better way? Thanks.
>
your example was interleaving each action across all hosts, but
still serializing each step -(which won't scale too well).
expect can listen to all your spawn_ids at once the -i flag can
be given a list of ids. Then just track the state fo each spawn
and as they respond you can respond
foreach ip $ipList {
spawn telnet $ip
lappend spawnIDs $spawn_id
set Info($spawn_id,state) INIT
set Info($spawn_id,ip) $ip
}
while { [llength $spwanIDs] > 0 } {
expect {
-i $spawn_IDs
"login:" {
set spid $expect_out(spawn_id)
send -i $spid "root\r"
if {$Info($spid,state) ne INIT} {
puts "Host $Info($spid,$ip) asking for login in state $Info($spid,state)"
}
set Info($spid,state) USER_SENT
}
"Password:" {
set spid $expect_out(spawn_id)
send -i $spid "mypassword\r"
if {$Info($spid,state) ne USER_SENT} {
puts "Host $Info($spid,$ip) asking for password in state $Info($spid,state)"
}
set Info($spid,state) PW_SENT
}
">" {
set spid $expect_out(spawn_id)
switch $Info($spid,state) {
PW_SENT {
send -i $spid "$command\r"
set $Info($spid,state) CMD_SENT
}
CMD_SENT {
send -i $spid "exit\r"
set $Info($spid,state) DONE
}
default {
puts "Saw unexpected prompt in state $Info($spid,state) for host $Info($spid,$ip)"
send -i $spid "exit\r"
set $Info($spid,state) DONE
}
}
}
eof {
set spid $expect_out(spawn_id)
if {$Info($spid,state) ne DONE} {
puts "Unexpected EOF from host $Info($spid,ip) in state $Info($spid,state)"
}
wait $spid
set idx [lsearch $spawn_IDs $spid]
set spawn_IDs [lreplace $spawn_IDs $idx $idx]
}
timeout {
puts "Haven't heard from anybody lately..."
foreach $spid $spawn_IDs {
close $spid
}
}
}
Note that this isn't tested/running code - it may have typos etc,
but should give the basic idea. This way you are never waiting on a
slow responding host if there are others ready to proceed. And if
you ever want more than a single command to run, you can add that
to the states to just keep sending the next command as each previous
one finishes & then exit after the last one.
As an aside, if you have a bunch of hosts, you may run out of ptys,
in that case instead of spawning ALL of your hosts at once you can just
spawn some configuarable number, and then keep a list of pending jobs.
As one of the current jobs finish, spawn the next pending & that way
you'll always have a bunch going at once. & they all can be at any
state along the processing.
Hope this helps,
Bruce
Dave P
Ok, I see now. For some reason, I got the impression from the book that
if you expect -i for a list of sessions, it will only deal with whichever
session responds first and then exit. I guess not. :) It works much
faster now. However, the script is hanging at the end of the script. It
doesn't hit the timeout and exit, so I'm wondering what might be going on.
Any thoughts?
Bruce Hartweg
Dave P wrote:
>
>
> Ok, I see now. For some reason, I got the impression from the book that
> if you expect -i for a list of sessions, it will only deal with whichever
> session responds first and then exit. I guess not. :)
yes it does, (exit the expect command, not the program) - that is why
the loop is there, to go back into the expect. you could also use the
exp_continue to keep the expect active, but since you need to handle
changes to the list of spawn_ids, the loop is a little clearer, at least
to me ;)
> It works much
> faster now. However, the script is hanging at the end of the script. It
> doesn't hit the timeout and exit, so I'm wondering what might be going on.
> Any thoughts?
do you see messages that each spawn if finished? hopefully the timeout case
is never hit. but if all processes spawned have finished the loop should exit.
try adding an extra message at the top and bottom of the loop (before
and after the expect statement) that prints out the current value of the
spawn id list and make sure it is getting cleaned up properly.
Bruce
Dave P
Sorry, that was sloppy coding by me. In any case, I now have it running,
but at the end, it always enters the eof portion of the switch statement,
and crashes with this error:
Connection closed by foreign host.
logout
Connection closed by foreign host.
Connection closed by foreign host.
Connection closed by foreign host.
can not find channel named "exp10"
while executing
"wait $spid"
invoked from within
"expect -nobrace -i {exp4 exp5 exp8 exp9 exp10} login: {
set spid $expect_out(spawn_id)
send -i $spid "root\r"
if {$Info($spid,state) != "INIT..."
invoked from within
"expect {
-i $spawn_IDs
"login:" {
set spid $expect_out(spawn_id)
send -i $spid "root\r"
if {$Info($spid,state) != "INIT"} {
puts "Ho..."
("while" body line 2)
invoked from within
"while {[llength $spawn_IDs] > 0} {
expect {
-i $spawn_IDs
"login:" {
set spid $expect_out(spawn_id)
send -i $spid "root\r"
if {$Info($s..."
(file "./login3.exp" line 23)
Any idea why this might be?
Thanks.
-Dave
Don Libes
You are correct that the default behavior of an expect command is to
find the first match, execute its action and that's it. Bruce's
solution works because he wrapped it in a "while {[llength $spawn_ids]}
If the while isn't exiting, perhaps you didn't express the while
expression correctly.
Don
Dave P
So it appears that once I enter CMD_SENT portion and send exit to the
telnet session, I immediately enter the EOF portion and the script hangs.
I can't figure out what might be causing this. How can I exit gracefully
once the command is sent to any spawn_id?
Thanks.
-Dave
Dave P
Ah, I'm getting stuck in the wait. I need to look into it a little more.
Bruce Hartweg
Dave P wrote:
like I said, i hadn't actually run that code ;)
and reviewing my docs it seems that wait $spid
should be wait -i $spid
see if that helps.
Bruce
Dave P
'
Ok here is the problem:
Length of spawn_IDs = 1; spawn_IDs = exp5
Connection closed by foreign host.
ENTERED EOF
Waiting for pid
can not find channel named "exp5"
while executing
"wait $spid"
invoked from within
"expect -nobrace -i exp5 login: {
set spid $expect_out(spawn_id)
send -i $spid "root\r"
if {$Info($spid,state) != "INIT"} {
puts "Host $In..."
invoked from within
"expect {
-i $spawn_IDs
"login:" {
set spid $expect_out(spawn_id)
send -i $spid "root\r"
if {$Info($spid,state) != "INIT"} {
puts "Ho..."
("while" body line 3)
invoked from within
"while {[llength $spawn_IDs] > 0} {
puts "Length of spawn_IDs = [llength $spawn_IDs]; spawn_IDs = $spawn_IDs"
expect {
-i $spawn_IDs
"login:" {
..."
(file "./login3.exp" line 18)
This is using the code you gave me and I just added some puts to see where
I am in the code. You can clearly see that exp5 exists when it enters the
EOF. I also do a If [[info exists spid]} {puts "Waiting for pid"; wait
$spid} but somehow by that time, spid doesnt exist which is why the scrip
tcrashes.
Don Libes
Try changing:
wait $spid
to
wait -i $spid
I'm a little embarrassed to find that wait doesn't issue diagnostics
about unexpected arguments.
Don
Dave P
When using this style where you use expect -i to control multiple sessions
ssimultaneously, how do you maintain it such that the output coming from
in from each terminal session is maintained in its own separate buffer?
It's very important that the data received from each session is not
corrupted with data from another session. Does this work this way?
Thanks.
-Dave
Don Libes
When expect makes a match, it also sets $expect_out(spawn_id) so you
can use that to distinguish the source of the data and maintain
separate buffers. Like this:
expect -i $spawnIds $pattern {
append buf($expect_out(spawn_id)) $expect_out(buffer)
}
... will end up with an array called buf that looks like this:
buf(exp1) "Login: "
buf(exp2) "% cat\r\n...."
buf(exp3) "..and so on..."
Don