Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

How to extract part of what matched an expect pattern?

4,125 views
Skip to first unread message

Ronald Fischer

unread,
Feb 6, 2006, 8:05:58 AM2/6/06
to
I studied several examples which come with expect, but did not find
anything suitable for my problem:

Part of my expect script searches on a remote host a text file,
named in variable $hosttab, for IP addresses, stored as, say

a.b.c.d HOSTNAME

where the HOSTNAME itself is stored in variable $hostname.
I would like to extract that IP address and then use this
address in further commands in my script.

I don't see how I can extract the IP address. Here is so
far I got:

spawn "telnet ..."
...
send "egrep ' $hostname\$' $hosttab\r"
expect -re "([0-9.]*) *$hostname" { set ipaddr ????? }

What do I have to put in the place of the question marks
in order to refer to the string matching the part of the
regexp in parentheses? In Perl, I would use $1, but I didn't
find a similar mechanism documented for 'expect'.

Or is my approach entirely wrong? All examples which I
studied, used $expect_out to catch part of the result,
but I don't think I can use it here, because expect_out
is documented as containing the output which has not
been matched yet, and I would like to catch part of
what has already matched.

Ronald


--
Sent by mn-pg-p-e-b-consultant-3.com from siemens in field com
This is a spam protected message. Please answer with reference header.
Posted via http://www.usenet-replayer.com

Bruce Hartweg

unread,
Feb 6, 2006, 3:10:59 PM2/6/06
to

no expect_out is what you want, read the man pages again.

Here's an excerpt from them:

Upon matching a pattern (or eof or full_buffer), any matching and
previously unmatched output is saved in the variable expect_out(buffer).
Up to 9 regexp substring matches are saved in the variables
expect_out(1,string) through expect_out(9,string). If the -indices flag
is used before a pattern, the starting and ending indices (in a form
suitable for lrange) of the 10 strings are stored in the variables
expect_out(X,start) and expect_out(X,end) where X is a digit,
corresponds to the substring position in the buffer. 0 refers to strings
which matched the entire pattern and is generated for glob patterns as
well as regexp patterns. For example, if a process has produced output
of "abcdefgh\n", the result of:

expect "cd"


is as if the following statements had executed:

set expect_out(0,string) cd
set expect_out(buffer) abcd


and "efgh\n" is left in the output buffer. If a process produced
the output "abbbcabkkkka\n", the result of:

expect -indices -re "b(b*).*(k+)"


is as if the following statements had executed:

set expect_out(0,start) 1
set expect_out(0,end) 10
set expect_out(0,string) bbbcabkkkk
set expect_out(1,start) 2
set expect_out(1,end) 3
set expect_out(1,string) bb
set expect_out(2,start) 10
set expect_out(2,end) 10
set expect_out(2,string) k
set expect_out(buffer) abbbcabkkkk


and "a\n" is left in the output buffer. The pattern "*" (and -re
".*") will flush the output buffer without reading any more output from
the process.
Normally, the matched output is discarded from Expect's internal
buffers. This may be prevented by prefixing a pattern with the
-notransfer flag. This flag is especially useful in experimenting (and
can be abbreviated to "-n" for convenience while experimenting).

ro.nald...@gmail.com

unread,
Feb 7, 2006, 5:08:19 AM2/7/06
to
Bruce Hartweg schrieb:

> Ronald Fischer wrote:
> > Part of my expect script searches on a remote host a text file,
> > named in variable $hosttab, for IP addresses, stored as, say
> >
> > a.b.c.d HOSTNAME
> >
> > where the HOSTNAME itself is stored in variable $hostname.
> > I would like to extract that IP address and then use this
> > address in further commands in my script.
> Upon matching a pattern (or eof or full_buffer), any matching and
> previously unmatched output is saved in the variable expect_out(buffer).
> Up to 9 regexp substring matches are saved in the variables

> If a process produced


> the output "abbbcabkkkka\n", the result of:
>
> expect -indices -re "b(b*).*(k+)"
>
> is as if the following statements had executed:
>
> set expect_out(0,start) 1
> set expect_out(0,end) 10
> set expect_out(0,string) bbbcabkkkk
> set expect_out(1,start) 2
> set expect_out(1,end) 3
> set expect_out(1,string) bb

Thank you for pointing this out! I tried to use it like this in my
script:

# Do something like:
# fgrep myhost /etc/hosts
# fgrep is supposed to return something like
# 10.168.40.24 myhost
# We use the regexp ([0-9.]+) *myhost to extract
# the IP address
send "fgrep $hostname $hosttab\r"
expect -indices -re [join {([0-9.]+) *} $hostname]
# The next two lines are only for testing
set result $expect_out(buffer)
puts "Received: <$result>"
puts "Matched part: <$expect_out(1,string)>"

After this, $result indeed contains the full expected output, including
the
IP address, but usage of $expect_out(1,string), which should be the IP
address only, fails:

can't read "expect_out(1,string)": no such element in array
while executing
"puts "Matched part: <$expect_out(1,string)>""

What am I doing wrong?

Ronald

Donal K. Fellows

unread,
Feb 7, 2006, 6:02:21 AM2/7/06
to
ro.nald...@gmail.com wrote:
> expect -indices -re [join {([0-9.]+) *} $hostname]

That use of [join] there can't be right. I don't know what you think it
is doing, but it isn't. (I wish I knew what I think it is doing, but it
just confuses me!) Try this:

expect -indices -re "(\[0-9.\]+) *$hostname"

OK, that might not work either, but it is at least probably closer to
what I suspect you intend...

Donal.

Ronald Fischer

unread,
Feb 7, 2006, 8:02:18 AM2/7/06
to
Dear Donal K. Fellows,

In your posting Re: How to extract part of what matched an expect


pattern? from Tue, 07 Feb 2006 11:02:21 +0000 you write:

>
> ro.nald...@gmail.com wrote:
> > expect -indices -re [join {([0-9.]+) *} $hostname]
>
> That use of [join] there can't be right. I don't know what you think it
> is doing, but it isn't. (I wish I knew what I think it is doing, but it
> just confuses me!)

The join has the following function:

I need a regexp pattern which looks like

([0-9.]+) *HOSTNAME

where HOSTNAME is the name of the host I'm looking for. I couldn't
use

"([0-9.]+) *$hostname"

because Tcl would then interpret [0-9.] as command interpolation.
I couldn't use

{([0-9.]+) *$hostname}

either, because $-Interpolation is not done between curly braces.
But I think I can use join, because join builds a string out
of its elements; hence,

[join {([0-9.]+) *} $hostname]

would take it's first argument, the string

([0-9.]+) *

append to it the hostname, and we have exactly the regexp for expect.

> Try this:
>
> expect -indices -re "(\[0-9.\]+) *$hostname"

I usually try to avoid \ escapes, because I personally find them
unpleasant to read, but if you prefer it this way, I implemented
this change. This has no effect on the result: I still get

can't read "expect_out(1,string)": no such element in array
while executing
"puts "Matched part: <$expect_out(1,string)>""

invoked from within

Note that the error is not with my regexp (because, as $expect_out(buffer)
shows, expect exactly matches what it is supposed to match). It is
that $expect_out(1,string) seems to be undefined.

Ronald

--
Spam protected message from:
Sent by mn-pg-p-e-b-consultant-3.com from siemens within field com
Posted via http://www.usenet-replayer.com

Bryan Oakley

unread,
Feb 7, 2006, 9:30:53 AM2/7/06
to
Ronald Fischer wrote:
> The join has the following function:
>
> I need a regexp pattern which looks like
>
> ([0-9.]+) *HOSTNAME
>
> where HOSTNAME is the name of the host I'm looking for. I couldn't
> use
>
> "([0-9.]+) *$hostname"
>
> because Tcl would then interpret [0-9.] as command interpolation.
> I couldn't use
>
> {([0-9.]+) *$hostname}
>
> either, because $-Interpolation is not done between curly braces.
> But I think I can use join, because join builds a string out
> of its elements; hence,
>
> [join {([0-9.]+) *} $hostname]
>
> would take it's first argument, the string
>
> ([0-9.]+) *
>
> append to it the hostname, and we have exactly the regexp for expect.
>

Have you actually tried that yet or are you speaking on theoritical
terms? What makes you think join appends strings to the end of another
string?

Join doesn't append anything, it joins list elements. It will treat its
first argument as a list and join them together with the second
argument. In the above case, ([0-9.])+) is the first list element, * is
the second list element.

To wit:

% set hostname "whatever"
whatever
% join {([0-9.]+) *} $hostname
([0-9.]+)whatever*

Notice how the space before the asterisk is lost and the asterisk comes
before the hostname. That is not what you say you want.

Doing something like the above is a bad habit to get into -- you
shouldn't use list commands on things that aren't a list. Your string
works, but there are many, many regexps that wouldn't work. If you think
this works and use the idiom regularly, eventually your code will break
on an unusual pattern.


--
Bryan Oakley
http://www.tclscripting.com

Ronald Fischer

unread,
Feb 7, 2006, 9:35:57 AM2/7/06
to
NEW FINDINGS!!!

Actually, I found out *why* expect_out(1,string) did not work:
expect ran on a timeout, i.e. my regexp didn't match at all!

OK, so now the interesting question is, why doesn't it match?
I have created the following simplified script:

#!/usr/bin/expect
send "fgrep localhost /etc/hosts\r"
expect {
-re {([0-9.]+) *localhost} {puts "IP address is $expect_out(1,string)"}
timeout {puts "timeout occurred"}
}

and indeed, I get a "timeout occured" message. But if I execute
the fgrep manually, I get a matching line

$ fgrep localhost /etc/hosts
127.0.0.1 localhost

This line certainly should match the expect result (although,
as I see now, it might only match the last digit, 1; so a better
regexp would be {^ *([0-9.]+) *localhost} - but at least it SHOULD
match something).

Why does expect fail to match here?

Ronald

--
Spam protected message from:

Sent by mn-pg-p-e-b-consultant-3.com from siemens element from com
Posted via http://www.usenet-replayer.com

Bruce Hartweg

unread,
Feb 7, 2006, 11:23:45 AM2/7/06
to

Ronald Fischer wrote:
> NEW FINDINGS!!!
>
> Actually, I found out *why* expect_out(1,string) did not work:
> expect ran on a timeout, i.e. my regexp didn't match at all!
>
> OK, so now the interesting question is, why doesn't it match?
> I have created the following simplified script:
>
> #!/usr/bin/expect
> send "fgrep localhost /etc/hosts\r"
> expect {
> -re {([0-9.]+) *localhost} {puts "IP address is $expect_out(1,string)"}
> timeout {puts "timeout occurred"}
> }
>
> and indeed, I get a "timeout occured" message. But if I execute
> the fgrep manually, I get a matching line
>
> $ fgrep localhost /etc/hosts
> 127.0.0.1 localhost
>
> This line certainly should match the expect result (although,
> as I see now, it might only match the last digit, 1; so a better
> regexp would be {^ *([0-9.]+) *localhost} - but at least it SHOULD
> match something).
>
> Why does expect fail to match here?
>

your expression is only matching spaces between the number and the name,
if there is a tab in there it will fail.

try {([0-9.]+)\s*localhost} and see if it works any better


Bruce

ro.nald...@gmail.com

unread,
Feb 8, 2006, 3:30:58 AM2/8/06
to
Bruce Hartweg schrieb:

> your expression is only matching spaces between the number and the name,
> if there is a tab in there it will fail.
>
> try {([0-9.]+)\s*localhost} and see if it works any better

No, it doesn't. Actually, the problem must be more fundamental: I
changed
my testscript to the following

#!/usr/bin/expect
send "fgrep localhost /etc/hosts\r"
expect {

-re {([0-9.]+)\s*localhost} {puts "IP address is
$expect_out(1,string)"}
-re {[0-9]} {puts "Number found"}
-re {.} {puts "Something found"}
timeout {puts "timeout occurred"}
}

and even here, timeout occurs! It does not even match the regexp "."

Maybe someone on a Linux/Unix machine could try to run the above script
on his/her machine? "localhost" should be in /etc/hosts, so fgrep will
return a suitable result.

Ronald

Donal K. Fellows

unread,
Feb 8, 2006, 6:30:13 AM2/8/06
to
ro.nald...@gmail.com wrote:
> No, it doesn't. Actually, the problem must be more fundamental: I
> changed my testscript to the following
>
> #!/usr/bin/expect
> send "fgrep localhost /etc/hosts\r"

You seem to be missing a [spawn] in there...

Donal.

ro.nald...@gmail.com

unread,
Feb 8, 2006, 8:58:34 AM2/8/06
to

Donal K. Fellows schrieb:

> > #!/usr/bin/expect
> > send "fgrep localhost /etc/hosts\r"
>
> You seem to be missing a [spawn] in there...

I thought I need spawn only if connecting to a process where I have to
conduct a dialogue (in my original version, I use spawn because I
execute the commands via telnet, but I omitted that part in my
demonstration program).

Indeed, the modified program works:

#!/usr/bin/expect
spawn bash --norc


send "fgrep localhost /etc/hosts\r"

expect {
-re {([0-9.]+)\s*localhost} {puts "IP address is
$expect_out(1,string)"}

timeout {puts "timeout occurred"}
}
send "exit"

(outputs: IP address is 127.0.0.1)

So I had combined two errors: In my first script, I didn't pay
attention to tab characters,
and in my simplified script, I didn't pay attention to spawning.

Unfortunately this did not help with my original script: I still didn't
get the IP address
in that context. Now with my new knowledge, I rewrote my original
program like this:

# ~/bin/+ is a one-line script, containing the telnet command to
connect to
# the remote host.
spawn "$env(HOME)/bin/+"
expect "#"


send "fgrep $hostname $hosttab\r"

expect "\n"
expect {
-re "(\[0-9.\]+)\s*$hostname" { puts "IP address is
$expect_out(1,string)" }
timeout { puts "timeout occurred"}
}

Executing it, I receive the following output (leaving out details about
the
login dialogue to the remote host):

root@sb1-1:~# fgrep actomm /etc/hosts
10.168.40.240 actomm
root@sb1-1:~# timeout occurred

So you see that the fgrep returns a correct value, but still I get the
error
message "timeout occured". But when I hardcode the $hostname
into the regexp, i.e.

-re "(\[0-9.\]+)\s*actomm" { puts "IP address is
$expect_out(1,string)" }

suddenly it matches. This means that in the statement

expect {
-re "(\[0-9.\]+)\s*$hostname" { puts "IP address is
$expect_out(1,string)" }
timeout { puts "timeout occurred"}
}

$hostname is not substituted by the content of the variable hostname -
probably
because of the outer braces around the expect parameters. Maybe expect
does not evaluate its parameters.

So, could some kind soul show me how to write this expect statement in
a correct way?

Ronald


Ronald

cullen...@themill.com

unread,
Apr 17, 2020, 2:12:33 PM4/17/20
to
Hi Ronald,

I'm 14 years behind the curve on this, but did you ever get this working?

Best,
Cullen

heinrichmartin

unread,
Apr 17, 2020, 3:24:48 PM4/17/20
to
tl;dr

You want to enable Expect's internal log [exp_internal -f log.dbg 0]. (I find the internal log on the screen quite confusing and always go for a file...)

This log.dbg shows all the input, all patterns and whether they matched, and all output.

It is a bit cumbersome to get used to the output, but I have syntax highlighting for the file format.

Cullen C

unread,
Apr 18, 2020, 11:03:25 PM4/18/20
to
Thanks for the reply.

Really the question I'm trying to answer is simple, i think... How do you take a ```set lindex argv *``` entry and make it in to a variable, and use that variable to either a: traverse an array for the $var, or b: search a table in a .csv for the values that corresponds to $var. and then pipe the info from the array, that you need, say an IP and outlet number for an old shite PDU to spawn a telnet session and run the other commands you want to run.

I have servers that need to be rebooted without ssh access, they also do not have obm. I've set the AC state in the BIOS to power on after power-off so I can remotely power-cycle via an expect script with the PDU. At the end of the day I want users to be able to do it them-selves. They can just say ```./script hostname``` and it will do a hard AC reboot of their machine. I hope that makes sense? I've tried doing it in bash and piping the output to an embedded expect script but expect wont read the variable. is there a way to do something like this all in expect or should I just give up?

Thank you for your time,
Cullen

Rich

unread,
Apr 19, 2020, 12:09:55 AM4/19/20
to
Cullen C <cullen...@themill.com> wrote:
> Really the question I'm trying to answer is simple, i think...

It probably is simple, but your explanation below is quite confusing,
so I'm not sure my answers are going to be to the questions you really
want to ask.

> How do you take a ```set lindex argv *``` entry and make it in to a
> variable,

If you mean, how to you access an element of the argv variable, and put
that element into a variable, then this is the way:

set new_variable [lindex $argv 4]

That will access the 5th element of argv (zero based addressing) and
extract it from the argv list and put it into a variable named
"new_variable".

> and use that variable to either a: traverse an array for the $var,

By 'traverse ... for the $var' do you mean "search the array for an
item that is equal to the contents of the variable"? And what in the
array do you want to search, the keys (names) or the values (contents)?

If you want to search names, this works:

foreach name [array names array_to_search] {
if {$name eq $search_var} {
# found - do something here with $name element
}
}

for values, you just change the 'if' to read:

if {$array_to_search($name) eq $search_var} {

and you search inside values.


> or b: search a table in a .csv for the values that corresponds to
> $var.

For this, to do it in Tcl, you first have to read in the contents of
the csv file and put those contents into some data structure within
which to search. The Tcllib CSV module will help (but will not do all
the work) with reading the csv file in. You have to write the code to
put it into a data structure that you'll search against.

> and then pipe the info from the array, that you need, say an IP and
> outlet number for an old shite PDU to spawn a telnet session and run
> the other commands you want to run.

I'm not at all sure what you mean here. But if you mean "output the
found array elements to stdout" then that is simply:

puts $array_to_search($name)

as the "do something here" part of the loop above.

> I have servers that need to be rebooted without ssh access, they also
> do not have obm. I've set the AC state in the BIOS to power on after
> power-off so I can remotely power-cycle via an expect script with the
> PDU. At the end of the day I want users to be able to do it
> them-selves. They can just say ```./script hostname``` and it will
> do a hard AC reboot of their machine. I hope that makes sense?

What you wish to do makes sense, how you describe what you are doing
makes less sense (because we don't have any background of how your
setup is operating).

> I've tried doing it in bash and piping the output to an embedded
> expect script but expect wont read the variable.

This sentence makes no sense to me. You mentioned argv at the start of
this post, but pipes do not set the argv variable (unless you also are
using xargs, but you've not said you are using xargs) so if you want to
read the data you are piping to the expect script over a pipe, you'd
have to use gets or read on stdin to obtain the pipe data.

> is there a way to do something like this all in expect or should I
> just give up?

There is very likely a way to do what you want to do. And likely a way
to do it using only Tcl. What do you mean by "I can remotely
power-cycle via an expect script with the PDU". If you can already do
a remote power cycle via expect, then if you ignore the potential
security issues of users doing the same, why can't they just fire off
the same expect script you use?

We might even be able to help, but we need to know a lot more than we
know right now to know how to be truly helpful.

CC

unread,
Apr 19, 2020, 8:57:46 AM4/19/20
to
##########################################

Thank you Rich, I'm going to try out some of your recommendations.

Yes, I think the way I explained things is convoluted. I'm new to programming/scripting.

I think there is a way to do this with Tcl, or bash or expect or pexpect or jest...the thing I'm trying to achieve is simple in concept(and execution for most); Restart an older server that does not have ipmi or idrac from a Networked PDU, using the PDU outlets and the server's AC state to create a caveman remote restart...Thank you covoid-19

I have an expect script that works for rebooting a machine this way but the person running the script needs to know the PDU's IP and outlet number to be able to use the script properly.

#######################################

#!/usr/bin/expect



#USE ---> ./apc_tel.sh ip outlet#

set timeout 9
set ip [lindex $argv 0]
set ol [lindex $argv 1]
set timeout -1

spawn telnet $ip

expect "User Name :" { send "apc\r" }
expect "Password :" { send "apc\r" }

send "olOff $ol\r"
expect "apc>"

sleep 1

send "olOn $ol\r"
expect "apc>"

send "bye\r"

expect eof
close $spawn_id

#######################################

I want to make something that either reads from a separate file with the hostname, PDU - IP and outLet # i.e.; host_list.csv

HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL

#######################################

Or some how put that info in an array in the expect script.--- This doesn't work because I'm base --- but maybe it will give you an idea?
#!/usr/bin/expect



# USE ---> ./apc_tel.sh hostname

set timeout 9
set hostname [lindex $argv 0]
#set ol [lindex $argv 1]

if {[llength $argv] == 0} {
send_user "Usage: apc_tel.sh hostname\n"
exit 1
}
# vvv these vvv are the pdu IPs and outlets that cooresspond to the machine's hostname

set names(host_blah1) { 10.xxx.xxx.164 1 }
set names(host_blah2) { 10.xxx.xxx.163 2 }
set names(host_blah3) { 10.xxx.xxx.164 3 }
set names(host_blah4) { 10.xxx.xxx.163 5 }
set names(host_blah5) { 10.xxx.xxx.164 7 }
set names(host_blah6) { 10.xxx.xxx.163 8 }
set names(host_blah7) { 10.xxx.xxx.164 9 }
set names(host_blah8) { 10.xxx.xxx.163 11 }
set names(host_blah9) { 10.xxx.xxx.164 15 }

set LIST [ array names ip outlet]

if {$hostname == $names($hostname)} {
set ip [lindex $LIST 0]
} else {
puts "$hostname does not match an existing hostname."
}

spawn telnet $ip

expect "User Name :" { send "apc\r" }
expect "Password :" { send "apc\r" }


#expect "User Name :"
#send "$user\r"
#expect "Password :"
#send "$password\r"

send "olOff $ol\r"
expect "apc>"

sleep 1

send "olOn $ol\r"
expect "apc>"

send "bye\r"

expect eof
close $spawn_id

###########################################

I know this is wrong, but my logic goes:

User enters: user% alias_reboot_script_name hostname

script takes hostname from argv 0 >>> parses correct PDU IP and oL# from the array >>> set IP to $ip >>> set oL # to $ol >>> spawn telnet $ip >>> pipes ol # for the rest of the command....

Rich

unread,
Apr 19, 2020, 12:01:02 PM4/19/20
to
No worries.

HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL


Ok, lets say you had an external file named "hostname_map" and it
contained exactly this:

HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL
HOSTNAME IPx.IPx.IPx.IPx oL

I.e., a host name, followed by a *single space* followed by an IP,
followed by a *single space* followed by "oL" (and, presumably none of
"hostname", "IP" or "oL" contain spaces (hostnames and IP's normally do
not, you'll have to tell me whether "oL"'s ever contain spaces). The
"no spaces in the data" and the "single space between items" are both
*very important*. Also, no blank lines at the end either.

And, you want to take an input hostname, find the line in the file with
a matching value for "HOSTNAME", and retreive the IP and oL values.
There are a couple ways to do this, but which data structure you use
depends upon what Tcl version your Expect is running within. Since
you've not said, I'll show the way that will work at least back to Tcl
8.4. If you are running a Tcl older than that then the first step you
will really want to do is upgrade your Expect/Tcl combo to a much more
modern version.

So, you'd read in this data file somewhere in the beginning of your Expect script by
doing something like this:

set fd [file open hostname_map {RDONLY}]
while {[gets $fd line] != -1} {
set temp [split $line " "]
set ::hostname_map([lindex $temp 0]) [lrange $temp 1 end]
}
close $fd

That reads in the file, and inserts its contents into a Tcl global
array named hostname_map, with the keys to the array being the first
column in the file (the hostname) and the values being a two element
list of "IP" and "oL".

Then, at the point where you want to extract the user's input hostname
from the command line you'd do something like:

set requested_hostname [lindex $argv 0]

Note, this above assumes the user always enters the hostname as the
first CLI argument. You'd adjust the 0 to something else if they enter
other CLI arguments before the hostname. If you want fancy command
line parsing with switches and flag parameters, then look to the Tcllib
cmdline library (http://tmml.sourceforge.net/doc/tcllib/cmdline.html)
What I show you here will just be the very basic, simple, variant.
Also note above that you might want to check to see if the user even
entered anything, so you can emit a suitable error message. You'd do
that by checking the length of $argv before you try to extract the
entered value:

if {[llength $argv] != 1} {
# suitable error message would go here instead of this comment
exit 1
}

If might also be good, right after you extract the argv CLI argument,
to see if the hostname they entered exists as a valid hostname in the
array, you'd do that like this:

if { ! [info exists ::hostname_map($requested_hostname)]} {
# suitable error message about "do not know hostname
# '$requested_hostname'" would replace these comments
exit 1
}

So, now you have the CLI parameter in the variable
'requested_hostname'. Now, you'd pass this variable along to the point in your
Expect script where you need to extract the IP and oL values, and you'd
do something like this:

set IP [lindex $::hostname_map($requested_hostname) 0]
set oL [lindex $::hostname_map($requested_hostname) 1]

and now you have the IP from your file in the variable "IP" and the oL
from your file in the "oL" variable (and the hostname already present
in the requested_hostname variable), and at this point, you'd do
whatever it is you do with an IP and oL value to do a server power
cycle.

Now, this above is the basic, simple, but also slightly fragile, way to
do things. The fragility is the strict requirement on the input file
of no spaces in any of the items, and single spaces separating items.
Your input file could be a CSV file if also utilize the Tcllib CSV
library
(https://core.tcl-lang.org/tcllib/doc/trunk/embedded/md/tcllib/files/modules/csv/csv.md)
to parse the CSV file. That will lift some of the really strict file
format requirements, and allow you to actually edit that file in a
spreadsheet (such as OpenOffice Calc) rather than by hand. Not
required of course, but it is an alternative if you want to go that
way.

Now, you also said:

> Or some how put that info in an array in the expect script.

If you don't need an external file to save your hostname IP oL mappings
(having one means you don't need to edit the expect script to
add/subtract them) you can instead embedd the mapping directly into the
script if you like. I'll give you the more verbose, but easier to
follow, way of initializing that array:

You'd do this, somewhere in the beginning of your script (in the same
spot where you'd otherwise read in the external file is good):

set ::hostname_map(HOSTNAME) [list IP oL]

And repeat that line for each new HOSTNAME, IP and oL combination,
changing the HOSTNAME, IP, and oL values to the appropriate ones for
each new hostname.

The rest of the code to retreive the CLI argument, and look it up in
the array to extract the IP and oL values, would remain the same.

Now, this method does have two tiny advantages over the external file
version. One, there's no external file. Two, if "oL" actually
contains spaces, then you can handle that by placing double quotes (")
around the oL value, like so:

set ::hostname_map(HOSTNAME) [list IP "oL"]

heinrichmartin

unread,
Apr 19, 2020, 4:23:11 PM4/19/20
to
On Sunday, April 19, 2020 at 6:01:02 PM UTC+2, Rich wrote:
> HOSTNAME IPx.IPx.IPx.IPx oL
> HOSTNAME IPx.IPx.IPx.IPx oL
>
> I.e., a host name, followed by a *single space* followed by an IP,
> followed by a *single space* followed by "oL" (and, presumably none of
> "hostname", "IP" or "oL" contain spaces (hostnames and IP's normally do
> not, you'll have to tell me whether "oL"'s ever contain spaces). The
> "no spaces in the data" and the "single space between items" are both
> *very important*. Also, no blank lines at the end either.

No spaces is actually much more important than single space: if the first is fulfilled, then every line is a Tcl list already.

> So, you'd read in this data file somewhere in the beginning of your Expect script by
> doing something like this:
>
> set fd [file open hostname_map {RDONLY}]
> while {[gets $fd line] != -1} {
> set temp [split $line " "]
> set ::hostname_map([lindex $temp 0]) [lrange $temp 1 end]
> }
> close $fd

Rich emphasized that his solution is suitable for older versions of Tcl. Let me add a few possibilities that might not work with 8.4 or require other handy packages.

Disclaimers:
All code untested.
If you are new to Tcl, be ready to read a few man pages.

A. If the file is not too large (like hundreds of MB), you could read it all at once. This eases the cleanup.

package require fileutil
set data [string trim [::fileutil::cat hostname_map]]

B. An array will definitely boost your lookup performance, but you could also lookup in a list of lists.

# $data as above
set hosts [lsort -index 0 [split $data \n]]
# retrieve for $host
lassign [lsearch -sorted -inline -index 0 $hosts $host] hostname ip oL
# values are in $hostname, $ip, and $oL - all of them are empty if $host was not found

C. Robustness: I assume it is very unlikely that the file is as clean as specified (unless the file is actually pre-processed). One devops engineer will at some point e.g. comment one line out with a leading hash. (Well, this particular example will just create a bogus hostname entry, but dormant bugs are nasty.)

# $data as above, but you can use the patterns also in [while {[gets]}] or [foreach]
set hosts [lmap line [split $data \n] {
# tune your regexp as desired
switch -regexp -matchvar match -- $line {
{^\s*(\S+)\s+(\S+)\s+(\S+)\s*$} {
lassign $match line host ip oL
# or assign the full match to a bogus variable as a hint to the reader
lassign $match -> host ip oL
# this will be the entry in the hosts list
list $host $ip $oL
}
{^\s*#} {
# ignore comment
continue
}
default {
# error message, then error out or [continue]
}
}
}]

You need not use reporting groups, if you go for the "is a Tcl list"-approach. When using [lmap] you could replay the value with [set list] instead of [list] as the last command.
Also [switch] can be replaced by one or more [if]s. And a simple [lsearch -all -inline -regexp] would do, if you need not be verbose.

CC

unread,
Apr 19, 2020, 6:00:15 PM4/19/20
to
################################################################

Thank you so much Rich. I ended up getting it working with the array! It ended up just being cleaner in the end. the mistake I was making was not setting the values for the $hostname_map before I ran my other if statements.

It works like a charm. Thank you so much for your help.

All the best,
Stay safe.
CC

CC

unread,
Apr 19, 2020, 6:03:14 PM4/19/20
to
Thank you Henrich for your reply, I ended up getting it working with the array, so going to go with that for now. Really appreciate your time though.

All the Best!
C

Rich

unread,
Apr 19, 2020, 9:12:27 PM4/19/20
to
CC <cullen...@themill.com> wrote:
>
>> Now, this method does have two tiny advantages over the external
>> file version. One, there's no external file. Two, if "oL" actually
>> contains spaces, then you can handle that by placing double quotes
>> (") around the oL value, like so:
>>
>> set ::hostname_map(HOSTNAME) [list IP "oL"]
>
> ################################################################
>
> Thank you so much Rich. I ended up getting it working with the
> array! It ended up just being cleaner in the end. the mistake I was
> making was not setting the values for the $hostname_map before I ran
> my other if statements.

Yep, order of operation is important when writing programs.

> It works like a charm. Thank you so much for your help.

You are welcome.

Bezoar

unread,
Apr 27, 2020, 9:05:26 PM4/27/20
to
Its a little late but FYI and for the future:

You can use subst to build your regexp:

%set hostname "myserver.com"
%set rgp [ subst -nocommands {([0-9.]*) *$hostname} ]
# rgp now contains {([0-9.]*) *myserver.com}

% foreach line { "34.56.44.23 x.com" "215.34.223.33 myserver.com" } {
if { [ regexp $rgp $line ] } {
puts "$line matches '$rgp'"
} else {
puts "$line does NOT match '$rgp'"
}
}
34.56.44.23 x.com does NOT match '([0-9.]*) *myserver.com'
215.34.223.33 myserver.com matches '([0-9.]*) *myserver.com'
%

- B


0 new messages