Expect and coroutines?

[Colin Macleod]

I'm updating an Expect-based script which collects information from a set of remote machines. Since this takes some time I process all the connections in parallel, using the expect_background command. Now the sequence of commands I need to run on each machine has become more complex. This makes managing the state of each connection in the usual event-handling style quite difficult.

So I have started looking at using a coroutine for each machine to manage its state and interaction in a more direct style. The tricky part is to integrate the coroutines with expect_background. I looked on the wiki but could not find anything relevant. Has anyone done this?

Thanks, Colin.

[Colin Macleod]

Nobody??

It does seem quite hard to do this in a general way, as expect_background has so many possibilities.

I ended up with something fairly limited, but which works for what I need at present. Basically on each machine I need to run a sequence of commands, and for each command process some of its output. When I see the machine prompt again, I'm finished with that command and can move on to the next.


I defined a helper:

proc co_expect {sid pattern} {

expect_background {
-i $sid
-re $pattern [list [info coroutine] 1]
$PROMPT [list [info coroutine] 0]
}
}

Then I run a coroutine for each machine, and within that use code like this for each command:

exp_send "ls -1 --color=never $::filepat\n"

set files {}

co_expect $spawn_id {\n([^\r]+)\r}

while {[yield]} {
set file $::expect_out(1,string)
lappend files $file
}


Colin.

[Don Porter]

On 01/24/2018 03:49 AM, Colin Macleod wrote:
> I'm updating an Expect-based script which collects information from a set of remote machines. Since this takes some time I process all the connections in parallel, using the expect_background command. Now the sequence of commands I need to run on each machine has become more complex. This makes managing the state of each connection in the usual event-handling style quite difficult.
>
> So I have started looking at using a coroutine for each machine to manage its state and interaction in a more direct style. The tricky part is to integrate the coroutines with expect_background. I looked on the wiki but could not find anything relevant. Has anyone done this?

Expect has not been actively maintained in a long time. That may be
changing now with adoption by a new lead maintainer. Time will tell.

I would be very very surprised if Expect has been revised so as to
support NRE operations as would be needed to interoperate with coroutines.

--
| Don Porter Applied and Computational Mathematics Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

[Colin Macleod]

On Wednesday, 24 January 2018 17:09:47 UTC, Don Porter wrote:
>
> Expect has not been actively maintained in a long time. That may be
> changing now with adoption by a new lead maintainer. Time will tell.
>
> I would be very very surprised if Expect has been revised so as to
> support NRE operations as would be needed to interoperate with coroutines.
>

Nevertheless, there do seem to be useful ways to interoperate at the script level. See my other post for a hack that works nicely for my current use case.

Cheers, Colin.

[Uwe Klein]

Am 24.01.2018 um 18:20 schrieb Colin Macleod:
> Nevertheless, there do seem to be useful ways to interoperate at the script level. See my other post for a hack that works nicely for my current use case.
>

You can keep state inside the bg_expect handling
using a fully referenced or global variable, can't you?

Over the years I've used (bg_)expect for rather complex tasks.

Usually with a per connection wrapper proc.


Uwe

[Colin Macleod]

On Friday, 26 January 2018 10:12:54 UTC, Uwe Klein wrote:
>
> You can keep state inside the bg_expect handling
> using a fully referenced or global variable, can't you?
>

Yes, that's what I was doing in previous versions of this script, I had code like:

expect_background {
-re \\n(\[^\\r]+)\\r {
set file $expect_out(1,string)
lappend ::sid_files($expect_out(spawn_id)) $file
}
$::PROMPT {run_grep $expect_out(spawn_id)}
}

- where run_grep was another proc to run the next step in the process, which would then read from ::sid_files($spawn_id).

But when you have a sequence of such operations to run on each connection, and the results of earlier steps determine the later steps, this sort of code gets quite hairy. Switching to running a coroutine per connection makes it much cleaner.

The equivalent of the code above becomes:

co_expect $spawn_id {\n([^\r]+)\r}
while {[yield]} {
set file $::expect_out(1,string)
lappend files $file
}

foreach file $files {
...next steps...

[Colin Macleod]

I cleaned up my code for this a little now and added it to the end of http://wiki.tcl-lang.org/21555

Colin.