Trouble with Expect
Keith Hanlan
I'm trying to use libexpect to write an application which manages
a dialogue with two pairs (4 total) of children concurrently.
I've discovered that Expect's use of global buffers makes it
impossible to leave anything in one child's output buffer while
doing an exp_expectl() on another child's output.
Unfortunately, this limits the utility of Expect for my purposes.
I've had to restructure my program to work around it and I've
sacrificed a fair bit of simplicity and elegance in order to do
so.
When I read the documentation, I realized that exp_buffer and the
rest were globals. However, I (incorrectly) assumed that
you used per-process buffers internally and artificially set these
globals after a call to expect to point to the appropriate child's
buffer. That is, that the global would only be useful until the next
expect call was made. Would it be feasible to modify the
implementation to behave this way? The way it is now, I could
have almost as easily used strtok() to do my parsing.
I've worked around this particular problem but have now encountered a
much more serious challenge. One of the things my program does is issue
a command to a utility which produces a list. The output looks like
this:
NUMIDS 29
IDNAME S1 A
CLASS variable : 4 1
IDNAME S1 AREA1
CLASS type : 0 1
IDNAME S1 AREA2
CLASS type : 0 1
IDNAME S1 INT
CLASS type : 0 1
...
end-of-data [SIC]
For each IDNAME, I need to get the symbol (A, AREA1, ...), the
class (variable, type, sym range elem, ...), and something called
the def_index (the first digit in the pair).
I use a front-end to exp_expectl() to generalize my parsing. It looks
like this:
=============================================================================
/* -----------------------------------------------------------*/
/* Get symbol name, class, and def_index */
/* */
/* NAMEID <Section> <Symbol> */
/* CLASS <class> : <def_index> <num_references> */
/* e.g. */
/* NAMEID MYSECT1 RED */
/* CLASS sym range elem : 4 1 */
/* */
Read_To_Pattern(xref, "IDNAME +([A-Z0-9]+) +([A-Z0-9]+)\r\n", 2);
symbol->id = strdup(Last_Buffer_Read(xref, 2));
Read_To_Pattern(xref, "CLASS +([a-z ]+) : ([0-9]+) +([0-9]+)\r\n", 3);
symbol->sclass = strdup(Last_Buffer_Read(xref, 1)); /* sym range elem */
symbol->def_index = strdup(Last_Buffer_Read(xref, 2)); /* 4 */
=============================================================================
Read_To_Pattern() does the following:
=============================================================================
regexp *re; /* compiled regular expression */
re = TclRegComp(pattern);
state = exp_expectl(child->in_fd,
exp_compiled, pattern, re, ES_MATCH,
exp_end);
switch (state)
{
case ES_MATCH:
{
int sub_pattern_index;
for (sub_pattern_index=1;
sub_pattern_index <= match_num;
sub_pattern_index++)
{
re->endp[sub_pattern_index][0]='\0';
child->last_response[sub_pattern_index-1]
= strdup(re->startp[sub_pattern_index]);
}
}
break;
case ES_APPL_ERROR: ...
case EXP_TIMEOUT: ...
case EXP_FULLBUFFER: ...
case EXP_EOF ...
default
}
=============================================================================
Now this all works. Except that after approximately 20 iterations
(i.e. it varies from one run to the next), it skips part of the
buffer and prematurely gets to the end of the output. It then
times out.
Here's a fragment of my debugging output (with exp_loguser on) to
illustrate:
=========================START SAMPLE OUTPUT===============================
...
IDNAME S1 SET2
CLASS type : 0 1
get_next_global_symbol: ID: SET2
get_next_global_symbol: CLASS: type
get_next_global_symbol: DEF_INDEX: 0
IDNAME S1 SET3
CLASS type : 0 1
get_next_global_symbol: ID: SET3
get_next_global_symbol: CLASS: type
get_next_global_symbol: DEF_INDEX: 0
IDNAME S1 SET4
CLASS type : 0 1
IDNAME S1 T12
CLASS type : 0 1
IDNAME S1 TABLE1
CLASS type : 0 1
IDNAME S1 TABLE3
CLASS type : 0 1
IDNAME S1 TABLE4
CLASS get_next_global_symbol: ID: SET4
get_next_global_symbol: CLASS: type
get_next_global_symbol: DEF_INDEX: 0
type : 0 1
IDNAME S1 TABLE5
CLASS type : 0 1
IDNAME S1 TS1
CLASS type : 0 1
IDNAME S1 TS2
CLASS type : 0 1
IDNAME S1 TS3
CLASS type : 0 1
IDNAME S1 U
CLASS variable : 0 1
IDNAME S1 X
CLASS variable : 1 1
IDNAME S1 Y
CLASS variable : 1 1
IDNAME S1 Z
CLASS variable : 1 1
end-of-data
get_next_global_symbol: ID: T12
get_next_global_symbol: CLASS: type
get_next_global_symbol: DEF_INDEX: 0
ERROR: Read_To_Pattern(4): TIMEOUT trying to match 'IDNAME +([A-Z0-9]+) +([A-Z0-9]+)
'
keithh%bcarh635 [98]:
==========================END SAMPLE OUTPUT===============================
Everything works for a while and then all of a sudden, Expect lets
more than the match get consumed. Notice in my code fragment
above how I've constructed the two regular expressions explicitly
to avoid this. There is no wild-card character in either
expression. In theory, only a single line of output can match.
No matter what I do, the problem persists. I've tried to speed the
processing up by turning off all my debugging output. I've turned off
exp_loguser. I've set exp_fullbuffer to True (and it never happens).
I've increased exp_match_max to 5000 and decreased it to 1000. Nothing
helped.
I'm in rather desperate straights and would appreciate your help.
Please reply directly to me as well as to the newsgroups because
my newsfeed is currently in a state of flux and is unreliable.
Thank you very much,
Keith Hanlan Kei...@bnr.ca Bell-Northern Research, Ottawa, Canada 613-765-4645