Flex 2.5.35: fatal flex scanner internal error--end of buffer missed

[Clint O]

Hi:

I received this message from flex when trying to write an EOF rule
looking for unterminated curly braces. I ended up writing a rule
like:

<curly><<EOF>> {
Token t("{", "{", yyextra->oldsrc());

yyextra->token() = t;

return 1;
}

This error happens only if I enable error recovery in Parse::Yapp (a
Perl clone of yacc). I would prefer if I could use error recovery to
emit more useful messages, but this has had me stumped for awhile.

Thanks,

-Clint
[I see notes saying that if you longjmp out of the lexer and call it
again without resetting the input stream, this error often results.
-John]

[Clint O]

On Saturday, January 22, 2011 3:38:13 PM UTC-8, Clint O wrote:
> [I see notes saying that if you longjmp out of the lexer and call it
> again without resetting the input stream, this error often results.
> -John]

Yes, I read something to the effect that once you reach an EOF you
must reset yyin (or something to that effect). This sort of begs the
question of how you'd do error recovery when you get a syntax error on
the last token of the input. Does this seem like the expected
behavior to you? I don't have any more input to give it, so I don't
know what I'm supposed to set yyin to once I've reached EOF.

Thanks,

-Clint
[You can call yyrestart(yyin) but based on offline discussion this sounds
to me like a smashed pointer bug. -John]