Note that there are more changes in the repo then just this fix.
Fix is not complete as it leaves buffer with last token(s) and I'm not sure yet how deal here.
In most cases it would be ok to skip last token in the buffer and return. In this situation you can continue parsing input as buffer is stored in lexer. This is helpful if you have input with repeated blocks you parse one by one.
If parser is ambiguos then several tokens may start in the same position, end in different positions and result in successful exhaustion. In this case you have several "correct" parse trees that end in different positions.
Most probably I will implement cutting buffer with the shortest match.
On Sat, Apr 7, 2012 at 15:29, Peter Stuifzand <peter.s...@gmail.com> wrote: > Hi, > > I'm not sure why this testcase dies with the message "exhausted". > I created a simple testcase on gist. > > My guess is that it $skips++ past the end of the buffer in this part and > should call end_input. Can anyone confirm? > > > -- > Peter Stuifzand > http://peterstuifzand.nl