Confused about the influence of the order of lexical rules on lexical processing

[Roger Costello]

Hi Folks,

Newbie here.

I am studying the XML lexer/parser example shown in the book, Definitive ANTLR.

It shows a lexer with these rules (and in this order):

OPEN             :   '<'                 -> pushMode(INSIDE) ;
XMLDeclOpen :   '<?xml' S        -> pushMode(INSIDE) ;

According to the book (p. 74) the lexical rule listed first is favored:

    ANTLR lexers resolve the ambiguities between lexical
    rules by favoring the rule specified first.

I interpret that to mean, if the input is this:

    <?xml ...

then the lexer will use this rule:

OPEN        :   '<'                      -> pushMode(INSIDE) ;

upon encountering the '<' in <?xml...

But that would result in an incorrect lexical processing.

Apparently I am misinterpreting the book. Would you clarify the relationship between the order of lexical rules and the rules that are used during lexical processing please?

/Roger

[Mike Lischke]

Hey Roger,

It shows a lexer with these rules (and in this order):

OPEN             :   '<'                 -> pushMode(INSIDE) ;
XMLDeclOpen :   '<?xml' S        -> pushMode(INSIDE) ;

According to the book (p. 74) the lexical rule listed first is favored:

    ANTLR lexers resolve the ambiguities between lexical
    rules by favoring the rule specified first.

That rule applies only to disambiquities. The above example is not ambiguous because the lexer always tries to match as much as possible (greedy matching), regardless of the rule order. The order plays a role if you have a match-all part (e.g. for letters) and define keywords that also consist of letters. In that case you have to list the keyword rule before the "catch-all" rule.

Mike

-- 

www.soft-gems.net

[Roger Costello]

Thanks Mike. That clears things up nicely.  /Roger

--
You received this message because you are subscribed to a topic in the Google Groups "antlr-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/antlr-discussion/qVoRtkM-1mk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to antlr-discussi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.