no viable alternative at input

[SD]

HI,

i am creating two rules and using two rules in a new rule. grammar file is 

// Filter.g4

grammar Filter;

// Tokens.

// LSTRING: ('a'..'z'| 'A'..'Z')+;

CONJUNCTION: 'AND'|'OR';

AND: 'AND';

META: ('meta.'('a'..'z'| 'A'..'Z'|'\\'|'/'|'_')+);

OBJECT:  (('a'..'z'| 'A'..'Z'|'\\'|'/'|'_')+);

OPERATOR: '=';

WHITESPACE: [ \r\n\t]+ -> skip;

NOT: 'NOT';

OPEN_CURLY: '(';

CLOSE_CURLY: ')';

fragment STRING:'"' (ESC|.)*? '"';

fragment UNICODE : ('\u0000'..'\u00FF')+;

fragment NUMBER: [0-9.]+;

fragment BOOL: 'true'|'false';

fragment ESC : '\\"' | '\\\\' ;

VALUE: STRING | NUMBER | BOOL;

metaExpression

    : META OPERATOR VALUE #metaComparison

    | metaExpression CONJUNCTION metaExpression #metaOperation

    | OPEN_CURLY metaExpression CLOSE_CURLY #metaNestedComparisons

    ;

fieldExpression

    : OBJECT #fieldLiteral

    | OBJECT OPERATOR VALUE #fieldComparison

    | fieldExpression CONJUNCTION fieldExpression #fieldOperation

    | OPEN_CURLY fieldExpression CLOSE_CURLY #fieldNestedComparisons

    ;

// Rules.

start: expression EOF;

expression

    : metaExpression #metaExp

    | metaExpression AND fieldExpression #operation

    | OPEN_CURLY expression CLOSE_CURLY  #nestedComparisons

    ;

and go parser snippet is

// Parse

func Parse(input string) (string, string, error) {

    // Setup the input.

    is := antlr.NewInputStream(input)

    // Create the Lexer.

    lexer := parser.NewFilterLexer(is)

    stream := antlr.NewCommonTokenStream(lexer, antlr.TokenDefaultChannel)

    

// Create the Parser.

    p := parser.NewFilterParser(stream)

    // Finally parse the expression (by walking the tree).

    listener := filterListener{}

    // Add err listener for the parser.

    p.AddErrorListener(&listener.errorListener)

    antlr.ParseTreeWalkerDefault.Walk(&listener, p.Start())

    // return the err message on encountering

    // err while parsing.

    if listener.errorListener.err != nil {

        return "", "", listener.errorListener.err

    }

   

    return "", "", nil

}

func main() {

    // q := `meta.test="125"` // success

    //q := `meta.test="125" AND meta.merrrr="myself"` // success

    // q := `(meta.test="125" AND meta.merrrr="myself")` // success

//q := `(meta.test="125" AND meta.merrrr="myself") AND field="other"` // failure

    q := `meta.test="125" AND meta.merrrr="myself" AND field="other"` // failure

    Parse(q)

}

Few expressions are failing with no viable alternative at input. expecting the failed expressions should match   metaExpression AND fieldExpression #operation. 

Could you please help me in identifying the issue ?

Thanks,

SD

[John B. Brodie]

Greetings!

you have conflicting lexer rules for CONJUNCTION and AND. both rules recognize the input string "AND". Antlr, when presented with two lexer rules that match the exact same input string, will select the lexer rule that appears first in the grammar. Thus the input "AND" will always be tokenized as CONJUNCTION, token AND will never be recognized.

Printing the token stream before calling the parser can help a lot for issues like this, in my opinion.

Hope this helps...

   -jbb

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/3486952c-5b8c-4dcc-85d5-e76201287210%40googlegroups.com.

[Mike Cargal]

I'd also add that you're tending to do some of the parser work in the tokenizer.

Example tokenizer rules:

CONJUNCTION: 'AND'|'OR';

fragment BOOL: 'true'|'false';

VALUE: STRING | NUMBER | BOOL;

drop the BOOL token and replace with

TRUE: true;

FALSE: false;

STRING, NUMBER, and BOOL don't need to be fragments.

Then change the following parser rules:

metaExpression

    : META EQUALS (STRING | NUMBER | BOOL) #metaComparison

    | metaExpression AND metaExpression #metaAnd    

    | metaExpression OR metaExpression #metaOR

    | OPEN_CURLY metaExpression CLOSE_CURLY #metaNestedComparisons

    ;

fieldExpression

    : OBJECT #fieldLiteral

    | OBJECT EQUALS (STRING | NUMBER | BOOL) #fieldComparison

    | fieldExpression AND fieldExpression #fieldAnd

    | fieldExpression OR fieldExpression #fieldOr

    | OPEN_CURLY fieldExpression CLOSE_CURLY #fieldNestedComparisons

   ;

This lexer rule seems oddly named as well... 

OPERATOR: '=';

I'd suggest EQUALS (as you'll likely have other "operators"

I see this tenancy all the time (usually in parser rules) all the time as people start out with ANTLR.  They try to get as many of the rules as possible encoded in the tokenizer of lexer.

The Tokenizer needs only the tokens you need to create an unambiguous stream of tokens that you can build parser rules from.  (note: by combining 'and' and 'or' into a single token, you forced them to have identical precedence in the parser rule.  This is almost certainly not what you would want.  Having them as separate tokens allows the definition of independent rules (and you can specify the precedence as a result).

The same thing is a tendency with folks in parser rules.  Here, my best guidance is to get enough "logic" in the rules to unambiguous construct the right interpretation of the input.  Then you can write validation that'll likely have much better error messages than what ANTLR is going to be able to generate for you.

(as a result, the suggested changes are really just a start, but hopefully this will point you in a more productive direction.)

To unsubscribe from this group and stop receiving emails from it, send an email to antlr-di...@googlegroups.com.

[SD]

Thank you.

i need only AND lexer rule between meta and non meta field groups. it should recognize the non meta field before AND rule. 

To unsubscribe from this group and stop receiving emails from it, send an email to antlr-di...@googlegroups.com.

[SD]

Thanks Mike. i will try suggested changes.

[SD]

HI Mike,

i have added all changes you suggested.Majority of scenarios are looks good. few scenarios are failing with ambiguity error.

Grammer :

// Filter.g4

grammar Filter;

// Tokens.

AND: 'AND';

OR: 'OR';

META: ('meta.'('a'..'z'| 'A'..'Z'|'\\'|'/'|'_')+);

OBJECT:  (('a'..'z'| 'A'..'Z'|'\\'|'/'|'_')+);

EQUALS: '=';

NOTEQUALS: '!=';

WHITESPACE: [ \r\n\t]+ -> skip;

NOT: 'NOT';

OPEN_CURLY: '(';

CLOSE_CURLY: ')';

TRUE: 'true';

FALSE: 'false';

STRING:'"' (ESC|.)*? '"';

fragment UNICODE : ('\u0000'..'\u00FF')+;

NUMBER: [0-9.]+;

fragment ESC : '\\"' | '\\\\' ;

metaExpression

    : META (EQUALS | NOTEQUALS) (STRING | NUMBER | TRUE | FALSE) #metaComparison

    | metaExpression AND metaExpression #metaAND

    | metaExpression OR metaExpression #metaOR

    | OPEN_CURLY metaExpression CLOSE_CURLY #metaNestedComparisons

    ;

fieldExpression

    : OBJECT #fieldLiteral

    | OBJECT (EQUALS | NOTEQUALS) (STRING | NUMBER | TRUE | FALSE) #fieldComparison

    | fieldExpression AND fieldExpression #fieldAND

    | fieldExpression OR fieldExpression #fieldOR

    | OPEN_CURLY fieldExpression CLOSE_CURLY #fieldNestedComparisons

    ;

// Rules.

start: expression EOF;

expression

    : metaExpression #metaExp

    | metaExpression AND fieldExpression #operation

    | OPEN_CURLY expression CLOSE_CURLY  #nestedComparisons

    ;

tested scenarios :

qs := []string{

        `meta.test="125"`,

        `(meta.test="125")`,

        `meta.test="125" AND meta.merrrr="myself"`,

        `(meta.test="125" AND meta.merrrr="myself")`,

        `(meta.test="125" AND meta.merrrr="myself") AND debug`,

        `(meta.test="125" AND meta.merrrr="myself") AND field="other"`,

        `((meta.test="125" OR meta.merrrr="myself") AND (field="other" OR field="some"))`,

    }

    for _, q := range qs {

        if _, _, err := Parse(q); err != nil {

            fmt.Printf("*********************\n\n failed - %s : %v\n\n*************************** \n", q, err)

        }

    }

metaExpression with open curly is failing with ambiguity error.

(meta.test="125")

(meta.test="125" AND meta.merrrr="myself")

 #metaExp could be causing the problem. i tried to provide explicit EOF for meta expression and didn't work.

Could you please help me in identifying the issue ? thanks.

SD

[SD]

fixed the problem by replacing  " | OPEN_CURLY expression CLOSE_CURLY  #nestedComparisons " following with "| OPEN_CURLY metaExpression AND fieldExpression CLOSE_CURLY  #nestedComparisons"

Thanks

[Mike Cargal]

expression

    : metaExpression #metaExp

    | metaExpression AND fieldExpression #operation

    | OPEN_CURLY expression CLOSE_CURLY  #nestedComparisons

    ;

says that an expression can be a metaExpression.  (first definition)

That means that:

  | OPEN_CURLY metaExpression CLOSE_CURLY #metaNestedComparisons

and :

  | OPEN_CURLY expression CLOSE_CURLY  #nestedComparisons

Both match OPEN_CURLY metaExpression CLOSE_CURLY

You'll need to decide if OPEN_CURLY metaExpression CLOSE_CURLY should be a metaExpression, or an expression.  I con't really sort out enough of what you're trying to accomplish to make a suggestion as to the right answer.

[SD]

Yes Mike. fixed it. Thanks for your suggestions.