Beginner Question - Walking the tree always returns first token matches

[Dave Pearce]

Hi

I am a newbie to Antlr and I'm trying to parse a C file using a set of rules for the C language.

When I call the Walk method, my overridden callback is executed as expected, but it only finds the first match and returns.

I've tried using listeners and visitors but the outcome is the same.

The top of my grammar file looks like:

grammar C;

primaryExpression
    :   Identifier
    |    preProcessorDirective
    |   Constant
    |   StringLiteral+
    |   '(' expression ')'
    |   genericSelection
    |   '__extension__'? '(' compoundStatement ')' // Blocks (GCC extension)
    |   '__builtin_va_arg' '(' unaryExpression ',' typeName ')'
    |   '__builtin_offsetof' '(' typeName ',' unaryExpression ')'
    ;

and the first few lines from the file I want to parse are:

#ifdef _LOOKUP_TABLES_H
#else

#define _LOOKUP_TABLES_H

// comment this next line out when testing against "eraweb.bc.jsplc.net" and "sf0105jhsteve.stbc2.jstest2.net"


My code is as follows:

        protected void ParseFile(StreamReader sr)
        {
            string strRuleName;
            List<string> lstTokens = new List<string>();

            // create a CharStream that reads from standard input
            AntlrInputStream inputStream = new AntlrInputStream(sr);

            // create a lexer that feeds off of input CharStream
            CLexer cLexer = new CLexer(inputStream);

            // create a buffer of tokens pulled from the lexer
            CommonTokenStream commonTokenStream = new CommonTokenStream(cLexer);

            // create a parser that feeds off the tokens buffer
            CParser cParser = new CParser(commonTokenStream);

            IParseTree tree = cParser.primaryExpression();
            ParseTreeWalker walker = new ParseTreeWalker();

            AntlrCListener listener = new AntlrCListener(this, cParser);
 //           AntlrCVisitor visitor = new AntlrCVisitor(inputStream, cParser);
            visitor.VisitPrimaryExpression(primaryExpressionContext);
            walker.Walk(listener, tree);

            ...
         }

My callback function is as follows:

        // Listen to matches of C Pre-Processor Directives

        public override void ExitPreProcessorDirective(CParser.PreProcessorDirectiveContext context)
        {
            string strOut = context.GetText();

            // get the tokens returned from the tree walk
            lstTokens.Clear();

            for (int i = 0; i < context.ChildCount; i++)
                lstTokens.Add(context.children[i].ToString());

        }

No matter what I do, I can only get the match for the #ifdef _LOOKUP_TABLES_H  line.

Can anyone point out what I'm doing wrong?

Thanks
Dave

[Eric Vergnaud]

Hi,

in your grammar, primaryExpression matches exactly one alternative, and does that once.

you need a higher level rule, say CompilationUnit, which will read multiple primaryExpressions, such as

compilationUnit:

     primaryExpression+

     ;

Eric

[Eric Vergnaud]

FYI, a number of grammars are available on the antlr web site. You would learn a lot from reading them.

Le mercredi 8 avril 2015 23:09:15 UTC+8, Dave Pearce a écrit :

[Dave Pearce]

Hi Eric

Thanks! That worked a treat.

Dave