Issues with the CSharp grammar preprocessor directives

[Angel Todorov]

Hello,

I am using this exact grammar (Java target):

https://github.com/antlr/grammars-v4/tree/master/csharp

I have the following questions / problems:

1) How do I specify the currently applied preprocessor directives , for example DEBUG;RELEASE, etc. Where are they set? 

2) I am using the following code and I am getting errors (below) for the #if directive:

List<Token> codeTokens = new ArrayList<Token>();

List<Token> commentTokens = new ArrayList<Token>();

CharStream cs = CharStreams.fromPath(path);

CSharpLexer lexer = new CSharpLexer(cs);

List<? extends Token> tokens = lexer.getAllTokens();

List<Token> directiveTokens = new ArrayList<Token>();

ListTokenSource directiveTokenSource = new ListTokenSource(directiveTokens);

CommonTokenStream directiveTokenStream = new CommonTokenStream(directiveTokenSource, CSharpLexer.DIRECTIVE);

CSharpPreprocessorParser preprocessorParser = new CSharpPreprocessorParser(directiveTokenStream);

int index = 0;

boolean compiliedTokens = true;

while (index < tokens.size())

{

    Token token = tokens.get(index);

    if (token.getType() == CSharpLexer.SHARP)

    {

        directiveTokens.clear();

        int directiveTokenIndex = index + 1;

        // Collect all preprocessor directive tokens.

        while (directiveTokenIndex < tokens.size() &&

              tokens.get(directiveTokenIndex).getType() != CSharpLexer.EOF &&

              tokens.get(directiveTokenIndex).getType() != CSharpLexer.DIRECTIVE_NEW_LINE &&

              tokens.get(directiveTokenIndex).getType() != CSharpLexer.SHARP)

        {

            if (tokens.get(directiveTokenIndex).getChannel() == CSharpLexer.COMMENTS_CHANNEL)

            {

                commentTokens.add(tokens.get(directiveTokenIndex));

            }

            else if (tokens.get(directiveTokenIndex).getChannel() != Lexer.HIDDEN)

            {

                directiveTokens.add(tokens.get(directiveTokenIndex));

            }

            directiveTokenIndex++;

        }

        directiveTokenSource = new ListTokenSource(directiveTokens);

        directiveTokenStream = new CommonTokenStream(directiveTokenSource, CSharpLexer.DIRECTIVE);

        preprocessorParser.setInputStream(directiveTokenStream);

        preprocessorParser.reset();

        // Parse condition in preprocessor directive (based on CSharpPreprocessorParser.g4 grammar).

        CSharpPreprocessorParser.Preprocessor_directiveContext directive = preprocessorParser.preprocessor_directive();

        // if true than next code is valid and not ignored.

        compiliedTokens = directive.value;

        index = directiveTokenIndex - 1;

    }

    else if (token.getChannel() == CSharpLexer.COMMENTS_CHANNEL)

    {

        commentTokens.add(token); // Colect comment tokens (if required).

    }

    else if (token.getChannel() != Lexer.HIDDEN && token.getType() != CSharpLexer.DIRECTIVE_NEW_LINE && compiliedTokens)

    {

        codeTokens.add(token); // Collect code tokens.

    }

    index++;

}

// At second stage tokens parsed in usual way.

ListTokenSource codeTokenSource = new ListTokenSource(codeTokens);

CommonTokenStream codeTokenStream = new CommonTokenStream(codeTokenSource);

CSharpParser parser =  new CSharpParser(codeTokenStream);

// Parse syntax tree (CSharpParser.g4)

parser.setBuildParseTree(true);

Compilation_unitContext compilationUnit = parser.compilation_unit();

When I have code like this:

using System.Text;

namespace MyProject {

public class test {

 public static void Main(string []args) {

 #if DEBUG

 int x = 2;

 #else

 int x = 10;

 #endif

 }

 }

}

I am getting the following errors:

line 11:13 mismatched input 'EOF' expecting {'as', 'is', '[', '(', '.', ',', ';', '+', '-', '*', '/', '%', '&', '|', '^', '<', '>', '?', '??', '++', '--', '&&', '||', '->', '==', '!=', '<=', '>=', '<<'}

Thanks in advance, 

Angel