Unknown token; why?

130 views
Skip to first unread message

Oskar

unread,
Oct 24, 2012, 8:43:06 AM10/24/12
to sab...@googlegroups.com
I'm using the grammar file below. It's a grammar for regular expressions where the terminals are clojure maps/json maps. When I try to parse the string "{}" (and strings like "{a b}" too) I get:

querylang.lexer.LexerException: [1,1] Unknown token: { 

But I was expecting {} to be an event, because: event = '{' clojurecode '}'; (in the grammar below). What's wrong?

My grammar:

Helpers

    all = [0 .. 0xFFFF];
    lowercase = ['a' .. 'z'];
    uppercase = ['A' .. 'Z'];
    digit = ['0' .. '9'];
    hex_digit = [digit + [['a' .. 'f'] + ['A' .. 'F']]];
    
    tab = 9;
    cr = 13;
    lf = 10;
    eol = cr lf | cr | lf; // This takes care of different platforms
    
    blank = (' ' | tab | eol)+;
    
    letter = lowercase | uppercase;
    symletters = letter | '+' | '-' | '_' | '*' | '?' | '/' | '\' | '=' | '.' | ':';
    clojurecodeletters = symletters | '[' | ']' | '(' | ')' | '#' | '`' | blank;
    clojurecode = clojurecodeletters*;


Tokens

    event = '{' clojurecode '}';
    bar = '|';
    plus = '+';
    star = '*';
    qmark = '?';
    opar = '(';
    cpar = ')';
    dot = '.';
    whitespace = blank;


Ignored Tokens

    whitespace;


Productions

    union {-> exp} = {concat} concat {-> concat.exp}
        | {union} union bar concat {-> New exp.union(union.exp, concat.exp)};
    concat {-> exp} = {closure} closure {-> closure.exp}
        | {concat} concat closure {-> New exp.concat(concat.exp, closure.exp)};
    closure {-> exp} = {atom} atom {-> atom.exp}
        | {qmark} atom qmark {-> New exp.qmark(atom.exp)}
        | {star} atom star {-> New exp.star(atom.exp)}
        | {plus} atom plus {-> New exp.plus(atom.exp)};
    atom {-> exp} = {dot} dot {-> New exp.dot()}
        | {event} event {-> New exp.event(event)}
        | {paren} opar union cpar {-> New exp.paren(union.exp)};


Abstract Syntax Tree

    exp = {union} [left]:exp [right]:exp
        | {concat} [left]:exp [right]:exp 
        | {qmark} exp
        | {star} exp
        | {plus} exp
        | {paren} exp
        | {event} event
        | {dot};

Oskar

unread,
Oct 24, 2012, 10:05:32 AM10/24/12
to sab...@googlegroups.com
My mistake. Even though I had deleted and replaced the parser, an old version was loaded in my clojure process.
Reply all
Reply to author
Forward
0 new messages