Problem with strange expectation and not matching negation

[Sascha Ziemann]

I would like to parse a SQL subset for table creation. I have attached my grammar "SqlDdl.g" and an SQL example "test.sql". When I run the default test rig:

    java org.antlr.v4.runtime.misc.TestRig SqlDdl data_definition < test.sql

I get the error:

    line 3:27 mismatched input 'NOT' expecting SYS

The problem seems to be in the way how I am parsing the column definition. Because I need only the column name and the column type I would like ignore everything else after the type.

Therefore I defined a column as an identifier, a data type and an optional list of ignored statements:

column_definition
    : ID data_type ignored_column_definition_statement*
    ;

I have defined a small number of SQL data types:

data_type
    : BLOB
    | CHAR '(' INT ')'
    | CLOB
    | NUMBER
    | RAW '(' INT ')'
    ¦ SYS '.' ANYDATA
    | TIMESTAMP WITH LOCAL TIME ZONE
    | VARCHAR2 '(' INT ( CHAR | BYTE )? ')'
    ;

And this is how I tried to ignore everything after the data type:

ignored_column_definition_statement
    : ~(',' | '(' | ')' )
    | '(' ignored_parenthesis_statement+ ')'
    ;

ignored_parenthesis_statement
    : ~('(' | ')')
    | '(' ignored_parenthesis_statement+ ')'
    ;

First I ignore anything which is neither a colon nor a parenthesis and I ignore a parenthesis expression containing itself parenthesis expressions.

But when I trace the test rig the ignore rules do not consume any tokens:

$ java org.antlr.v4.runtime.misc.TestRig SqlDdl data_definition -trace < test.sql
enter   data_definition, LT(1)=CREATE
enter   table_definition, LT(1)=CREATE
consume [@0,0:5='CREATE',<19>,1:0] rule table_definition alt=1
consume [@1,7:11='TABLE',<24>,1:7] rule table_definition alt=1
consume [@2,13:20='"PERSON"',<6>,1:13] rule table_definition alt=1
consume [@3,27:27='(',<4>,2:4] rule table_definition alt=1
enter   column_definition, LT(1)="PERSON_ID"
consume [@4,35:45='"PERSON_ID"',<6>,3:5] rule column_definition alt=1
enter   data_type, LT(1)=RAW
consume [@5,47:49='RAW',<22>,3:17] rule data_type alt=5
consume [@6,51:51='(',<4>,3:21] rule data_type alt=5
consume [@7,52:53='16',<7>,3:22] rule data_type alt=5
consume [@8,54:54=')',<1>,3:24] rule data_type alt=5
line 3:27 mismatched input 'NOT' expecting SYS
exit    data_type, LT(1)=NOT
enter   ignored_column_definition_statement, LT(1)=NOT
exit    ignored_column_definition_statement, LT(1)=NULL
enter   ignored_column_definition_statement, LT(1)=NULL
exit    ignored_column_definition_statement, LT(1)=,
exit    column_definition, LT(1)=,

The data type gets consumed but the NOT NULL constraint does not get consumed. But in order to consume it I have defined keyword rule which matches every unquoted keyword.

KEYWORD
    : ('a'..'z'|'A'..'Z'|'_'|'-')+
    ;

First I do not understand why the negation rule ~(',' | '(' | ')') does not lead to a match of the keyword rule.

And second I do not understand why a SYS is expected although the data_type has already consumed and the surrounding data_definition rule does not contain any iteration of the data_type.

Can anybody help my explaining this?

[Sascha Ziemann]

Am Freitag, 17. Mai 2013 19:22:17 UTC+2 schrieb Sascha Ziemann:

data_type
    : BLOB
    | CHAR '(' INT ')'
    | CLOB
    | NUMBER
    | RAW '(' INT ')'
    ¦ SYS '.' ANYDATA
    | TIMESTAMP WITH LOCAL TIME ZONE
    | VARCHAR2 '(' INT ( CHAR | BYTE )? ')'
    ;

Sorry my fault. In front of the SYS is a broken pipe symbol.