ANTLR 4 SQL Grammar

4,748 views
Skip to first unread message

cdu...@twitter.com

unread,
Mar 7, 2014, 4:45:51 PM3/7/14
to antlr-di...@googlegroups.com
Hey guys,

Does anyone know if there's a working ANTLR 4 SQL grammar out there somewhere? I know of the antlr/grammars-v4 repository where all v4 grammars live, but I'm crossing my fingers and hoping someone else out there has come across one and is willing to share it. The particular flavor of SQL isn't that important - Postgres, MySQL, TransactSQL, it's all good.

I've found a few v3 SQL grammars but they all fall short of what I need. One completely ignores JOINs, the other doesn't generate an actual tree, just a stream of tokens.

Any help you can offer is much appreciated.

-Cameron

Sam Harwell

unread,
Mar 7, 2014, 7:18:53 PM3/7/14
to antlr-di...@googlegroups.com

Hi Cameron,

 

The grammar that doesn’t generate a tree may actually work for you. ANTLR 4 doesn’t support ASTs like ANTLR 3 did; instead it automatically produces parse trees that match the shape defined by the rules in the grammar. During the conversion to ANTLR 4, it’s actually necessary to remove the AST operators and rewrite rules from ANTLR 3 grammars.

 

For example, consider commit aacf1d4 in my example conversion of an application from ANTLR 3 to ANTLR 4.

 

Thanks,

Sam

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Terence Parr

unread,
Mar 7, 2014, 7:24:29 PM3/7/14
to antlr-di...@googlegroups.com
I would love to have a working SQL grammar available and I would be willing to help, but it’s a pretty big job I think. my biggest problem was I couldn’t figure out where to get appropriate sample input for the dialect of SQL I was building a grammar for. Actually it would not be too bad if you had a reference manual for a particular SQL and valid sample inputs. The splintering of the SQL world is maddening.

unfortunately we do not have one as of yet…

Ter

Bence Erős

unread,
Mar 13, 2014, 2:43:39 PM3/13/14
to antlr-di...@googlegroups.com
Hello,

do you think it would be a reasonable way to create a .g4 file based on the SQL 2003 standard BNF ?

There may be 2 problems with it:
- it would mean omitting the offset and limit clauses, since those are not the part of the standard. It is a bad point, since both mysql and postgres implements them (and maybe more)
- still no sample input (as Terence pointed out)


regards,


2014-03-08 1:24 GMT+01:00 Terence Parr <pa...@cs.usfca.edu>:
I would love to have a working SQL grammar available and I would be willing to help, but it's a pretty big job I think. my biggest problem was I couldn't figure out where to get appropriate sample input for the dialect of SQL I was building a grammar for.  Actually it would not be too bad if you had a reference manual for a particular SQL and valid sample inputs.  The splintering of the SQL world is maddening.

unfortunately we do not have one as of yet...


Ter
On Mar 7, 2014, at 1:45 PM, cdu...@twitter.com wrote:

> Hey guys,
>
> Does anyone know if there's a working ANTLR 4 SQL grammar out there somewhere? I know of the antlr/grammars-v4 repository where all v4 grammars live, but I'm crossing my fingers and hoping someone else out there has come across one and is willing to share it. The particular flavor of SQL isn't that important - Postgres, MySQL, TransactSQL, it's all good.
>
> I've found a few v3 SQL grammars but they all fall short of what I need. One completely ignores JOINs, the other doesn't generate an actual tree, just a stream of tokens.
>
> Any help you can offer is much appreciated.
>
> -Cameron
>
> --
> You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Bence Erős
CyclonePHP core developer

Cameron Dutro

unread,
Mar 13, 2014, 2:56:25 PM3/13/14
to antlr-di...@googlegroups.com
Hey guys,

Thanks for all the input here everyone! I actually found an ANTLR v4 SQL grammar (separate lexer and parser) as part of the Apache Tajo project available here. I was able to successfully compile it into java code, which I published on Github.  I cannot personally vouch for the accuracy or completeness of this parser, but it works flawlessly for the use cases I've thrown at it so far. Please don't ask me technical questions about the parser because frankly I won't know. I have very little knowledge of how ANTLR actually works, and this grammar is freaking complicated anyway :)

-Cameron


--
You received this message because you are subscribed to a topic in the Google Groups "antlr-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/antlr-discussion/TSBqzfLsBQg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to antlr-discussi...@googlegroups.com.

David Whitten

unread,
Mar 13, 2014, 11:15:09 PM3/13/14
to antlr-di...@googlegroups.com
you can find all source code with a g4 extension on github.
You have to search for antlr because it won't do an attribute only search.
Currently (as of 5:30 EST on 13-Mar-2014) there are 312 such files.

David Whitten

unread,
Mar 14, 2014, 11:52:01 AM3/14/14
to antlr-di...@googlegroups.com
you can find all source code with a g4 extension on github.
You have to search for antlr because it won't do an attribute only search.
Currently (as of 5:30 EST on 13-Mar-2014) there are 312 such files.
On Thu, Mar 13, 2014 at 2:56 PM, Cameron Dutro <cdu...@twitter.com> wrote:

Terence Parr

unread,
Mar 21, 2014, 12:38:11 PM3/21/14
to antlr-di...@googlegroups.com
 hi guys, sorry for the delay. I just received word that salesforce.com will be releasing their SQL grammar, hopefully soon.  maybe we can even get some sample input out of them

Ter


On Thu, Mar 13, 2014 at 11:43 AM, Bence Erős <eben...@gmail.com> wrote:



--
Dictation in use. Please excuse homophones, malapropisms, and nonsense. 
Reply all
Reply to author
Forward
0 new messages