Re: PyParsing
Andi Albrecht
IMO after a (not too extensive) pyparsing experiment at this time
would show if it's an option to rewrite the engine using pyparsing.
Besides basic SQL statements it would be nice to see how it could
handle some edge cases where the current parser has problems. Some of
the are in the tests, some as examples in issues (and somewhere I've
got a "SQL of death" with which the parser as biiiig troubles *gg*).
--
Andi
On Fri, Jan 20, 2012 at 11:23 PM, pir...@gmail.com <pir...@gmail.com> wrote:
> It's true... What do you think would be better, give PyParser a test, try to
> improve the current parser, or write a new one?
>
> Sended from my Android cell phone, please sorry the lack of format on the
> text and my fat thumbs :-P
>
> El 20/01/2012 22:11, "Andi Albrecht" <albrec...@googlemail.com>
> escribió:
>
>> On Fri, Jan 20, 2012 at 4:01 PM, pir...@gmail.com <pir...@gmail.com>
>> wrote:
>> > I know it's non validating and i don't want it, it's just a little bit
>> > more
>> > intelligence on the parser for example like on the issue i have submit
>> > :-D
>>
>> Would be interesting to give it a quick try to see which problems
>> occur when trying to implement a (more or less) flexible parser using
>> pyparsing.
>>
>> >
>> > Sended from my Android cell phone, please sorry the lack of format on
>> > the
>> > text and my fat thumbs :-P
>> >
>> > El 20/01/2012 14:53, "Andi Albrecht" <albrec...@googlemail.com>
>> > escribió:
>> >
>> >> Hi,
>> >>
>> >> I'm aware of pyparsing and considered it as a base for sqlparse. But
>> >> as you've already notices the problem is the SQL compliance. sqlparse
>> >> is non-validating and it is one of it's features to parse even invalid
>> >> SQL statements.
>> >>
>> >> Ideally sqlparse ends up with a tree structure (which isn't
>> >> implemented very well yet). But it also can end up with a partial tree
>> >> structure.
>> >>
>> >> Andi
>> >>
>> >> On Fri, Jan 20, 2012 at 12:15 PM, pir...@gmail.com <pir...@gmail.com>
>> >> wrote:
>> >> > Hi Andi, i have found recently the library PyParsing that is intented
>> >> > to develop parsers in the same way that lex and i thought it would be
>> >> > interesting to improve tree generation of sqlparse... In fact, one of
>> >> > the examples is regarding a SELECT parser. What do you think? I'm
>> >> > having some problems with the SQL trees in my filesystem project and
>> >> > i
>> >> > don't know if it would be a good idea to use this library or try to
>> >> > debug and improve the current implementation...
>> >> >
>> >> > http://pyparsing.wikispaces.com/
>> >> > http://pyparsing.wikispaces.com/file/view/select_parser.py
>> >> >
>> >> > On the other way, they make a reference to the sqlite language
>> >> > specification for develop the select parser, it would be a good idea
>> >> > to use it as basis since it's trying to be SQL-92 compliant... :-)
>> >> >
>> >> > http://www.sqlite.org/lang_select.html
>> >> >
>> >> > --
>> >> > "Si quieres viajar alrededor del mundo y ser invitado a hablar en un
>> >> > monton de sitios diferentes, simplemente escribe un sistema operativo
>> >> > Unix."
>> >> > – Linus Tordvals, creador del sistema operativo Linux
pir...@gmail.com
>
Ok.
> IMO after a (not too extensive) pyparsing experiment at this time
> would show if it's an option to rewrite the engine using pyparsing.
> Besides basic SQL statements it would be nice to see how it could
> handle some edge cases where the current parser has problems. Some of
> the are in the tests, some as examples in issues (and somewhere I've
> got a "SQL of death" with which the parser as biiiig troubles *gg*).
Althought it was my idea i don't know if it would be a good idea to do
a full rewrite using PyParsing, or just a clean-up and optimization.
The most important thing is define the grammar, and this would maybe
lead to make sqlparse a little bit less non-validating... This is not
mandatory since the basic gramar structures are the same on all
dialects, it only depends on if we want to parse somewhat random sql
snippets (that it would only check for little more than keywords...)
or if we would assume to some extend that are being used full (not
necesary valid) sql statements...
P.D.: Andi, i have push a fix for the issue i post the other day, take
a look ;-)