BNF grammar for Delphi
Ralph Oudejans
Is there anyone who knows where I might find a BNF grammar for Delphi?
TIA,
Ralph Oudejans
Ira D. Baxter
See http://www.semdesigns.com/Products/DMS/DMSToolkit.html.
It also provides an AST builder, a transformation engine,
an attribute evaluator for computing analysis, and a prettyprinter.
Works for/with other languages, too, even at the same time.
--
Ira Baxter, Ph.D., CTO idba...@semdesigns.com 512-250-1018x140
Semantic Designs, Inc., www.semdesigns.com FAX 512-250-1191
12636 Research Blvd #C214, Austin, Texas 78759
Ralph Oudejans <R.Oud...@firstresult.com> wrote in message
news:8gtsfq$d7g$1...@spectra.a2000.nl...
Bruce Roberts
"Ralph Oudejans" <R.Oud...@firstresult.com> wrote in message
news:8gtsfq$d7g$1...@spectra.a2000.nl...
> Hello,
>
> Is there anyone who knows where I might find a BNF grammar for Delphi?
I've always thought of BNF as a syntax notation. What's a BNF grammar?
>
> TIA,
>
>
> Ralph Oudejans
>
>
Stefan Franke
I think it is obvious that he meant "an Object Pascal grammar definition
in a BNF-like notation".
Ralph,
I guess you won't have much luck with this. Some time ago I did an extensive
search for a Delphi grammar in various news and list archives. I wasn't able to
locate anything complete or non-commercial.
The grammar provided in the built-in help is far from being complete and full of
errors (e.g. exception handling is missing completely). Trying to use this as a
starting point really can drive you nuts (see below).
IMHO it looks more or less like a two-days effort of an underemployed student
trainee at Borland's starting from an ISO Pascal grammar and trying to add
Object Pascal's features from the hearsay ;-)
I'm rather convinced that even the Borland guys have no formal definition of
their language -- the compiler is the definition. It performs a hand-tuned mix
of what is traditionally called scanning and parsing, always with greatest efforts
to remain being parsable in one pass.
Note for example that you cannot tokenize Delphi sources without context
information: If the expression ^i is a pointer to a type named i in
var v: ^i;
or a control sequence inside a string literal in
const s: string = ^i;
is undecidable at the token level. The same applies to identifiers in traditional
Pascal -- the usual [a-zA-Z_] [a-zA-Z0-9_]* regex -- and identifiers in an
OLE automation context which may contain any wide-char characters, like
var v: variant;
v.überkráß();
(with lots of european special characters).
* * *
Commercial offerings I found so far:
1. http://www.semdesigns.com/Products/DMS/DMSToolkit.html. (see the other reply)
I didn't not find anything I could have a look at on their website.
2. Cocolsoft (http://www.cocolsoft.com.au) sell a parser for Delphi. They claim that it
is even possible to independently verify Object Pascal sources with it and offer a test
program to download but no grammer definition (of course -- that's their capital).
As far as I can tell from the test app, the parser is OK for usual 'nice' source code
but does only approximate Delphi's grammar when it comes to details. During various
tests I constructed a really ill-formed source file which is nonetheless perfectly parsed by
Delphi 4, but at which Cocolsoft's parser stops after 30 (different) syntax errors having
read 1/3 of the file.
* * *
Finally, I decided to construct my own Object Pascal grammar -- at least for interface
definitions -- and started from the help file 'grammar', naively thinking it could be done
within two or three days.
I was (and still am) planning to write an interface generator (like SWIG -- see www.swig.org)
to be able to use scripting languages (especially the unbelievably clever Python language
-- www.python.org) together with Delphi without having to create the usual wrapper functions
and shadow classes by hand, but instead mapping them to the facilities to 'Python for Delphi'
framework provides (http://www.multimania.com/marat).
After approx. two weeks rewriting 80% of the original grammar I came up with a parser
that accepted all .pas/.dpr/.dpk files in the full Delphi Pro distro along with our
company's DirectX based animation toolkit containing around 9 MB of own and foreign
sources -- except those files with one of the two scanner ambiguities mentioned above
(which are rare, luckily).
The big drawback is that my parser is very slow, consumes unjustifiable amounts of memory
and is written using a quite esoteric parsing toolkit (SPARK for Python --
http://csr.uvic.ca/~aycock/python/) that utilizes a very general context-free parsing
algorithm (Early's algo, to be precise). That means it would still be a significant amount of work
to rewrite the grammar far enough to be usable as an input for, say, common LALR ot LR(n)
toolkits. My intention was first to get the language right and optimize later.
In the moment I had interrupted this project, mainly due to job-related involvement, but partly
because I'm re-assessing the Python/Delphi integration relying more on M$'s COM. I'm
hesitating to release my stuff as open source in that stadium, but that's what I originally
intended.
Ideally, I would either find a company that has a strong enough interest in a Delphi parser
to pay me for programming a YACC/BISON/ANTLR/whatever-based parser (preferably
still allowing the whole thing or the interface section parts to be open-sourced), or get in
contact with a compiler guy who is willing and able to do continue this project or wants
to collaborate.
Suggestions welcome.
Stefan Franke
www.meso.net
Bruce Roberts
"Stefan Franke" <spamf...@bigfoot.de> wrote in message
news:3934184...@news.btx.dtag.de...
> On Tue, 30 May 2000 00:19:19 -0400, "Bruce Roberts"
<no.junk.p...@attcanada.net> wrote:
>
> >"Ralph Oudejans" <R.Oud...@firstresult.com> wrote in message
> >news:8gtsfq$d7g$1...@spectra.a2000.nl...
> >> Hello,
> >>
> >> Is there anyone who knows where I might find a BNF grammar for Delphi?
> >
> >I've always thought of BNF as a syntax notation. What's a BNF grammar?
>
> I think it is obvious that he meant "an Object Pascal grammar definition
> in a BNF-like notation".
>
>
The words are obvious. What I was curious about is how one uses BNF to
express a grammar. Obviously it can be used to express syntax.
I haven't actually verified it, but I think the OPLG railroad diagrams cover
all of the language syntax. Converting those to BNF should only be a matter
of a few hours.
Ralph Oudejans
I'll try the OPLG railroad diagrams as a starting point for specifying the
Object Pascal syntax.
- Ralph
Ira D. Baxter
<no.junk.p...@attcanada.net> wrote:
>
> >"Ralph Oudejans" <R.Oud...@firstresult.com> wrote in message
> >news:8gtsfq$d7g$1...@spectra.a2000.nl...
> >> Hello,
> >>
> >> Is there anyone who knows where I might find a BNF grammar for Delphi?
>
> I guess you won't have much luck with this. Some time ago I did an
extensive
> search for a Delphi grammar in various news and list archives. I wasn't
able to
> locate anything complete or non-commercial.
>
> The grammar provided in the built-in help is far from being complete and
full of
> errors (e.g. exception handling is missing completely). Trying to use this
as a
> starting point really can drive you nuts (see below).
Agreed. We did use it as a starting point, and it did drive us nuts.
But then, that's why we're a commercial company.
> I'm rather convinced that even the Borland guys have no formal definition
of
> their language -- the compiler is the definition.
This is pretty typical for non-Standardized languages (if you think
ObjectPascal
is bad, you should try VisualBasic.
The same applies to identifiers in traditional
> Pascal -- the usual [a-zA-Z_] [a-zA-Z0-9_]* regex -- and identifiers in an
> OLE automation context which may contain any wide-char characters, like
>
> var v: variant;
> v.überkráß();
>
> (with lots of european special characters).
Our tools are UNICODE based, so we tend to do this kind of thing easily.
> Commercial offerings I found so far:
>
> 1. http://www.semdesigns.com/Products/DMS/DMSToolkit.html.
That's us.
> Finally, I decided to construct my own Object Pascal grammar -- at least
for interface
> definitions -- and started from the help file 'grammar', naively thinking
it could be done
> within two or three days.
>
> After approx. two weeks rewriting 80% of the original grammar
>
> The big drawback is that my parser is very slow, consumes unjustifiable
amounts of memory
> and is written using a quite esoteric parsing toolkit (SPARK for Python --
> http://csr.uvic.ca/~aycock/python/) that utilizes a very general
context-free parsing
> algorithm (Early's algo, to be precise). That means it would still be a
significant amount of work
> to rewrite the grammar far enough to be usable as an input for, say,
common LALR ot LR(n)
> toolkits.
We use GLR ("Tomita") parsing techniques, which, like Early's algorigthm,
can parse
any context-free language, but in practice, runs in linear time on the 99.5%
of a BNF
which is LALR(1). (It can go exponential on the other .5%, but in most
language
instances, these regions are extremely short and so you don't notice or
care.)
This gives us quite a nice, fast parser for the full language.
The DMS Toolkit does more than parse the language. It also automatically
builds
ASTs, capturing comments, converting literal values to native binary
formats,
prettyprints, and can carry out source-to-source transforms. And, it works
with other languages too.
VBDis
<no.junk.p...@attcanada.net> schreibt:
>I haven't actually verified it, but I think the OPLG railroad diagrams cover
>all of the language syntax. Converting those to BNF should only be a matter
>of a few hours.
The Case statement, in the current specification with the Else part, cannot be
converted into a railroad diagram. Also you should prepare for many cases,
where a ';' is allowed, but not mandatory.
Unfortunately I don't remember the URL, where the solutions from the previous
discussion of the grammar reside. Perhaps in the homepage of
r...@stud.mw.tu-muenchen.de (Robert Zierer).
DoDi
P.S.: I have some grammar related references in my new homepage
<www.homepages.compuserve.de/DrDiettrich/Refs>.