PGE now supports grammars, built-in rules

[Patrick R. Michaud]

I've just checked in changes to PGE that enable it to support
grammars, as well as some more built-in rules (<alpha>,
<digit>, <alnum>, <ident>, etc.). To create a new grammar,
just create a subclass of "PGE::Rule" and install new rules into
the new grammar's namespace:

.sub main @MAIN
load_bytecode "PGE.pbc"
.local pmc p6rule
p6rule = find_global "PGE", "p6rule"

$P0 = getclass "PGE::Rule"
$P1 = subclass $P0, "MyClass"

# create a custom MyClass::alpha rule
$P0 = p6rule("<[abcde]>")
store_global "MyClass", "alpha", $P0

$P0 = p6rule("^ <MyClass::ident>")
$P1 = $P0("ab23") # matches "ab23"
$I0 = $P1.__get_bool()
print $I0
print "\n"

$P1 = $P0("zz23") # doesn't match zz23
$I0 = $P1.__get_bool()
print $I0
print "\n"
.end

I've added some more tests for the above, but many many more
are needed. If anyone wants to request (or better yet, submit
patches for) other of the default built-in rules, I'll be glad
to add them.

Pm

[Allison Randal]

On Jun 25, 2005, at 20:38, Patrick R. Michaud wrote:

> I've just checked in changes to PGE that enable it to support

> grammars, as well as some more built-in rules...

These are totally awesome. On the plane today, I converted the Punie
parser over to use the shiny new grammars. My one frustration so far is
that subrule calls for a rule from a grammar always have to be fully
specified, even if the subrule is in the same grammar. It makes for
some verbose constructs:

p6rule("1", "PunieGrammar", "term")
p6rule("print \s* <PunieGrammar::expr>", "PunieGrammar", "gprint")
p6rule("<PunieGrammar::gprint> | <PunieGrammar::term>",
"PunieGrammar", "expr")
p6rule("<PunieGrammar::expr>;", "PunieGrammar", "line")

instead of:

p6rule("1", "PunieGrammar", "term")
p6rule("print \s* <expr>", "PunieGrammar", "gprint")
p6rule("<gprint> | <term>", "PunieGrammar", "expr")
p6rule("<expr>;", "PunieGrammar", "line")

Early days yet, so this is mainly saying "This is something I'm looking
forward to."

Thanks, Patrick!
Allison

[Patrick R. Michaud]

On Thu, Jun 30, 2005 at 11:46:55PM -0400, Allison Randal wrote:
> On Jun 25, 2005, at 20:38, Patrick R. Michaud wrote:
>
> >I've just checked in changes to PGE that enable it to support
> >grammars, as well as some more built-in rules...
>
> These are totally awesome. On the plane today, I converted the Punie
> parser over to use the shiny new grammars. My one frustration so far is
> that subrule calls for a rule from a grammar always have to be fully
> specified, even if the subrule is in the same grammar.

Well, it's not *supposed* to work that way -- it's supposed to work
the way you expect it to -- i.e., "<expr>" in a rule calls the the "expr"
rule in the current grammar.

> p6rule("1", "PunieGrammar", "term")
> p6rule("print \s* <expr>", "PunieGrammar", "gprint")
> p6rule("<gprint> | <term>", "PunieGrammar", "expr")
> p6rule("<expr>;", "PunieGrammar", "line")

It seems to work for me (r8486)...

[pmichaud@localhost pge]$ cat punie.pir
.sub main @MAIN
.local pmc p6rule
.local pmc myrule
.local pmc match

load_bytecode 'PGE.pir'

p6rule = find_global "PGE", "p6rule"

p6rule("1", "PunieGrammar", "term")

p6rule("print \s* <expr>", "PunieGrammar", "gprint")
p6rule("<gprint> | <term>", "PunieGrammar", "expr")
p6rule("<expr>;", "PunieGrammar", "line")

myrule = p6rule("<PunieGrammar::line>")

match = myrule("print print 1;")
match."dump"()
.end
$ parrot punie.pir
: <print print 1; @ 0> 0
<PunieGrammar::line>: <print print 1; @ 0> 0
<PunieGrammar::line><expr>: <print print 1 @ 0> 0
<PunieGrammar::line><expr><gprint>: <print print 1 @ 0> 0
<PunieGrammar::line><expr><gprint><expr>: <print 1 @ 6> 0
<PunieGrammar::line><expr><gprint><expr><gprint>: <print 1 @ 6> 0
<PunieGrammar::line><expr><gprint><expr><gprint><expr>: <1 @ 12> 1
<PunieGrammar::line><expr><gprint><expr><gprint><expr><term>: <1 @ 12> 1
$

Perhaps you can send me a copy of your code so I can take a look...?

Pm

[Will Coleda]

When I was compiling the rules, they weren't going where I thought
they were going. This was because I stole code from compilers/pge/
demo.pir improperly; there is a gname variable in there that's a PMC.
If you change it to a string instead, it seems to work fine.
Apparently as a PMC it was installing my rules in the wrong location.
(My clue was your sample, which was calling the p6rule_compile with a
literal string to describe the parent grammar, not a PMC as in
demo.pir).

Of course, now I don't know why demo.pir works, but it does. =-)

Regards.

On Jul 3, 2005, at 9:11 AM, Will Coleda wrote:

> I'm having a similar problem trying to integrate this in partcl
>
> Haven't been able to come up with a short test case yet, but i
> suspect that it has something to do with different namespaces in
> the original compile vs. when it's invoked.
>
> I'll try to come up with a test case that doesn't involve all of
> partcl.
>
> As a side note, it'd be nice if the code that demo.pir uses to read
> in a perl6 rules file was part of PGE's interface.
>

[Patrick R. Michaud]

On Sun, Jul 03, 2005 at 11:08:56AM -0400, Will Coleda wrote:
> When I was compiling the rules, they weren't going where I thought
> they were going. This was because I stole code from compilers/pge/
> demo.pir improperly; there is a gname variable in there that's a PMC.

No, it's a string:

[pmichaud@localhost pge]$ grep gname demo.pir
.local string gname
p6rule_compile($S2, gname, $S1)
print gname
gname = $P9
[pmichaud@localhost pge]$

> (My clue was your sample, which was calling the p6rule_compile with a
> literal string to describe the parent grammar, not a PMC as in
> demo.pir).
>
> Of course, now I don't know why demo.pir works, but it does. =-)

Well, it works because (AFAICT) gname is in fact a string. :-)
Maybe there's something else bizarre going on here?

> >As a side note, it'd be nice if the code that demo.pir uses to read
> >in a perl6 rules file was part of PGE's interface.

I considered adding this at one point and then decided it might be
a little too much creeping featurism. But I can see how it'd be
very useful, so I'll see about re-incorporating it.

Pm