Optional productions in ply.yacc
eliben
In complex grammars I find myself writing dozens of productions of the
type:
def p_XXX_opt(self, p):
""" XXX_opt : empty
| XXX
"""
p[0] = p[1]
For many different XXXs.
It really gets tiresome, and I was wondering about the alternatives.
* Has anyone attempted some meta-programming magic to generate such
methods automatically from a list of productions?
* Dave, have you considered adding "optional" as a feature to PLY? It
would be really useful, not hard to implement, and the syntax is
really simple:
""" YYY : XXX?"""
The question mark can mean "XXX or empty", and in case of empty the
relevant item of the p[] array just gets None.
Eli
David Beazley
PLY-3.0 release might make it significantly easier to explore
such possibilities--namely because I refactored the yacc
internals to make different parts of it more self-contained.
It might be possible to build this kind of functionality into
an intermediate step.
I suppose one could revisit the question of rules that allow
repeats (e.g., TOKEN*) as well. Hmmm. Interesting.
Cheers,
Dave
On Mon 09/02/09 1:25 PM , eliben eli...@gmail.com sent:
>
> Hello,
>
> In complex grammars I find myself writing dozens of productions of the
> type:
>
> def p_XXX_opt(self, p):
> """ XXX_opt : empty
> | XXX
> """
> p[0] = p[1]
>
> For many different XXXs.
> It really gets tiresome, and I was wondering about the alternatives.
>
> * Has anyone attempted some meta-programming magic to generate such
> methods automatically from a list of productions?
> * Dave, have you considered adding "optional" as a feature to
> PLY? Itwould be really useful, not hard to implement, and the syntax is
> really simple:
>
> """ YYY : XXX?"""
>
> The question mark can mean "XXX or empty", and in case of empty
> therelevant item of the p[] array just gets None.
>
> Eli
>
> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups
> "ply-hack" group.To post to this group, send email to ply
> -h...@googlegroups.comTo unsubscribe from this group, send email to ply-hack+
> unsub...@googlegroups.comFor more options, visit this group at http://groups.google.com/group/ply-hack?hl=en-~----------~----~----~----
~------~----~------~--~---
>
>
>
>
eliben
David Beazley wrote:
> I haven't really spent much time thinking about this, but the
> PLY-3.0 release might make it significantly easier to explore
> such possibilities--namely because I refactored the yacc
> internals to make different parts of it more self-contained.
> It might be possible to build this kind of functionality into
> an intermediate step.
>
> I suppose one could revisit the question of rules that allow
> repeats (e.g., TOKEN*) as well. Hmmm. Interesting.
>
of subproductions (i.e. p[N] where N is the index of a subproduction
with * is a list of length 0, 1 or more).
This together with ? would considerably reduce the amount of
boilerplate code currently required for complex PLY grammars.
Eli
D.Hendriks (Dennis)
If you add TOKEN* (0 or more) and TOKEN? (0 or 1), you could also
add TOKEN+ (1 or more). Furthermore, brackets would be nice.
For instance: (A B)* would be 0 or more times 'A followed by B'.
Dennis
eliben
class PLYParser(object):
def _create_opt_rule(self, rulename):
""" Given a rule name, creates an optional ply.yacc rule
for it. The name of the optional rule is
<rulename>_opt
"""
optname = rulename + '_opt'
def optrule(self, p):
optrule.__name__ = 'p_%s' % optname
setattr(self.__class__, optrule.__name__, optrule)
if __name__ == '__main__':
pp = PLYParser()
pp._create_opt_rule('java')
ar = [4, 6]
pp.p_java_opt(ar)
print ar
print pp.p_java_opt.__doc__
print dir(pp)
---------------------
._create_opt_rule creates a ply.yacc valid rule, with a correct
docstring.
Tried it in my parsing code (using PLY 2.5) - it works, and saves
quite a bit of boilerplate code. So I'm putting it here in the hope
that it will be useful.
Dave - adding rule? and rule* to PLY would still be much cooler :-)
Eli
A.T.Hofkamp
> I've ended up with this concoction:
>
> class PLYParser(object):
> def _create_opt_rule(self, rulename):
> """ Given a rule name, creates an optional ply.yacc rule
> for it. The name of the optional rule is
> <rulename>_opt
> """
> optname = rulename + '_opt'
>
> def optrule(self, p):
> p[0] = p[1]
>
> optrule.__doc__ = '%s : empty\n| %s' % (optname, rulename)
> optrule.__name__ = 'p_%s' % optname
> setattr(self.__class__, optrule.__name__, optrule)
Can you explain what 'empty' is?
The usual way to write optional rules is
X_opt :
| X
In this situation, 'p[0] = p[1]' will fail in the case of the empty alternative.
Before we jump to implementation, it may be good to first define what we
should be able to express, and how to do that.
The one thing that bothers me with these extensions, is that we may loose the
power to express all cases explicitly.
In the case of 'X?', how does one distinguish between both cases?
In particular when 'None' is a legal associated value for X?
Sincerely,
Albert
eliben
>
> The usual way to write optional rules is
>
> X_opt :
> | X
>
> In this situation, 'p[0] = p[1]' will fail in the case of the empty alternative.
>
def p_empty(self, p):
'empty : '
p[0] = None
It's recommended for use by Dave in the PLY documentation.
Eli