Getting back to the DSL

118 views
Skip to first unread message

Tarek Ziadé

unread,
Sep 27, 2011, 6:44:23 AM9/27/11
to ply-...@googlegroups.com
Hey

I am happy with my DSL, I can yacc.parse() it it and create an AST in memory.

I am now trying to add a feature where, for a given AST, I can
generate the corresponding portion of DSL.

The AST I've created in memory is a simple tuple for every expression
I am visiting, and I was wondering what is the best way to do the
revert operation.

The problem I am facing is that I don't want to maintain on one side a
grammar to parse my DSL, and on the other side a function to parse the
AST and do the revert operation.

I was wondering how this is usually solved, or if PLY provides some
helpers for this

Thanks
Tarek

--
Tarek Ziadé | http://ziade.org

Michael Foord

unread,
Sep 27, 2011, 7:29:30 AM9/27/11
to ply-...@googlegroups.com

If the AST contains (stores) the original tokens the collapsing an AST back to "code" should be trivial. At resolver systems we used PLY to rewrite a custom language to Python via AST transformations. Each AST node had a "collapse" method (recursive) that would return a string representing the node.

Having parsed you could get code back by calling collapse on the top level node.

All the best,

Michael Foord




 

Thanks
Tarek

--
Tarek Ziadé | http://ziade.org

--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
To post to this group, send email to ply-...@googlegroups.com.
To unsubscribe from this group, send email to ply-hack+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.




--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Tarek Ziadé

unread,
Sep 27, 2011, 8:01:06 AM9/27/11
to ply-...@googlegroups.com
On Tue, Sep 27, 2011 at 1:29 PM, Michael Foord <fuzz...@gmail.com> wrote:
..

>
> If the AST contains (stores) the original tokens the collapsing an AST back
> to "code" should be trivial. At resolver systems we used PLY to rewrite a
> custom language to Python via AST transformations. Each AST node had a
> "collapse" method (recursive) that would return a string representing the
> node.
>
> Having parsed you could get code back by calling collapse on the top level
> node.

That sounds like the best idea. I'll try this.

Do you happen to have a public example ?

Cheers

--
Tarek Ziadé | http://ziade.org

David Beazley

unread,
Sep 27, 2011, 8:18:10 AM9/27/11
to ply-...@googlegroups.com, David Beazley
I think you just do a depth-first traversal of the parse tree. For example something sort of like this (shown for a node representing a binary operator):

class BinaryOperator(object):
def __init__(self, operator, left, right):
self.operator = operator
self.left = left
self.right = right
def collapse(self):
return self.left.collapse() + self.operator + self.right.collapse()

Note: You'll need to careful about too much recursion depending on the depth of your parse tree.

Cheers,
Dave

Tarek Ziadé

unread,
Sep 27, 2011, 8:23:58 AM9/27/11
to ply-...@googlegroups.com
On Tue, Sep 27, 2011 at 2:18 PM, David Beazley <da...@dabeaz.com> wrote:
> I think you just do a depth-first traversal of the parse tree.   For example something sort of like this (shown for a node representing a binary operator):
>
> class BinaryOperator(object):
>      def __init__(self, operator, left, right):
>              self.operator = operator
>              self.left = left
>              self.right  = right
>      def collapse(self):
>              return self.left.collapse() + self.operator + self.right.collapse()
>
> Note:  You'll need to careful about too much recursion depending on the depth of your parse tree.

Thanks, that's useful

Cheers

Michael Foord

unread,
Sep 27, 2011, 8:49:39 AM9/27/11
to ply-...@googlegroups.com

You may want your grammar to preserve whitespace in the tokens (rather than discarding) - unless you're happy to lose / change whitespace when round tripping.

Michael
 

Cheers
Tarek
--
Tarek Ziadé | http://ziade.org

--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
To post to this group, send email to ply-...@googlegroups.com.
To unsubscribe from this group, send email to ply-hack+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.

Tarek Ziadé

unread,
Sep 27, 2011, 8:53:59 AM9/27/11
to ply-...@googlegroups.com
On Tue, Sep 27, 2011 at 2:49 PM, Michael Foord <fuzz...@gmail.com> wrote:
...

>
> You may want your grammar to preserve whitespace in the tokens (rather than
> discarding) - unless you're happy to lose / change whitespace when round
> tripping.

Yeah ideally I'd like the round trip to look similar, including the
level indentation.

The use case is a web editor where you can change pieces of an already
loaded AST, and have the DSL rendered back and forth.

If the first sync changes the initial text after the first
tokenization, it's fine I guess.

--
Tarek Ziadé | http://ziade.org

Reply all
Reply to author
Forward
0 new messages