[groovy-user] Questions about global AST transformations

[Daniel Henrique Alves Lima]

Hi, everybody.

Sorry for bother you with my newbie questions :-(
I'm using groovy 1.7.4 or 1.7.5.

1. Is there a rich guide (with a lot of examples) about global AST
transformations? I'm trying to use Groovy docs and some random pages
that i've found, but they aren't enough.

2. Is it known that if i try to use AstBuilder.buildFromString to
generate some code, this could trigger an infinite loop by indirect
recursion?
For example, suppose i've built a AST transformation that will replace
"println 'some_string'" by "println '123: some_string'". I can do this
changing the parameters of the original MethodCallExpression, but, for
illustration purpose, let's consider i will try to use the returned
value of AstBuilder.buildFromString("println '123: some_string'"). This
call you generate a new temporary Groovy class (i'm guessing here), this
Groovy class will be parsed/compiled and a new AST transformation will
occur before the first one get resumed.
I think this is the desirable behavior for the majority of the cases.
But is there an way of avoiding this using AstBuilder.buildFromString?
Or should i use another AstBuilder method instead?

3. Is there a rich documentation about AstBuilder.buildFromSpec usage?
I'm struggling with "methodCall" expression. I don't why, but some of
the examples that i've found don't seem to work. I think maybe i'm
missing something...

4. Is StatementReplacingVisitorSupport the best way of replacing method
call expressions inside another methods?

Thanks!

Best regards,

Daniel.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Hamlet D'Arcy]

Hi Daniel,

Sorry about the delay... hopefully these answers help.

>        1. Is there a rich guide (with a lot of examples) about global AST
> transformations? I'm trying to use Groovy docs and some random pages
> that i've found, but they aren't enough.

There are not too many documents about global annotations because
locals are much more common.
Be sure to check out the examples folder in Groovy. Otherwise, I don't
know of too many global examples.

>        2. Is it known that if i try to use AstBuilder.buildFromString to
> generate some code, this could trigger an infinite loop by indirect
> recursion?

No, this should not be a problem. No classes are generated.

> Or should i use another AstBuilder method instead?

You can always access the ASTNode subclasses directly.

>        3. Is there a rich documentation about AstBuilder.buildFromSpec usage?

There is! Check out the unit tests. I wrote these myself with the
intent that they would serve as documentation.
https://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/org/codehaus/groovy/ast/builder/AstBuilderFromSpecificationTest.groovy

>        4. Is StatementReplacingVisitorSupport the best way of replacing method
> call expressions inside another methods?

It is an implementation class. As such, I would not use it (as Peter
said earlier today)

--
Hamlet D'Arcy
haml...@gmail.com

[Daniel Henrique Alves Lima]

On Thu, 2011-01-13 at 18:04 +0100, Hamlet D'Arcy wrote:
> Hi Daniel,

Hi, Hamlet.

>
> Sorry about the delay...

No problem at all.

> hopefully these answers help.

I'm sure they will.

>
> > 2. Is it known that if i try to use AstBuilder.buildFromString to
> > generate some code, this could trigger an infinite loop by indirect
> > recursion?
>
> No, this should not be a problem. No classes are generated.

Hamlet, are you sure about this one? I'll try to write a sample class
later and mailing it to the list. BTW, i'm using Groovy 1.7.x.

[Daniel Henrique Alves Lima]

Try this one.

groovyc -cp /tmp/println_g_ast.jar Run.groovy

The only explanation that i've found was: .buildFromString("") is
creating a new Groovy class and this class is getting compiled. But i
can be doing something veeeeeeeeeeeeeeeeeeeeeeeeeeeery stupid :-)

JAVA_HOME=/usr/local/jdk1.6.0_20/
GROOVY_HOME=/usr/local/devel/groovy-1.7.4
GROOVY_HOME=/usr/local/devel/groovy-1.7.5

Thanks!

[Daniel Henrique Alves Lima]

Attached files

http://groovy.329449.n5.nabble.com/Questions-about-global-AST-transformations-tp3332519p3340529.html

On Thu, 2011-01-13 at 20:21 -0200, Daniel Henrique Alves Lima wrote:
> Try this one.
>
> groovyc -cp /tmp/println_g_ast.jar Run.groovy
>

---------------------------------------------------------------------

[Hamlet DArcy]

When I run this I get the following output:

visit(): this pack.PrintlnTransformation@d2e55e
visitClass(): node Run
visitMethod(): node MethodNode@32331490[void main([Ljava.lang.String;)]
visitMethod(): node MethodNode@15111830[java.lang.Object this$dist$invoke$2(java.lang.String, java.lang.Object)]
visitMethod(): node MethodNode@11045826[void this$dist$set$2(java.lang.String, java.lang.Object)]
visitMethod(): node MethodNode@24086409[java.lang.Object this$dist$get$2(java.lang.String)]
1 compilation error:

Exception thrown

org.codehaus.groovy.syntax.RuntimeParserException: Cannot use return statement with an expression on a method that returns void
. At [1:1] Run.groovy
at org.codehaus.groovy.classgen.AsmClassGenerator.throwException(AsmClassGenerator.java:513)
at org.codehaus.groovy.classgen.asm.StatementWriter.writeReturn(StatementWriter.java:569)

The printlns show that your global transformation was successfully called. CONGRATULATIONS! We are through the first hurdle :)

The exception gives a clue about what is wrong: "Cannot use return statement".

I ran this script in GroovyConsole to investigate:
import org.codehaus.groovy.ast.builder.AstBuilder;
def x = new AstBuilder().buildFromString("println('ok')");
x.each { println x.text}

The result shows the AST we produced:
[{ return this.println(ok) }]

So buildFromString("println('ok')") produces the code "return this.println('ok')".

This is almost what we want but not quite.

Your buildFromString should be:
def x = new AstBuilder().buildFromString("println('ok')");
x[0].statements[0].expression // this is a MethodCallExpression

x is a List<ASTNode> with one element. The First element is a BlockStatement, which has only one statement: a ReturnStatement, whose expression is a MethodCallExpression. For simple one-liners like this, AstBuilder is overkill, but for more advanced usages it is helpful.

--
Hamlet D'Arcy
hamlet...@canoo.com

[Daniel Henrique Alves Lima]

Hamlet, this is different from the output that i'm getting.

Check this out:

visit(): this pack.PrintlnTransformation@564ac216
visitClass(): node Run
visitMethod(): node MethodNode@454514340[void main([Ljava.lang.String;)]
visit(): this pack.PrintlnTransformation@e0d5eb7
visitClass(): node script1294994955229
visitMethod(): node MethodNode@264587158[void main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1625215216[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@5a20f443
visitClass(): node script1294994955262
visitMethod(): node MethodNode@696551663[void main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1010440244[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@a68cb6b
visitClass(): node script1294994955267

(...) a lot of visitClass later

visitMethod(): node MethodNode@380449691[java.lang.Object run()]
>>> a serious error occurred: null
>>> stacktrace:
java.lang.StackOverflowError
at java.lang.Exception.<init>(Exception.java:77)
at
java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:54)
at sun.reflect.GeneratedConstructorAccessor1.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at antlr.ASTFactory.create(ASTFactory.java:262)
at antlr.ASTFactory.create(ASTFactory.java:153)
at antlr.ASTFactory.create(ASTFactory.java:194)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.constant(GroovyRecognizer.java:13142)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.primaryExpression(GroovyRecognizer.java:11028)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.pathExpression(GroovyRecognizer.java:10920)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.postfixExpression(GroovyRecognizer.java:13087)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.unaryExpressionNotPlusMinus(GroovyRecognizer.java:13056)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.powerExpressionNotPlusMinus(GroovyRecognizer.java:12760)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.multiplicativeExpression(GroovyRecognizer.java:12692)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.additiveExpression(GroovyRecognizer.java:12362)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.shiftExpression(GroovyRecognizer.java:9886)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.relationalExpression(GroovyRecognizer.java:12267)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.equalityExpression(GroovyRecognizer.java:12191)
at
org.codehaus.groovy.antlr.parser.GroovyRecognizer.regexExpression(GroovyRecognizer.java:12139)

On Fri, 2011-01-14 at 09:40 +0100, Hamlet DArcy wrote:
> When I run this I get the following output:
>
>
> visit(): this pack.PrintlnTransformation@d2e55e
> visitClass(): node Run
> visitMethod(): node MethodNode@32331490[void main([Ljava.lang.String;)]
> visitMethod(): node MethodNode@15111830[java.lang.Object this$dist$invoke$2(java.lang.String, java.lang.Object)]
> visitMethod(): node MethodNode@11045826[void this$dist$set$2(java.lang.String, java.lang.Object)]
> visitMethod(): node MethodNode@24086409[java.lang.Object this$dist$get$2(java.lang.String)]
> 1 compilation error:
>
> Exception thrown

(...)

[Hamlet DArcy]

What is the exact command line you used to get this output?
Instead of groovyc, can you try just groovy.

Thanks,

--
Hamlet D'Arcy
hamlet...@canoo.com

[Daniel Henrique Alves Lima]

[daniel@any_host tmp]$ groovy --version
Groovy Version: 1.7.4 JVM: 1.6.0_20

The same happens to:

[daniel@any_host tmp]$ groovy --version
Groovy Version: 1.7.5 JVM: 1.6.0_20

On Fri, 2011-01-14 at 06:51 -0200, Daniel Henrique Alves Lima wrote:
> Hamlet, this is different from the output that i'm getting.
>
> Check this out:

[Daniel Henrique Alves Lima]

[daniel@any_host tmp]$ groovyc -cp /tmp/println_g_ast.jar Run.groovy

On Fri, 2011-01-14 at 06:55 -0200, Daniel Henrique Alves Lima wrote:
> [daniel@any_host tmp]$ groovy --version
> Groovy Version: 1.7.4 JVM: 1.6.0_20
>
> The same happens to:
>
> [daniel@any_host tmp]$ groovy --version
> Groovy Version: 1.7.5 JVM: 1.6.0_20
>
>
>

---------------------------------------------------------------------

[Hamlet DArcy]

What is the output when you run this:

groovy -cp /tmp/println_g_ast.jar Run.groovy

--
Hamlet D'Arcy
hamlet...@canoo.com

[Daniel Henrique Alves Lima]

[daniel@magnaopus tmp]$ groovy -cp /tmp/println_g_ast.jar Run.groovy
visit(): this pack.PrintlnTransformation@627a4489
visitClass(): node Run
visitMethod(): node MethodNode@984103443[void main([Ljava.lang.String;)]
visit(): this pack.PrintlnTransformation@5bf99eea
visitClass(): node script1294995677622
visitMethod(): node MethodNode@1630553042[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@666157527[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@67f31652
visitClass(): node script1294995677653
visitMethod(): node MethodNode@138319567[void main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1159656515[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@30e34726
visitClass(): node script1294995677658
visitMethod(): node MethodNode@424945885[void main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1682362920[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@74b957ea
visitClass(): node script1294995677663
visitMethod(): node MethodNode@876281732[void main([Ljava.lang.String;)]
visitMethod(): node MethodNode@224218598[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@38827968
visitClass(): node script1294995677668
visitMethod(): node MethodNode@2112927699[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1186250301[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@638bd7f1
visitClass(): node script1294995677671
visitMethod(): node MethodNode@1913537093[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@102824579[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@5adf48c4
visitClass(): node script1294995677673
visitMethod(): node MethodNode@2032298615[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1115416770[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@301db5ec
visitClass(): node script1294995677684
visitMethod(): node MethodNode@1855107489[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@1623980477[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@77546dbc
visitClass(): node script1294995677688
visitMethod(): node MethodNode@1206947544[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@36842446[java.lang.Object run()]
visit(): this pack.PrintlnTransformation@6da5db4b
visitClass(): node script1294995677691

(...) sometime in the near future

visitClass(): node script1294995678620
visitMethod(): node MethodNode@1309289016[void
main([Ljava.lang.String;)]
visitMethod(): node MethodNode@2103044750[java.lang.Object run()]
Caught: java.lang.StackOverflowError
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)
at
org.codehaus.groovy.ast.builder.AstStringCompiler.compile(AstStringCompiler.groovy:46)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy:100)
at
org.codehaus.groovy.ast.builder.AstBuilder.buildFromString(AstBuilder.groovy)