[groovy-user] Intercept/monitor statements in a groovy script

[Magnus Rundberget]

Hi !

I'm working on a plugin for Light Table (https://github.com/rundis/LightTable-Groovy). One thing Light Table tries to address is providing developers with more value than current editors do. One particular area is reducing the need for log/printlns for debugging purposes. One of the ways to address that is to try and devise clever ways to show results inline alongside code, where possible even how values flow through the code/methods/functions/statements. The clojure plugin for Light Table manages to do some pretty funky stuff, and I'm curious about what would be possible to do with some groovy magic :-)

So far I've done some really basic stuff using GroovyShell to get a return value, whatever is printed to std out  and retrieve and binding variables set. Its a start, but certainly not kicking butt quite yet.

1) Would it be possible to intercept each statement of a groovy script  to extract information about:

- assignments (name and value set) 

- method-calls/function-calls with param names, values and returnvalue

- ... with reference to line number (and line col where applicable) from the originating script-file/text

2)  Which avenue would quickly get me further than I am, and which avenue would take me to new heights in the long term : (interceptors? mop funkyness ? ast ?) 

3) Any suggestions on where I can find inspiration in terms of implementation ? OpenSource projects, the groovy console code  ?

Let me provide a fairly simple but still interesting example:

1: def myDouble(x) { it * 2}

2: x = 4

3: y = 6

4: z = x + y

5: a = myDouble([x, y, z].sum()) 

6: if(new Date()[Calendar.MONTH] % 2 == 0) { 

7:     a = a + 1

8: }

- Step 1: Show value of assignments. This alone would be great and a huge step forward for me !

1: def myDouble(x) { it * 2} 

2: x = 4 // -> 4 

3: y = 6 //  -> 6

4: z = x + y // -> 10

5: a = myDouble([x, y, z].sum()) // -> 40

6: if(new Date()[Calendar.MONTH] % 2 == 0) { 

7:     a = a + 1 // -> 41

8: }

- Step 2: Show values of variables used in expressions: lines 4, 5 and 6

- Step 3: Show input param(s) and return value for last invocation of function defined on line 1. (this example happily only calls function once so it would be possible to display values without a doubt, obviously more tricky to display anything sensible if invoked several times

- Step 4. Being able to evaluate this as you type (Instarepl is the term used in Light Table)

Not completely related sample from clojure plugin:

 

 

any help / ideas would be greatly appreciated

cheers

Magnus

[Jim White]

Hi Magnus!

You're in luck.  I'd like to help out with Light Table and I've been hacking on Groovy Script AST transformations the last few days.  I've whipped up a little AST transformation that instruments statements in a Groovy script.

https://github.com/jimwhite/TransformationSandbox 

Jim

[Tim Yates]

Very nice :-D

[Jim White]

I've pushed a much nicer version that uses ExpressionTransformer to do the job of walking the AST.

Jim

[Magnus Rundberget]

This is just awesome ! 

I'll have a closer look and if you don't mind (?), start incorporating into a 0.0.2 release of the plugin.

-magnus

---

Date: Wed, 19 Feb 2014 17:38:38 -0800
From: james.pa...@gmail.com
To: us...@groovy.codehaus.org
Subject: Re: [groovy-user] Intercept/monitor statements in a groovy script

[Cédric Champeau]

On a similar infrastructure (but not goal), the "runtime" secure AST customizer we've been working on at a hackathon might be of help. It proposes a general approach on rewriting expressions so that we can make runtime access checks. The very same strategy can be used to intercept calls and memoize values.

See https://github.com/rivieragug/groovy-core/tree/secureruntime

-- 
Cédric Champeau
SpringSource - Pivotal
http://twitter.com/CedricChampeau
http://melix.github.io/blog
http://spring.io/ http://www.gopivotal.com/

[Jim White]

Yes, please do use it as you see fit with the plugin.  Let me know if you have any questions at all.  I'm also available via Skype for chat (id:JamesPaulWhite).

Jim

[Jim White]

Looks like we are indeed working in the same milieu.  I've just whipped up a base script that uses GROOVY-6585 so that a security manager can be used to log all file access as part of my Gondor (Groovy Condor) project.

https://github.com/jimwhite/Gondor/blob/master/src/org/ifcx/gondor/SecuredScript.java 

Jim

[Simon Temple]

Hi Magnus

I also wrote an AST Transformation which may help you:

http://simontemple.blogspot.co.uk/2013/12/groovy-in-cloud-sandboxing-and-more.html

https://github.com/simon-temple/groovy-sandbox

Feel free to contact me if you have questions.

SimonT

[Cédric Champeau]

Hi Simon!

We referenced it when implementing the SecureRuntimeASTCustomizer. We are currently working on finishing the implementation, I think you will be quite happy with the result :)

[Simon Temple]

I don't want to hijack this thread but.... that's great new Cédric.  Thank you.

[Fabrice Matrat]

Hi,

The RivieraGUG are currently working with Cedric Champeau to integrate a solution for Runtime Check, it's still a work in progress !

https://github.com/rivieragug/groovy-core/tree/secureruntime

We took into consideration the work of Simon, Koshuke to make sure it will also fit those problematics. And they are great implementation

Our goal is to finish it for end of March for a first version.

This can be a source of inspiration for your problem. If you have any questions, do not hesitate

Fabrice.

--
Best regards,

http://rivieragug.org/
http://3musket33rs.github.com/

[Fabrice Matrat]

Hi Simon,

Just also to clarify why we didn't take the existing solutions:

This solution is highly inspired from Groovy Sandbox and Simon's solution but will wrap the intercepted call inside a closure to be as Groovy as possible and will be forward compatible with evolution.

The existing solutions are great but they are too intrusive (especially on the Groovy way to invoke a method). The Groovy sandbox is the most advanced one but is also the most intrusive by trying to replicate call site caching mechanism and probably using class and method that could be modified or deleted in the future.

The solution we propose is to wrap the intercepted call within a closure. It has the advantage to be more compatible with call site, static typing or invoke dynamic.

Intercepting Groovy calls, however, have a performance impact on script execution but this is not the primary goal of the implementation.

But I think I may hijack the thread here so we could open a new one to discuss those points together.

Fabrice.

[Jochen Theodorou]

Am 20.02.2014 09:54, schrieb Fabrice Matrat:
[...]

> Intercepting Groovy calls, however, have a performance impact on script
> execution but this is not the primary goal of the implementation.

The wrapping in a closure and additional dynamic execution paths as well
as additional boxing is what causes the performance loss.

This is exactly why other solutions try to work on the call site
directly. Groovy's callsite internals are not very friendly, but the
invokedynamic version would, if it supported a custom bootstrap method,
easily support that kind of thing. And I would not expect much
performance loss in that case either.

as for static compilation... it afaik has an influence on the static
compiler if you wrap something, especially for flow typing.

for example if you compare

def i = "d"
i=1
println i.MAX_VALUE

and

def i = "d"
def tmp = {i=1}
tmp()
println i.MAX_VALUE

In the first case the compiler can determine i being an int and thus
i.MAX_VALUE is valid. In the second case the usage of an open block
prevents the clear flow and only the LUB of String and Integer can be
used later on. So i.MAX_VALUE becomes invalid, as there is no common
type that has MAX_VALUE defined.

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Jim White]

I've gone ahead and merged the logging code into my fork of your plugin and sent you a pull request.

Jim

[Magnus Rundberget]

Jim !

I really appreciate your effort. Thanks a bunch. It's really cool stuff and I've learned so much from what you've done.

Previously I was short of information, the issue now is more in terms of how to present the information without overloading :-)

The next big area I'd like to venture into would be an instarepl kind of experience (continuous evaluation while you type). Haven't given it much thought so far, but I can imagine some interesting challenges ahead. I'd love to hear your thoughts on alternatives to attack this area (and others , there are certainly enough areas one could address for a groovy IDE plugin... but I figure my interest for now is trying to figure out areas that makes a difference, something innovative. ) I have no commercial interests, this is just a ride/learning experience for me. Who knows where it goes !

It's been real fun so far. 

I'm not sure if a groovy plugin for a particular IDE is a proper topic for the groovy user group ? But if no one is complaining I'm more than happy to continue !

@jim: you on twitter btw ? 

So once again big thx to Jim, and thx to everyone else for your input. I have and will investigate and learn from what you have suggested !

cheers

Magnus

---

Date: Thu, 20 Feb 2014 18:36:11 -0800

[Jim White]

GroovyShell should probably be able to do instarepl although as it is currently implemented it doesn't let you define members (that persist anyhow) and instead you need to use bindings.  I have thoughts about how to fix that.  Also worthwhile would be to use the Clojure instarepl code for control and have an adapter for differing languages.  I'm going to be cleaning up GroovyMain (to fix http://jira.codehaus.org/browse/GROOVY-6561 - which I've asked for advice on but haven't received any so far) and that should give me a better idea about the best way to do something like instarepl with Groovy.  

Discussion here is fine with me, although for some focused things it might be better to use issues and/or wiki on your github for the plugin.

I do have twitter but I don't really tweet much.  I will probably use the handle @oohtml if I do.

Jim

[Magnus Rundberget]

Cool !

I've added some issues on the github for the plugin (https://github.com/rundis/LightTable-Groovy).

Feel free to chip in with ideas, new issues or indeed take on issues if you feel the calling :-)

cheers

Magnus 

---

Date: Sat, 22 Feb 2014 10:36:34 -0800