Lift-lint

[Mads Hartmann Jensen]

Hey guys,

I've been playing around with a compiler plugin today that creates a new compiler phase which will look through all of your lift templates and make sure that your snippet invocations are actually calling methods on classes that exist :) 

I've hit an issue though so if you have any knowledge on compiler plugins please go over tohttp://article.gmane.org/gmane.comp.lang.scala/22073 and help me out :) 

I'm far from done ofc. but I think it's a pretty cool idea. What do you think? 

The little code I have is hiding on github: 
https://github.com/mads379/lift-lint/blob/master/src/main/scala/com/sidewayscoding/LiftLint.scala

Thanks,
Mads Hartmann

[Andy Czerwonka]

Brilliant idea.  The more type-checking the better.

[Naftoli Gugenheim]

Sounds like a great thing to have!

But why does it need to be a compiler plugin? Why not an sbt build step? It could just use reflection, similar to what lift is going to do at run time. (Or maybe you could tap into the lift snippet processing api and catch an exception? Don't know if that would require any refactoring, or a dummy session, but if it does already or could be made to work without either, it would be a generally useful API, e.g., to use from the REPL. Something like:

scala> processTemplate(<span class="lift:Snippet"><button /></span>)

(stack trace of an InvalidTemplateException, or the processed template)

--
You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.

[Stephen Tu]

Mads,

three things:

1) for your current issue, since there's no "post-phase" hook, heres a simple workaround:
A) move the remainingInvocations out to be a global variable (gross i know).
B) run your current component "LiftLintPhase" as is, but don't issue any errors.
C) create a new component, something like "LiftLintErrorPhase", which runs after "LiftLintPhase". have this phase check remainingInvocations and print the errors there. you can have this done with a guard, so it only happens once.

yes its not pretty but it would do exactly what you are looking for

2) i agree with naftoli that i think this might be better suited as an API call/sbt build step and it can just use reflection. reason being, you're only really using the compiler plugin as a means to access methods, but reflection provides all this for you already. you usually venture into the world of compiler plugins only when you want to do some code generation or something super crazy. otherwise the pains involved in understanding the quirks of the scala compiler might not be worth it (well, its a fun learning exercise :))

3) if you are interested in trying to add some compile time support for lints in lift, i've got another idea that i think would be well suited for a plugin. i'm not too familiar with mapper, but I was reading the wiki for mapper and there seem to be some requirements such as "fields must be objects within the model class" which are obviously not enforced by scalac. also there might be some other reqs which i cannot remember immediately, such as having to have a singleton meta mapper. regardless, it would be relatively straightforward IMO to implement a plugin which checked for these requirements and threw errors. i'm not sure if one already exists out there or if this is of any real use, just throwing ideas around. the grander vision i guess is a set of tools which provided more compile time checking for lift-specific things. thoughts?

[Mads Hartmann Jensen]

Hey Stephen & Naftoli

Thanks for the feedback:

@Stephen:
The solution works like a charm. I agree It's not very pretty but till PaulP implements a post-phase hook it will have to do :)

@both of you:
The only advantage I see of writing a compiler plugin over, say, an SBT task is that the compiler plugin is easily used with any build system and not just SBT. I use ant where I work and on our hudson instance so it's nicest for me if it could run it there as well :)

Also if we go further and extend the plugin (which would be very cool) with some more checks as described in point 3 in Stephen's mail it might be nice to have all the lift-specific checks gathered one place instead of spread across compiler plugins and build tool tasks :) Do you see any disadvantage to writing it as a compiler plugin besides the slight increase in complexity?

Another advantage of writing it as a compiler plugin is you get to write a compiler plugin ;) (Tautology ftw)

[Stephen Tu]

On Mon, Dec 20, 2010 at 9:08 PM, Mads Hartmann Jensen <mad...@gmail.com> wrote:

@Stephen:
The solution works like a charm. I agree It's not very pretty but till PaulP implements a post-phase hook it will have to do :)

you should also try paul's sugesstion w/ subclassing Run. i am not sure what exactly this entails but it might yield a better solution
 

@both of you:
The only advantage I see of writing a compiler plugin over, say, an SBT task is that the compiler plugin is easily used with any build system and not just SBT. I use ant where I work and on our hudson instance so it's nicest for me if it could run it there as well :) 

Also if we go further and extend the plugin (which would be very cool) with some more checks as described in point 3 in Stephen's mail it might be nice to have all the lift-specific checks gathered one place instead of spread across compiler plugins and build tool tasks :) Do you see any disadvantage to writing it as a compiler plugin besides the slight increase in complexity?

what about a case where people invoke snippets from 3rd party libraries / dependencies (do people do this / can you even do this?), ie classes which live in the classpath from some jar. with a compiler plugin. this wouldn't work, because you're requiring all snippets be defined in source code

another disadvantage is what about incremental compilation? i'm not really sure how this works in scalac, but i could imagine it only passing relevant classes through the compiler phases, in which case your plugin would throw a lot of false errors during these runs. that would be very misleading.
 

Another advantage of writing it as a compiler plugin is you get to write a compiler plugin ;) (Tautology ftw)

ya compelling point lol

[Mads Hartmann Jensen]

On Dec 21, 2010, at 7:27 AM, Stephen Tu wrote:

On Mon, Dec 20, 2010 at 9:08 PM, Mads Hartmann Jensen <mad...@gmail.com> wrote:

@Stephen:
The solution works like a charm. I agree It's not very pretty but till PaulP implements a post-phase hook it will have to do :)

you should also try paul's sugesstion w/ subclassing Run. i am not sure what exactly this entails but it might yield a better solution
 

@both of you:
The only advantage I see of writing a compiler plugin over, say, an SBT task is that the compiler plugin is easily used with any build system and not just SBT. I use ant where I work and on our hudson instance so it's nicest for me if it could run it there as well :) 

Also if we go further and extend the plugin (which would be very cool) with some more checks as described in point 3 in Stephen's mail it might be nice to have all the lift-specific checks gathered one place instead of spread across compiler plugins and build tool tasks :) Do you see any disadvantage to writing it as a compiler plugin besides the slight increase in complexity?

what about a case where people invoke snippets from 3rd party libraries / dependencies (do people do this / can you even do this?), ie classes which live in the classpath from some jar. with a compiler plugin. this wouldn't work, because you're requiring all snippets be defined in source code

I know people do it all the time when using lift:surround, lift:embed etc. which are defined in lift. My slightly hacky solution would be to white-list those snippets.

another disadvantage is what about incremental compilation? i'm not really sure how this works in scalac, but i could imagine it only passing relevant classes through the compiler phases, in which case your plugin would throw a lot of false errors during these runs. that would be very misleading.

Damn. I think that puts the last nail in the coffin for doing it as a compiler plugin. WIll try to do it as an SBT task then as a proof of concept ;)

 

Another advantage of writing it as a compiler plugin is you get to write a compiler plugin ;) (Tautology ftw)

ya compelling point lol

[Timothy Perrett]

> I know people do it all the time when using lift:surround, lift:embed etc. which are defined in lift. My slightly hacky solution would be to white-list those snippets.
>
>

I actually do this a lot with my own JARs for common operations within
my problem domain, so yeah, it could be tricky.

>
> > another disadvantage is what about incremental compilation? i'm not really sure how this works in scalac, but i could imagine it only passing relevant classes through the compiler phases, in which case your plugin would throw a lot of false errors during these runs. that would be very misleading.
>
> Damn. I think that puts the last nail in the coffin for doing it as a compiler plugin. WIll try to do it as an SBT task then as a proof of concept ;)
>

Would be really interested in seeing this :-)

[Mads Hartmann Jensen]

Tim, you've created SBT plugins before - Do you think this would be well suited as a SBT plugin? I'm thinking it would simply add a check-templates command.

[Sander Mak]

I would also be interested in creating a Maven (sorry...) plugin for this functionality. So if the core checking logic is modularized I could reuse it for a Maven plugin... Hm, I'll keep an eye on this and see how it unfolds.

[Naftoli Gugenheim]

Those arguments don't really rule out having the checks run as a plugin, they just rule out using the plugin  system to find out available snippets, leaving the possibility of using some other technique like reflection, inside a compiler plugin.

I think that at then end of the day, a missing snippet should be a warning, not an error. For one thing, how will you deal with templates that rely on S.mapSnippet or snippetDispatch? (Although perhaps [and perhaps not] it's good style to use names that correspond to actual reflection-invocable method names?) Not to mention DispatchSnippets (including StatefulSnippet) (although in that case you could call snippet.dispatch.isDefinedAt).

I wonder if it's a bad idea to do this not as a build step, but an API to use from your test suites, or from Boot in dev/test mode. The disadvantage is that you won't get results as quickly necessarily, and perhaps it would require more manual work, but the advantage is that the tests could go through the actual lift pipeline on a running app, thus getting access to all snippet information. Well, mapSnippet wouldn't end up being invoked. On the other hand a compiler plugin could find mapSnippet invocations...

Or maybe a combination: In dev/test mode, the app would have a url that when hit would run the checks. Then there would be a build step, that when invoked while the app is running, hits that url! This way you can use the usual jrebel/jetty-run/~prepare-webapp development cycle, get instant feedback on template typos, but the feedback will be accurate!

[Peter Petersson]

Great :) I was starting to get a bit worried it would turn out to be a sbt feature only ...
   
best regards
   peter petersson

[Stephen Tu]

On Tue, Dec 21, 2010 at 6:57 AM, Naftoli Gugenheim <nafto...@gmail.com> wrote:

Those arguments don't really rule out having the checks run as a plugin, they just rule out using the plugin  system to find out available snippets, leaving the possibility of using some other technique like reflection, inside a compiler plugin.

true, although quite odd.

so i think the conclusion is that this functionality should be developed as an API which just uses vanilla reflection. then many different kinds of wrappers can be built around it (ie maven plugin, sbt plugin, compiler plugin, test service, etc).

[David Pollak]

I really love the idea of snippet lint.

However, there are so many ways to declare snippets (by convention, global registration, by Loc, per session, per request, and call stack), I think there may be issues with getting it all right.

--

You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

[Mads Hartmann Jensen]

David, Unless you think it's an impossible task I'm going to take a swing at this - Not going for perfection the first time around :) 

Just talked to Ross on IM and there's really no compelling reason to do it as a compiler plugin and it raises a lot of annoying issues. 

What I need is the following (if I remember correctly)

- Access to LiftRules after instantiation  

- Access to all the jars that the project depends on & compiled classes

- All the templates of the app

I will either implement this as a standalone jar or as a sbt plugin - haven't decided yet.

Cheers

[David Pollak]

On Tue, Dec 21, 2010 at 11:52 AM, Mads Hartmann Jensen <mad...@gmail.com> wrote:

David, Unless you think it's an impossible task I'm going to take a swing at this - Not going for perfection the first time around :) 

I don't know if it's impossible and I certainly want you to try.  What I'm saying is that if you try and don't succeed, then it's a learning experience.  If you try and do succeed, then you'll have blown my socks off!

 

Just talked to Ross on IM and there's really no compelling reason to do it as a compiler plugin and it raises a lot of annoying issues. 

What I need is the following (if I remember correctly)

- Access to LiftRules after instantiation  

- Access to all the jars that the project depends on & compiled classes

- All the templates of the app

Yeah... you're probably going to need the code that walks the templates split out so you can access it without a session so you can walk all the templates and find the referenced snippets.  If you open a ticket for me, I'll work on that (it'll also be better for testing Lift).

[Mads Hartmann Jensen]

Done :) 

http://www.assembla.com/spaces/liftweb/tickets/798-make-code-that-walks-the-templates-accessible-without-a-session

[TylerWeir]

Mads, you're a beast!

[Mads Hartmann Jensen]

Lol, I'm going to interpret that as a compliment :P

On Dec 22, 2010, at 12:18 PM, TylerWeir wrote:

> Mads, you're a beast!
>

[TylerWeir]

You should:
http://www.youtube.com/watch?v=9vvuLAl99ec

[Mads Hartmann Jensen]

Lol I'm in beast mode!!

On Dec 22, 2010, at 10:17 PM, TylerWeir wrote:

> You should:
> http://www.youtube.com/watch?v=9vvuLAl99ec

[Mads Hartmann Jensen]

Okay,

So I've been playing around with it a bit to create a proof of concept snippet linter. Here's some example output>

[warn] No snippet named 'FunctionalDictionary'. Invocation found in src/test/webapp/subfolder/test2.html
[warn] No snippet named 'Test'. Invocation found in src/test/webapp/subfolder/test2.html

I've just coded enough for the code to work together. Nothing works optimally yet and it can only validate a template if the snippet is declared in the project. The reason for this is because I have no idea how to figure out the fully qualified class name of a snippet invoked like this <lift:Snippet.render> so the best I can do is to try to prepend the package of the project and see if that class exists.

Just wanted to keep you updated.

Thanks,
Mads Hartmann

[Sander Mak]

On Sat, Dec 25, 2010 at 3:21 PM, Mads Hartmann Jensen <mad...@gmail.com> wrote:

 The reason for this is because I have no idea how to figure out the fully qualified class name of a snippet invoked like this  <lift:Snippet.render> so the best I can do is to try to prepend the package of the project and see if that class exists.

How about scanning for the LiftRules.addToPackages calls to see what package prefixes are given for the project and using that in the analysis (or is that what you're saying here)? 

[Mads Hartmann Jensen]

I'll have to figure out a way to access LiftRules after instantiation so I can get the proper package prefixes at some point :) 

[David Bernard]

May be a stupid idea,... why not 

* run the lint as part of the test phase (like AppTest.testXml check if *.html/*.xml are well-formed) and included it in project skeleton

* make the code part of lift-testkit

During test, you can run Lift's Boot (like from console), access LiftRules, compiled classes, classpath.

No impact on startup time for dev/staging/prod mode.

my 2c

/davidB

[David Bernard]

As is every builder have it run for free,

and user can custimize it (eg, disable, register snippet not detected,...)

and later tools' developper will be able create editor assistant,... based on same API.

/davidB