CoFoJa failing when used with other APT processors (such as Lombok)

[David Barri]

Hi! I'm having a problem where if I use cofoja in conjunction with lombok (which can generate getter/setter methods from annotations) then cofoja fails with a "cant find symbol" message.

I've banged up a really simple example. (Downloadable with build script at https://github.com/japgolly/sandbox)

Say I have 3 Java files:

@Accessors(chain = true)
public class Apple {
@Setter public Long id;
public void hello() { setId(null); }
}
public interface EatContract<T> {
@Ensures("result == t")
T eat(T t);
}
public class EatImpl implements EatContract<Apple> {
public Apple eat(Apple p) { return p; }
}

And I add both apt processes to javac like -processor lombok.core.AnnotationProcessor,com.google.java.contract.core.apt.AnnotationProcessor

Then compilation fails with:

error: error in contract: ./Apple.java:10: cannot find symbol
symbol  : method setId(<nulltype>)
location: class Apple
1 error

If I allow Lombok to convert the files to java (instead of classes) first, and then run cofoja over those classes everything works.

Any ideas on how I can get these APT processors to work together?

Thanks,

David

[Nhat Minh Lê]

Hi,

Could you try running Cofoja with the annotation processor option
com.google.java.contract.dump turned on? It should dump the contract
code Java files in the folder contracts_for_java.out, so we can look
at them and see what it's trying to compile wrong.

To be honest, I don't have a machine that runs Java, let alone Cofoja,
right now (waiting for new dev box) and I haven't touched the Cofoja
code base in quite some time, but if you can help with the live
debugging, I'll see what I can do.

Cheers,
Nhat

[David Barri]

Hi Nhat,

Thanks for helping mate!

I switched on that flag and it generated a dir called contracts_for_java.out which you can see here: https://github.com/japgolly/sandbox/tree/lombok_cofoja/contracts_for_java.out

The APT output when building still shows lombok and cofoja running in round 1 and then nothing happening in round 2. Do you know if there's a way to force cofoja to run in round 2? Or is there something I could change in the cofoja code to get it to run in round 2? Maybe I could patch it so that you could specify an optional round param.

Cheers,

David

[Nhat Minh Lê]

On Wed, Nov 7, 2012 at 10:58 PM, David Barri <japg...@gmail.com> wrote:
> Hi Nhat,
>
> Thanks for helping mate!
>
> I switched on that flag and it generated a dir called contracts_for_java.out
> which you can see here:
> https://github.com/japgolly/sandbox/tree/lombok_cofoja/contracts_for_java.out
>
> The APT output when building still shows lombok and cofoja running in round
> 1 and then nothing happening in round 2. Do you know if there's a way to
> force cofoja to run in round 2? Or is there something I could change in the
> cofoja code to get it to run in round 2? Maybe I could patch it so that you
> could specify an optional round param.

OK, I think I get it.


Short answer: The problem is probably not the rounds or the order. Try
running the two annotation processors in two different invocations of
javac. Lombok first, Cofoja second; you don't need to have Lombok
generate Java files, just compile your sources directly to class files
with Lombok enabled, then give them to Cofoja with -proc:only, so that
the class files are not regenerated, but the contracts will (and
should use the class files you have already generated using Lombok,
which will be correct, with accessors and everything). Something like
the following:

$ javac -processor lombok.core.AnnotationProcessor ...
$ javac -processor
com.google.java.contract.core.apt.AnnotationProcessor -proc:only ...

Don't forget to add the class output directory to the classpath of the
second invocation if necessary. Generally speaking, Cofoja doesn't
need to touch your classes themselves, it will generate its own
metadata, so it's fine to run it *after* the normal compilation
process, whatever it is. The only (pretty big) downside of this
approach is that javac won't know how to track dependencies correctly
and will just recompile contracts for everything you put on the
command line (all the .java files you specify), so if you want proper
dependency tracking, you'll have to roll your own.


Long answer: The problem as I see it is as follows: as you can see
from the contract files in contracts_for_java.out, the contract code
itself doesn't contain any reference to the Lombok setter we're
missing, so the issue is that when compiling the contract code, the
Apple is needed, hence Apple.java gets pulled into the mix; however,
Apple.java itself is never modified, because that's you own source
file.

One thing you might not know about annotation processing in Java 6 is
that annotation processors can't (normally) touch your classes. That's
(one of the reasons) why Cofoja generates a bunch of auxiliary classes
instead, and doesn't need to be run as part of your normal compilation
chain; it can (should) be run as an extra, afterwards, especially if
you have complex compilation rules.

So, you might wonder how Lombok does its magic. Well, it does change
your classes, but that's not standard; it uses internal classes
exposed through arbitrary casting objects to their concrete
Sun-javac-specific implementations and working with that. Cofoja
doesn't expect that to happen, and will choke hard. It's not to say
it's Lombok's fault; Cofoja has its own quirks: in this case, the
unfortunate consequence of contracts being compiled in the background.
What that means is that, for various that would be a bit long to
explain here, Cofoja does not use the annotation API to compile
contract classes, but rather invokes a child Java compiler through the
tools API. What that means for you is that even if Cofoja did run in
round 2, that wouldn't solve anything, because Lombok changes
something internal that Cofoja doesn't expect to see, and Cofoja
compiles its stuff in an environment Lombok is not aware of. There's
really no ideal solution to this problem, though, as the very design
of Java annotation processing requires some kind of work around or
another for what we're trying to do, both Lombok and us.


Side note: From the round information, I can see that our
AnnotationProcessor.process() shouldn't be returning true, that's a
bug, since we capture all annotations, we should always return false.
Probably a leftover from the time when we didn't use * as the set of
annotations (IIRC we switched to * because we wanted to see classes
that are not contract-annotated but might need contracts regardless).


Hope that helps,
Nhat

[David Barri]

Nhat, mate, thanks a lot for taking the time to explain all of that to me! Your proc:only suggestion worked!

Also that explanation made sense to me and I now have some thinking to do about how I approach this from a build automation perspective. Without that great info I would've wasted time on a solution based on mere assumptions and it sounds as though it would've have worked in the end.

Thanks again Nhat!