Javac, -sourcepath, and false positives

792 views
Skip to first unread message

Thomas Broyer

unread,
Feb 8, 2015, 8:20:04 AM2/8/15
to pants...@googlegroups.com
Hi all,

TL;DR: Pants should probably pass a -sourcepath to javac. I have repro cases for Buck and Gradle but someone would have to check if Pants has that same bug and it's not easy to setup.

It all started with “javax.inject:javax.inject:1:sources on classpath during javac compilation / annotation processor execution […] will cause an annoying warning: "Implicitly compiled files were not subject to annotation processing." and you end up with an implicitly compiled Provider.class in the compile output.” [1]
That made me think about an issue we had in GWT a few years ago [2] where gwt-user.jar included javax.servlet sources and users complained that they were being compiled implicitly because source timestamps were more recent than class timestamps; and the various javac options I came to learn last year, specifically -sourcepath, -implicit, -Xprefer.

The thing is that, when you explicitly list all sources that you want compiled, your build tool should instruct javac to only compile those: it should never try to compile java sources present in dependencies.
I started looking at the mainstream build tools, namely Maven, Gradle, Buck, and Pants.
  • Maven passes a -sourcepath, but that's not without issues, and it cannot decide what's the appropriate behavior [3]
  • Gradle [4] and Buck [5] don't pass a -sourcepath (it can however be configured in Gradle, so there's a workaround)
  • Pants apparently does not pass a -sourcepath either, but it's too complicated to set it up for me to make a repro case.
Starting from the Buck repro-case [6] you should be able to easily make a repro-case for Pants (it could be as easy as renaming the BUCK file to BUILD, after setting up Pants in the source tree) and possibly file a bug and/or come with a fix.

John Sirois

unread,
Feb 8, 2015, 10:53:57 AM2/8/15
to Thomas Broyer, pants-devel


On Feb 8, 2015 6:25 AM, "Thomas Broyer" <t.br...@gmail.com> wrote:
>
> Hi all,
>
> TL;DR: Pants should probably pass a -sourcepath to javac. I have repro cases for Buck and Gradle but someone would have to check if Pants has that same bug and it's not easy to setup.
>
> It all started with “javax.inject:javax.inject:1:sources on classpath during javac compilation / annotation processor execution […] will cause an annoying warning: "Implicitly compiled files were not subject to annotation processing." and you end up with an implicitly compiled Provider.class in the compile output.” [1]

That error is caused having resources on the compile classpath instead of the runtime classpath iiuc.  I'm not positive if pants gets this right from my phone, but supplying a sourcepath would be wrong since a target can own a single file in a direction of many java files.

You have varied examples though in the links below - perhaps you could make clear which class of issues you cate about (is the apt issue hitting you too or is that just an ~unrelated example?).

Thomas Broyer

unread,
Feb 8, 2015, 1:47:56 PM2/8/15
to pants...@googlegroups.com, t.br...@gmail.com


On Sunday, February 8, 2015 at 4:53:57 PM UTC+1, John Sirois (Unaffiliated) wrote:


On Feb 8, 2015 6:25 AM, "Thomas Broyer" <t.br...@gmail.com> wrote:
>
> Hi all,
>
> TL;DR: Pants should probably pass a -sourcepath to javac. I have repro cases for Buck and Gradle but someone would have to check if Pants has that same bug and it's not easy to setup.
>
> It all started with “javax.inject:javax.inject:1:sources on classpath during javac compilation / annotation processor execution […] will cause an annoying warning: "Implicitly compiled files were not subject to annotation processing." and you end up with an implicitly compiled Provider.class in the compile output.” [1]

That error is caused having resources on the compile classpath instead of the runtime classpath iiuc.


That's one way to look at it. It's caused by having source files with annotations in the compile classpath, for classes referenced from the compiled source files (and those extra source files are either newer than their corresponding compiled classes or don't have corresponding compiled classes). The question of why you have such things in your classpath can have many answers, but ultimately the build tool shouldn't care (particularly if it supports sucking dependencies out of external / remote repositories such as Maven Central)..
 

I'm not positive if pants gets this right from my phone, but supplying a sourcepath would be wrong since a target can own a single file in a direction of many java files.


Hmm, not sure I understand; can you give more details? Maybe an example?
 

You have varied examples though in the links below - perhaps you could make clear which class of issues you cate about (is the apt issue hitting you too or is that just an ~unrelated example?).


There are 2 kinds of issues, which can manifest or not depending on the build tool.
  • there's the case of dependencies containing source files; this is what my Gradle and Buck repro case exhibit. This happens when the build tool doesn't pass a -sourcepath to javac (so javac will look for source files in the classpath).
  • then there's the case of source files that are present in the sourcepath but that you didn't expect to compile; this is what we can see with Maven. It's important to keep it in mind when trying to fix the first issue; for example, with Buck or Pants, it would be a bad idea to use source_roots as the sourcepath.
Now you're right that it's a complex issue and "just passing an empty sourcepath" is possibly not the solution; particularly when you have annotation processors that use processingEnv.getFiler().getResource(StandardLocation.SOURCE_PATH, …). If you want to support those cases, then I think you only have two choices (with Pants-like source tree layouts):
  • copy or symlink all input files into a temporary directory that you can use as the sourcepath
  • use a special JavaFileManager with the CompilationTask that will hook into getFileForInput using the input files as a whitelist for the SOURCE_PATH location
Or you could maybe just make the sourcepath configurable on a java_library or similar?
(note: for Filger#getResource, you may want to use a java_library's resources in the sourcepath [7], but that'd be a bad idea if they contain java sources, so it shouldn't at least be the default; did I say it's a complex issue?)
Reply all
Reply to author
Forward
0 new messages