better dependency tracking
Hubert Plociniczak
someone posted on the scala-ide mailing list a simple example of
dependency tracking that sbt does, which roughly looked like
following:
deptest/A.scala:
package deptest
trait A {
def a = "A:f"
}
trait B {
def b = "B:f"
}
-----------------------
deptest/ClassA.scala:
package deptest
class ClassA extends A{
println("Hello "+a)
}
-----------------------
deptest/ClassB.scala:
package deptest
class ClassB extends B {
println("Class B "+b)
}
Now, the issue is that even if you change something in trait A (or B)
all 3 files are recompiled due to dependency tracking that sbt does,
instead of just ClassA (or classB respectively).
Can we do better than this? i.e. instead of tracking dependency per
file, do it at least per class?
Thanks,
hubert
Eugene Vigdorchik
What happens if the modification you make in A (or B) makes the code
contain an error? Both classfiles should be deleted prior to
compilation, so both ClassA and ClassB are in error now.
Just thinking,
Eugene.
> --
> You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
> To post to this group, send email to simple-b...@googlegroups.com.
> To unsubscribe from this group, send email to simple-build-t...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en.
>
Hubert Plociniczak
On Jan 26, 10:21 am, Eugene Vigdorchik <eugene.vigdorc...@gmail.com>
wrote:
> What happens if the modification you make in A (or B) makes the code
> contain an error? Both classfiles should be deleted prior to
> compilation, so both ClassA and ClassB are in error now.
>
classfiles prior to compilation, I think. I'm not saying it would be
an easy change just that it is worth considering because right now it
doesn't make sense to have multiple classes in a file under sbt.
Harald Meland
<hubert.pl...@gmail.com> wrote:
> That's true for sbt. But for example Eclipse (for Java) doesn't remove
> classfiles prior to compilation, I think. I'm not saying it would be
> an easy change just that it is worth considering because right now it
> doesn't make sense to have multiple classes in a file under sbt.
As scala tends to generate more .class files than there are "class",
"object" or "trait" keywords in your source code, the "not remove
classfiles prior to (re-)compilation" can produce some rather
surprising results -- especially if you're using e.g. Spring to do
things like "scan the present classfiles for things annotated in this
way, and produce beans from them".
By moving from Maven (whose scala plugin doesn't delete classfiles
before re-compilation) to SBT, the amount of "clean" build actions
I've had to do to has gone from more than 10 times a day to be all but
eliminated. Please don't kill this correctness for speed.
To further optimize SBT's strategy for recompilation while keeping
things as correct as they are, I think it'd have to recompile the
modified file(s), note what classfiles this generated, compare this
set of classfiles with the pre-existing set, and for files that
already existed also compare the contents of the old and the new
classfile. There are likely tricky issues to do with synthetic
classfiles, and there might well be performance considerations for
"how complicated should the recompile-detection algorithm be" versus
"how many potentially affected files might just as well be recompiled
in one go".
--
Harald
Jason Zaugg
Mark Harrah
Jason Zaugg <jza...@gmail.com> wrote:
> AFAIR this is a conservative approach to account for interactions between
> templates defined in a source file, namely the contents of a sealed type,
> and possibly the companionship status between a class and an object
>
> sealed abstract class A
> case object B extends A
> case object C extends A
>
> Adding `object D extends A` requires a recompile of code that pattern
> matches a scrutinee of type A.
Yes.
> Perhaps this can be refined to a more fine-grained level (although the
> additional complexity of the implementation must be weighed against the
> benefits).
Right. Moving to finer granularity is as big a step as the 0.7->0.10 shift. There are several improvements that can be made on the current approach before such a move would be worth it, in my opinion. A serious obstacle is that we had to drop the full API in favor of a hash for memory reasons. We'd need the full API for a finer approach or we'd need to come up with some other system. Also, we have macros coming up and those are likely problematic for incremental compilation.
-Mark
Hubert Plociniczak
On Jan 26, 7:46 pm, Mark Harrah <dmhar...@gmail.com> wrote:
> On Thu, 26 Jan 2012 06:12:00 -0800 (PST)
>
> Jason Zaugg <jza...@gmail.com> wrote:
> > AFAIR this is a conservative approach to account for interactions between
> > templates defined in a source file, namely the contents of a sealed type,
> > and possibly the companionship status between a class and an object
>
> > sealed abstract class A
> > case object B extends A
> > case object C extends A
>
> > Adding `object D extends A` requires a recompile of code that pattern
> > matches a scrutinee of type A.
>
> Yes.
>
> > Perhaps this can be refined to a more fine-grained level (although the
> > additional complexity of the implementation must be weighed against the
> > benefits).
>
> Right. Moving to finer granularity is as big a step as the 0.7->0.10 shift. There are several improvements that can be made on the current approach before such a move would be worth it, in my opinion. A serious obstacle is that we had to drop the full API in favor of a hash for memory reasons. We'd need the full API for a finer approach or we'd need to come up with some other system. Also, we have macros coming up and those are likely problematic for incremental compilation.
Hubert Plociniczak
On Jan 26, 11:50 am, Harald Meland <haral...@gmail.com> wrote:
> On Thu, Jan 26, 2012 at 11:18, Hubert Plociniczak
>
> > That's true for sbt. But for example Eclipse (for Java) doesn't remove
> > classfiles prior to compilation, I think. I'm not saying it would be
> > an easy change just that it is worth considering because right now it
> > doesn't make sense to have multiple classes in a file under sbt.
>
> As scala tends to generate more .class files than there are "class",
> "object" or "trait" keywords in your source code, the "not remove
> classfiles prior to (re-)compilation" can produce some rather
> surprising results -- especially if you're using e.g. Spring to do
> things like "scan the present classfiles for things annotated in this
> way, and produce beans from them".
>
> By moving from Maven (whose scala plugin doesn't delete classfiles
> before re-compilation) to SBT, the amount of "clean" build actions
> I've had to do to has gone from more than 10 times a day to be all but
> eliminated. Please don't kill this correctness for speed.
of sbt) but speed is already becoming a real pain and sooner or later
something will need to be done about it.
>
> To further optimize SBT's strategy for recompilation while keeping
> things as correct as they are, I think it'd have to recompile the
> modified file(s), note what classfiles this generated, compare this
> set of classfiles with the pre-existing set, and for files that
> already existed also compare the contents of the old and the new
> classfile. There are likely tricky issues to do with synthetic
> classfiles, and there might well be performance considerations for
> "how complicated should the recompile-detection algorithm be" versus
> "how many potentially affected files might just as well be recompiled
> in one go".
manager, but that had its own drawbacks.
> --
> Harald
nafg
Mark Harrah
No, the files have to be removed from the classpath for compilation to properly fail on references to stale classes.
-Mark
Naftoli Gugenheim
--
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
Mark Harrah
Naftoli Gugenheim <nafto...@gmail.com> wrote:
> Ah, I thought it was a problem for running after, which of course makes no
> sense.
> Could they be removed from the classpath without being deleted?
How? You'd need both javac and scalac to support this and they both only accept directories and jars on the classpath as far as I know.
This wouldn't fix the problem regardless. sbt does multi-step compilation, so class files get generated at different times anyway.
-Mark
Paul Phillips
>> Could they be removed from the classpath without being deleted?
>
> How? You'd need both javac and scalac to support this and they both only accept directories and jars on the classpath as far as I know.
My work in progress allows arbitrary classpath filtering, so you could
actually remove individual classes from the classpath were it
desirable. (You're on your own with java.)