incremental compilation when compiling sbt itself

[Johannes Rudolph]

Hi,

when working on the sbt 0.13 source code I noticed that changing insignificant lines in the main project often causes all of the project to recompile. E.g. adding just a line (adding `println`) not belonging to the public API in a method inside of BuiltinCommands (Main.scala) causes two recompilations of the complete subproject.

Before I start to look into it: Is it a known issue and just hard to fix or has noone investigated what the problem is here?

Johannes

[Jason Zaugg]

On Friday, November 9, 2012 12:21:59 PM UTC+1, Johannes Rudolph wrote:

when working on the sbt 0.13 source code I noticed that changing insignificant lines in the main project often causes all of the project to recompile. E.g. adding just a line (adding `println`) not belonging to the public API in a method inside of BuiltinCommands (Main.scala) causes two recompilations of the complete subproject.

Before I start to look into it: Is it a known issue and just hard to fix or has noone investigated what the problem is here?

This one was discovered recently:

  https://issues.scala-lang.org/browse/SI-6596

-jason 

[Johannes Rudolph]

On Fri, Nov 9, 2012 at 12:48 PM, Jason Zaugg <jza...@gmail.com> wrote:
> This one was discovered recently:
>
> https://issues.scala-lang.org/browse/SI-6596

Thanks. Not sure if it is exactly that.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

[Johannes Rudolph]

Something strange. I added this to SameAPI.apply:

def apply(a: Source, b: Source): Boolean = {

val hashesEqual = a.apiHash == b.apiHash

val hashesNotEmpty = (a.hash.length > 0 && b.hash.length > 0)

val apiEqual = apply(a.api, b.api)

val res = hashesEqual && hashesNotEmpty && apiEqual

if (!res) {

println("Not equal. %s %s %s" format (hashesEqual, hashesNotEmpty, apiEqual))

println("Api hashes: a1: %d, a2: %d, b1: %d, b2: %d" format (

xsbt.api.HashAPI(a.api),

xsbt.api.HashAPI(a.api),

xsbt.api.HashAPI(b.api),

xsbt.api.HashAPI(b.api)

))

}

res

}

[info] Compiling 1 Scala source to /data/git/opensource/xsbt/main/target/classes...

[warn] there were 10 deprecation warnings; re-run with -deprecation for details

[warn] there were 1 unchecked warnings; re-run with -unchecked for details

[warn] two warnings found

Not equal. false true true

Api hashes: a1: -461092469, a2: -461092469, b1: -461092469, b2: -461092469

[info] Compiling 32 Scala sources to /data/git/opensource/xsbt/main/target/classes...

That means that for this case the stored apiHash and api disagreed. Any idea what that could be?

Johannes

--
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
To view this discussion on the web visit https://groups.google.com/d/msg/simple-build-tool/-/n06Xj3w4SrQJ.
To post to this group, send email to simple-b...@googlegroups.com.
To unsubscribe from this group, send email to simple-build-t...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en.

[Mark Harrah]

On Fri, 9 Nov 2012 03:48:33 -0800 (PST)
Jason Zaugg <jza...@gmail.com> wrote:

> On Friday, November 9, 2012 12:21:59 PM UTC+1, Johannes Rudolph wrote:
>
> > when working on the sbt 0.13 source code I noticed that changing
> > insignificant lines in the main project often causes all of the project to
> > recompile. E.g. adding just a line (adding `println`) not belonging to the
> > public API in a method inside of BuiltinCommands (Main.scala) causes two
> > recompilations of the complete subproject.
> >
> > Before I start to look into it: Is it a known issue and just hard to fix
> > or has noone investigated what the problem is here?

This hasn't been investigated, no.

> This one was discovered recently:
>
> https://issues.scala-lang.org/browse/SI-6596

It could be that one, although I would be surprised if there were refinement types in Main.scala.

-Mark

>
> -jason

>
> --
> You received this message because you are subscribed to the Google Groups "simple-build-tool" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/simple-build-tool/-/dR6f07qo7zAJ.

[Mark Harrah]

The stored API is actually a skeleton. It turned out to be too expensive to serialize/deserialize the full information, so sbt only stores the tiny part of it in order to detect main classes and tests.

-Mark

[Johannes Rudolph]

On Fri, Nov 9, 2012 at 3:49 PM, Mark Harrah <dmha...@gmail.com> wrote:
> The stored API is actually a skeleton. It turned out to be too expensive to serialize/deserialize the full information, so sbt only stores the tiny part of it in order to detect main classes and tests.

Aha, so comparing the api structure itself is only a weak indicator
that nothing has changed because it has been minified. The hash, in
contrast, is calculated over the complete data and thus should be the
more accurate indicator.

[Johannes Rudolph]

The culprit seems to be `BuiltinCommands.extractLast` whose type is
inferred and contains `sbt._$5` in one case and `sbt.Extracted._$5` in
the other.

Not sure, what that means, though.

Johannes

[Grzegorz Kossakowski]

On Friday, November 9, 2012 7:18:44 AM UTC-8, Johannes Rudolph wrote:

The culprit seems to be `BuiltinCommands.extractLast` whose type is
inferred and contains `sbt._$5` in one case and `sbt.Extracted._$5` in
the other.

Not sure, what that means, though.

Scala compiler (in 2.9.x series) has a bug in pickler/unpickler related to existentials. See https://issues.scala-lang.org/browse/SI-6692 for details.

-- 

Grzegorz

[Johannes Rudolph]

On Tue, Nov 20, 2012 at 9:10 AM, Grzegorz Kossakowski
<grzegorz.k...@gmail.com> wrote:
> Scala compiler (in 2.9.x series) has a bug in pickler/unpickler related to
> existentials. See https://issues.scala-lang.org/browse/SI-6692 for details.
>

Thanks for looking into it.

[Grzegorz Kossakowski]

On Wednesday, November 21, 2012 11:48:12 PM UTC-8, Johannes Rudolph wrote:

On Tue, Nov 20, 2012 at 9:10 AM, Grzegorz Kossakowski
<grzegorz.k...@gmail.com> wrote:
> Scala compiler (in 2.9.x series) has a bug in pickler/unpickler related to
> existentials. See https://issues.scala-lang.org/browse/SI-6692 for details.
>

Thanks for looking into it.

No problem. Incremental compiler is under my attention at the moment so if you run into problems/bugs please report them here (and CC me) . I'm happy to look into them.

-- 

Grzegorz