Improved incremental compilation shipped!

[Grzegorz Kossakowski]

Hi,

If you have been bitten by slow Scala compilation in Play project you probably want to read this post.

I wanted to let Play community know that improved incremental compiler has been shipped as part of sbt 0.13.2-M1 release. Below, there's a comparison of incremental compiler behavior in sbt 0.13.1 and 0.13.2-M1 with name hashing enabled for some small Play project. Note that actual compilation times are not that relevant because JVM running sbt wasn't fully warmed up which greatly reduces compilation times in regular development mode. What's important is number of files (and number of compiler runs) being performed:

// sbt 0.13.1

Now simulate adding a method to a controller

[info] Compiling 1 Scala source to [...]/inc-compile/target/scala-2.10/classes...

[info] Compiling 3 Scala sources to [...]/inc-compile/target/scala-2.10/classes...

[success] Total time: 6 s, completed Jan 9, 2014 4:54:33 PM

Now simulate adding a new route to the routes file

[info] Compiling 2 Scala sources and 1 Java source to [...]/inc-compile/target/scala-2.10/classes...

[info] Compiling 29 Scala sources and 1 Java source to [...]/inc-compile/target/scala-2.10/classes...

[success] Total time: 8 s, completed Jan 9, 2014 4:54:42 PM

// sbt 0.13.2-M1 with name hashing enabled

Now simulate adding a method to a controller

[info] Compiling 1 Scala source to [...]/inc-compile/target/scala-2.10/classes...

[success] Total time: 3 s, completed Jan 9, 2014 4:56:30 PM

Now simulate adding a new route to the routes file

[info] Compiling 2 Scala sources and 1 Java source to [...]/inc-compile/target/scala-2.10/classes...

[success] Total time: 5 s, completed Jan 9, 2014 4:56:35 PM

It's very easy to test name hashing algorithm on your own Play project. Just edit project/build.properties file and change sbt version to be 0.13.2-M1:

sbt.version=0.13.2-M1

Then in build.sbt file add following lines:

// enable improved (experimental) incremental compilation algorithm called "name hashing"

incOptions := incOptions.value.withNameHashing(true)

You'll need to restart Play and do full clean build after turning on name hashing. If you discover any issues, you can easily go back to sbt 0.13.1 by reverting those two changes.

Give it a whirl and report back if it helps in your Play project!

-- 

Grzegorz

[Matt]

Is there a way to apply this to projects for versions before v2.2?  I've tried bumping the sbt version in build.properties and adding the option to Build.scala in a Play 2.1.4 project as below but after the clean and restart and successful download of the new sbt jars I get an error.

<Build.scala>
object ApplicationBuild extends Build {

    

    ....

    val main = play.Project(appName, appVersion, appDependencies).settings(

      

      ...

      

      // Experiment with new incremental compilation:

      incOptions := incOptions.value.withNameHashing(true)

    )

}

<Error stack trace>

java.lang.NoClassDefFoundError: scala/StringContext

        at sbt.CommandStrings$.<init>(CommandStrings.scala:80)

        at sbt.CommandStrings$.<clinit>(CommandStrings.scala)

        at sbt.BuiltinCommands$.defaults(Main.scala:208)

        at sbt.xMain.run(Main.scala:30)

        at xsbt.boot.Launch$.run(Launch.scala:55)

        at xsbt.boot.Launch$$anonfun$explicit$1.apply(Launch.scala:45)

        at xsbt.boot.Launch$.launch(Launch.scala:69)

        at xsbt.boot.Launch$.apply(Launch.scala:16)

        at xsbt.boot.Boot$.runImpl(Boot.scala:31)

        at xsbt.boot.Boot$.main(Boot.scala:20)

        at xsbt.boot.Boot.main(Boot.scala)

Caused by: java.lang.ClassNotFoundException: scala.StringContext

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        ... 11 more

[Grzegorz Kossakowski]

Hi Matt!

If I understand things correctly, Play 2.1 ships with sbt 0.12.x. It probably won't work to just bump the version number to 0.13.x because Play has an sbt plugin that won't be compatible.

I hope someone from Play project can confirm this.

-- 

Grzegorz Kossakowski

Scalac hacker at Typesafe

twitter: @gkossakowski

github: @gkossakowski

[Marcin Bablok]

Hello Grzegorz,

I have a problem running the developer mode (invoked by entering run on play's sbt console) - when I enter the webapp from browser I get

play.PlayExceptions$UnexpectedException: Unexpected exception[UnsupportedOperationException: The `++` operation is not supported for relations with different values of `nameHashing` flag.]

        at play.PlayReloader$$anon$1$$anonfun$reload$2$$anonfun$apply$14.apply(PlayReloader.scala:309) ~[na:na]

        at play.PlayReloader$$anon$1$$anonfun$reload$2$$anonfun$apply$14.apply(PlayReloader.scala:298) ~[na:na]

        at scala.Option.map(Option.scala:145) ~[scala-library.jar:na]

        at play.PlayReloader$$anon$1$$anonfun$reload$2.apply(PlayReloader.scala:298) ~[na:na]

        at play.PlayReloader$$anon$1$$anonfun$reload$2.apply(PlayReloader.scala:296) ~[na:na]

        at scala.util.Either$LeftProjection.map(Either.scala:377) ~[scala-library.jar:na]

java.lang.UnsupportedOperationException: The `++` operation is not supported for relations with different values of `nameHashing` flag.

        at sbt.inc.MRelationsDefaultImpl.$plus$plus(Relations.scala:367) ~[na:na]

        at sbt.inc.MAnalysis.$plus$plus(Analysis.scala:135) ~[na:na]

        at play.PlayCommands$$anonfun$12$$anonfun$apply$5.apply(PlayCommands.scala:58) ~[na:na]

        at play.PlayCommands$$anonfun$12$$anonfun$apply$5.apply(PlayCommands.scala:58) ~[na:na]

        at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111) ~[scala-library.jar:na]

        at scala.collection.immutable.List.foldLeft(List.scala:84) ~[scala-library.jar:na]

[warn] play - No application found at invoker init

I have configured the project according to instructions and the compile times are lower: "adding method to controller" and "adding a route to routes file" have changed from 12 to 7 seconds and 19 to 16 respectively on my 2.2.1 project. One strange thing though - when hit I compile when I changed nothing it takes 6 seconds compared to 1 second on sbt 0.13

Regards,

Marcin Bablok

[virtualeyes]

Thanks, couple of observations out of the box:

1) after a clean/compile (3 in fact, wanted to see if any build time improvement) ~run surprsingly triggers another compilation (or 2)
2) seems pretty borken (broken) for sub projects, does the exact opposite, slows things waaaaay down as a single change to a subproject X controller triggers recompilation of all other subprojects -- yikes bad memories of pre-sub project days ;-)

Not sure if I'm missing something, have this in my parent aggregator project section of Buiuld.scala:

incOptions := incOptions.value.withNameHashing(true)

Maybe the above needs an

in ThisBuild

or apply it to all sub project .settings(...)?

[Fernando Correia]

Hi Grzegorz,

Performance improvements are great news.

To collaborate with this effort, I created this repository:

https://github.com/fernandoacorreia/sbt-name-hashing-cases

In there you'll find test scripts for the ZenTasks project and the SecureSocial project, and the test results.

Unless I'm doing something wrong, on these test cases the new feature is actually compiling more files.

[Grzegorz Kossakowski]

On 10 January 2014 10:41, Marcin Bablok <marcin...@gmail.com> wrote:

Hello Grzegorz,

Hi Marcin!

Thanks for testing and reporting back!

 

I have a problem running the developer mode (invoked by entering run on play's sbt console) - when I enter the webapp from browser I get

play.PlayExceptions$UnexpectedException: Unexpected exception[UnsupportedOperationException: The `++` operation is not supported for relations with different values of `nameHashing` flag.]

I just tried it with a sample app create with `play new` and it works. I'm guessing you have more complicated application with subprojects, right?

If that's the case you'll need to make sure that all of your subprojects have the same value for `nameHashing` set. You can't mix them as the error message says.

I have configured the project according to instructions and the compile times are lower: "adding method to controller" and "adding a route to routes file" have changed from 12 to 7 seconds and 19 to 16 respectively on my 2.2.1 project. One strange thing though - when hit I compile when I changed nothing it takes 6 seconds compared to 1 second on sbt 0.13

The slowdown is not expected.

Are there any files being recompiled during those 6 seconds? Or are there any messages printed?

[Grzegorz Kossakowski]

On 10 January 2014 10:47, virtualeyes <sit...@gmail.com> wrote:

Thanks, couple of observations out of the box:

1) after a clean/compile (3 in fact, wanted to see if any build time improvement) ~run surprsingly triggers another compilation (or 2)

If you turn off nameHashing but keep sbt 0.13.2-M1 set do you still see that behavior? If so, then it's probably this problem:

https://github.com/playframework/playframework/pull/2237

 

2) seems pretty borken (broken) for sub projects, does the exact opposite, slows things waaaaay down as a single change to a subproject X controller triggers recompilation of all other subprojects -- yikes bad memories of pre-sub project days ;-)

Huh. This is not expected. Could you show what `last compile` prints? I added some logging that was meant to be readable for regular users so you might see yourself why it's recompiling subprojects.

 

Not sure if I'm missing something, have this in my parent aggregator project section of Buiuld.scala:

incOptions := incOptions.value.withNameHashing(true)

Maybe the above needs an

in ThisBuild

or apply it to all sub project .settings(...)?

Yes, you need all subprojects to have the same value for nameHashing set.

If you do `show incOptions` it will print the value of incOptions in all projects defined in sbt build. The expected output should be:

[info] IncOptions(3,0.5,false,false,5,None,<function0>,true,true) //the last boolean is for nameHashing

[Marcin Bablok]

Thanks! Subprojects  - this is it! I have another problem though: after having modified build.sbt for every subproject (build.properties as well but I don't think that is required), I get 

java.lang.UnsupportedOperationException: The `memberRef` source dependencies relation is not supported when `nameHashing` flag is disabled.

on some of subprojects. When I do show incOptions on the main project I get the same on all of them.

Re 6 seconds: There was no message other than standard [success] Total time 6 sec. so I guess there no file was compiled. 

Best regards,

Marcin

[Grzegorz Kossakowski]

On 10 January 2014 16:31, Marcin Bablok <marcin...@gmail.com> wrote:

Thanks! Subprojects  - this is it! I have another problem though: after having modified build.sbt for every subproject (build.properties as well but I don't think that is required), I get 

java.lang.UnsupportedOperationException: The `memberRef` source dependencies relation is not supported when `nameHashing` flag is disabled.

Trying cleaning in every subproject. This message means that you have incremental compiler state saved when the `nameHashing` flag was disabled. Now you have it enabled and the previous state is incompatible.

  

Re 6 seconds: There was no message other than standard [success] Total time 6 sec. so I guess there no file was compiled. 

 

It would be great to check it again once we are sure `nameHashing` value (and saved state of incremental compiler) is consistent for all subprojects. If it's still 6s then we'll need to dig deeper because it's certainly not expected.

[Grzegorz Kossakowski]

Hi Fernando!

Thanks for excellent tests! I had to tweak them a little bit to be able to run them on Mac.

Other than that, incOptions were not set for subprojects. I just saw you fixed that already an hour ago. Thanks!

I'll rerun those tests and dig deeper but I'm afraid that this issue might be killing our testing efforts: https://github.com/playframework/playframework/pull/2237

[Marcin Bablok]

I can't rely on my memory today too much - I thought I cleaned all subprojects before. Nevertheless I have done it now, along with 'cleanFiles' and it worked, thank you ;)

I was mistaken on the '6 seconds' - on every compile sbt compiles all 5 views I have in the project - though they are not changed, so there is something to it. The views are pretty standard, sitting in app/views. They are only in the main project, the subprojects are not play, just vanilla sbt.

Regards,

Marcin Bablok

[virtualeyes]

Doesn't work, each sub project gets:

incOptions := incOptions.value.withNameHashing(true)

via a trait helper method and as a result I get the error described by Marcin:

java.lang.UnsupportedOperationException: The `memberRef` source dependencies relation is not supported when `nameHashing` flag is disabled

This occurs even after cleaning every sub project, blowing away sbt compile target, etc.

Not sure what the deal is, running last give me this stack trace:

java.lang.UnsupportedOperationException: The `memberRef` source dependencies relation is not supported when `nameHashing` flag is disabled.

    at sbt.inc.MRelationsDefaultImpl.inheritance(Relations.scala:333)
    at sbt.inc.IncrementalNameHashing.invalidatedPackageObjects(Incremental.scala:503)
    at sbt.inc.IncrementalCommon.cycle(Incremental.scala:88)
    at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:38)
    at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:37)
    at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:58)
    at sbt.inc.Incremental$.compile(Incremental.scala:37)
    at sbt.inc.IncrementalCompile$.apply(Compile.scala:27)
    at sbt.compiler.AggressiveCompile.compile2(AggressiveCompile.scala:150)
    at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:70)
    at sbt.compiler.AggressiveCompile.apply(AggressiveCompile.scala:45)
    at sbt.Compiler$.apply(Compiler.scala:70)
    at sbt.Defaults$.sbt$Defaults$$compileTaskImpl(Defaults.scala:741)
    at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:735)
    at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:735)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
    at sbt.std.Transform$$anon$4.work(System.scala:64)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:244)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

[Grzegorz Kossakowski]

This is rather strange. The stacktrace is telling you that name hashing algorithm is enabled (we are in IncrementalNameHashing class) but it tries to read data from Analysis object that was created with name hashing flag disabled. There must be part of your application that has the flag disabled.

What's the output of `show incOptions` ran right after starting Play? This should print the value of incOptions setting in all scopes.

--
You received this message because you are subscribed to a topic in the Google Groups "play-framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/play-framework/S_-wYW5Tcvw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[virtualeyes]

For show incOptions each subproject has:
IncOptions(3,0.5,false,false,5,None,<function0>,true,true)

Somehow the application compiles now o_O

The surprising behaviors remain, however:
1) after a ;clean;compile, ~run triggers another compile (does not occur on < 0.13.2)
2) changing a controller in subproject X triggers the recompilation of all subprojects

#2 is particularly painful, basically negates one of the primary benefits of using subprojects.

BTW, application is Play 2.2.1, 10 subprojects, all play.Projects (i.e. no pure sbt Projects)

Had a similar issue with subprojects and SBT native packaging manager that Play swtiched to with the 2.2 branch. Hopefully there's quick fix in there to address this issue. Would like to take it for a spin as full ;clean;compile cycles are about 10% faster, and incremental improvements to boot would make for some nice WIN ;-)

Thanks for pushing scalac performance forward.

[Marcin Bablok]

Hi Grzegorz,

I noticed that the additional recompilation of all views after every compile was connected with upgrading sbt from version 0.13.0 (I observed the same behaviour in sbt 0.13.1). It looks like play must be changed to avoid it, (possibly done in 2.2.2RC1, I haven't tested it).

Thanks for speeding efforts, much appreciated!

Best regards,

Marcin 

[Grzegorz Kossakowski]

On 11 January 2014 10:43, virtualeyes <sit...@gmail.com> wrote:

For show incOptions each subproject has:
IncOptions(3,0.5,false,false,5,None,<function0>,true,true)

Somehow the application compiles now o_O

The surprising behaviors remain, however:
1) after a ;clean;compile, ~run triggers another compile (does not occur on < 0.13.2)

Actually, it does occur with sbt 0.13.1 as well and it's not related to the name hashing work I've done.

This is a bug in Play itself and it's fixed now in Play 2.2 snapshots. See https://github.com/playframework/playframework/pull/2237 for details. I just built Play locally (it's very easy, just clone and run `framework/build publish-local`) and incremental compilation works fine with any version of sbt 0.13.

 

2) changing a controller in subproject X triggers the recompilation of all subprojects

#2 is particularly painful, basically negates one of the primary benefits of using subprojects.

This is not expected at all. If you could try to use Play 2.2-SNAPSHOT (built locally) with name hashing that would be great. Sbt 0.13.2-M1 ships with better logging for incremental compiler (especially when name hashing is enabled) that should be readable to regular user. You can check the log by running `last compile` which prints debug log of the last `compile` task execution. The log should include information why all subprojects are being recompiled.

 

BTW, application is Play 2.2.1, 10 subprojects, all play.Projects (i.e. no pure sbt Projects)

Had a similar issue with subprojects and SBT native packaging manager that Play swtiched to with the 2.2 branch. Hopefully there's quick fix in there to address this issue. Would like to take it for a spin as full ;clean;compile cycles are about 10% faster, and incremental improvements to boot would make for some nice WIN ;-)

Thanks for pushing scalac performance forward.

You are very welcome! Thanks for testing!

With the fix of Play bug I think we should have a chance to test name hashing for real.

[Grzegorz Kossakowski]

On 14 January 2014 18:28, Marcin Bablok <marcin...@gmail.com> wrote:

Hi Grzegorz,

I noticed that the additional recompilation of all views after every compile was connected with upgrading sbt from version 0.13.0 (I observed the same behaviour in sbt 0.13.1). It looks like play must be changed to avoid it, (possibly done in 2.2.2RC1, I haven't tested it).

Thanks for speeding efforts, much appreciated!

See my other response. It's indeed a bug in Play. All I can add is that 2.2.2-RC1 doesn't contain the fix. You need to build Play snapshot locally.

[Grzegorz Kossakowski]

On 10 January 2014 15:16, Fernando Correia <fernando...@gmail.com> wrote:

Hi Fernando!

Thanks again for great test case. It allowed me to dig more and find out that the fix to Play's incremental compiler support I referred to before in this thread is incomplete.

After fixing those issues I see that name hashing is never worse and sometimes it can improve a lot. See:

https://github.com/gkossakowski/sbt-name-hashing-cases/tree/local-play-build/results

[Fernando Correia]

That's great news! It seems that we can expect shorter build times when 2.2.2 is released.

[Patrick Mahoney]

Is there a snapshot available here?

[Patrick Mahoney]

Hello Grzegorz,
Thanks for the great work here!

I am running into a snag here in sbt-13.2-M1 and a macros project:

sbt test
[info] Loading global plugins from /Users/patrick/.sbt/0.13/plugins
[info] Loading project definition from /Users/patrick/npl/project/project
[info] Loading project definition from /Users/patrick/npl/project
[info] Set current project to root (in build file:/Users/patrick/npl/)
java.lang.AssertionError: assertion failed: When `nameHashing` is disabled `names` relation should be empty: $names
    at scala.Predef$.assert(Predef.scala:179)
    at sbt.inc.TextAnalysisFormat$RelationsF$.read(TextAnalysisFormat.scala:228)
    at sbt.inc.TextAnalysisFormat$$anonfun$1.apply(TextAnalysisFormat.scala:73)
    at sbt.inc.TextAnalysisFormat$$anonfun$1.apply(TextAnalysisFormat.scala:73)
    at sbt.inc.FormatTimer$.aggregate(TextAnalysisFormat.scala:19)
    at sbt.inc.FormatTimer$.time(TextAnalysisFormat.scala:26)
    at sbt.inc.TextAnalysisFormat$.read(TextAnalysisFormat.scala:73)
    at sbt.inc.FileBasedStore$$anon$1$$anonfun$getUncaught$1.apply(FileBasedStore.scala:19)
    at sbt.inc.FileBasedStore$$anon$1$$anonfun$getUncaught$1.apply(FileBasedStore.scala:19)
    at sbt.Using.apply(Using.scala:25)
    at sbt.inc.FileBasedStore$$anon$1.getUncaught(FileBasedStore.scala:19)
    at sbt.inc.FileBasedStore$$anon$1.get(FileBasedStore.scala:17)
    at sbt.inc.AnalysisStore$$anon$1.get(AnalysisStore.scala:25)
    at sbt.inc.AnalysisStore$$anon$2.get(AnalysisStore.scala:31)
    at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:64)

    at sbt.compiler.AggressiveCompile.apply(AggressiveCompile.scala:45)
    at sbt.Compiler$.apply(Compiler.scala:70)
    at sbt.Defaults$.sbt$Defaults$$compileTaskImpl(Defaults.scala:741)
    at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:735)
    at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:735)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
    at sbt.std.Transform$$anon$4.work(System.scala:64)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:244)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)

    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:695)
[error] (macros/compile:compile) java.lang.AssertionError: assertion failed: When `nameHashing` is disabled `names` relation should be empty: $names
[error] Total time: 2 s, completed Jan 17, 2014 2:26:38 AM

Funny thing is, sbt clean compile works fine-only a follow up sbt compile triggers this error.

I removed the target directory in the macros project itself. Any ideas here?

Thank you,
-Patrick

[Patrick Mahoney]

@Grzegorz: Just a note: I tested this with play framework 2.2.2-RC2 - same stacktrace is produced.

Here is the output of "show incOptions" for the macro project:
[info] macros/*:incOptions
[info]     IncOptions(3,0.5,false,false,5,None,<function0>,true,true)

It matches that of all other subprojects within the build. Anything I should try?

-P

[Grzegorz Kossakowski]

Hi Patrick,

This looks correct. Did you run `clean` and did you restart sbt after turning on name hashing?

The restart is required due to this (recently discovered) bug: https://github.com/sbt/sbt/issues/1081

-- 

Grzegorz

--
You received this message because you are subscribed to a topic in the Google Groups "play-framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/play-framework/S_-wYW5Tcvw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Patrick Mahoney]

Hello Grzegorz,

When I initially run 'sbt clean compile', success-the build completes successfully, and all lights are green. If I then run 'sbt compile', this error occurs.

I did restart sbt - exit. I also tried cleaning the build project targets by hand - ; reload plugins; clean; compile;

I also killed the target folder in the affected project by hand.

No luck- the second compile is enough that this stacktrace occurs again. What can I provide that might assist further troubleshooting?

-Patrick

You received this message because you are subscribed to the Google Groups "play-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.

[Grzegorz Kossakowski]

On 17 January 2014 22:27, Patrick Mahoney <pat...@auvik.com> wrote:

Hello Grzegorz,

When I initially run 'sbt clean compile', success-the build completes successfully, and all lights are green. If I then run 'sbt compile', this error occurs.

I did restart sbt - exit. I also tried cleaning the build project targets by hand - ; reload plugins; clean; compile;

I also killed the target folder in the affected project by hand.

No luck- the second compile is enough that this stacktrace occurs again. What can I provide that might assist further troubleshooting? 

Oh, ok. That sounds more likely to be a real bug. You mention you are using macros. Are you using Scala compiler macro paradise plugin by any chance? Also, can you reproduce this problem without any macros involved? I would be very surprised if macros were relevant here at all.

The error you are seeing has (so far) only one explanation: for some reason the value of name hashing flag is inconsistent: some parts see it enabled and some see it disabled which results in a crash.

I'm happy to debug it but I need a reproducible test case. It doesn't need to be minimized.

[Patrick Mahoney]

Hello Grzegorz,

While we are using macros, we are not using the macro paradise plugin. I'm going to work at getting you a reproducible test case-it's a tad difficult as our build is quite large and hard to pare down.

-Patrick

[Grzegorz Kossakowski]

Hi!

I want to give you some update on incremental compilation in Play. Thanks for all the testing you've done! It helped us to identify issues both in sbt and Play related to incremental compilation. We fixed all bugs reported by you. All fixes are incorporated into sbt 0.13.2-M2 and Play 2.2.2 releases. Please upgrade to that combo if you want to test incremental compilation improvements.

Those who tested name hashing algorithm are probably familiar with problems reported here:

https://github.com/sbt/sbt/issues/1071

https://github.com/sbt/sbt/issues/1081

From now on, restart of Play/sbt is not required when you switch to name hashing algorithm. However, clean is still required. Also, please remember that you need consistent value of name hashing flag for all subprojects in your project.

I updated excellent tests by Fernando Correia: https://github.com/fernandoacorreia/sbt-name-hashing-cases/pull/2

In all tests name hashing is either on par with the old algorithm or better. A few people have tested name hashing at their companies and reported much better incremental compilation behavior on large Play projects.

I'd love to hear if it works well for you now and if it yields noticeable improvements!

[Fernando Correia]

That's great! 

I've merged your pull request in the test project.

Fernando Correia