[2.0.X - 2.0.4] Compilation duration problem (and stack overflow exception)

[Joe Wyrembelski]

Summary: The time it takes to compile our app is almost intolerable and we're looking for a solution.

My application is currently a single module project (default Java app structure) and when executing 'play clean compile' compiling currently says 180 Scala sources and 230 Java sources.  The compile time got so bad, I went out and bought a much faster machine and the compile time is holding steady at 41s.  (My partner's machine is 80+ seconds.)

While the time to compile is pretty bad, we're living with it.  The real problem here is that recently we've started seeing a Stack Overflow Exception thrown from time to time during reload or from a fresh compile.  This is a bit alarming, but when we compile without seeing that problem everything seems to work great.

Research: From all I've read, it seems that the problem lies with routes compilation bug but unless you want to switch to a different template engine (japid42?) you're stuck waiting for the bug to be resolved.  There are also threads that suggest the problem is in the hierarchies used in the project (model classes that extend a base model class; controllers that extend a base controller; etc.).

Possible Solutions:

• Split the project into separate projects (say we have three 'areas' of the app, we'd create three separate play applications with a common model library. The assumption would be that we could get to having 1/3 of the compile time for each.)  
  
• Reorganize the project to use sub-projects (documentation link here). I'm not finding anything that suggests this would actually improve our situation.  I'm also not finding anything that suggests what a good reorganization would be.

Request for help:

Some validation of my assumptions would be very helpful. I haven't been able to find anyone here with a similar StackOverflowException during compile time (mostly due to the fact that searching on StackOverflow almost always finds the popular website with the same name). I'd *really* like to see some discussion about splitting the project into either side-by-side (separated) projects or sub-projects. 

Thanks in advance

[Sean Brady]

Joe,

Our experience is the same.  Our stats are as follows:

273 Scala sources, 706 Java sources, 76s to compile (quad-core i7, 8 GB RAM).

Regarding StackOverflow, the workaround is to increase the JVM max stack size in the Play build file (under play directory it is /framework/build).  In fact, we have increased all of our memory settings to run incremental compiles and unit tests more smoothly.  Here are our settings:

-Xms1024M -Xmx4096M -Xss2M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=1024M

Regarding research, we hired a Scala/sbt consultant to do some analysis for us.  It turns out that the most significant contributor to compile time is the Scala templates.  There are two factors at play: 1) each template takes a long time to compile; and, 2) sbt is aggressive about what to recompile, so a lot of templates get recompiled during incremental recompiles.

As for solutions, we have seriously considered moving our templates to Japid42.  I am also curious to see how the latest scala/sbt/Play versions will affect compile times.  

When we get the time, I plan to publish our analysis for the community's benefit.  I'd like to see some real plans to improve compile time since it is our largest productivity drain.  The path forward sounds like either 1) the community (and Typesafe) adopts Japid42 as a de facto standard, or 2) the existing templating system is somehow improved (if at all possible).

Sean

[Joe Wyrembelski]

Thanks for the JVM settings. I've applied them to avoid the overflow error, but they haven't really sped up the process. Still between 39-42s for a complete reload. 

I spent a few hours this morning looking into and playing with sub-projects and think we could really gain some ground there.  It seems that if we were to break things up into sub-projects that we'd avoid a "whole world reload" and only build the templates in each module while we work in that module. (Addressing the very aggressive nature of SBT's compile choices.)

I've started another thread (https://groups.google.com/d/topic/play-framework/dZnYiQsvs4k/discussion) requesting some assistance in getting a POC working. 

[Sean Brady]

The JVM settings do not improve speed for us either.  It just prevents running out of memory.  

We haven't gone down the path of sub-modules yet because Play 2.0 only supports a single routes file (Play 2.1 supports multiple).  Once we can separate our app into sub-apps (including multiple routes files), we will likely go down this path.

On Nov 11, 2012, at 6:08 AM, Joe Wyrembelski <j...@liftpattern.com>

 wrote:

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To view this discussion on the web visit https://groups.google.com/d/msg/play-framework/-/FuuwpCniOKwJ.
To post to this group, send email to play-fr...@googlegroups.com.
To unsubscribe from this group, send email to play-framewor...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/play-framework?hl=en.