Slow compilation in some cases

227 views
Skip to first unread message

Alexey Romanchuk

unread,
Feb 17, 2015, 5:42:11 AM2/17/15
to scala-i...@googlegroups.com
Hey all!

I am new to scala compiler discussions. If it is not right mail list to write to please tell the right one.

I am trying to understand why scala compiler performs very slow in some cases. I have scala 2.11.5 and working with akka-stream. Right now I have file which compiles 10+ minutes. If I change line or two in this file compilation tooks usual 5-10 seconds. In idea I saw "Phase typer in ***.scala" for 10 minutes. I can not easily provide minimal reproducible example because file contains some internal libraries and after commenting/uncommenting one line bug disapperas or appers again.

Is it any diagnostic instructions like AST dump, or "profiled run" or something else to reproduce this bug? Here it is stacktrace of working thread from sbt compiler.

Thanks!

"pool-1444-thread-2" #5814 prio=5 os_prio=0 tid=0x00007f5ce1b9e000 nid=0x10c9 runnable [0x00007f5c48de8000]
   java.lang.Thread.State: RUNNABLE
at scala.reflect.internal.tpe.TypeMaps$SubstMap.apply(TypeMaps.scala:698)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:149)
at scala.reflect.internal.tpe.TypeMaps$SubstMap.apply(TypeMaps.scala:700)
at scala.reflect.internal.Types$Type.subst(Types.scala:705)
at scala.reflect.internal.Types$Type.instantiateTypeParams(Types.scala:470)
at scala.reflect.internal.Types$class.appliedType(Types.scala:3551)
at scala.reflect.internal.SymbolTable.appliedType(SymbolTable.scala:16)
at scala.reflect.internal.Types$TypeRef.transformInfo(Types.scala:2192)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1985)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:148)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.applyToSymbolInfo(TypeMaps.scala:218)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.loop$1(TypeMaps.scala:227)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.noChangeToSymbols(TypeMaps.scala:229)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:243)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:133)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$Type.memberInfo(Types.scala:683)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1982)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:148)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:134)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$Type.memberInfo(Types.scala:683)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1982)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:148)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:134)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$Type.memberInfo(Types.scala:683)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1982)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:148)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:134)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$TypeRef.asSeenFromOwner(Types.scala:2197)
at scala.reflect.internal.Types$TypeRef.transformInfo(Types.scala:2192)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1985)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:149)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.applyToSymbolInfo(TypeMaps.scala:218)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.loop$1(TypeMaps.scala:227)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.noChangeToSymbols(TypeMaps.scala:229)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:243)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:133)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$TypeRef.asSeenFromOwner(Types.scala:2197)
at scala.reflect.internal.Types$TypeRef.transformInfo(Types.scala:2192)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1985)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:441)
at scala.collection.immutable.List.loop$1(List.scala:172)
at scala.collection.immutable.List.mapConserve(List.scala:188)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:115)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:149)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:134)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$TypeRef.asSeenFromOwner(Types.scala:2197)
at scala.reflect.internal.Types$TypeRef.transformInfo(Types.scala:2192)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1985)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:148)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.applyToSymbolInfo(TypeMaps.scala:218)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.loop$1(TypeMaps.scala:227)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.noChangeToSymbols(TypeMaps.scala:229)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:243)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:133)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$TypeRef.asSeenFromOwner(Types.scala:2197)
at scala.reflect.internal.Types$TypeRef.transformInfo(Types.scala:2192)
at scala.reflect.internal.Types$NonClassTypeRef$class.scala$reflect$internal$Types$$relativeInfo(Types.scala:1985)
at scala.reflect.internal.Types$AbstractArgsTypeRef.scala$reflect$internal$Types$$relativeInfo(Types.scala:2327)
at scala.reflect.internal.Types$class.baseTypeOfNonClassTypeRef(Types.scala:1997)
at scala.reflect.internal.SymbolTable.baseTypeOfNonClassTypeRef(SymbolTable.scala:16)
at scala.reflect.internal.Types$NonClassTypeRef$class.baseType(Types.scala:1991)
at scala.reflect.internal.Types$AbstractArgsTypeRef.baseType(Types.scala:2327)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.loop$4(TypeMaps.scala:647)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.thisTypeAsSeen(TypeMaps.scala:653)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:464)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:110)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:149)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:134)
at scala.reflect.internal.tpe.TypeMaps$AsSeenFromMap.apply(TypeMaps.scala:468)
at scala.reflect.internal.Types$Type.asSeenFrom(Types.scala:663)
at scala.reflect.internal.Types$Type.memberInfo(Types.scala:683)
at scala.tools.nsc.typechecker.Typers$Typer.checkNonCyclic(Typers.scala:293)
at scala.tools.nsc.typechecker.Typers$Typer.checkNonCyclic(Typers.scala:310)
at scala.tools.nsc.typechecker.Typers$Typer.checkNonCyclic(Typers.scala:320)
at scala.tools.nsc.typechecker.Typers$Typer.typedValDefImpl(Typers.scala:1927)
at scala.tools.nsc.typechecker.Typers$Typer.typedValDef(Typers.scala:1917)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5205)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5258)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5295)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5322)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5269)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5273)
at scala.tools.nsc.typechecker.Typers$Typer.typedByValueExpr(Typers.scala:5351)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedStat$1(Typers.scala:2977)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:3081)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:3081)
at scala.collection.immutable.List.loop$1(List.scala:172)
at scala.collection.immutable.List.mapConserve(List.scala:188)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3081)
at scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2340)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$typedOutsidePatternMode$1$1.apply(Typers.scala:5217)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$typedOutsidePatternMode$1$1.apply(Typers.scala:5217)
at scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:5216)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5252)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5259)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5295)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5322)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5269)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5273)
at scala.tools.nsc.typechecker.Typers$Typer.transformedOrTyped(Typers.scala:5504)
at scala.tools.nsc.typechecker.Typers$Typer.typedDefDef(Typers.scala:2167)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5207)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5258)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5295)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5322)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5269)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5273)
at scala.tools.nsc.typechecker.Typers$Typer.typedByValueExpr(Typers.scala:5351)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedStat$1(Typers.scala:2977)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:3081)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:3081)
at scala.collection.immutable.List.loop$1(List.scala:172)
at scala.collection.immutable.List.mapConserve(List.scala:188)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3081)
at scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:1880)
at scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:1767)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5209)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5258)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5295)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5322)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5269)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5273)
at scala.tools.nsc.typechecker.Typers$Typer.typedByValueExpr(Typers.scala:5351)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedStat$1(Typers.scala:2977)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:3081)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:3081)
at scala.collection.immutable.List.loop$1(List.scala:172)
at scala.collection.immutable.List.mapConserve(List.scala:188)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3081)
at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:4918)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5211)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5258)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5295)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5322)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5269)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5273)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5347)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:102)
at scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:428)
at scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:419)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:428)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:94)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:93)
at scala.collection.Iterator$class.foreach(Iterator.scala:743)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1195)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.run(Analyzer.scala:93)
at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1338)
at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1325)
at scala.tools.nsc.Global$Run.compileSources(Global.scala:1320)
at scala.tools.nsc.Global$Run.compile(Global.scala:1418)
at xsbt.CachedCompiler0.run(CompilerInterface.scala:123)
at xsbt.CachedCompiler0.run(CompilerInterface.scala:99)
- locked <0x00000000c46cb350> (a xsbt.CachedCompiler0)
at xsbt.CompilerInterface.run(CompilerInterface.scala:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102)
at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:48)
at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.apply$mcV$sp(AggressiveCompile.scala:99)
at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:99)
at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:99)
at sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:166)
at sbt.compiler.AggressiveCompile$$anonfun$3.compileScala$1(AggressiveCompile.scala:98)
at sbt.compiler.AggressiveCompile$$anonfun$3.apply(AggressiveCompile.scala:143)
at sbt.compiler.AggressiveCompile$$anonfun$3.apply(AggressiveCompile.scala:87)
at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:39)
at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:37)
at sbt.inc.IncrementalCommon.cycle(Incremental.scala:99)
at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:38)
at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:37)
at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:65)
at sbt.inc.Incremental$.compile(Incremental.scala:37)
at sbt.inc.IncrementalCompile$.apply(Compile.scala:27)
at sbt.compiler.AggressiveCompile.compile2(AggressiveCompile.scala:157)
at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:71)
at sbt.compiler.AggressiveCompile.apply(AggressiveCompile.scala:46)
at sbt.Compiler$.apply(Compiler.scala:75)
at sbt.Compiler$.apply(Compiler.scala:66)
at sbt.Defaults$.sbt$Defaults$$compileTaskImpl(Defaults.scala:770)
at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:762)
at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:762)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
at sbt.std.Transform$$anon$4.work(System.scala:64)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
at sbt.Execute.work(Execute.scala:244)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)



Jason Zaugg

unread,
Feb 17, 2015, 6:16:52 AM2/17/15
to scala-i...@googlegroups.com
On Tue, Feb 17, 2015 at 8:42 PM, Alexey Romanchuk <alexey.r...@gmail.com> wrote:
Hey all!

I am new to scala compiler discussions. If it is not right mail list to write to please tell the right one.

I am trying to understand why scala compiler performs very slow in some cases. I have scala 2.11.5 and working with akka-stream. Right now I have file which compiles 10+ minutes. If I change line or two in this file compilation tooks usual 5-10 seconds. In idea I saw "Phase typer in ***.scala" for 10 minutes. I can not easily provide minimal reproducible example because file contains some internal libraries and after commenting/uncommenting one line bug disapperas or appers again.

Is it any diagnostic instructions like AST dump, or "profiled run" or something else to reproduce this bug? Here it is stacktrace of working thread from sbt compiler.

Hi Alexey,

Thanks for reporting the problem. This is a suitable mailing list.

To start with, I would suggest to run with the compiler option -Ytyper-debug to pinpoint the last expression that is typechecked before it grinds to a halt.

At that point, I would try progressively adding in explicit type arguments and type annotations to that part of the program to see if that helps. You can retrieve those with -Xprint:typer (although you will have to wait for the slow compile!)


Please let me know what you find.

-jason

Alexey Romanchuk

unread,
Feb 17, 2015, 6:36:57 AM2/17/15
to scala-i...@googlegroups.com
Jason, thanks for reply. With this option I have billion lines of debug. All of them valuable or only slow ones?


вторник, 17 февраля 2015 г., 17:16:52 UTC+6 пользователь Jason Zaugg написал:

Alexey Romanchuk

unread,
Feb 17, 2015, 6:56:49 AM2/17/15
to scala-i...@googlegroups.com
The most slowest one line is 

result : pt=akka.stream.scaladsl.Flow[ru.dgis.casino.bss.message.v1.BSSMessage,gypsy.upload.SearchTextFact] EXPRmode (site: method apply in SearchFactFlow)


My file consist of manipulations with Flow from akka-stream. Something like

Flow[String].map(???).filter(???).map(???).transform(() => ???) and so on. One flow chains 10-20 combinators. When I separated them into small parts compiler worked much faster - 10-15 sec. Finally it looks like

val f1 = Flow[String].map(???).map(???)
val f2 = Flow[String].map(???)...
val f3 = Flow[String].map(???)...
f1.via(f2).via(f3)

вторник, 17 февраля 2015 г., 17:36:57 UTC+6 пользователь Alexey Romanchuk написал:

Jason Zaugg

unread,
Feb 17, 2015, 9:26:28 AM2/17/15
to scala-i...@googlegroups.com
On Tue, Feb 17, 2015 at 9:56 PM, Alexey Romanchuk <alexey.r...@gmail.com> wrote:
The most slowest one line is 

result : pt=akka.stream.scaladsl.Flow[ru.dgis.casino.bss.message.v1.BSSMessage,gypsy.upload.SearchTextFact] EXPRmode (site: method apply in SearchFactFlow)


My file consist of manipulations with Flow from akka-stream. Something like

Flow[String].map(???).filter(???).map(???).transform(() => ???) and so on. One flow chains 10-20 combinators. When I separated them into small parts compiler worked much faster - 10-15 sec. Finally it looks like

val f1 = Flow[String].map(???).map(???)
val f2 = Flow[String].map(???)...
val f3 = Flow[String].map(???)...
f1.via(f2).via(f3)

Thanks, that is helpful. I’ve lodged SI-9157 to track this. I can reproduce the slow compilation with:

trait FlowOps[+Out] {
  type Repr[+O] <: FlowOps[O]
}

trait Flow[-In, +Out] extends FlowOps[Out] {
  override type Repr[+O] <: Flow[In, O]
  def map[T](f: Out => T): Repr[T] /* workaround: expand alias Flow[In, T] */
}

class Test {
  def slowFlow {
    (null: Flow[String, String])
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b) // takes an age to compile
  }
}
​

-jason

martin odersky

unread,
Feb 17, 2015, 10:46:11 AM2/17/15
to scala-internals
Thanks for the condensation, Jason! For what it's worth, this seems to
work fine in Dotty. So it's most likely some accidental exponential
blowup.

- Martin
> --
> You received this message because you are subscribed to the Google Groups
> "scala-internals" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scala-interna...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Martin Odersky
EPFL

Jason Zaugg

unread,
Feb 18, 2015, 9:51:33 PM2/18/15
to scala-i...@googlegroups.com
On Wed, Feb 18, 2015 at 1:45 AM, martin odersky <martin....@epfl.ch> wrote:
Thanks for the condensation, Jason! For what it's worth, this seems to
work fine in Dotty. So it's most likely some accidental exponential
blowup.

We’ve fixed the performance by improving the cache in NonClassTypeRef#relativeInfo to avoid calling memberInfo every time. Instead, it uses the current phase as the cache.

@martin: Could you please double check this change?

​
-jason

martin odersky

unread,
Feb 19, 2015, 2:25:07 AM2/19/15
to scala-internals
Hi Jason,

The change looks good to me! It also makes it clearer why dotc did not
have the same problem, because it always caches infos of TypeRefs.

In which info transformer phase did the blowup occur?

Cheers

- Martin

Jason Zaugg

unread,
Feb 19, 2015, 2:32:47 AM2/19/15
to scala-i...@googlegroups.com
On Thu, Feb 19, 2015 at 5:24 PM, martin odersky <martin....@epfl.ch> wrote:
Hi Jason,

The change looks good to me! It also makes it clearer why dotc did not
have the same problem, because it always caches infos of TypeRefs.

In which info transformer phase did the blowup occur?

It actually occurred in the typechecker, when calling baseClasses on a type like:

  T[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]

Given:

  trait T[A] { type A[X] <: T[X] }
​
-jason

martin odersky

unread,
Feb 20, 2015, 3:14:04 AM2/20/15
to scala-internals
This makes me wonder whether we should have a watchdog that flags
compilations making too little progress as errors. Exponential
worst-case behavior is inevitable. Every major language with type
inference has it or something even worse, like undecidable type
checking. Even Hindley-Milner is DEPXTIME complete. It's true that in
practice these catastrophic behaviors occur only rarely, most programs
are well behaved. But the more people use this stuff in anger the more
often we will see exponential blowups.

So, how about we have a configurable timeout? If the compiler makes no
progress during a timeout, we could run a full GC and try again. If
there's still no progress, we issue an error saying that the compiled
term takes unreasonably long to compile and should be simplified or
split up. There are a number of possible progress indicators. A simple
one would be to declare progress each time a new tree node is
typechecked.

What do people think?

- Martin

Roland Kuhn

unread,
Feb 20, 2015, 4:16:48 AM2/20/15
to scala-i...@googlegroups.com
This sounds like a reasonable thing to do, but I’d like to suggest that “time” should not be “wall clock time” or “cpu cycles”, it should be something abstract that does not vary across compilation runs (for the same compiler and source). Otherwise we get into situations where, say, the Akka codebase can only be compiled on computers with performance > X.

Regards,

Roland
Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn


Grzegorz Kossakowski

unread,
Feb 20, 2015, 6:39:46 AM2/20/15
to scala-internals scala
Agreed. Would counting the number of TypeRefs created work in this case?
Grzegorz Kossakowski
Scalac hacker at Typesafe
twitter: @gkossakowski

Miles Sabin

unread,
Feb 22, 2015, 7:19:22 AM2/22/15
to scala-internals
On Fri, Feb 20, 2015 at 11:39 AM, Grzegorz Kossakowski
<grzegorz.k...@gmail.com> wrote:
> Agreed. Would counting the number of TypeRefs created work in this case?

How many TypeRefs is too many? Asking for a for a friend.

Cheers,


Miles

--
Miles Sabin
tel: +44 7813 944 528
skype: milessabin
gtalk: mi...@milessabin.com
g+: http://www.milessabin.com
http://twitter.com/milessabin

martin odersky

unread,
Feb 22, 2015, 7:59:33 AM2/22/15
to scala-internals
On Fri, Feb 20, 2015 at 12:39 PM, Grzegorz Kossakowski
<grzegorz.k...@gmail.com> wrote:
> Agreed. Would counting the number of TypeRefs created work in this case?
>

Maybe a measure involving several key operations. Number of TypeRefs
created is a good component, number of subtype invocations +
findMember operations + asSeenFroms could be others. If we base it on
number of key operations we also do not need any contortions for
normalizing wrt GC, which is another advantage.

Cheers

- Martin

Simon Schäfer

unread,
Feb 25, 2015, 7:05:21 AM2/25/15
to scala-i...@googlegroups.com

On 20.02.2015 09:13, martin odersky wrote:
> This makes me wonder whether we should have a watchdog that flags
> compilations making too little progress as errors. Exponential
> worst-case behavior is inevitable. Every major language with type
> inference has it or something even worse, like undecidable type
> checking. Even Hindley-Milner is DEPXTIME complete. It's true that in
> practice these catastrophic behaviors occur only rarely, most programs
> are well behaved. But the more people use this stuff in anger the more
> often we will see exponential blowups.
>
> So, how about we have a configurable timeout? If the compiler makes no
> progress during a timeout, we could run a full GC and try again. If
> there's still no progress, we issue an error saying that the compiled
> term takes unreasonably long to compile and should be simplified or
> split up. There are a number of possible progress indicators. A simple
> one would be to declare progress each time a new tree node is
> typechecked.
>
> What do people think?
I timeout/cancellation subscription for the typechecker is not only
useful for exponential blowups. Actually, in an IDE/other tools it would
be very useful if the typechecker could be aborted. When the
presentation compiler is invoked it could happen that users already
started doing something different (continue typing in an editor for
example). In such cases there is no need to wait until the presentation
compiler is completed - its result needs to be ignored anyway.

martin odersky

unread,
Feb 25, 2015, 7:42:39 AM2/25/15
to scala-internals
That's true. But I believe that IDEs already have the type-checking on
a separate thread, different from the user thread. At least I recall
the xclipse IDE working that way and I suspect the others are similar.
On the other hand, hyperlinking and completion do rely on type
information, so not having to wait for anomalously long type
derivations could be a win for them.

Cheers

- Martin
--
Martin Odersky
EPFL

Rodrigo Cano

unread,
Feb 25, 2015, 10:28:58 AM2/25/15
to scala-i...@googlegroups.com
They do have it on a background thread, but if they want to ask the complier something again they still have to wait for that stale result to be complete, yielding unusable ides for tenths of seconds. At leasts that's what happens to Netbeans, which simply uses the presentation compiler.

Haoyi Li

unread,
Feb 25, 2015, 10:32:54 AM2/25/15
to scala-internals
Yeah being able to kill stuff early when it's no longer necessary is totally orthogonal to it being on a separate thread. Unless you have infinite cores to spend on useless computations!

Naftoli Gugenheim

unread,
Feb 25, 2015, 4:23:07 PM2/25/15
to scala-i...@googlegroups.com
On Wed Feb 25 2015 at 10:29:04 AM Rodrigo Cano <ioni...@gmail.com> wrote:
They do have it on a background thread, but if they want to ask the complier something again they still have to wait for that stale result to be complete, yielding unusable ides for tenths of seconds. 
At leasts that's what happens to Netbeans, which simply uses the presentation compiler.

Not to mention unusable IDEs for tens of seconds. https://groups.google.com/d/msg/scala-ide-user/Sbng5R43ZOw/NKwC0qvaPwUJ

 


>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "scala-internals" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> For more options, visit https://groups.google.com/d/optout.



--
Martin Odersky
EPFL

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-internals+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-internals+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages