the stack overflowing build failures

[Paul Phillips]

We have quite a few of these to choose from now.  Based on the ones I have looked at so far, it is arising from some interaction between synthetic methods generated for case classes and the virtpatmat's translation of them.   The CaseDef shown here is deeply involved.  It's from the synthetic productElement method, which looks something like this:

    <synthetic> def productElement(x$1: Int): Any = x$1 match {

      case 0 => Foo.this.x

      case 1 => Foo.this.y

      case _ => throw new IndexOutOfBoundsException(x$1.toString())

    };

Here is the tree which appears in multiple failures (but for different case classes.)

[scalacfork] CaseDef( // tree.tpe=Nothing

[scalacfork]   "_" // tree.tpe=Int

[scalacfork]   Throw( // tree.tpe=Nothing

[scalacfork]     ApplyConstructor( // def <init>(x$1: String): IndexOutOfBoundsException in class IndexOutOfBoundsException, tree.tpe=IndexOutOfBoundsException

[scalacfork]       new IndexOutOfBoundsException."<init>" // def <init>(x$1: String): IndexOutOfBoundsException in class IndexOutOfBoundsException, tree.tpe=(x$1: String)IndexOutOfBoundsException

[scalacfork]       Apply( // def toString(): String in class Any, tree.tpe=String

[scalacfork]         "x$1"."toString" // def toString(): String in class Any, tree.tpe=()String

[scalacfork]         Nil

[scalacfork]       )

[scalacfork]     )

[scalacfork]   )

[scalacfork] )

The stack trace line frequency list shows a classic 2-2-1-1-1-1 pattern.  There are two calls to undoUnless in Types and they both appear to be participants in it.

 152 [scalacfork] at scala.reflect.internal.Types$UndoLog.undoUnless(Types.scala:164)

 152 [scalacfork] at scala.reflect.internal.Types$TypeConstraint$$anonfun$isWithinBounds$1.apply(Types.scala:3771)

  76 [scalacfork] at scala.runtime.BoxesRunTime.equals2(Unknown Source)

  76 [scalacfork] at scala.runtime.BoxesRunTime.equals(Unknown Source)

  76 [scalacfork] at scala.reflect.internal.Types$class.scala$reflect$internal$Types$$isSameType1(Types.scala:5206)

  76 [scalacfork] at scala.reflect.internal.Types$class.isSubType(Types.scala:5400)

  76 [scalacfork] at scala.reflect.internal.Types$class.isSameType(Types.scala:5047)

  76 [scalacfork] at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:799)

  76 [scalacfork] at scala.reflect.internal.Types$Type.$eq$colon$eq(Types.scala:850)

  76 [scalacfork] at scala.reflect.internal.Types$SubTypePair.equals(Types.scala:4924)

  76 [scalacfork] at scala.reflect.internal.Types$$anonfun$isSubType$1.apply$mcZ$sp(Types.scala:5403)

  76 [scalacfork] at scala.reflect.internal.Types$$anonfun$isSameType$1.apply$mcZ$sp(Types.scala:5048)

  76 [scalacfork] at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:13)

  76 [scalacfork] at scala.reflect.internal.SymbolTable.isSameType2(SymbolTable.scala:13)

  76 [scalacfork] at scala.reflect.internal.SymbolTable.isSameType(SymbolTable.scala:13)

  76 [scalacfork] at scala.collection.mutable.HashSet.containsEntry(HashSet.scala:41)

  76 [scalacfork] at scala.collection.mutable.HashSet.contains(HashSet.scala:58)

  76 [scalacfork] at scala.collection.mutable.FlatHashTable$class.containsEntry(FlatHashTable.scala:122)

  76 [scalacfork] at scala.collection.mutable.AbstractSet.apply(Set.scala:45)

  76 [scalacfork] at scala.collection.immutable.List.forall(List.scala:77)

  76 [scalacfork] at scala.collection.LinearSeqOptimized$class.forall(LinearSeqOptimized.scala:69)

  76 [scalacfork] at scala.collection.GenSetLike$class.apply(GenSetLike.scala:43)

  74 [scalacfork] at scala.reflect.internal.Types$class.isSameType2(Types.scala:5332)

  74 [scalacfork] at scala.reflect.internal.Types$TypeVar.registerTypeEquality(Types.scala:3076)

  74 [scalacfork] at scala.reflect.internal.Types$TypeConstraint.isWithinBounds(Types.scala:3771)

[Adriaan Moors]

thanks for distilling it (it sounds almost poetic when you put it like that)

I now also have one of those traces sitting in my terminal -- looking into it

[Paul Phillips]

On Mon, May 14, 2012 at 12:04 AM, Adriaan Moors <adriaa...@epfl.ch> wrote:

thanks for distilling it (it sounds almost poetic when you put it like that)

I now also have one of those traces sitting in my terminal -- looking into it

I think it's this line in 7a5aaa9e23 :

-      val nextBinderTp = glb(List(patBinder.info.widen, pt))

+      val nextBinderTp = glb(List(patBinder.info.widen, pt)).normalize

Or something else in that commit, but that's my guess.  I offer no explanation.

[Adriaan Moors]

here's what happens when you uncomment the debug println in registerTypeEquality:

[scalacfork] regTypeEq: (?_1723,Symbols.this.Annotatable[?_1723],false)

[scalacfork] regTypeEq: (?_1723,forSome [type _1723] Symbols.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._1723],false)

[scalacfork] regTypeEq: (?_1723,Symbols.this.Annotatable[?_1723],false)

[scalacfork] regTypeEq: (?_1723,forSome [type _1723] Symbols.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._1723],false)

[scalacfork] regTypeEq: (?_1723,Symbols.this.Annotatable[?_1723],false)

[scalacfork] regTypeEq: (?_1723,forSome [type _1723] Symbols.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._1723],false)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

[scalacfork] regTypeEq: (?_1723,?_1723,true)

(...)

a classic 1-pattern

[Paul Phillips]

On Mon, May 14, 2012 at 12:17 AM, Adriaan Moors <adriaa...@epfl.ch> wrote:

here's what happens when you uncomment the debug println in registerTypeEquality:

Oh excellent, I know all about that one! It's one of those ridiculous pathological lubs.  I have a branch somewhere which chases it a ways but I never caught it.  You can see Annotatable arise here:

scala> List(ListClass, ListClass.tpe)

res0: List[$r.intp.global.Annotatable[_ >: $r.intp.global.Type with $r.intp.global.Symbol <: $r.intp.global.Annotatable[_ >: $r.intp.global.Type with $r.intp.global.Symbol <: Object]]] = List(class List, List[A])

 

And sometimes that type is screenfuls, where Annotatable keeps being pushed further and further up the bounds.

[Paul Phillips]

On Mon, May 14, 2012 at 12:24 AM, Paul Phillips <pa...@improving.org> wrote:

And sometimes that type is screenfuls, where Annotatable keeps being pushed further and further up the bounds.

Maybe this unusual type parameter / self-type relationship is involved.

  trait Annotatable[Self] {

    self: Self =>

 

The self-type does not actually appear to be utilized, so I removed it.  The fun of bugs like this of course is never knowing with confidence if it's fixed or being sure what fixed it.

[Adriaan Moors]

and to make it even worse, it's non-deterministic... at least on my machine it doesn't fail every time

[Adriaan Moors]

changing registerTypeEquality to:

    def registerTypeEquality(tp: Type, typeVarLHS: Boolean): Boolean = {
      println("regTypeEq: "+(safeToString, debugString(tp), tp.getClass, if (typeVarLHS) "in LHS" else "in RHS", if (suspended) "ZZ" else if (constr.instValid) "IV" else "")) //@MDEBUG
      println("constr: "+ constr)

makes the SO impossible (for now) to reproduce for me... so could it be an uninitialized symbol?

[Paul Phillips]

On Mon, May 14, 2012 at 12:41 AM, Adriaan Moors <adriaa...@epfl.ch> wrote:

makes the SO impossible (for now) to reproduce for me... so could it be an uninitialized symbol?

How are you reproducing it when it isn't SO impossible?

[Adriaan Moors]

How are you reproducing it when it isn't SO impossible?

so, SO shuns stdout, seemingly

spring, storm

shouts!

[Paul Phillips]

By the way, maybe this is relevant, and even if it isn't, any idea where this would have regressed? See my last comment.

https://issues.scala-lang.org/browse/SI-3528?focusedCommentId=57612&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-57612

[Adriaan Moors]

I hereby retract my shoddy haiku:

quick.comp:

    [mkdir] Created dir: /Users/adriaan/git/scala/build/quick/classes/compiler

    [javac] Compiling 77 source files to /Users/adriaan/git/scala/build/quick/classes/compiler

    [javac] Note: Some input files use unchecked or unsafe operations.

    [javac] Note: Recompile with -Xlint:unchecked for details.

[scalacfork] Compiling 516 files to /Users/adriaan/git/scala/build/quick/classes/compiler

[scalacfork] regTypeEq: (?_15,Definitions.this.Annotatable[?_15],class scala.reflect.internal.Types$TypeRef$$anon$5,in RHS,)

[scalacfork] constr: [ <: (Definitions.this.Type)]

[scalacfork] regTypeEq: (?_15,Definitions.this.Annotatable[?_15],class scala.reflect.internal.Types$TypeRef$$anon$5,in RHS,)

[scalacfork] constr: [ <: (Definitions.this.Type)]

[scalacfork] regTypeEq: (?_15,Definitions.this.Annotatable[?_15],class scala.reflect.internal.Types$TypeRef$$anon$5,in RHS,)

[scalacfork] constr: [ >: (Definitions.this.Type) |  <: (Definitions.this.Type)]

[scalacfork] regTypeEq: (?_16,forSome [type _16] Definitions.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._16],class scala.reflect.internal.Types$ExistentialType,in RHS,)

[scalacfork] constr: [ <: (Definitions.this.Type)]

[scalacfork] regTypeEq: (?_16,forSome [type _16] Definitions.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._16],class scala.reflect.internal.Types$ExistentialType,in RHS,)

[scalacfork] constr: [ <: (Definitions.this.Type)]

[scalacfork] regTypeEq: (?_16,forSome [type _16] Definitions.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._16],class scala.reflect.internal.Types$ExistentialType,in RHS,)

[scalacfork] constr: [ >: (Definitions.this.Type) |  <: (Definitions.this.Type)]

[scalacfork] regTypeEq: (?_52.type,?scala.reflect.internal.Types$NoPrefix$?._52.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?_53.type,ReflectGlobal.this.erasure.type,class scala.reflect.internal.Types$UniqueSingleType,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?_53.type,ReflectGlobal.this.erasure.type,class scala.reflect.internal.Types$UniqueSingleType,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?x.type,GenICode.this.global.type.platform.type,class scala.reflect.internal.Types$UniqueSingleType,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?c.type,?scala.reflect.internal.Types$NoPrefix$?.c.type,class scala.reflect.internal.Types$TypeRef$$anon$4,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (?_1728,Types.this.Type,class scala.reflect.internal.Types$TypeRef$$anon$6,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (=Types.this.Type,Types.this.Type,class scala.reflect.internal.Types$TypeRef$$anon$6,in RHS,IV)

[scalacfork] constr: [] _= Types.this.Type

[scalacfork] regTypeEq: (=Types.this.Type,Types.this.Type,class scala.reflect.internal.Types$TypeRef$$anon$6,in RHS,IV)

[scalacfork] constr: [] _= Types.this.Type

[scalacfork] regTypeEq: (?_1729,forSome [type _1729] Types.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._1729],class scala.reflect.internal.Types$ExistentialType,in RHS,)

[scalacfork] constr: [ <: (Types.this.Type)]

[scalacfork] regTypeEq: (?_1729,forSome [type _1729] Types.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._1729],class scala.reflect.internal.Types$ExistentialType,in RHS,)

[scalacfork] constr: [ <: (Types.this.Type)]

[scalacfork] regTypeEq: (?_1729,forSome [type _1729] Types.this.Annotatable[?scala.reflect.internal.Types$NoPrefix$?._1729],class scala.reflect.internal.Types$ExistentialType,in RHS,)

[scalacfork] constr: [ >: (Types.this.Type) |  <: (Types.this.Type)]

[scalacfork] regTypeEq: (?_1736,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (=scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,IV)

[scalacfork] constr: [] _= scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type

[scalacfork] regTypeEq: (=scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,IV)

[scalacfork] constr: [] _= scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type

[scalacfork] regTypeEq: (?_1737,?scala.reflect.internal.Types$NoType$?.<refinement>,class scala.reflect.internal.Types$RefinementTypeRef,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (=RefinedBuildManager.this.compiler.Symbol,?scala.reflect.internal.Types$NoType$?.<refinement>,class scala.reflect.internal.Types$RefinementTypeRef,in RHS,IV)

[scalacfork] constr: [] _= RefinedBuildManager.this.compiler.Symbol

[scalacfork] regTypeEq: (=RefinedBuildManager.this.compiler.Symbol,?scala.reflect.internal.Types$NoType$?.<refinement>,class scala.reflect.internal.Types$RefinementTypeRef,in RHS,IV)

[scalacfork] constr: [] _= RefinedBuildManager.this.compiler.Symbol

[scalacfork] regTypeEq: (?_1736,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (=scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,IV)

[scalacfork] constr: [] _= scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type

[scalacfork] regTypeEq: (=scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,IV)

[scalacfork] constr: [] _= scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type

[scalacfork] regTypeEq: (?_1736,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,)

[scalacfork] constr: []

[scalacfork] regTypeEq: (=scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,IV)

[scalacfork] constr: [] _= scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type

[scalacfork] regTypeEq: (=scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type,<refinement>.this,class scala.reflect.internal.Types$UniqueThisType,in RHS,IV)

[scalacfork] constr: [] _= scala.tools.nsc.interactive.RefinedBuildManager.<refinement>.type

[scalacfork] regTypeEq: (?_1737,?scala.reflect.internal.Types$NoType$?.<refinement>,class scala.reflect.internal.Types$RefinementTypeRef,in RHS,)

[scalacfork] constr: []

ad stackoverflowium

[Adriaan Moors]

so, my current theory is that the =:= in

  class SubTypePair(val tp1: Type, val tp2: Type) {

    override def hashCode = tp1.hashCode * 41 + tp2.hashCode

    override def equals(other: Any) = other match {

      case stp: SubTypePair =>

        (tp1 =:= stp.tp1) && (tp2 =:= stp.tp2)

should not mutate TypeVars, since it's simply checking whether this pair of types is awaiting their subtype judgment

since it shows up in the SO, it seems so likely that temporarily suspending TVs will fix the problem

[Adriaan Moors]

https://github.com/adriaanm/scala/commit/c10ba974eac06bc40af3c5897e8f42e6f70837f7

[Adriaan Moors]

reviewed with Martin: it's not performance critical since it's only run when we hit the subtype-recursion-limit,

so refactoring to nicer code and adding dots to avoid postfix-feature warnings