Scala 2.11.7: AssertionError in compilation

[Martin Studer]

Hi,

I ran into an AssertionError during incremental compilation of the following code ("minimal" reproducible example):

import breeze.linalg._

import breeze.linalg.operators._

import breeze.macros.expand

 

object VectorOps {

  import breeze.math.PowImplicits._

  @expand

  @expand.valify

  implicit def dv_dv_Op[@expand.args(Int, Double) T,

  @expand.args(OpAdd, OpSub, OpDiv, OpSet, OpMod, OpPow) Op <: OpType]

  (implicit @expand.sequence[Op]({_ + _}, {_ - _}, {_ / _}, {(a, b) => b}, {_ % _}, {_ pow _})

  op: Op.Impl2[T, T, T]): Op.Impl2[DenseVector[T], DenseVector[T], DenseVector[T]] = {

    new Op.Impl2[DenseVector[T], DenseVector[T], DenseVector[T]] {

      def apply(a: DenseVector[T], b: DenseVector[T]): DenseVector[T] = {

        (a.length, b.length) match {

          case (1, bl) =>

            val s = a(0)

            val result = DenseVector.zeros[T](bl)

            var i = 0

            while (i < bl) {

              result(i) = op(s, b(i))

              i += 1

            }

            result

          case _ => // irrelevant

            a

        }

      }

  

      implicitly[BinaryRegistry[Vector[T], Vector[T], Op.type, Vector[T]]].register(this)

    }

  }

}

The resulting error is:

[error] (compile:compileIncremental) java.lang.AssertionError: assertion failed: List(method apply$mcI$sp, method apply$mcI$sp)

Attached is an sbt script runner script that can be used for reproduction of the issue. Would this be considered a compiler bug?

Thanks & Best regards,

Martin

[Martin Studer]

Hi,

I found that if I add the following def to VectorOps, the AssertionError disappears. The def, however, needs to be placed _before_ the implicit:

def run(): Unit = {

    val x = DenseVector(1.0, 2.0, 3.0)

val y = DenseVector(1.0, 2.0, 3.0, 4.0, 5.0)

println(x + y)

  }

Regards,

Martin

[Jason Zaugg]

Before we could really dig into this, we'd need to see a reproduction of this with the macro annotation boilerplate hand expanded. As an experimental flavour of an experimentation feature, they are guilty until proven innocent.

-jason

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Hall]

Just to add one note: I run into that particular error in a bunch of random places in breeze at random times, even without macros being involved. I've never been able to minimize it.  It is always with Int specialization (you only see $mcI$) and never with any other type.

[David Hall]

one more: i believe it has to do with the fact that trait Vector[@specialized(Int, Float, Long, Double) V]  extends Tensor[Int, V], with Tensor[@specialized(Int) K, @specialized(Int, Float, Double, Long) V]

and Tensor has 

def apply(k: K):V

and so I think the mangling is getting confused between apply(k: Int):V and apply(k: K):Int (with K and V being generics/erased)

-- David

[Martin Studer]

For the moment I can pin it down to the following (unfortunately I didn't manage to get rid of the macro annotation; see updated repro.scala attached):

import breeze.linalg.DenseVector

import breeze.linalg.operators.OpAdd

import breeze.macros.expand

 

object VectorOps {

  @expand

  implicit def dv_dv_Op[@expand.args(Int) T]: OpAdd.Impl2[DenseVector[T], DenseVector[T], DenseVector[T]] = {

    new OpAdd.Impl2[DenseVector[T], DenseVector[T], DenseVector[T]] {

      def apply(a: DenseVector[T], b: DenseVector[T]): DenseVector[T] = {

        val s = a(0) // if you remove that line it compiles ok

        DenseVector.zeros[T](1)

      }

    }

  }

}

[Adriaan Moors]

To compound the exotic nature, the assertion error occurs during specialization. A bit more of the stack trace:

java.lang.AssertionError: assertion failed: List(method apply$mcI$sp, method apply$mcI$sp)

at scala.reflect.internal.Symbols$Symbol.suchThat(Symbols.scala:1922)

at scala.tools.nsc.transform.SpecializeTypes$$anon$2.matchingSymbolInPrefix$1(SpecializeTypes.scala:1474)

at scala.tools.nsc.transform.SpecializeTypes$$anon$2.transformSelect$1(SpecializeTypes.scala:1497)

at scala.tools.nsc.transform.SpecializeTypes$$anon$2.transform1(SpecializeTypes.scala:1593)

at scala.tools.nsc.transform.SpecializeTypes$$anon$2$$anonfun$transform$3.apply(SpecializeTypes.scala:1442)

at scala.tools.nsc.transform.SpecializeTypes$$anon$2$$anonfun$transform$3.apply(SpecializeTypes.scala:1442)

at scala.tools.nsc.transform.SpecializeTypes$$anon$2.reportError(SpecializeTypes.scala:1434)

[David Hall]

As I mentioned, it is possible to trigger this kind of bug without the macro, but I'm struggling to find cases where I was able to trigger it.

FWIW, the macro is fairly trivia in case it wasn't obvious: it just substitutes the arguments in expand.args at each occurrence of the symbol. It largely (but not entirely) exists to get around the abstraction penalty for arithmetic on primitives.

[David Hall]

Martin could you try it with a breeze build where Tensor's K parameter isn't @specialized(Int)? You might need to take out QuasiTensor's specialization on Int as well

[Martin Studer]

Hi David,

when removing @spec(Int) for K in Tensor and TensorLike the issue seems to indeed disappear. Removing @spec(Int) on Tensor only is not enough.

Best regards,

Martin

[Martin Studer]

On the other hand, removing @spec(Int) for K on TensorLike and QuasiTensor (but keeping it on Tensor) seems to be ok as well.

[Adriaan Moors]

Here's a smaller version (crashes on Scala 2.10.x/2.11.y, breeze 0.11.z forAll x, y, z)

class Test {

  def foo = {

   val dv = breeze.linalg.DenseVector[Int]()

   dv.apply(0) // java.lang.AssertionError: assertion failed: List(method apply$mcI$sp, method apply$mcI$sp)

  }

}

On Thu, Sep 17, 2015 at 11:49 AM, Martin Studer <martin.re...@gmail.com> wrote:

On the other hand, removing @spec(Int) for K on TensorLike and QuasiTensor (but keeping it on Tensor) seems to be ok as well.

--

[Adriaan Moors]

Workaround, avoid overloading of DenseVector's apply method: (dv: QuasiTensor[Int,Int]).apply(0)

(BTW, I confirmed scalac also crashes when compiling against breeze 0.12-SNAPSHOT)

[Adriaan Moors]

Tracked as https://issues.scala-lang.org/browse/SI-9578