Question about type class design pattern of ImmutableNumericOps in Breeze

[meng....@gmail.com]

Hi all,

This is Meng Pan. I'm now using Scala's Type Class design pattern in developing my own Deep Learning framework in Scala. However, I have encountered a few problems and can't find solution to solve them. I found that implicits are also used intensively in Breeze, but the same problems I have met seem to not exist in the implementation of Breeze. Hence could anyone help me to solve these issues?

A simplified version of the problems I have met is as follows:

Suppose I have a trait CanAutoInit for initializing my neural network's layer:

-------------------------------------------------
trait CanAutoInit[For] {
  def init(f: For): Unit
}
-------------------------------------------------

Then there are a trait Layer and a trait LayerLike:

-------------------------------------------------
trait Layer extends LayerLike[Layer]

trait LayerLike[+Repr <: Layer] extends Any{
  def repr: Repr = this.asInstanceOf[Repr]
  final def init[TT >: Repr](implicit op: CanAutoInit[TT]): Unit = op.init(repr)
}
-------------------------------------------------

Here I pick two implimentations of the trait Layer, namely DropoutLayer and PoolingLayer, as an example to state my question:

-------------------------------------------------
class DropoutLayer extends Layer with LayerLike[DropoutLayer]

object DropoutLayer {
  implicit def dropoutCanAutoInit = new CanAutoInit[DropoutLayer] {
    override def init(f: DropoutLayer): Unit = println("Init inside Dropout Layer")
  }
}

class PoolingLayer extends Layer with LayerLike[PoolingLayer]

object PoolingLayer {
  implicit def poolingCanAutoInit = new CanAutoInit[PoolingLayer] {
    override def init(f: PoolingLayer): Unit = println("Init inside PoolingLayer")
  }
}
-------------------------------------------------

Then I create instances of DropoutLayer and PoolingLayer seperately, and initialize them, everything works properly:

-------------------------------------------------
val a = new DropoutLayer
a.init
val b = new PoolingLayer
b.init
-------------------------------------------------

When executing the above code, I will see "Init inside Dropout Layer" and "Init inside PoolingLayer" printed on screen as expected.

However, if I combine a and b into a list, of the type List[Layer], then compiler will complain with "could not find implicit value for parameter op: CanAutoInit[Layer]":

-------------------------------------------------
val list: List[Layer] = List(a, b)
list.foreach(_.init)
-------------------------------------------------

In order to find solution to this problem, I have tested CSCMatrix and DenseMatrix in Breeze using the same logic, and I can get the correct result I want:

-------------------------------------------------
import breeze.linalg.{CSCMatrix, DenseMatrix}

val sparse = new CSCMatrix[Double]((1 to 100).map(_.toDouble).toArray, 9, 9, (0 to 9).toArray, (0 to 9).toArray)
val dense = DenseMatrix.ones[Double](10, 10)

sparse.t
dense.t

val list = List(sparse, dense)
list.map(_.t)
-------------------------------------------------

Thus I really want to know has anyone encountered the above problem? And If so, how did you resolve it?

Thank you very much for anyone's vire and help, I am really looking forward to your reply.

Sincerely,

Meng

[David Hall]

Hi Meng,

We did encounter this in Breeze, and we have a solution, though it's maybe not the best. Basically, there's a supertrait Matrix that has its own capability type class implementations, some of which inherit from a "Registry" that allows for virtual function dispatch. It works ok and I think it's probably fine for what you're trying to do.

I can think of two alternatives. One involves using Shapeless HLists instead of lists, so that you maintain static type information. HLists can be a bit difficult to work with, but I think that should be ok.

The other looks a bit more like what Rust does implicitly. Basically, define another trait `Can[T, X[T]]` and an implicit conversion `implicit canHas[T, X[T]](t: T)(implicit x: X[T])`. Then make the list be `List[Can[_, CanAutoInit]]` I've never tried this before and I've never seen it, but it might work fine (though it might not work with scala's type inference, who knows)

-- David

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze+unsubscribe@googlegroups.com.
To post to this group, send email to scala-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/e932007c-0814-4035-a96f-122d9542b4a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[meng....@gmail.com]

Hi David, 

Thank you very much for your help! I will try the solutions today. Thanks again! 

Sincerely, 

Meng 

[meng....@gmail.com]

Hi David, 

I'm sorry to bother you again. But after working with Scala after such a long time, I still cannot find a way to deal with the above problem I mentioned last year. More specifically, I don't understand what "virtual function dispatch" means in your word "some of which inherit from a "Registry" that allows for virtual function dispatch". Would you mind giving me a more specific example about it? 

I'm so appreciating for your help! Thank you very much!! 

Sincerely, 

Meng 

在 2017年12月28日星期四 UTC+8上午3:33:50，David Hall写道：

To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze...@googlegroups.com.

[David Hall]

Hi Meng,

Sorry for the slow response!

So, typeclasses are usually basically static dispatch: they provide compile-time polymorphism by allowing you to say type T must come with some some bundle of functionality. However, sometimes, you might want dynamic dispatch, where subtypes of type T might have different implementations of that functionality. For instance, SparseVectors and DenseVectors are both Vectors, but (performant) implementations of dot product for SV/SV, DV/DV and SV/DV are fairly different. So, in Breeze, we sometimes have "type class registries" (BinaryRegistry, etc) that can take an object of type T (or two objects of type T, or a type T and a type U) and select the appropriate runtime implementation via dynamic dispatch (i.e. looking up their runtime classes in a hash table).

You can look at the BinaryRegistry type in Breeze (and all the places it's used) to get a sense.

-- David

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/7ba5919a-9291-4989-9d77-d3a05b599f8a%40googlegroups.com.

[meng....@gmail.com]

Hi David, 

I've looked into BinaryRegistry type and got a clear sense about it! Thank you very much for your help! 

Have a nice day and thanks again for your excellent work in Breeze! 

Sincerely, 

Meng 

在 2018年5月19日星期六 UTC+8上午1:57:49，David Hall写道：