Riddle -- how can contravariant types be specialized at all?

57 views
Skip to first unread message

Sciss

unread,
Nov 7, 2011, 3:31:37 PM11/7/11
to scala-user
hi,

i don't get the :javap output here -- can someone tell me in the following, Test is fully specialized? my gutts say it can't because of the variance annotation of type A in trait KeyObserver -- but then shouldn't scalac complain about this?

```scala

trait KeyObserver[ @specialized( Int, Long ) -A ] { def keyUp( key: A ) : Unit }
object NoKeyObserver extends KeyObserver[ Any ] { def keyUp( key: Any ) {} }
defined module NoKeyObserver

trait Test[ @specialized( Int, Long ) A ] {
def lala : A
def obs : KeyObserver[ A ]

// test specialization
obs.keyUp( lala )
}


should i change this to

```scala

trait KeyObserver[ @specialized( Int, Long ) A ] { def keyUp( key: A ) : Unit }
def NoKeyObserver[ A ] : KeyObserver[ A ] = new KeyObserver[ A ] { def keyUp( key: A ) {} }

instead?

thanks, -sciss-

Aleksey Nikiforov

unread,
Nov 7, 2011, 4:42:18 PM11/7/11
to Sciss, scala-user
Specialization kicks in when compiler knows that all the relevant types fit the specialized annotation. When you call obs.keyUp( lala ) in trait Test, compiler cannot possibly know what A will be, so it will not use specialization. This has nothing to do with variance.

On the other hand, when all the types in question are know, variance does not matter when it comes to specialization:

 class IntObs extends KeyObserver[Int] { def keyUp( key: Int ) {  } }
 class AnyObs extends KeyObserver[Any] { def keyUp( key: Any ) {  } }
 
 class Test1 extends Test[Int] {
   def lala = 1
   def obs = new IntObs
   
   obs.keyUp( lala ) // specialized
 }
 
 class Test2 extends Test[Int] {
   def lala = 1
   def obs = new AnyObs

   obs.keyUp( lala ) // not specialized

 }

Sciss

unread,
Nov 7, 2011, 5:21:17 PM11/7/11
to Aleksey Nikiforov, scala-user
so, are you saying specialization doesn't compose at all??

trait KeyObserver[ @specialized( Int ) A ] { def keyUp( k: A ) : Unit }
class Test2[ @specialized( Int ) A ]( obs: KeyObserver[ A ], lala: A ) { obs.keyUp( lala )}

scala> :javap -v Test2

public Test2(KeyObserver, java.lang.Object);
Code:
Stack=2, Locals=3, Args_size=3
0: aload_0
1: aload_1
2: putfield #17; //Field obs:LKeyObserver;
5: aload_0
6: aload_2
7: putfield #19; //Field lala:Ljava/lang/Object;
10: aload_0
11: invokespecial #24; //Method java/lang/Object."<init>":()V
14: aload_0
15: getfield #17; //Field obs:LKeyObserver;
18: aload_0
19: getfield #19; //Field lala:Ljava/lang/Object;
22: invokeinterface #30, 2; //InterfaceMethod KeyObserver.keyUp:(Ljava/lang/Object;)V // !!
27: return

i can't believe it.

so what is the purpose of specialization -- do i need to write for every primitive still concrete classes, and i must under no circumstances call into generic methods?

something must be wrong here, i hope?

best, -sciss-

Sciss

unread,
Nov 7, 2011, 5:53:10 PM11/7/11
to Sciss, Aleksey Nikiforov, scala-user
it seems, i still have hope:

trait Key[ @specialized( Int ) A ] { def up( v: A ) : Unit }

class KeyTest[ @specialized( Int ) A ] extends Key[ A ] { def up( v: A ) { sys.error( "here" )}}

final case class Value[ @specialized( Int ) A ]( v: A )

trait Test[ @specialized( Int ) A ] {
def value: Value[ A ]
def key: Key[ A ]
def test { key.up( value.v )}
}

object TestFactory {
def apply[ @specialized( Int ) A ]( _key: Key[ A ], _value: Value[ A ]) : Test[ A ] = new Test[ A ] { def key = _key; def value = _value }
}

val x = TestFactory( new KeyTest[ Int ], Value( 33 ))

x.test

java.lang.RuntimeException: here
at scala.sys.package$.error(package.scala:27)
at KeyTest$mcI$sp.up$mcI$sp(<console>:8)
at KeyTest$mcI$sp.up(<console>:8)
at KeyTest$mcI$sp.up(<console>:8)
at Test$class.test(<console>:13)
at TestFactory$$anon$2.test(<console>:12)
at .<init>(<console>:15)
at .<clinit>(<console>)
at .<init>(<console>:11)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:704)
at scala.tools.nsc.interpreter.IMain$Request$$anonfun$14.apply(IMain.scala:920)
at scala.tools.nsc.interpreter.Line$$anonfun$1.apply$mcV$sp(Line.scala:43)
at scala.tools.nsc.io.package$$anon$2.run(package.scala:25)
at java.lang.Thread.run(Thread.java:637)

so can it be that there is a limitation for the constructor body of specialized traits? otherwise this seems to compose nicely and as expected.... however i don't see in the stack trace that actually a specialized version of method `test` has been called...?

best, -sciss-

Aleksey Nikiforov

unread,
Nov 7, 2011, 5:56:48 PM11/7/11
to Sciss, scala-user
Specialization does compose, but in a slightly different way. The choice to use specialization is made at the CALL SITE. The limitations of specialization are the consequences of that. What is boils down to is that specialization only composes at the method level, and your methods must have at least one argument that have the type marked as specialized.


object Main {
  def main(args: Array[String]) {
    val t = new Test2(new IntObs, 1)
    t.bar // not specialized
    t.foo // not specialized
    t.foo[Int] // still not specialized
    t.foo(1) // specialized
  }
}

trait KeyObserver[ @specialized( Int ) A ] { def keyUp( k: A ) : Unit }

class IntObs extends KeyObserver[Int] { def keyUp( key: Int ) {  } }
class AnyObs extends KeyObserver[Any] { def keyUp( key: Any ) {  } }
 
class Test2[ @specialized( Int ) A ]( obs: KeyObserver[ A ], lala: A ) {
  obs.keyUp( lala )
  
  def bar() {
    obs.keyUp( lala )
  }
  
  def foo[X <: A]() { // will produce a specialized method that is never used.
    obs.keyUp( lala )
  }
  
  def foo(a: A) {
    obs.keyUp(a)
  }
}

Constructors look like methods with specialized signatures, but they are not specialized.

Of course specialization also composes by extending specialized traits. Extending specialized classes is somewhat buggy and requires intimate knowledge of what goes on under the hood. If you try extending a specialized class you will get a compiler warning.

Overall, using specialization is a bit like walking on the minefield. One wrong step - and primitives get boxed. So I suggest you get a decent decompiler and verify your code. Here is the one I use: http://java.decompiler.free.fr/

Sciss

unread,
Nov 7, 2011, 6:16:15 PM11/7/11
to Aleksey Nikiforov, scala-user
(was missing the scala-user CC)

oh.... so it did _not_ work?

here is another example that i thought should work:

final case class Compare[ @specialized( Int ) A ]( fun: (A, A) => Int ) { def apply( a: A, b: A ) : Int = fun( a, b )}

trait SkipList[ @specialized( Int ) A ] {
def compareFun: Compare[ A ]
private class Node( var value: A ) {
def compare( b: Node ) : Int = compareFun( value, b.value )
}

def test( a: A, b: A ) : Int = {
val n = new Node( a )
val m = new Node( b )
n compare m
}
}

object SkipList { def apply[ A ]( _compareFun: Compare[ A ]) : SkipList[ A ] = new SkipList[ A ] { def compareFun = _compareFun }}

val l = SkipList( Compare[ Int ] { (a, b) => sys.error( "here" )})

l.test( 1, 2 )

java.lang.RuntimeException: here
at scala.sys.package$.error(package.scala:27)

at $anonfun$1.apply(<console>:11)
at $anonfun$1.apply(<console>:11)
at scala.Function2$class.apply$mcIII$sp(Function2.scala:33)
at scala.runtime.AbstractFunction2.apply$mcIII$sp(AbstractFunction2.scala:12)
at Compare$mcI$sp.apply$mcI$sp(<console>:7)
at Compare$mcI$sp.apply(<console>:7)
at Compare$mcI$sp.apply(<console>:7)
at SkipList$Node.compare(<console>:12)
at SkipList$class.test(<console>:18)
at SkipList$$anon$1.test(<console>:10)
at SkipList$class.test$mcI$sp(<console>:15)
at SkipList$$anon$1.test$mcI$sp(<console>:10)
at .<init>(<console>:13)
...

again here is your pattern:

at Compare$mcI$sp.apply$mcI$sp(<console>:7)
at Compare$mcI$sp.apply(<console>:7)
at Compare$mcI$sp.apply(<console>:7)

however not in the actual function call (scala.runtime.AbstractFunction2.apply$mcIII$sp)

--- so i don't know what to make of this. is there boxing happing?

if so, why is that? because Node.compare doesn't contain a "naked" A? that would truly suck. should i add a dummy argument? (that would be terribly messy)


thanks again, -sciss-

On 7 Nov 2011, at 23:04, Aleksey Nikiforov wrote:

> Seem you have posted more while I was typing the response. When you see something like this in the stack trace, it means specialization did not work out:


>
> at KeyTest$mcI$sp.up$mcI$sp(<console>:8)
> at KeyTest$mcI$sp.up(<console>:8)
> at KeyTest$mcI$sp.up(<console>:8)
>

> When specialization works you will see only one specialized call:
>
> at KeyTest$mcI$sp.up$mcI$sp(<console>:8)
>
>

Aleksey Nikiforov

unread,
Nov 7, 2011, 6:28:03 PM11/7/11
to Sciss, scala-user
  private class Node( var value: A ) {
     def compare( b: Node ) : Int = compareFun( value, b.value )
  }

class Node is not specialized. You would have to do: class Node[@specialized(Int) B <: A]( var value: B ) or something along those lines.

Sciss

unread,
Nov 7, 2011, 6:31:19 PM11/7/11
to Aleksey Nikiforov, scala-user
but Node is an _inner_ class of a type which is specialized in A. i really cannot add the type parameter again, that would mean i rewrite 90% of my code, plus it seriously damaged readability.

Aleksey Nikiforov

unread,
Nov 7, 2011, 7:15:12 PM11/7/11
to Sciss, scala-user
Specialization works by generating every combination of primitive types. This can cause an explosion of type signatures. Scala takes a very conservative approach and makes it an opt-in feature rather than opt-out. As a result you have to annotate every class and every type parameter you want specialized.

Try compiling this one-liner to get the idea how crazy specialization can get:

class S[@specialized T0, @specialized T1, @specialized T2, @specialized T3](v0: T0, v1: T1, v2: T2, v3: T3)

It will take a while, generate 6564 classes, and consume 5.4 Mb of disk space. All from one line of code.

Sciss

unread,
Nov 7, 2011, 7:43:30 PM11/7/11
to Aleksey Nikiforov, scala-user
i understand. but if i have

trait X[ @specialized( Int ) A ] {
private class Node {
var payload: A
}
}

i don't get why that wouldn't be specialized. this is by no means creating any sort of explosion, because Node as an inner class is inherently tied to X, so the maximum number of classes equals the number of specializations in A, not to the power of two.

this just shows for me that type parameters and path-dependent types are second-class citizens in scala, and i find that very implausible.

best, -sciss-

Sciss

unread,
Nov 7, 2011, 7:45:55 PM11/7/11
to Sciss, Aleksey Nikiforov, scala-user
abstract type _members_ i wanted to say.

trait X {
type Y
}

versus

trait X[ Y ]

Aleksey Nikiforov

unread,
Nov 7, 2011, 11:58:16 PM11/7/11
to Sciss, scala-user
Abstract types and inner classes are different concepts. So I am not quite sure what abstract and path-dependent types have to do with specialization.

As for inner classes, each inner class becomes a java class file after compilation. If you Node would keep specialization of the parent class, it would effectively double the total number of classes, provided it uses all the specialized types. Now, if you have 10 inner classes, you get 10 times more classes.

Specialization works well when you have a limited number of specialized types, otherwise the number of combinations can explode very quickly. So specialization is made explicit, to make sure you know what you are doing, and the very least prevent accidental explosion in generated number of classes. Also this approach makes it easy to track down the causes by simply searching for @specialized annotation.

Overall, given how unpredictable specialization results can be, I think it was the right decision to make it so explicit. I do not believe it has anything to do with second-class citizens, but more to do with potential problems with specialization when used without caution.

Sciss

unread,
Nov 8, 2011, 10:59:13 AM11/8/11
to Aleksey Nikiforov, scala-user
i disagree with judging this situation as explosion. that for me implies exponential behaviour. as if you'd have 4 * 4 = 16 combinations for an inner and an outer class specializing in four types, where you would only need 4 + 4. so the number of classes generated is linear. wouldn't it be possible at least to get proper specialization like this:

trait SkipList[ @specialized( Int, Long) A ] {
@specialized private class Node {
var value A
...
}
}

the idea of specialization for me is to get maximum performance, that's the only reason why you do the effort to put the annotations. so if get a half-way specialized result with numerous boxing, unboxing, that is clearly unintended and counter-intuitive. the compiler should at least tell me

Warning: inner class Node is not specialized in A.
enforce specialization by annotating this class as @specialized

or similar

?

The current behaviour just means Scala is unsuitable for creating elegant high-performance data structures. When i start now to move around my classes and annotate methods and so forth to eventually end up in a fully specialized data structure -- which seems almost impossible to observe --, probably the result would be more readable if i go back to Java. I'm very disappointed by this.

best, -sciss-

Matthew Pocock

unread,
Nov 8, 2011, 11:24:56 AM11/8/11
to Sciss, Aleksey Nikiforov, scala-user
On 8 November 2011 15:59, Sciss <con...@sciss.de> wrote:

The current behaviour just means Scala is unsuitable for creating elegant high-performance data structures.

Unfortunately, I came to the same conclusion for my applications. @specialize is currently too buggy, not calling specialized calls in several important cases. The compiler doesn't help you track down issues where non-specialized things are called from specialized code and vice-versa - you end up crawling through (decompiled) bytecode and multi-megabyte compiler logs. It also doesn't give you fine-grained control over what combinations are specialized for - it expands the cross-product of the type parameters, but often you only want the 'same type' cases specialized. Specialization and derived classes/traits seem to not play at all well with each other. None of this can be fixed without some resources being thrown at it, but the pool of people who understand the specialization plugin is small.
 
When i start now to move around my classes and annotate methods and so forth to eventually end up in a fully specialized data structure -- which seems almost impossible to observe --, probably the result would be more readable if i go back to Java. I'm very disappointed by this.


For now, when I find that boxing is the bottleneck and @specialize doesn't work, I end up hand-rolling an implementation for the concrete primitives that I need for that application. It's far from ideal.

I'd be happy to work with someone to fix the specialization plugin, but I'd need mentoring, and I expect the pool of available mentors is even smaller than those who understand the plugin to begin with.

Matthew
 
best, -sciss-


Matthew
 

--
Dr Matthew Pocock
Integrative Bioinformatics Group, School of Computing Science, Newcastle University
skype: matthew.pocock
tel: (0191) 2566550

Rex Kerr

unread,
Nov 8, 2011, 11:33:24 AM11/8/11
to Sciss, Aleksey Nikiforov, scala-user
On Tue, Nov 8, 2011 at 10:59 AM, Sciss <con...@sciss.de> wrote:

The current behaviour just means Scala is unsuitable for creating elegant high-performance data structures. When i start now to move around my classes and annotate methods and so forth to eventually end up in a fully specialized data structure -- which seems almost impossible to observe --, probably the result would be more readable if i go back to Java. I'm very disappointed by this.

I sympathize.  I've found the same thing.  Specialization was a good idea, but the current implementation falls on its face in so many cases that it's almost unusable for anything with nontrivial structure.  Also, the bugs do not seem to be getting fixed very quickly.  Still, if you find a new bug that isn't in JIRA, it would be great if you could add it for the benefit of the rest of us who have tried to use specialization.

I write code generators instead, most of the time.

I think I may have managed to write a specialized mutable tuple library using specialization without it completely breaking, but my efforts to create a specialized mutable collection library have thus far been pretty unsuccessful.

  --Rex
 

Sciss

unread,
Nov 8, 2011, 12:02:49 PM11/8/11
to Matthew Pocock, Aleksey Nikiforov, scala-user
another 'solution' is to copy the source code, use the very well working refactoring tool of IDEA to translate the type parameter 'A' to 'Int' and 'Long', and then rename SkipListImpl[Int] to IntSkipListImpl.

but that doesn't really scale, because once you need to change code afterwards, you need to change it three times.

i wonder if there is maybe a capable source-rewriting tool around that can do this refactoring automatically, like a nice little sbt plugin... ?

best, -sciss-

Sciss

unread,
Nov 8, 2011, 12:10:10 PM11/8/11
to Rex Kerr, Aleksey Nikiforov, scala-user
tada..... that is what i was thinking now. can you recommend an approach here that i could use?

best, -sciss-

Rex Kerr

unread,
Nov 8, 2011, 12:39:59 PM11/8/11
to Sciss, scala-user
For some code I just use search-and-replace (generally using $x$ or $$x$$ as the target pattern); just write the code in strings (and include the compile-and-pipe-to-output as an initial step for compiling the code).  For example,

  lowup.zipWithIndex.map { case ((l,u),i) =>
    "trait Oople%s[@specialized %s] extends Muple%s[%s] with Oople {\n".format(u,u,u,u) +
    "  def op%s = if (ok) Some(%s) else None\n".format(u,l) +
    "  def map%s(fn: %s => %s) = { if (ok) %s = fn(%s); this }\n".format(u,u,u,l,l) +
    "}"
  }

produces a bunch of traits from other traits.  (In this case, specialization seems to work, but I generated code for different numbers of arguments; if you *just* want to use it to replace specialization, this approach is usually good enough, I've found.)

For cases with more complex structure, I typically create a class hierarchy representing the types I want to specialize and manipulations between.  For example,

  object GenericType extends RType("G","Generic") {
    override def defmods = "abstract "; def supertype = None
  }
  object ImmutableType extends RType("I","Immutable") {
    override def defmods = "final case "; def supertype = Some(GenericType)
  }
  object AssignableType extends RType("A","Assignable") {
    override def mutates = true; override def defmods = "abstract "
    def supertype = Some(GenericType)
  }
  object MutableType extends RType("M","Mutable") {
    override def mutates = true; override def defmods = "final "
    def supertype = Some(AssignableType)
  }
  object BackedType extends RType("B","Backed") {
    override def mutates = true; def supertype = Some(AssignableType)
  }
  val reps = List(GenericType, ImmutableType, AssignableType, MutableType, BackedType)
  val canon = reps.map(r =>
    r -> (if (r==GenericType || r==ImmutableType) ImmutableType else MutableType)
  ).toMap

is part of a system to represent the inheritance hierarchy that I want to automatically generate.

So far I haven't really needed a powerful generic system for code generation; Scala is sufficiently powerful so I can just come up with an ad-hoc solution each time.

I am hoping that macro facilities will obviate the need for some of these things.

  --Rex

Sciss

unread,
Nov 15, 2011, 9:33:43 AM11/15/11
to scala-user
of course, i invested energy again to remove the inner classes in favour of outer classes with type parameters so specialization kicks in again. i should have known:

[info] Compiling 1 Scala source to /Users/hhrutz/Documents/devel/TreeTests/target/scala-2.9.1/classes...
[error] {file:/Users/hhrutz/Documents/devel/TreeTests/}default-92482a/compile:compile: scala.tools.nsc.symtab.Types$TypeError: type mismatch;
[error] found : de.sciss.collection.txn.HASkipList.Node[S(in class Leaf),A]
[error] required: de.sciss.collection.txn.HASkipList.Child[S(in class Leaf$mcI$sp),Int]
[error] Total time: 1 s, completed Nov 15, 2011 2:28:05 PM

are there any tricks to get more information from this -- like which is the line that crashes the compiler?

best, -sciss-


On 8 Nov 2011, at 16:33, Rex Kerr wrote:

Aleksey Nikiforov

unread,
Nov 15, 2011, 10:01:27 AM11/15/11
to Sciss, scala-user
I was told to use  -Ylog:icode -Ydebug to find where compiler has crashed, but depending at what stage it crashed you may not get any useful info. The only 100% reliable method is removing parts of your project until it start compiling again. Rinse and repeat with the last removed part until you narrow down the cause.

You could also try scala nightly. I have encountered several compiler crashes recently, and most of them were fixed in unrleased 2.10.

Sciss

unread,
Nov 15, 2011, 10:05:23 AM11/15/11
to Aleksey Nikiforov, scala-user
ok, the first thing doesn't give me any additional info:

[info] Compiling 47 Scala sources and 11 Java sources to /Users/hhrutz/Documents/devel/TreeTests/target/scala-2.9.1/classes...
[info] [running phase parser on 58 compilation units]
[info] [running phase namer on 58 compilation units]
[info] [running phase packageobjects on 58 compilation units]
[info] [running phase typer on 58 compilation units]
[info] [running phase superaccessors on 58 compilation units]
[info] [running phase pickler on 58 compilation units]
[info] [running phase refchecks on 58 compilation units]
[info] [running phase liftcode on 58 compilation units]
[info] [running phase uncurry on 58 compilation units]
[info] [running phase tailcalls on 58 compilation units]
[info] [running phase specialize on 58 compilation units]


[error] {file:/Users/hhrutz/Documents/devel/TreeTests/}default-92482a/compile:compile: scala.tools.nsc.symtab.Types$TypeError: type mismatch;

[error] found : HASkipList.this.Node[S(in class Leaf),A]
[error] required: HASkipList.this.Child[S(in class Leaf$mcI$sp),scala.this.Int]
[error] Total time: 9 s, completed Nov 15, 2011 3:03:34 PM


i will see if i can build with 2.10, but i have various dependencies, so probably not. might need to add -no-specialization and screw this wasted attempt...

Sciss

unread,
Nov 15, 2011, 10:19:32 AM11/15/11
to Sciss, Aleksey Nikiforov, scala-user
ok, i was able to 'avoid' the bug. it seems to have been a problem with specialization and pattern matching. before i had:

@tailrec def step( num: Int, n: Node[ S, A ]) : Int = n match {
case Leaf( _ ) => num
case Branch( b ) => step( num + 1, b.down( 0 ))
}

with my extractor objects like this to avoid warnings due to type erasure:

object Node {
def unapply[ S <: Sys[ S ], @specialized( Int ) A ]( child: Child[ S, A ]) : Option[ Node[ S, A ]] = child.nodeOption
}

so this was somehow crashing the compiler/specialization. how if i go back to crappy warnings:

@tailrec def step( num: Int, n: Node[ S, A ]) : Int = n match {
case _: Leaf[ _, _ ] => num
case b: Branch[ S, A ] => step( num + 1, b.down( 0 ))
}

i get a strange warning:

warn] /Users/hhrutz/Documents/devel/TreeTests/src/main/scala/de/sciss/collection/txn/HASkipList.scala:155: non variable type-argument S in type pattern de.sciss.collection.txn.HASkipList.Branch[S,A] is unchecked since it is eliminated by erasure
[warn] case b: Branch[ S, A ] => step( num + 1, b.down( 0 ))
[warn] ^
...
[warn] missing combination Leaf
[warn] missing combination Leaf$mcI$sp
[warn] @tailrec def step( num: Int, n: Node[ S, A ]) : Int = n match {
[warn] ^

a guess this latter warning is a bug with the interaction between pattern matching and specialization?

also i'm curious why i only get a warning about the erasure of `S` (my list's transactional system), but not because of `A` (my list's element type, specialized)


best, -sciss-

Aleksey Nikiforov

unread,
Nov 15, 2011, 10:22:09 AM11/15/11
to Sciss, scala-user
I tend to lose my cool when encountering rough edges in Scala, espectially if the problem involves a compiler crash. However, I have figured that Scala makes me about twice as productive when compared to Java (a factor of 2.0). So if I spend as much as 10% of my overall time chasing down and working around bugs in the Scala itself, then my overall efficiency is: 0.9*2.0 = 1.8. Still 80% higher than using Java.

That said, you should try to track down the problem and file the bug report. As a bonus, once you know the problem you may be able to work around it.

Sciss

unread,
Nov 15, 2011, 12:18:36 PM11/15/11
to Aleksey Nikiforov, scala-user
specialization is like an invitation to shoot yourself in the foot

class Test1[ @specialized( Int ) A ] {
def test( a: A ) {
sys.error( "here" )
}
}

val t1 = new Test1[ Int ]
t1.test(0)
// ok:
// java.lang.RuntimeException: here
// at scala.sys.package$.error(package.scala:27)
// at Test1$mcI$sp.test$mcI$sp(<console>:34)

abstract class Test2[ @specialized( Int ) A ] {
def test( a: A ) {
sys.error( "here" )
}
}


def ano[ @specialized( Int ) A ] : Test2[ A ] = new Test2[ A ] {}

val t2 = ano[ Int ]
t2.test( 1 )
// not ok:
// java.lang.RuntimeException: here
// at scala.sys.package$.error(package.scala:27)
// at Test2.test(<console>:34)
// at Test2.test$mcI$sp(<console>:33)


:-(

best, -sciss-


On 7 Nov 2011, at 23:04, Aleksey Nikiforov wrote:

Rex Kerr

unread,
Nov 15, 2011, 12:36:20 PM11/15/11
to Sciss, Aleksey Nikiforov, scala-user
Is that one not in the bug tracker?  Better add it, at least to have it throw out the same "can't inherit specialization" warning that it does when you try
  class Test3[@specialized(Int) A] extends Test2[A] {}
(which is really what's going on here).

  --Rex

Sciss

unread,
Nov 15, 2011, 12:42:08 PM11/15/11
to Rex Kerr, Aleksey Nikiforov, scala-user
btw, this seems independent of whether an abstract class or a trait is extended:

trait Test4[ @specialized( Int ) A ] {


def test( a: A ) {
sys.error( "here" )
}
}

def ano2[ @specialized( Int ) A ] : Test4[ A ] = new Test4[ A ] {}
val t4 = ano2[ Int ]
t4.test(0) // OK!

class Test5[ @specialized( Int ) A ] extends Test4[ A ]
val t5 = new Test5[ Int ]
t5.test(0) // Not OK!

Aleksey Nikiforov

unread,
Nov 15, 2011, 2:36:30 PM11/15/11
to Sciss, scala-user
Everything worked as expected.

// at scala.sys.package$.error(package.scala:27) <-- Your error call happens last
//      at Test2.test(<console>:34) <- Generic test call happens just before the error call.
//      at Test2.test$mcI$sp(<console>:33) <- Specialized call happens, first, but it delegates to generic implementation.

So specialized call happened first, means it worked. The big question is: can you really do anything with "a: A" at the abstract level without avoiding boxing? I mean, you dont know what A will be, so there is no way to specialize around that. The only thing you can do is override test(), and provide a specialized implementation. But in that case the specialized call will delegate to your override instead of the generic implementation of the abstract class.

iulian dragos

unread,
Nov 16, 2011, 5:27:30 PM11/16/11
to Aleksey Nikiforov, Sciss, scala-user
On Mon, Nov 7, 2011 at 11:56 PM, Aleksey Nikiforov <lex...@gmail.com> wrote:
Specialization does compose, but in a slightly different way. The choice to use specialization is made at the CALL SITE. The limitations of specialization are the consequences of that. What is boils down to is that specialization only composes at the method level, and your methods must have at least one argument that have the type marked as specialized.


object Main {
  def main(args: Array[String]) {
    val t = new Test2(new IntObs, 1)
    t.bar // not specialized
    t.foo // not specialized
    t.foo[Int] // still not specialized
    t.foo(1) // specialized
  }
}

trait KeyObserver[ @specialized( Int ) A ] { def keyUp( k: A ) : Unit }

class IntObs extends KeyObserver[Int] { def keyUp( key: Int ) {  } }
class AnyObs extends KeyObserver[Any] { def keyUp( key: Any ) {  } }
 
class Test2[ @specialized( Int ) A ]( obs: KeyObserver[ A ], lala: A ) {
  obs.keyUp( lala )
  
  def bar() {
    obs.keyUp( lala )
  }
  
  def foo[X <: A]() { // will produce a specialized method that is never used.
    obs.keyUp( lala )
  }
  
  def foo(a: A) {
    obs.keyUp(a)
  }
}

Constructors look like methods with specialized signatures, but they are not specialized.

Of course specialization also composes by extending specialized traits. Extending specialized classes is somewhat buggy and requires intimate knowledge of what goes on under the hood. If you try extending a specialized class you will get a compiler warning.

Overall, using specialization is a bit like walking on the minefield. One wrong step - and primitives get boxed. So I suggest you get a decent decompiler and verify your code. Here is the one I use: http://java.decompiler.free.fr/

You could also use this compiler plugin:


You'd need to build it yourself and it was written for 2.8, but it should work out of the box with 2.9.

iulian



--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

Sciss

unread,
Nov 16, 2011, 5:30:33 PM11/16/11
to iulian dragos, Aleksey Nikiforov, scala-user
ah that's great! thanks!

[...]


On 16 Nov 2011, at 22:27, iulian dragos wrote:
>
> You could also use this compiler plugin:
>
> https://github.com/dragos/noboxing-plugin
>
> You'd need to build it yourself and it was written for 2.8, but it should work out of the box with 2.9.
>
> iulian

[...]

Reply all
Reply to author
Forward
0 new messages