Recursive Types (F-Bounded Polymporhism) and Java Interop

[Toby]

Hi,

I have found what I consider to be an inconsistency in the compiler and specifically how it interoperates with Java. I am using scala 2.11.5 and java 1.8.

A brief summary, full example below:

I have a java interface of the form

interface Upper<T extends <Upper<T>>

If a java method returns Upper<?>, the Scala compiler correctly interprets this as Upper[_$1] forSome { type _$1 <: Upper[_$1] }

However, a java interface with a method that accepts Upper<?> as an argument, implemented in Scala, must accept the argument as Upper[_], which expands to Upper[Any] and loses the recursiveness of the type.

I was for example not able to write the method as 

def consume(upper: Upper[T] forSome {type T <: Upper[T]})

even though that is really what the Java version means.

Full example below (a Java and a Scala class), any comments/thoughts/explanations and particularly solutions much appreciated.

Java:

public class TestRecursiveTypes

{

// Base interface for F-Bounded polymorphism

public interface Upper<T extends Upper<T>>

{

public T map();

}

// Implementation of F-Bounded polymorphism

public static class UpperImpl implements Upper<UpperImpl>

{

@Override

public UpperImpl map()

{

System.out.println(String.format("map@%s",this));

return new UpperImpl();

}

}

// Consumer interface

public interface UpperConsumer

{

void consume(Upper<?> upper_);

}

// Consumer implementation

public static class Consumer implements UpperConsumer

{

@Override

public void consume(Upper<?> upper_)

{

upper_.map().map().map();

}

}

// Upper factory method, returns interface

public Upper<?> provideUpper() 

{

return new UpperImpl();

}

public static void main(String[] args)

{

new TestRecursiveTypes().run();

}

private void run()

{

Upper<?> upper = new UpperImpl();

// This is also an upper, without conversio

Upper<?> mapped = upper.map();

Upper<?> upper2 = provideUpper();

new Consumer().consume(upper2.map().map().map());

}

}

Scala (does not compile for reasons explained above):

import TestRecursiveTypes._

import language.existentials

object TestRecursiveTypesScala {

  

  def main(args: Array[String]): Unit = {

    val impl = new UpperImpl()

    

    impl.map().map().map()

   

    val upper = new TestRecursiveTypes().provideUpper()

    

    upper.map().map().map()

  }

  

  class Consumer extends UpperConsumer {

    def consume(upper: Upper[_]): Unit = {

      upper.map().map()

    }

  }

}

[Juha Heljoranta]

Are you absolute sure you have to use forSome keyword?

I think this has all the types you requested and everything works just fine:

$ echo 'trait Upper[T <: Upper[T]]; object Foo {def consume[T <: Upper[T]](u:
Upper[T]): T = ???; def bar[T <: Upper[T]](u: Upper[_]): T = ???}' >
Upper.scala
$ scalac Upper.scala
$ javap Upper
Compiled from "Upper.scala"

public interface Upper<T extends Upper<T>> {
}

$ javap Foo
Compiled from "Upper.scala"
public final class Foo {
public static <T extends Upper<T>> T bar(Upper<?>);
public static <T extends Upper<T>> T consume(Upper<T>);
}


Cheers,
Juha

On Monday, January 12, 2015 01:59:29 Toby wrote:
> Hi,
>
> I have found what I consider to be an inconsistency in the compiler and
> specifically how it interoperates with Java. I am using scala 2.11.5 and
> java 1.8.
>
> A brief summary, full example below:
>
> I have a java interface of the form
>
> interface Upper<T extends <Upper<T>>
>
>
> If a java method returns Upper<?>, the Scala compiler correctly interprets

> this as* Upper[_$1] forSome { type _$1 <: Upper[_$1] }*

>
> However, a java interface with a method that accepts Upper<?> as an
> argument, implemented in Scala, must accept the argument as Upper[_], which
> expands to Upper[Any] and loses the recursiveness of the type.
>
> I was for example not able to write the method as
>
> def consume(upper: Upper[T] forSome {type T <: Upper[T]})
>
>
> even though that is really what the Java version means.
>
> Full example below (a Java and a Scala class), any
> comments/thoughts/explanations and particularly solutions much appreciated.
>

> *Java:*

> *Scala (does not compile for reasons explained above):*

[Toby]

Hi Juha,

I certainly don't want the forSome keyword, in fact I would positively love to avoid it.

I'm not sure I see how your example helps. Your example does not replicate the problem of a Java interface having a method which accepts Upper<?> as an argument, and a Scala implementation of that interface. In the Scala implementation one has to use Upper[_] as the argument type but this loses the recursiveness of the type information (or in other words, the upper bound), and one runs into problems as for example when trying to call map().map() in my original example.

The issue concisely:

Java interface:

void consume(Upper<?> upper_);

Scala Implementation:

override def consume(upper: Upper[_]) ={...} 

The former means "Upper<T extends<Upper<T>>", the latter means "Upper[Any]", yet this is the only version I have succeeded in persuading the compiler to accept as an override. It means I subsequently have to cast upper to Upper[T] forSome {type T <: Upper[T]}. 

[Juha Heljoranta]

trait Upper[T <: Upper[T]] {
def map(): T
}
class UpperImpl extends Upper[UpperImpl] {
def map: UpperImpl = new UpperImpl
}
class Consumer {
def consume(upper: T forSome { type T <: Upper[T] }) =
upper.map.map.map
}
class Test {
def provideUpper() = new UpperImpl()
val upper = new UpperImpl()
val mapped = upper.map()
val upper2 = provideUpper()
new Consumer().consume(upper2.map().map().map())
}

Cheers,
Juha

On Monday, January 12, 2015 12:13:46 Toby wrote:
> Hi Juha,
>
> I certainly don't want the forSome keyword, in fact I would positively love
> to avoid it.
>
> I'm not sure I see how your example helps. Your example does not replicate
> the problem of a Java interface having a method which accepts Upper<?> as
> an argument, and a Scala implementation of that interface. In the Scala
> implementation one has to use Upper[_] as the argument type but this loses
> the recursiveness of the type information (or in other words, the upper
> bound), and one runs into problems as for example when trying to call
> map().map() in my original example.
>
> The issue concisely:
>
> Java interface:
>
> void consume(Upper<?> upper_);
>
>
> Scala Implementation:
>
> override def consume(upper: Upper[_]) ={...}
>
>

> The former means *"Upper<T extends<Upper<T>>"*, the latter means
> *"Upper[Any]", *yet this is the only version I have succeeded in persuading

[Toby]

Still not implementing the Java interface…

[Jason Zaugg]

On Tue, Jan 13, 2015 at 8:42 AM, Toby <to...@meliorbis.com> wrote:

Still not implementing the Java interface…

On Monday, 12 January 2015 22:51:38 UTC+1, Juha Heljoranta wrote:

trait Upper[T <: Upper[T]] {
  def map(): T
}
class UpperImpl extends Upper[UpperImpl] {
  def map: UpperImpl = new UpperImpl
}
class Consumer {
  def consume(upper: T forSome { type T <: Upper[T] }) =
    upper.map.map.map
}
class Test {
  def provideUpper() = new UpperImpl()
  val upper = new UpperImpl()
  val mapped = upper.map()
  val upper2 = provideUpper()
  new Consumer().consume(upper2.map().map().map())
}

Cheers,
Juha

I think you need to resort to a cast.

  class Consumer extends UpperConsumer {
    def consume(upper: Upper[_]): Unit

 = {
      fbound(upper).map().map()
    }
  }
  // Workaround for the limbo state given that:
  // a) Java wildcard types propagate bounds (after the fix for https://issues.scala-lang.org/browse/SI-6169)
  // b) Scala existentials don't https://issues.scala-lang.org/browse/SI-1786
  def fbound(u: Upper[_]) = u.asInstanceOf[Upper[A] forSome { type A <: Upper[A] }]

​

I've added your excellent test case to the comments of SI-1786. We think we have a means of improving the situation for Scala 2.12. We have been using experimental dotty compiler to try out improved bounds propagation; I just checked and your test case is accepted by dotc. 

-jason

[Toby]

Thanks, Jason - no elegant solution for now then, but hope for the future.

Purely as a matter of interest, do you have any insight on the inconsistency I mentioned earlier: the compiler correctly infers a return type (from a Java API) of Upper<?> to be Upper[A] forSome { type A <: Upper[A] }, but when implementing a method from a Java interface with an argument of the same type only Upper[_] is accepted?

[Jason Zaugg]

On Tue, Jan 13, 2015 at 9:25 AM, Toby <to...@meliorbis.com> wrote:

Thanks, Jason - no elegant solution for now then, but hope for the future.

Purely as a matter of interest, do you have any insight on the inconsistency I mentioned earlier: the compiler correctly infers a return type (from a Java API) of Upper<?> to be Upper[A] forSome { type A <: Upper[A] }, but when implementing a method from a Java interface with an argument of the same type only Upper[_] is accepted?

I think it is because the fix for SI-6169 is limited to Java wildcard types.

-jason

[Toby]

Thanks, Jason.