traits, binary compatibility, and the protection racket

[Paul Phillips]

Consider the perversion taking place in the following tale.

    trait A {

      private[this] val bippy = 5

      def dingus = bippy * bippy

    }

    class B extends A

trait A has a single public method and a private[this] immutable field

which does not escape and which the language prohibits anything outside

of A from accessing. Nevertheless, we generate not only a public getter,

but a public setter, thus making the name of the private[this] field a

central piece of A's binary compatibility story. May as well be a public

var for all the insulation A has from its implementation.

Now class B comes along and extends A. It receives the private field and

the public getter and setter. To populate the field, B's constructor

calls into A$class, which turns around and calls the public setter on

B, the need for which is now apparent. To implement the one interface

method which A intentionally exposed, we again send B packing to A$class

to perform the actual operation. Having left the warm confines of B

where we could have accessed the field directly, we must resort to using

the public getter which also is trying to justify its existence.

I understand why this machinery is in place and that (at least given the

current encoding) it's necessary to support access which is scala-legal

but jvm-illegal. But I can't see why it can't be done more selectively,

with the especially interesting case being the private field which never

hurt a fly. Separate compilation can hardly be banking on the presence

of private fields which can't be separately accessed.

Optimally:

 

  interface A {

    def dingus(): Int

  }

  class B extends A {

    private int bippy = 5

    def dingus = bippy * bippy

  }

In actual fact:

  interface A {

    def dingus(): Int

    def A$$bippy(): Int

    def A$_setter_$A$$bippy_$eq(x: Int): Unit

  }

  class B extends A {

    private int A$$bippy = 5

    def A$$bippy(): Int = A$$bippy

    def A$_setter_$A$$bippy_$eq(x: Int): Unit = A$$bippy = x

    def dingus(): Int = A$class.dingus(this)

  }

  class A$class {

    def dingus(x: A) = x.A$$bippy() * x.A$$bippy()

  }

Bytecode:

// interface A

public abstract int A$$bippy();

public abstract void A$_setter_$A$$bippy_$eq(int);

public abstract int dingus();

// class B

private final int A$$bippy;

public int A$$bippy();

   0: aload_0

   1: getfield #11; //Field A$$bippy:I

   4: ireturn

public void A$_setter_$A$$bippy_$eq(int);

   0: aload_0

   1: iload_1

   2: putfield #11; //Field A$$bippy:I

   5: return

public int dingus();

   0: aload_0

   1: invokestatic #19; //Method A$class.dingus:(LA;)I

   4: ireturn

// Implementation class A$class:

public static int dingus(A);

   0: aload_0

   1: invokeinterface #12,  1; //InterfaceMethod A.A$$bippy:()I

   6: aload_0

   7: invokeinterface #12,  1; //InterfaceMethod A.A$$bippy:()I

   12: imul

   13: ireturn

public static void $init$(A);

   0: aload_0

   1: iconst_5

   2: invokeinterface #26,  2; //InterfaceMethod A.A$_setter_$A$$bippy_$eq:(I)V

   7: return

[Geoff Reedy]

On Wed, Dec 21, 2011 at 10:34:38PM -0800, Paul Phillips said

> I understand why this machinery is in place and that (at least given the
> current encoding) it's necessary to support access which is scala-legal
> but jvm-illegal. But I can't see why it can't be done more selectively,
> with the especially interesting case being the private field which never
> hurt a fly. Separate compilation can hardly be banking on the presence
> of private fields which can't be separately accessed.

I think it can make a difference. Say A was in some library xyz that has
two different versions

Version 1:

trait A {
private[this] val bippy = 5
def dingus = bippy * bippy
}

Version 2:

trait A {
private[this] val bippy = 10

def dingus = bippy * bippy
}

and B was in some app using xyz

and we also had

object C {
def main(args: Array[String]) { println(new B.dingus) }
}

As it stands now, with no recompilation of the app inbetween

% scala -cp xyz-1.jar:app.jar C
25

and

% scala -cp xyz-2.jar:app.jar C
100

> Optimally:
>
> interface A {
> def dingus(): Int
> }
> class B extends A {
> private int bippy = 5
> def dingus = bippy * bippy
> }

Then we would have, with no recompilation of the app inbetween

% scala -cp xyz-1.jar:app.jar C
25

and

% scala -cp xyz-2.jar:app.jar C
25

-- Geoff

[Paul Phillips]

On Thu, Dec 22, 2011 at 10:35 AM, Geoff Reedy <ge...@programmer-monk.net> wrote:

I think it can make a difference.

Oh, I didn't say it couldn't make a difference, I said that separate compilation shouldn't be banking on it.  I think there are all kinds of hijinx you can bring out with selective recompilation and private members.

I think anything where different semantics arise for A based on implementation details of the trait encoding should be a guarantee-free zone.

[Adriaan Moors]

This has become (even) more relevant with our focus on incremental compilation.

As it stands, the trait encoding makes it very hard to compile incrementally when traits change.

[martin odersky]

--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967

[Paolo G. Giarrusso]

Well, if you want any Scala library to offer any binary compatibility, you need to document which changes are binary incompatible. Java does that: either the JVM or the language spec has a chapter on binary compatibility, which in fact is essentially a tutorial on the consequences of the compilation process.

I'd rephrase Geoff Reedy's example by saying that with the encoding you propose, initializers for private fields of a trait become part of the ABI of the library. In other words, it seems to me that with your encoding, public traits of interface-conscious libraries should stop using private values.

I'd propose instead a compromise:

  interface A {

    def dingus(): Int

    $synthetic protected def bippy_initializer = 5 //In fact, this method goes in A$class.

  }

  class B extends A {

    private val bippy: Int = bippy_initializer

    def dingus = bippy * bippy

  }
This should avoid making the field's setters public and hide bippy_initializer from the ABI. To support the implementation of dingus, the getter is still public, but I don't see how that can be fixed with the current encoding; Java 8 defender methods could allow merging A$class within A (if the semantics are close enough to Scala traits) while improving binary compatibility - thanks to JVM support (or load-time bytecode rewriting), you can add defender methods to an interface without recompiling the inheriting classes.

Java 8 doesn't plan to allow state in interfaces, but it seems this can be solved as follows:

  interface A {

    def dingus(): Int

    $synthetic protected def bippy_initializer = 5 //In fact, this method goes in A and is a defender method.

    def dingus = bippy * bippy //Ditto

    $synthetic protected def A$$$bippy

  }

  class B extends A {

    private val bippy: Int = bippy_initializer

    $synthetic protected def A$$$bippy = bippy

  }