Objects and InnerClasses

[Grzegorz Kossakowski]

Hi,

This e-mail is an attempt to describe what we should be doing about nested objects and InnerClasses attribute in corresponding class files.

I've been postponing this e-mail for a long time in a hope of some enlightenment coming to me. I don't aspire to truly enlightened on all aspects of InnerClasses issue anymore. I think incremental approach should bring some improvement to current rather broken situation. Thus I'll focus my attention only on nested objects and classes (including generic ones) and InnerClasses attribute. Specifically, I put out of consideration following things: Java reflection (should not be more broken than it is right now), objects/classes nested in anonymous classes of any kind.

Problem statement

Consider following code:

class A[T] {

  object B {

    def m(x: T): T = x

  }

}

object B is going to be represented by 'A$B$' class that is not declared as an inner class of A. However, in definition of A$B$ there will be some references to parameter T in generic signatures. This leads to reference to type parameter not declared anywhere.

Proposed solution + discussion

We should be always declaring nested objects as inner classes. It's worth noting that classes declared in object (at any level of nesting) have correct InnerClasses table. E.g.

object A {

  class B

}

will give us correct InnerClasses table for both A and A$B classes. Specifically:

Compiled from "inner.scala"

public final class A extends java.lang.Object

[...]

  InnerClass: 

   public #13= #10 of #12; //B=class A$B of class A

Compiled from "inner.scala"

public class A$B extends java.lang.Object implements scala.ScalaObject

[...]

  InnerClass: 

   public #17= #14 of #16; //B=class A$B of class A

Note that B is declared as inner class of A and not A$. Two classes are generated only for top-level objects so let's see what happens in following case:

class A {

  object B {

    class C

  }

}

Compiled from "inner.scala"

public final class A$B$ extends java.lang.Object implements scala.ScalaObject

[...]

  InnerClass: 

   public #23= #20 of #22; //C=class A$B$C of class A$B$

Compiled from "inner.scala"

public class A$B$C extends java.lang.Object implements scala.ScalaObject

[...]   

  InnerClass: 

   public #30= #10 of #29; //C=class A$B$C of class A$B$

You can see that C is declared as being InnerClass of A$B$ class. Thus, we found one irregularity: classes might be declared as inner classes of either dollar-postfixed class or without dollar-postfixed class corresponding to an object. Another interesting observation is that full name of C is 'A$B$C' and not 'A$B$$C' (one dollar sign for B$ and another one inner class separator). I checked that briefly against Eclipse and javac and they both seems to be fine with that. I guess reflection is not going to like it as per SI-4316 but since Scala reflection is coming we can easily ignore this issue.

Armed with those observations we can make a more precise rule: dollar-postfixed classes corresponding to objects should always be declared as inner classes and should be declared as outer classes for other members in an object (classes or objects). For example, following code:

object A {

  class B {

    object C {

       class D

    }

  }

}

Should result in following chain: A$ <- A$B <- A$B$C$ <- A$B$C$D of nesting declarations. And only that one. Specifically, 'A' class should put out of considerations when it comes to inner classes declaration.

Name clashes

Some time ago Paul brought a point that current name mangling scheme might result in conflicting names. I thought a bit about it and came to conclusion that it's not true. Let me discuss when it could happen and explain why it doesn't happen. Let's assume for a while that inner classes of objects get two dollar signs in their name contrary to what we observed above. Consider following code:

class A {

  class B {

    class `#`

  }

}

Classes we get are: 

• A //class A
• A$B //class A.B
• A$B$$hash //class A.B.`#`

Now let's introduce companion object:

class A {

  object B {

    class hash

  }

  class B {

    class `#`

  }

}

Classes we get are:

• A //class A
• A$B //class A.B
• A$B$ //object A.B
• A$B$$hash //class A.B.`#`
• A$B$$hash //class A.B.hash

We've got a conflict. All that was under assomption made above. However, actual implementation will produce following list of classes:

• A //class A
• A$B //class A.B
• A$B$ //object A.B
• A$B$$hash //class A.B.`#`
• A$B$hash //class A.B.hash

Thus current scheme seems to be safe. It's worth stressing out that this proposal doesn't touch name mangling scheme. What I propose is purely changing contents of InnerClasses attribute.

Empirical evidence

Ideally I'd like to compile my first example using current trunk to produce broken class files. Then edit them according to rules I outlined above and check using badsigs tool me and Paul has been working on. This way we could easily check if changes proposed here bring anticipated results. However, I failed to find any convenient java class file editor. I tried this one: http://sourceforge.net/projects/classeditor/ but it seems to be incapable of editing InnerClasses attribute. Also, I'd like to check following code:

class A[T] {

  object B {

    class C {

      def m(x: T): T = x

    }

  }

}

so we make sure that inner classes with additional dollar sign do not confuse Eclipse compiler when it tries to resolve generic signatures. Again, I need class file editor to verify that. Any tips would be welcome.

Summary

It turns out that, if this proposal has any merit, changes to class files are pretty straightforward. Essentially, we just need to declare dollar sign-postfixed classes corresponding to objects as inner classes. We don't need to change name mangling schem or anything like that. Overall, impact of this proposal is just change to InnerClass attribute.

Paul, I'm interested in your thoughts.

--
Grzegorz Kossakowski

[Simon Ochsenreither]

Hi,

thanks for your great work!

This e-mail is an attempt to describe what we should be doing about nested objects and InnerClasses attribute in corresponding class files.

Problem statement

Consider following code:

class A[T] {

  object B {

    def m(x: T): T = x

  }

}

object B is going to be represented by 'A$B$' class that is not declared as an inner class of A. However, in definition of A$B$ there will be some references to parameter T in generic signatures. This leads to reference to type parameter not declared anywhere.

This very much looks like it is the cause of https://issues.scala-lang.org/browse/SI-4023

Not sure if that is where you actually come from, but if not, would it be possible to have that use-case in mind when testing?

Thanks and bye!


Simon

[Paul Phillips]

On 8/16/11 7:00 AM, Grzegorz Kossakowski wrote:
> *Name clashes*

> Some time ago Paul brought a point that current name mangling scheme
> might result in conflicting names. I thought a bit about it and came to

> conclusion that it's not true. Let me discuss when it /could/ happen and
> explain why it /doesn't/ happen.

I didn't say it might; I said it does already.

https://issues.scala-lang.org/browse/SI-2806

Unless you mean only the specific question of inner class nesting. But
the name mangling scope is significantly larger.

scala> :paste
// Entering paste mode (ctrl-D to finish)

class A_+
class A_ { class plus }

// Exiting paste mode, now interpreting.

defined class A_$plus
defined class A_

scala> val a = new A_
a: A_ = A_@402c3549

scala> new a.plus
res0: a.plus = A_$plus@284f2189

scala> new A_+
java.lang.NoSuchMethodError: A_$plus: method <init>()V not found
at .<init>(<console>:9)
at .<clinit>(<console>)
at .<init>(<console>:11)

(More later)

[Grzegorz Kossakowski]

On 18 August 2011 08:04, Paul Phillips <pa...@improving.org> wrote:

On 8/16/11 7:00 AM, Grzegorz Kossakowski wrote:

*Name clashes*

Some time ago Paul brought a point that current name mangling scheme
might result in conflicting names. I thought a bit about it and came to
conclusion that it's not true. Let me discuss when it /could/ happen and
explain why it /doesn't/ happen.

I didn't say it might; I said it does already.

 https://issues.scala-lang.org/browse/SI-2806

Unless you mean only the specific question of inner class nesting.  But the name mangling scope is significantly larger.

Yeah, I meant specific question about name mangling and inner class nesting. Thanks for that example, it improves my understanding of limitations of current name mangling.

However, since my proposal doesn't affect name mangling I'd like to exclude it from this discussion. It would be good to fix one issue at the time.

Could you comment on the rest of the proposal? Do you see any obvious flaws in it?

--
Grzegorz Kossakowski

[iulian dragos]

I think it's worth doing. If we can improve on the current state of affairs without breaking anything, we should do it. Note that declaring classes as being members of the $-class may break Java code that relied on instantiating them as 'new A.B' (I say 'may' because I am not sure). Not a big deal (if we have tangible gains), since that was never specified, but to be mentioned in the release notes. Also, we need to test your assumption that the JVM will not choke on having A$B as a member of A$ (and not A$$B, which would be the 'right' name according to its mangling scheme).

 

I believe you can use http://jasmin.sourceforge.net/ to test all these before you implement them in the backend.

cheers,

iulian

--
Grzegorz Kossakowski

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

[Grzegorz Kossakowski]

On 25 August 2011 15:42, iulian dragos <jagu...@gmail.com> wrote:

I think it's worth doing. If we can improve on the current state of affairs without breaking anything, we should do it. Note that declaring classes as being members of the $-class may break Java code that relied on instantiating them as 'new A.B' (I say 'may' because I am not sure). Not a big deal (if we have tangible gains), since that was never specified, but to be mentioned in the release notes. Also, we need to test your assumption that the JVM will not choke on having A$B as a member of A$ (and not A$$B, which would be the 'right' name according to its mangling scheme).

 

I believe you can use http://jasmin.sourceforge.net/ to test all these before you implement them in the backend.

Thanks. Will test it.

Also, I'll go ahead and try to implement it. Stay tuned.

--
Grzegorz Kossakowski

[Grzegorz Kossakowski]

On 26 August 2011 10:10, Grzegorz Kossakowski <grzegorz.k...@gmail.com> wrote:

Thanks. Will test it.

Also, I'll go ahead and try to implement it. Stay tuned.

I implemented my fix and pushed to trunk. Great news are that my little tool called 'badsigs' that tries to import types from scala library into java files and compile then using ecj does not fail anymore, see:

https://scala-webapps.epfl.ch/jenkins/view/Misc/job/scala-signatures/69/console

:-)

Thanks to Paul for contributing various bits of that tool.

--
Grzegorz Kossakowski