Package protected and virtual/non-virtual dispatch

[Brian Frank]

In Fantom, I have been using invokespecial as poor-man's non-virtual
dispatch to avoid method naming conflicts between modules. But it
appears the Andriod's VM requires the ACC_SUPER flag and won't let you
call a method unless it is actually marked as private. So I need to
switch to invokevirtual, but still want to avoid that subclasses can
accidentally override a module scoped method in a base class.

Experimenting with Java, it appears that the JVM dispatches
invokevirtual correctly in the case of package scoped methods. That
is given these two classes:

package alpha;
public class A { String foo() { return "A.foo"; }

package beta;
public class B extends A { String foo() { return "B.foo"; }

It seems that calling A.foo on an instance of B will always correctly
dispatch to A.foo (B's version does not truly override A.foo). I
tried to find where this behavior is dictated by the JVM language
spec, but couldn't find any mention of it.

Is this the specified behavior that can be counted on across different
JVMs?

[Jeroen Frijters]

I don't know the answer, but I had to reply because this is one of my (many) pet peeves about the JVM spec. It used to be the case that method dispatching was completely unspecified in the JVM spec (I think it, implicitly, delegated this to the JLS, which is completely unacceptable IMNSHO). I seem to recall that recently there have been some efforts to remove the JLS dependency from the JVM spec, but I don't know what became of that. I just tried googling the 3rd edition of JVM spec, but couldn't find anything recent.

Also, because of the lack of specification, the behavior is sometimes a bit odd. For example, if you modify your example so that A has a base class (Base) with a public foo method, now A.foo will override Base.foo and B.foo will override A.foo.

Regards,
Jeroen

[digo]

> Is this the specified behavior that can be counted on across different
> JVMs?

http://psc.informatik.uni-jena.de/languages/Java/javaspec-3.pdf
Last paragraph on pg 358 of the pdf above mentions the following:

Unlike C++, the Java programming language does not specify altered
rules for method dispatch during the creation of a new class instance.
If methods are invoked that are overridden in subclasses in the object
being initialized, then these overriding methods are used, even before
the new object is completely initialized.

[John Rose]

On Oct 18, 2011, at 5:59 AM, Brian Frank wrote:

Is this the specified behavior that can be counted on across different
JVMs?

Yes, the current behavior can be relied upon.  The spec. issue was identified in 1999 when Hotspot did something different from Sun's previous VM.

  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4248990

Since then it's been stable behavior (probably across all JVMs) but a weak spot in the JVM spec., which JVM implementors had to "just know".

Alex Buckley has been diligently tracking this and dozens of other small defects in previous versions of the spec.  The results can be seen in the Java SE 7 Edition of the JVM Specification.  This is in JSR 336's Final Release (Annex 3) and at http://download.oracle.com/javase/cmn/spec_index.html.  It has a modest little section called "5.4.5 Method overriding", which documents the behavior you are asking about.

On Oct 18, 2011, at 6:45 AM, Jeroen Frijters wrote:

I just tried googling the 3rd edition of JVM spec, but couldn't find anything recent.

See above!  Let us know what you think, Jeroen.

Best wishes,

-- John

[Jeroen Frijters]

John Rose wrote:
> Alex Buckley has been diligently tracking this and dozens of other small
> defects in previous versions of the spec. The results can be seen in
> the Java SE 7 Edition of the JVM Specification. This is in JSR 336's
> Final Release (Annex 3) and at
> http://download.oracle.com/javase/cmn/spec_index.html. It has a modest
> little section called "5.4.5 Method overriding", which documents the
> behavior you are asking about.

Thanks for the link.

Maybe I'm misunderstanding what it says, but I'm thoroughly confused by HotSpot behavior:

package p1;
class Z { public void foo() { } }

package p2;
class A extends Z { void foo() { } }

package p3;
class C extends A { void foo() { } }

Here A.foo clearly overrides Z.foo and according to my understand of 5.4.5 C.foo should not override A.foo.

Now the weird part, on HotSpot when C has class file version 51 C.foo does override A.foo, but for earlier class file versions it doesn't.

It looks to me as if the spec documents the pre-7 HotSpot behavior.

Regards,
Jeroen

[Jeroen Frijters]

Hi,

Found another issue. When you the run attached code on JDK 7u1 (java -cp test.zip pkg1.Base) you get:

A.foo
A.foo
A.foo
A.foo
B.foo
C.foo
C.foo
Exception in thread "main" java.lang.NullPointerException
at pkg3.B.invoke(B.java:10)
at pkg1.Base.main(Base.java:20)

Note that pkg3/B.class has class file version 50, the other classes are all 51.

Regards,
Jeroen

> --
> You received this message because you are subscribed to the Google
> Groups "JVM Languages" group.
> To post to this group, send email to jvm-la...@googlegroups.com.
> To unsubscribe from this group, send email to jvm-
> languages+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/jvm-languages?hl=en.

[Jeroen Frijters]

(resent without attachment)

Hi,

Found another issue. When you the run [1] on JDK 7u1 (java -cp test.zip pkg1.Base) you get:

A.foo
A.foo
A.foo
A.foo
B.foo
C.foo
C.foo
Exception in thread "main" java.lang.NullPointerException
at pkg3.B.invoke(B.java:10)
at pkg1.Base.main(Base.java:20)

Note that pkg3/B.class has class file version 50, the other classes are all 51.

Regards,
Jeroen

[1] http://www.frijters.net/test-20111019.zip

> -----Original Message-----
> From: jvm-la...@googlegroups.com [mailto:jvm-
> lang...@googlegroups.com] On Behalf Of Jeroen Frijters
> Sent: Wednesday, October 19, 2011 13:55
> To: jvm-la...@googlegroups.com
> Subject: RE: [jvm-l] Package protected and virtual/non-virtual dispatch
>

[John Rose]

On Oct 19, 2011, at 6:00 AM, Jeroen Frijters wrote:

> (resent without attachment)
>
> Hi,
>
> Found another issue. When you the run [1] on JDK 7u1 (java -cp test.zip pkg1.Base) you get:
>
> A.foo
> A.foo
> A.foo
> A.foo
> B.foo
> C.foo
> C.foo
> Exception in thread "main" java.lang.NullPointerException
> at pkg3.B.invoke(B.java:10)
> at pkg1.Base.main(Base.java:20)
>
> Note that pkg3/B.class has class file version 50, the other classes are all 51.
>
> Regards,
> Jeroen
>
> [1] http://www.frijters.net/test-20111019.zip
>

It looks like you have found at least one JVM bug. Please submit a report at http://bugreport.sun.com/bugreport/ . If you mail me the bug ID I'll put myself on the CC list for it.

Thanks!
-- John

P.S. Here's an approximate source-based version of your test that I played with. I don't have time to track down the source of the behavior difference.

/*
$ javac -d .. ../????/*.java
$ emacs Base.class # change f00 to foo everywhere
$ java -cp .. pkg1.Base
Base.foo on A: A.foo
A.foo on A: A.foo
Base.foo on B: B.foo
A.foo on B: B.foo
B.foo on B: B.foo
Base.foo on C: C.foo
A.foo on C: C.foo
B.foo on C: C.foo
C.foo on C: C.foo
$ javac -d .. -cp .. ../pkg?/?.java
$ java -cp .. pkg1.Base
Base.foo on A: Base.foo
A.foo on A: A.foo
Base.foo on B: Base.foo
A.foo on B: A.foo
B.foo on B: B.foo
Base.foo on C: Base.foo
A.foo on C: A.foo
B.foo on C: B.foo
C.foo on C: C.foo
*/

package pkg1;
import pkg2.A; // A <: Base
import pkg3.B; // B <: A <: Base
import pkg4.C; // C <: B <: A <: Base

public class Base
{
public static void main(String[] strings) {
A a = new A();
Base.invoke(a);
A.invoke(a);

B b = new B();
Base.invoke(b);
A.invoke(b);
B.invoke(b);

C c = new C();
Base.invoke(c);
A.invoke(c);
B.invoke(c);
C.invoke(c);
}
public String f00() {
return "Base.foo";
}
public static void invoke(Base x) {
System.out.println("Base.foo on "+x.getClass().getSimpleName()+": "+x.f00());
}
}

/*
==> ../pkg2/A.java <==
package pkg2;
import pkg1.Base;

public class A extends Base
{
String foo() {
return "A.foo";
}
public static void invoke(A a) {
System.out.println("A.foo on "+a.getClass().getSimpleName()+": "+a.foo());
}
}

==> ../pkg3/B.java <==
package pkg3;
import pkg2.A;

public class B extends A
{
String foo() {
return "B.foo";
}

public static void invoke(B b) {
System.out.println("B.foo on "+b.getClass().getSimpleName()+": "+b.foo());
}
}

==> ../pkg4/C.java <==
package pkg4;
import pkg3.B;
public class C extends B
{
String foo() {
return "C.foo";
}
public static void invoke(C c) {
System.out.println("C.foo on "+c.getClass().getSimpleName()+": "+c.foo());
}
}
*/

[Jeroen Frijters]

John Rose wrote:
> It looks like you have found at least one JVM bug. Please submit a
> report at http://bugreport.sun.com/bugreport/ . If you mail me the bug
> ID I'll put myself on the CC list for it.

Last time I filed a JVM bug (which was admittedly long ago) that only reproduced from .class files it was closed "because I didn't provide Java source", so I sort of have a policy against bug filing (unless security related). Don't take it personally, I also don't file .NET bugs for similar reasons, so it could just be me :-).

But beyond obscure HotSpot bugs, what I care more about is the JVM spec, any comments on my (or HotSpot's) confusion regarding 5.4.5?

Regards,
Jeroen

[Attila Szegedi]

Sorry, I just came across this post now, and couldn't help but wonder: shouldn't the correct JVM operation here be to refuse to load p2.A with IncompatibleClassChangeError?

After all, the source code as such is rejected by the compiler, because the visibility of Z.foo() was reduced in the subclass A.

If you trick it into compiling (by removing the "public" modifier from p1.Z before compiling p2.A, then only recompiling p1.Z with "public" added back in), I think the JVM should just refuse to load the class with IncompatibleClassChangeError, isn't that right?

Attila.

> --
> You received this message because you are subscribed to the Google Groups "JVM Languages" group.
> To post to this group, send email to jvm-la...@googlegroups.com.

> To unsubscribe from this group, send email to jvm-language...@googlegroups.com.

[Jeroen Frijters]

I don't think disallowing this would be great either. To me the easiest solution would be to simply make package accessible method inhabit a totally different namespace (one for each package). This corresponds much better with people's intuition about package private members, I believe.

The only complication is a package that contains both package private and public (or protected) methods with the same signature. You can make a case that this should be disallowed. Another option would be to use two vtable slots in this case (one public slot and one package slot).

Regards,
Jeroen

> -----Original Message-----
> From: jvm-la...@googlegroups.com [mailto:jvm-

> languages+...@googlegroups.com.

> > For more options, visit this group at
> http://groups.google.com/group/jvm-languages?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "JVM Languages" group.
> To post to this group, send email to jvm-la...@googlegroups.com.
> To unsubscribe from this group, send email to jvm-

> languages+...@googlegroups.com.