method handles
John Rose
the dynamic language implementor is to use reflection for connecting
dynamic callers to dynamic callees. The next step, which is painful
in a different way, is to reinvent a somewhat better reflection, by
byte-compiling lots of wrapper classes which connect via various
language-specific interfaces.
The JVM really should give more help for this sort of thing.
Inspired in part by some of the conversations in this group, I think
there is a way:
http://blogs.sun.com/jrose/entry/method_handles_in_a_nutshell
Want to help refine the design?
There need to be be both native and emulation implementations, so
implementors can use method handles on current and future JVMs.
By the way, this is not the same as invokedynamic, which is concerned
with the logic of call site linkage and dynamic overloading. But
method handles are a necessary foundation piece for invokedynamic,
because they provide the vocabulary for linking a dynamic call site.
Best wishes,
-- John
Martin Probst
very interesting article. I'm not sure if I should rather ask here or
in your blog, I'll try here first :-)
In the article you use a function H(Object.equals) in pseudocode,
which is supposed to give you a method handle. I'm curious, with the
new uses of annotations for meta programming in Java (using reflection
or byte code modifications like AspectJ), you'll see stuff like this
all over the place:
@Entity
@Index(on = "bar")
class Foo {
private String bar;
}
I.e., the field bar is referenced by it's name as a String, which is
ugly for many reasons. Stuff like this can be found in Hibernate
annotations, JPA annotations, JSR-311 and so on.
I think while your at it defining a way to obtain a handle to a
method, maybe it would be nice to introduce something like a Smalltalk
symbol to Java? I'd like to be able to write "@Index(on = &Foo.bar)",
or something similar. We'd need the equivalents of a local symbol
(resolvable in the context of a class), a resolved symbol that maps to
a fully qualified name, and then maybe a way to differentiate between
methods and fields.
Something like this would make many meta programming APIs a lot easier
to use, including yours about method handles.
Regards,
Martin
Alex Tkachman
Daniel Yokomizo
>
> Hi,
>
> very interesting article. I'm not sure if I should rather ask here or
> in your blog, I'll try here first :-)
>
> In the article you use a function H(Object.equals) in pseudocode,
> which is supposed to give you a method handle. I'm curious, with the
> new uses of annotations for meta programming in Java (using reflection
> or byte code modifications like AspectJ), you'll see stuff like this
> all over the place:
>
> @Entity
> @Index(on = "bar")
> class Foo {
> private String bar;
> }
>
> I.e., the field bar is referenced by it's name as a String, which is
> ugly for many reasons. Stuff like this can be found in Hibernate
> annotations, JPA annotations, JSR-311 and so on.
>
> I think while your at it defining a way to obtain a handle to a
> method, maybe it would be nice to introduce something like a Smalltalk
> symbol to Java? I'd like to be able to write "@Index(on = &Foo.bar)",
> or something similar. We'd need the equivalents of a local symbol
> (resolvable in the context of a class), a resolved symbol that maps to
> a fully qualified name, and then maybe a way to differentiate between
> methods and fields.
A couple of the closure proposals include some form of method/field
literal, it would look like this:
-fields: Foo#bar
-methods: Foo#bar(), Foo#bar(int)
It's better to reuse whatever syntax these proposals use.
> Something like this would make many meta programming APIs a lot easier
> to use, including yours about method handles.
>
> Regards,
> Martin
Best regards,
Daniel Yokomizo
John Rose
> I think while your at it defining a way to obtain a handle to a
> method, maybe it would be nice to introduce something like a Smalltalk
> symbol to Java? I'd like to be able to write "@Index(on = &Foo.bar)",
> or something similar.
Yes, that notation would be nice in several places. For annotations,
I think you want something a little different from method handles,
since &Foo.bar above is a symbolic reference, rather than an opaque
method.
I left out several such things in the "nutshell" note for two reasons.
1. It is a language-oriented feature, not a JVM-oriented one.
2. It is not the core of the proposal, but rather a detail to fill in.
Consider Neal's recent admission of such a construct to closures.
It's a common thing to ask for (I asked for it 2 years ago http://
blogs.sun.com/jrose/entry/better_closures ). But it is best to
concentrate on the core features first and then fill out the boundary
of the design as necessary.
In that spirit, and for the JVM, I am thinking that "ldc" of
a*Methodref constant pool entry should yield a method handle.
Also, maybe we should slightly temper the opaqueness of method
handles, and allow a "MethodHandles.reflect" operation which will
recover the full java.lang.reflect.Method, in all its symbolic
glory. I don't think that causes a security problem, and it makes
some uses of method handles (e.g., debugging them) smoother.
Basically, I've left open the question of how the Core Reflection API
interoperates with method handles, but full reflection is obviously
the most plausible option. (Yes, it goes against the grain of what
function objects do, but reflection is an equal-opportunity offender
in that respect: It also threatens object abstraction boundaries.)
-- John
John Rose
Is any particular plans when it will be available for usage?
John Rose
It's better to reuse whatever syntax these proposals use.
public static MethodHandle unreflectFieldGetter(java.lang.reflect.Field f);
public static MethodHandle unreflectFieldSetter(java.lang.reflect.Field f);
... }
Something language designers usually miss at this point is that a handle to a variable (like a field above) is most naturally represented as a two-sided extension to a normal function type, nameless as a regular function, but with an additional setter aspect. It may be called an accessor; it is a subtype of function, and is parameterized exactly the same as function:
class Foo { int val } foo; accessor {=> int} acc1 = (&foo.val or foo#val or something); int i = acc1(); acc1() = i+1; Map<K,V> map; accessor {K=>V} acc2 = (&<K>map.get or map#get(K) or something, with put annotated in somehow) V x = acc2(k); acc2(k2) = v2;
The design of accessor types is (AFAIK) insufficiently appreciated. It was at the heart of "esh" Scheme[1] and allowed clean access-oriented programming patterns, and better integration with slot-oriented languages (CLOS-lite, C++ lvalues).
Rémi Forax
C++ lvalues works on local variables too.
Please don't add a way to create a MethodHandle on local vars ?
passing by address semantic is the root of all evil.
> -- John
>
> [1] http://homepage.mac.com/rose00/work/esh.html
Rémi
John Rose
Please don't add a way to create a MethodHandle on local vars ?
Rémi Forax
I tried to understand how to implement closure with method handles.
So correct me if i'm wrong:
- a method handle is a Java Object with a special header
let say it's a 3 words header, the last word is a pointer to
the signature of the method handle.
- in order to use covariance/contravariance, you need
to use an adapt method that create a *new* method handle
Because adapt create a new handle, closures are not method handles.
Example using the BGGA syntax:
private static void f({String=>void} closure) {
closure.invoke("hello");
}
public static void print(Object o) {
System.out.println(o);
}
...
{Object=>void} h1=#print(Object);
f(h1); // here the method handle need to be adapted
// so the local parameter closure is not == to h1
So closures are not method handles.
Ok, let now say a closure is an object that contains a method handle and
invoke is performed by using the new invokedynamic bytecode.
public class Closure {
private final MethodHandle handle;
private MethodHandle invokeDynamicFailure(MethodType signature) {
checkIfSignatureCompatible(signature);
return MethodHandles.adaptArguments(handle,signature);
}
}
When invoke is called using the invoke dynamic bytecode,
the default behavior failed (if there is one), the VM called
invokeDynamicFailure
with the method signature of the invoke dynamic call site.
This method returns the method handle that should be used as fallback.
The returned handle is stored in an inline cache to avoid to lookup each
time.
ok, this seems to work, is is the general idea ?
Rémi
John Rose
> I tried to understand how to implement closure with method handles.
> So correct me if i'm wrong:
> - a method handle is a Java Object with a special header
> let say it's a 3 words header, the last word is a pointer to
> the signature of the method handle.
That's a good model for the low-level or primitive method handles
(MHD, MHB). A method handle invocation must first do a dynamic
signature check (signatures are JVM-level proxies for an infinite
schema of function-like interfaces), possibly do some argument
wrangling, and then jump to the target method.
The other kind is any object that implements the MethodHandle
interface and supplies an invoke method that matches getType. That
kind is not a new kind of JVM data structure, but an ordinary Java
object. See this blog paragraph:
> So, at the risk of adding a chore to the JVM implementor’s list, I
> think an object of such a type should serve (uniformly in the
> contexts described above) as a method handle. (It is may be
> necessary to ask that R implement the marker interface and the
> getType method; but is something the system could also figure out
> well enough on its own.) I admit that this is not a necessary
> feature, but it could cut in half the number of small method-like
> objects running around in some systems.
>
You are right about the differences between closures and method handles.
> - in order to use covariance/contravariance, you need
> to use an adapt method that create a *new* method handle
That would be true if method handle signatures were identical to
closure signatures.
But method handles are JVM-level objects. To implement closures, I
think it would be better to erase the function signatures first, then
work with method handles on the erased signatures.
> Because adapt create a new handle, closures are not method handles.
...Unless the type changes are compile-time only, or unless we are
talking about closure conversion instead of mere subtyping.
> Ok, let now say a closure is an object that contains a method
> handle and
> invoke is performed by using the new invokedynamic bytecode.
Setting aside invokedynamic for the moment, you could factor a
significant chunk of closures implementation into two parts. First,
a template-generated schema of implementation classes, one per
function interface type. The classes embody the semantics of
closures, referring on invoke to a private method handle (as in your
sketch). The method handle would have a signature identical to the
erased (JVM-level) signature of the function type.
Second, when a closure expression is evaluated, the compiler puts the
code into a private method on a nearby class, and generates a method
handle to it. (If the JVM is really accommodating, this ought to be
an "ldc" instruction!) It then wraps the standard function-type
around the method handle.
This division is labor is in most ways better than what we get
without method handles: The compiler must generate not just a
private method but a whole private class, which implements a function
type and defines an invoke method. The point of method handles (in
this application) is to allow closure methods to be grouped naturally
in the module that uses them, instead of spread out as 'invoke'
methods in a bunch of inner classes.
I'm sure we don't need invokedynamic (i.e., programmatically defined
call site linkage) to implement closures, with or without method
handles.
-- John
hlovatt
would be great - so don't get me wrong in the text below I am not
against their introduction. Below I am musing on using them for inner
classes/closures.
To use a method handle for an inner class/closure would require
certain, not necessarily desirable, semantics for the inner class/
closure. To give an example I need some syntax because Java currently
does not have any suitable syntax, I have chosen some syntax not
particularly aligned with any inner class/closure proposal to
illustrate my thoughts, because I don't want any semantic baggage from
existing proposals. If the keyword 'shared' denoted a local variable
that had a lifetime beyond the enclosing method call and allowed write
access and if the keyword 'method' introduced a local method inside
another method then:
class Foo {
int bar( int[] is ) {
shared int sum = 0;
method void summer( int x ) { sum += x; }
for ( int i : is ) { summer( i ); }
return sum;
}
}
Then the compiler could translate this into:
class Foo {
private int bar$sum; // shared
private final void bar$summer( int x ) { bar$sum += x; } // inner
method
private final MethodHandle bar$summer$MH =
insertArgument( findVirtual( Foo.class, "bar$summer", ...), this ); //
create a handle to the method
int bar( int[] is ) {
sum = 0;
for ( int i : is ) { bar$summer$MH.invoke( i ); }
return bar$sum;
}
}
There are two things to note:
1. The method handle isn't needed in this example since you could call
bar$summer directly. However for something executed later, e.g. an
event, the handle is needed.
2. The code isn't thread safe and therefore not suitable for
invocation later, e.g. as an event (though OK for some Swing events).
If you use a separate class then you get the thread safety (one
possible transformation below):
class Foo$bar$Sum{
int value = 0; // shared local variable
}
class Foo {
private final static void bar$summer( Foo$bar$sum sum, int x )
{ sum.value += x; } // inner method
int bar( int[] is ) {
final Foo$bar$Sum sum = new Foo$bar$Sum();
for ( int i : is ) { bar$summer( sum, i ); }
return sum.value;
}
}
The disadvantage of this separate class approach, is that class Foo$bar
$sum is unlikely to be garbage collected.
Is there a way to use method handles for inner classes/closures?
Just to reiterate I think speeding up reflection is a great idea, I am
just not sure of other uses.