Tool to convert call sites into invokedynamic?

[Charles Oliver Nutter]

We're making a performance push for JRuby, and today's item is getting all our dynamic calls from Java to be more efficient. At the moment, the best I can do writing Java code is to use our non-invokedynamic call site objects to provide a monomorphic cache in each place. The ideal fix would be to replace those virtual calls with invokedynamic calls at the bytecode level, leveraging our existing indy call site logic from both Ruby and Java.

I need suggestions for tools that might be able to do some or part of this for me. Of course there's ASM, with which I could do the modification by hand, but I'm hoping there's some smarter bytecode-reengineering tool focused on invokedynamic. If there isn't, I'll have to make one.

So...any thoughts?

- Charlie

[John Rose]

On Jul 19, 2016, at 8:07 AM, Charles Oliver Nutter <hea...@headius.com> wrote:

So...any thoughts?

Remi has some stuff that can re-weave all normal invokes to indy's.

You'll want to put a big filter on it.  I anticipate a cool design for that filter.

Another thought:  Use annotations to mark some arguments as static,

and require those to be ldc-able constants, just like some C intrinsics

require some of their arguments to be C constants.  (The C compiler

backend object to them if they aren't really constants; your bytecode

transform would object similarly.)

I wish we had a good PIC combinator off the shelf.  Idea:  Use a short

@Stable array and ensure constant folding of current method bindings.

It's an abuse of @Stable, though, since the PIC contents would change

over time.

— John

[Remi Forax]

De: "John Rose" <john....@oracle.com>
À: jvm-la...@googlegroups.com
Envoyé: Mardi 19 Juillet 2016 18:12:07
Objet: Re: [jvm-l] Tool to convert call sites into invokedynamic?

On Jul 19, 2016, at 8:07 AM, Charles Oliver Nutter <hea...@headius.com> wrote:

So...any thoughts?

Remi has some stuff that can re-weave all normal invokes to indy's.

You'll want to put a big filter on it.  I anticipate a cool design for that filter.

yes, the rewriting part is easy, the question is more how to find the call you

want to rewrite and how to find the bootstrap method and the bootstrap

arguments associated with that call.

Charles, given that i will be locked up in a flying box for a little more that 10 hours soon,

if you have precise answers to these two questions, i can write a tool for you during that time.

A fun way to design such filter is to create a class that declares methods that uses the @PolymorphicSignature,

with that the compiler will not try to box the arguments and it will be easier to substitute a call to

a method of this call by an invokedynamic.

Another thought:  Use annotations to mark some arguments as static,

and require those to be ldc-able constants, just like some C intrinsics

require some of their arguments to be C constants.  (The C compiler

backend object to them if they aren't really constants; your bytecode

transform would object similarly.)

I wish we had a good PIC combinator off the shelf.  Idea:  Use a short

@Stable array and ensure constant folding of current method bindings.

It's an abuse of @Stable, though, since the PIC contents would change

over time.

— John

Rémi

[Charles Oliver Nutter]

On Wed, Jul 20, 2016 at 12:41 PM, Remi Forax <fo...@univ-mlv.fr> wrote:

yes, the rewriting part is easy, the question is more how to find the call you

want to rewrite and how to find the bootstrap method and the bootstrap

arguments associated with that call.

At the moment I am using my own non-indy call sites, one per actual site in the Java code. The call sites are all very straightforward:

sites(context).some_method.call(context, caller, target, args...) 

Where sites is a private static method that gets the right box of call sites, some_method is one of those call sites for calling "some method" and the call logic does the monomporphic cache.

It should not be hard to see this pattern in code and rewrite it, but it won't be a simple invokevirtual => invokedynamic.

Charles, given that i will be locked up in a flying box for a little more that 10 hours soon,

if you have precise answers to these two questions, i can write a tool for you during that time.

What I want is a way to say "take all call sites that look like X and turn them into indy call sites that do X the right way".

 

A fun way to design such filter is to create a class that declares methods that uses the @PolymorphicSignature,

with that the compiler will not try to box the arguments and it will be easier to substitute a call to

a method of this call by an invokedynamic.

There's lots of annotations I'd love to play with from java.lang.invoke, but is there any way to do it without runtime tricks?

FWIW, a contrived benchmark that uses a lot of dynamic calling from Java is already 3x faster by introducing simple monomorphic caches.

This work is the first step.

* Get dynamic calls from Java caching in SOME way.

* Get dynamic calls from Java using invokedynamic using some tooling. SURE WISH I COULD EMIT INDY FROM JAVA :-D

* Profile Java-based Ruby core methods and find the worst poly/megamorphic cases. Move them to Ruby.

Once code is in Ruby, I already have a POC to re-emit the bytecode on a per-callsite basis. We should also be able to feed expected types into IR and have it emit specialized versions for e.g. math.

Lots of stuff finally coming together, but Java is the biggest thing holding me back right now.

- Charlie

[Remi Forax]

De: "Charles Oliver Nutter" <hea...@headius.com>
À: "JVM Languages" <jvm-la...@googlegroups.com>
Envoyé: Jeudi 21 Juillet 2016 03:21:36
Objet: Re: [jvm-l] Tool to convert call sites into invokedynamic?

On Wed, Jul 20, 2016 at 12:41 PM, Remi Forax <fo...@univ-mlv.fr> wrote:

yes, the rewriting part is easy, the question is more how to find the call you

want to rewrite and how to find the bootstrap method and the bootstrap

arguments associated with that call.

At the moment I am using my own non-indy call sites, one per actual site in the Java code. The call sites are all very straightforward:

sites(context).some_method.call(context, caller, target, args...) 

Where sites is a private static method that gets the right box of call sites, some_method is one of those call sites for calling "some method" and the call logic does the monomporphic cache.

It should not be hard to see this pattern in code and rewrite it, but it won't be a simple invokevirtual => invokedynamic.

Charles, given that i will be locked up in a flying box for a little more that 10 hours soon,

if you have precise answers to these two questions, i can write a tool for you during that time.

What I want is a way to say "take all call sites that look like X and turn them into indy call sites that do X the right way".

here is a prototype:

https://github.com/forax/jruby-methodcall-optimizer

it recognize

  sites(context).some_method.call(context, caller, target, "arg1", "arg2");

and transform it too

       6: aload_1
       7: pop
       8: aload_1
       9: aload_2
      10: aload_3
      11: iconst_2
      12: anewarray     #5                  // class java/lang/Object
      15: dup
      16: iconst_0
      17: ldc           #6                  // String arg1
      19: aastore
      20: dup
      21: iconst_1
      22: ldc           #7                  // String arg2
      24: aastore
      25: invokedynamic #74,  0             // InvokeDynamic #0:some_method:(LContext;Ljava/lang/Object;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;

the pop at line 7 is here because the pattern load the context twice, and as you can see the arguments are boxed in an array.

A fun way to design such filter is to create a class that declares methods that uses the @PolymorphicSignature,

with that the compiler will not try to box the arguments and it will be easier to substitute a call to

a method of this call by an invokedynamic.

There's lots of annotations I'd love to play with from java.lang.invoke, but is there any way to do it without runtime tricks?

as far as i know, no,

but you want a tool that rewrite bytecode (so introducing annotation can be done at the same time, i.e. translating your custom annotation into jdk internal one).

FWIW, a contrived benchmark that uses a lot of dynamic calling from Java is already 3x faster by introducing simple monomorphic caches.

This work is the first step.

* Get dynamic calls from Java caching in SOME way.

* Get dynamic calls from Java using invokedynamic using some tooling. SURE WISH I COULD EMIT INDY FROM JAVA :-D

* Profile Java-based Ruby core methods and find the worst poly/megamorphic cases. Move them to Ruby.

Once code is in Ruby, I already have a POC to re-emit the bytecode on a per-callsite basis. We should also be able to feed expected types into IR and have it emit specialized versions for e.g. math.

Lots of stuff finally coming together, but Java is the biggest thing holding me back right now.

- Charlie

Rémi

[John Rose]

On Jul 21, 2016, at 3:53 PM, Remi Forax <fo...@univ-mlv.fr> wrote:

here is a prototype:

https://github.com/forax/jruby-methodcall-optimizer

We've gotta lock you in a flying metal box more often.

[Charles Oliver Nutter]

This is excellent, Remi! I think I should be able to take it from here.

- Charlie (mobile)

--
You received this message because you are subscribed to the Google Groups "JVM Languages" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jvm-language...@googlegroups.com.
To post to this group, send email to jvm-la...@googlegroups.com.
Visit this group at https://groups.google.com/group/jvm-languages.
For more options, visit https://groups.google.com/d/optout.

[John Rose]

On Jul 21, 2016, at 6:24 PM, Charles Oliver Nutter <hea...@headius.com> wrote:

This is excellent, Remi! I think I should be able to take it from here.

- Charlie (mobile)

FTR, I sent a brain dump of possible next steps to mlvm-dev:

http://mail.openjdk.java.net/pipermail/mlvm-dev/2016-July/006661.html

Enjoy!

— John