Service Loader optimizations

[Georgios Andrianakis]

Hi folks,

I wanted to give an update on a small experiment I ran on the side and solicit feedback.

The idea is essentially to optimize the use of the Java Service Loader in Quarkus, since all the services to be loaded are already known at build time. A few of us have discussed this in the past, but we never got around to prototyping anything.

I essentially prototyped what is the most basic kind of optimization: Using bytecode transformers to replace specific/well-known instances of the code that use the Service Loader with constructor invocations. The prototype can found at: https://github.com/quarkusio/quarkus/pull/25219 

The advantages of this approach are:

• It's rather straightforward to implement as there are no new concepts introduced, just reuse of existing Quarkus functionality
  
• It provides the best possible runtime performance as service loader calls are replaced with simple constructor invocations

The disadvantages are:

• It can be tedious to implement as writing ASM code is not to most people's liking
• Every new instance of the Service Loader we want to eliminate requires a new optimization
  
• The optimizations (essentially ASM transformations) would require maintenance as they can get out of sync with the original code. It's important to note however that even if this happens, the application simply reverts to the default JVM behavior. Moreover, we can likely introduce tests that use ClassLoader events (which Sanne had added in the past) to ensure that the optimizations don't break.

I benchmarked the effect of these optimizations on the startup time using our getting-started quickstart (using Java 11) and got the following results:

	Original	Optimized
Average	0.47739	0.46453
Median	0.476	0.464
70th Percentile	0.485	0.468
80th Percentile	0.49	0.474
90th Percentile	0.4951	0.48

I also have Java 17 numbers that show a very similar difference.

Furthermore, also checked the effect on the size of the native binary and got the following:

Original	Optimized
36881K	36865K

As you can see, the effects aren't mind blowing by any means.

What do you think about this? Is it something worth pursuing in this form or another?

[Stuart Douglas]

Looking at those numbers my gut feeling is that this is not worth the complexity it brings. I am assuming there is no measurable difference in native mode?

Also for some of those classes (e.g. the Resteasy Reactive ones) we have full control of the class, so we could probably just add a service loader free option.

Stuart

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALeTM-kyM%2BeCk3GVkKeeJ0FW-yt%2BsxzZ94Rw3hS6HeaaXdJCZQ%40mail.gmail.com.

[Georgios Andrianakis]

On Wed, May 4, 2022 at 11:51 AM Stuart Douglas <sdou...@redhat.com> wrote:

Looking at those numbers my gut feeling is that this is not worth the complexity it brings. I am assuming there is no measurable difference in native mode?

No difference in native mode...

Also for some of those classes (e.g. the Resteasy Reactive ones) we have full control of the class, so we could probably just add a service loader free option.

Yeah, we definitely could. For this round, I just wanted to get a baseline doing the quickest thing possible.

[Stephane Epardaud]

Besides the numbers, I wonder why this is so complex. I would have thought that with Jandex being able to report use-sites of the `ServiceLoader` API, we could just replace the calls to `ServiceLoader.load(Class)` with calls to `$generated$loadServices` (containing `new Service[]{ new Impl(), new Impl2()}` if `Class` is a constant. No?

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALeTM-nP%2B0xoSbsrK5JZwo1M_e2WfgdfLXrKc0PrYMGDmvkh_A%40mail.gmail.com.

--

Stéphane Épardaud

[Georgios Andrianakis]

On Wed, May 4, 2022 at 12:00 PM Stephane Epardaud <stephane...@gmail.com> wrote:

Besides the numbers, I wonder why this is so complex. I would have thought that with Jandex being able to report use-sites of the `ServiceLoader` API, we could just replace the calls to `ServiceLoader.load(Class)` with calls to `$generated$loadServices` (containing `new Service[]{ new Impl(), new Impl2()}` if `Class` is a constant. No?

Not really, because most of the instances are not Jandex-ed and at this point, we probably don't want them to be.

[Stephane Epardaud]

Well, OK, so perhaps you hard-code specific classes that need to be transformed if we can't index/find them, but still they should all have the same transformer, no?

--

Stéphane Épardaud

[Martin Kouba]

On 04. 05. 22 11:00, Stephane Epardaud wrote:
> Besides the numbers, I wonder why this is so complex. I would have
> thought that with Jandex being able to report use-sites of the
> `ServiceLoader` API, we could just replace the calls to
> `ServiceLoader.load(Class)` with calls to `$generated$loadServices`
> (containing `new Service[]{ new Impl(), new Impl2()}` if `Class` is a
> constant. No?

Hm, but this is not supported by Jandex yet, or?

> * It's rather straightforward to implement as there are no

> new concepts introduced, just reuse of existing Quarkus
> functionality

> * It provides the best possible runtime performance as

> service loader calls are replaced with simple
> constructor invocations
>
> The disadvantages are:
>

> * It can be tedious to implement as writing ASM code is

> not to most people's liking

> * Every new instance of the Service Loader we want to

> eliminate requires a new optimization

> * The optimizations (essentially ASM transformations)

> would require maintenance as they can get out of sync
> with the original code. It's important to note however
> that even if this happens, the application simply
> reverts to the default JVM behavior. Moreover, we can
> likely introduce tests that use ClassLoader events
> (which Sanne had added in the past) to ensure that the
> optimizations don't break.
>
> I benchmarked the effect of these optimizations on the
> startup time using our getting-started quickstart (using
> Java 11) and got the following results:
>
>

> *Original* *Optimized*

> Average 0.47739 0.46453
> Median 0.476 0.464
> 70th Percentile 0.485 0.468
> 80th Percentile 0.49 0.474
> 90th Percentile 0.4951 0.48
>
>
> I also have Java 17 numbers that show a very similar difference.
> Furthermore, also checked the effect on the size of the
> native binary and got the following:
>

> *Original* *Optimized*

> 36881K 36865K
>
>
> As you can see, the effects aren't mind blowing by any means.
>
> What do you think about this? Is it something worth pursuing
> in this form or another?
>
> --
> You received this message because you are subscribed to the
> Google Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/quarkus-dev/CALeTM-kyM%2BeCk3GVkKeeJ0FW-yt%2BsxzZ94Rw3hS6HeaaXdJCZQ%40mail.gmail.com

> <https://groups.google.com/d/msgid/quarkus-dev/CALeTM-kyM%2BeCk3GVkKeeJ0FW-yt%2BsxzZ94Rw3hS6HeaaXdJCZQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/CALeTM-nP%2B0xoSbsrK5JZwo1M_e2WfgdfLXrKc0PrYMGDmvkh_A%40mail.gmail.com
> <https://groups.google.com/d/msgid/quarkus-dev/CALeTM-nP%2B0xoSbsrK5JZwo1M_e2WfgdfLXrKc0PrYMGDmvkh_A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> Stéphane Épardaud

>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/CAKU9E9tqSuLk5iUWvJ2QUnbELbSGiMi7aR5v%2BjKHspk9NbzZOQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/quarkus-dev/CAKU9E9tqSuLk5iUWvJ2QUnbELbSGiMi7aR5v%2BjKHspk9NbzZOQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Martin Kouba
Software Engineer
Red Hat, Czech Republic

[Georgios Andrianakis]

On Wed, May 4, 2022 at 12:06 PM Stephane Epardaud <stephane...@gmail.com> wrote:

Well, OK, so perhaps you hard-code specific classes that need to be transformed if we can't index/find them, but still they should all have the same transformer, no?

In a lot of the cases, yes, but sometimes we can do better (as with the RESTEasy Reactive case where you essentially only need one).

But in any case, this was just a prototype to see what the actual effect was in practice as I don't think any of us had a good estimate.

[Stephane Epardaud]

On Wed, 4 May 2022 at 11:10, Martin Kouba <mko...@redhat.com> wrote:

Hm, but this is not supported by Jandex yet, or?

It's been for a while, yes, this is how we transform Panache use-sites and Logger use-sites and Renarde reverse-router use-sites. It's _very_ useful. 

[Martin Kouba]

AFAIK the panache logging is only using the
IndexView.getKnownUsers(DotName) to find the classes that need to be
transformed and then the usual ASM hell...

[Stephane Epardaud]

Sure, but that's what I was saying, we could find all the classes that use `ServiceLoader` and register the transformer for them, and do it all automatically with no special-code.

--

Stéphane Épardaud

[Emmanuel Bernard]

Once this thread converges, it looks like a good candidate for an ADR so we remember this.

--