Native-image support

113 views
Skip to first unread message

Kamil Toman

unread,
Feb 17, 2019, 10:46:54 AM2/17/19
to HikariCP
Hi all,

I experimented lately with Graal's SubstrateVM to create executable images. One of the goals was a small application that accesses a database using a connection pool.

SubstrateVM is in the very early stage so most things don't work yet. Plus there is a big list of various limitattions that programs compiled into native code must work with.

I tried to adapt HikariCP to fit into SubstrateVM's closed-world assumptions and the good news is that it mostly works.

The biggest pain points are:
1) unsupported features in JDBC driver code,
2) dynamic loading of JDBC drivers,
3) reflective setup that includes dynamic classloading,
4) optional dependencies.

Let's look at those individually:

1) I found out that, unlike many others, PostgresQL JDBC driver mostly does work - the most problematic code only takes care of cleaning up badly handled resources. Limited but workable.

2) The driver code can be explicitly loaded as a dependency and manually registered. OK.

3) The reflective setup in HikariCP is a tough nut for Graal to break. But I figured out that if I just replace the dynamic code with some repetitive (and a bit ugly) static code Graal can compile the code without problems.
   See this commit.  It doesn't end there - the program must prepare and pass the datasource to the pool directly bacause of other limitations. OK.

4) Optional dependencies are another thing that Graal's native-image can't handle well. Even if you don't use those features native-image will try to compile them. For now I just stubbed those. TBD.

You can have a look at the small fork (just a few patches really) that can be compiled into a native binary, here https://github.com/leafclick/HikariCP.
And the small (clojure) program that uses it to run database migrations, here https://github.com/leafclick/pgmig.

What I'd like to see is a single version of HikariCP that would fully work in either traditional JVM or natively compiled code. The question is how to get there. What do you think?

Kamil



Brett Wooldridge

unread,
Feb 20, 2019, 12:27:17 PM2/20/19
to HikariCP
Hi Kamil,

First, let me say it is a great goal and I appreciate your investigating it.  I too am interested in Graal, but not for native-image generation.  Still, I understand your goal and motivation.

Having said that ... man is some of that code ugly! :-)  That is not a comment on your coding style, but a result of the necessary "unrolling" of the reflection.

I've made some comments on your code.  Note, this is not a committment to merge this, just my feedback on the code in a purely acedemic context.  Let's keep up the dialog.

Brett

Kamil Toman

unread,
Feb 22, 2019, 3:26:47 PM2/22/19
to HikariCP
Hi Brett!

Thanks for looking at those patches. In the meantime I looked into optional dependencies and I found out that if the pool only accesses them by reflection Graal's native-image doesn't go deep into that part of the dependency tree. But if you want them included you can add a configuration feature and it will pick them up.
With the latest Graal's support for incomplete classpath the process is much smoother.

I reworked previous patches and made them less aggressive (taking your comments into account). See https://github.com/leafclick/HikariCP/commits/dev

- "Actually use maven-compiler-plugin to set java source and target to 1.8." - a workaround the force IDEA to import the correct java versions; mvn seems to use the same settings (no change)
- "Replace some introspective code by static code." - manually unrolled config and config loading
- "Remove direct references to classes defined in optional dependencies." - only referencing HikariCP classes that don't import optional dependencies; lazy loading connectors in setters

The good news is that even a native image can be configured via standard configuration options now - preparing a pre-configured datasource is no longer needed if the JDBC driver is configured for reflection access. I updated PGMig accordingly (and it works).

Kamil
Reply all
Reply to author
Forward
0 new messages