Native dependency handling in Java

[Lukács T. Berki]

Hey there,

I was recently made aware of a bug and a pull request to fix it in the way our Java code handles native libraries.

Now, the pull request looks reasonable, at least after a quick glance, but the code that supports native library loading in Java is very crufty and it's essentially how things work at Google and when it was written, non-Google use was not even considered.

Therefore, I thought that instead of fixing the bug, we could just remove that functionality, but I don't know if anyone uses it or not, and if so, for what purpose. 

Robert, if this functionality went away, would you have a substitute? What are you using it for?

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Dmitry Lomov]

It does seem weird that while we say that Bazel is a multi-language build system, Java->C++ builds do not really work..

--
You received this message because you are subscribed to the Google Groups "Bazel/JVM Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-sig-jv...@googlegroups.com.
To post to this group, send email to bazel-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-sig-jvm/CAOu%2B0LUNFHgb%2BpX2WV5F-sTpx0V7tSVzZsDLUeMp2m0zL9uG6Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Google Germany GmbH

Erika-Mann-Straße 33, 80636 München, Germany

[Lukács T. Berki]

Can't disagree with that, but I'd much rather not promise functionality before we know what functionality is needed. If we "officially" support Java->C++ dependencies, we suddenly have to have a good story for how it works on all supported operating systems, in various deployment scenarios, etc.

[Robert Brown]

Bazel's java_binary rule arranges for the java.library.path Java system

property to include all directories containing native libraries that a binary

depends on.  This allows Java libraries to load native library dependencies

using System.loadLibrary().  At Two Sigma we have very widely used Java

libraries that dynamically load C shared libraries to implement authentication,

RPC, etc.

 

If the above described functionality went away, I think we would have to run

all of our applications using some sort of wrapper script.  Although we may be

able to use an aspect to discover what native libraries a binary depends on in

order to set java.library.path ourselves, we also need the functionality in the

pull request I submitted.  That change sets LD_LIBRARY_PATH to the same set of

directories as java.library.path, so that native libraries with dependencies on

other native shared libraries can be loaded by Java.

 

I'd prefer Bazel to have good built-in support for Java binaries that depend on

native shared libraries.  It would seem to be a fairly common need.  Also, I

would expect to see a similar need with Python binaries that depend on native

shared libraries.  This kind of Bazel feature is more important outside of

Google, where it's much more common to use native libraries compiled by others,

instead of building all artifacts from source code.

[Ulf Adams]

I think it would be a mistake to introduce more divergence between the rules Google is using and the rules we're shipping in Bazel, regardless of how crufty the code seems to be, and that doesn't take into account all the Bazel users who are already relying on the functionality.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-sig-jvm/c68655e07c874317a02a94f201b4bbb2%40EXMBNJE6.ad.twosigma.com.

[Lukács T. Berki]

Yeah... I'm not that happy about this state of affairs, but on second thought, let's not gratuitously break working things. I'll try to make passing C++ libraries through the Java sandwich work. I *think* it's possible.

[Kevin Bierhoff]

Keep in mind we probably need C++ to work in the Java sandwich for Google as well!

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-sig-jvm/CAOu%2B0LWkRLWj2WJvW3_xZGTsDPPat%3DSmfV%2Bj66k2Deb08aNGKA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kevin Bierhoff

Google

[Lukács T. Berki]

Indeed!