JNA and OSGi

[Scooter Morris]

Hi all,

I've been beating my head against the wall for a couple of days and have gotten nowhere so I figured it was time to ask for help.  I'm using a JNA package called jnainchi in my app, which is a plugin to Cytoscape.  Cytoscape uses OSGi.  I can get OSGi to load the native code provided by jnainchi by adding a Bundle-NativeCode line to the MANIFEST and then calling System.loadLibrary().  Great, so far so good.  The problem is that the JNA package want's to load osname-machine/library, so it can't find it.  System.loadLibrary doesn't take a path and because the native code is part of an OSGi bundle (and not on a standard classpath), that's the only way I can load it.

If anyone has any hints as to how to coerce JNA and OSGi to work together, it would be really appreciated.

-- scooter

[Tres Finocchiaro]

The problem is that the JNA package want's to load osname-machine/library, so it can't find it.  System.loadLibrary doesn't take a path and because the native code is part of an OSGi bundle (and not on a standard classpath), that's the only way I can load it.

Hi,

Can you try to set jna.library.path?

Per:

https://java-native-access.github.io/jna/5.6.0/javadoc/com/sun/jna/NativeLibrary.html#library_search_paths 

In my experience, this can be set at runtime if needed.

[Matthias Bläsing]

Hi,

JNA provides OSGI Headers in the MANIFEST and it should also be possible to automatically load the native library part.

It would be helpful to know why you think you need to do something special. What errors do you see when JNA initializes?

Greetings

Matthias

[Scooter Morris]

Thanks for getting back to me.  Unfortunately, that didn't work.  Here is the code I added to my bundle activator:

     Properties props = System.getProperties();
     props.setProperty("jna.library.path", "/home/scooter/CytoscapeConfiguration/3/apps/installed/chemViz2-1.2.jar");

where chemViz2-1.2.jar is my bundle.  In that bundle is the native code:

META-INF/
META-INF/MANIFEST.MF
META-INF/maven/
META-INF/maven/edu.ucsf.rbvi/
META-INF/maven/edu.ucsf.rbvi/chemViz2/
META-INF/maven/edu.ucsf.rbvi/chemViz2/pom.properties
META-INF/maven/edu.ucsf.rbvi/chemViz2/pom.xml
<--- SNIP --->
jna-5.10.0.jar
jna-inchi-api-1.2.1.jar
jna-inchi-core-1.2.jar
jna-inchi-darwin-aarch64-1.2.jar
jna-inchi-darwin-x86-64-1.2.jar
jna-inchi-linux-aarch64-1.2.jar
jna-inchi-linux-arm-1.2.jar
jna-inchi-linux-x86-1.2.jar
jna-inchi-linux-x86-64-1.2.1.jar
jna-inchi-smiles-1.2.1.jar
jna-inchi-win32-x86-1.2.jar
jna-inchi-win32-x86-64-1.2.jar
junit-4.10.jar
linux-x86-64/
linux-x86-64/libjnainchi.so
signatures-1.1.jar
vecmath-1.5.2.jar

And this is what I get when I attempt to call the INCHI functions:

java.lang.RuntimeException: Error loading InChI native code. Please check that the binaries for your platform (linux-x86-64) have been included on the classpath.
    at io.github.dan2097.jnainchi.JnaInchi.checkLibrary(JnaInchi.java:828) ~[?:?]
    at io.github.dan2097.jnainchi.JnaInchi.getInchiInputFromInchi(JnaInchi.java:650) ~[?:?]
    at io.github.dan2097.jnainchi.JnaInchi.getInchiInputFromInchi(JnaInchi.java:646) ~[?:?]
    at org.openscience.cdk.inchi.InChIToStructure.<init>(InChIToStructure.java:137) ~[?:?]
    at org.openscience.cdk.inchi.InChIToStructure.<init>(InChIToStructure.java:148) ~[?:?]
    at org.openscience.cdk.inchi.InChIGeneratorFactory.getInChIToStructure(InChIGeneratorFactory.java:211) ~[?:?]
    at edu.ucsf.rbvi.chemViz2.internal.model.Compound.convertInchiToSmiles(Compound.java:397) ~[?:?]
    at edu.ucsf.rbvi.chemViz2.internal.model.Compound.createStructure(Compound.java:318) ~[?:?]
    at edu.ucsf.rbvi.chemViz2.internal.model.Compound.<init>(Compound.java:132) ~[?:?]
    at edu.ucsf.rbvi.chemViz2.internal.model.Compound.<init>(Compound.java:109) ~[?:?]
    at edu.ucsf.rbvi.chemViz2.internal.tasks.GetCompoundTask.call(GetCompoundTask.java:107) ~[?:?]
    at edu.ucsf.rbvi.chemViz2.internal.tasks.GetCompoundTask.call(GetCompoundTask.java:56) ~[?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
    at java.lang.Thread.run(Thread.java:840) ~[?:?]
Caused by: java.lang.UnsatisfiedLinkError: Unable to load library 'jnainchi':
libjnainchi.so: cannot open shared object file: No such file or directory
libjnainchi.so: cannot open shared object file: No such file or directory
Native library (linux-x86-64/libjnainchi.so) not found in resource path (/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/branding-3.11.0-SNAPSHOT.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/karaf-launcher-3.11.0-SNAPSHOT.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/org.apache.karaf.diagnostic.boot-4.3.9.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/org.apache.karaf.jaas.boot-4.3.9.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/org.apache.karaf.main-4.3.9.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/org.apache.karaf.specs.activator-4.3.9.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/boot/osgi.core-7.0.0.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/jdk9plus/istack-commons-runtime-3.0.10.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/jdk9plus/jakarta.xml.bind-api-2.3.3.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/jdk9plus/javax.annotation-api-1.3.2.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/jdk9plus/jaxb-runtime-2.3.3.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/jdk9plus/org.apache.servicemix.specs.activation-api-1.2.1-1.2.1_3.jar:/usr/local/src/Cytoscape/cytoscape3/cy3/project/cytoscape/gui-distribution/assembly/target/cytoscape/framework/lib/jdk9plus/txw2-2.3.3.jar)
    at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:301) ~[?:?]
    at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:461) ~[?:?]
    at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:403) ~[?:?]
    at io.github.dan2097.jnainchi.inchi.InchiLibrary.<clinit>(InchiLibrary.java:33) ~[?:?]
    at io.github.dan2097.jnainchi.JnaInchi.<clinit>(JnaInchi.java:65) ~[?:?]
    ... 13 more

Note that I *can* load the library by going through OSGi's loader:

System.loadLibrary("jnainchi");

and I've tried that, but apparently the NativeLibrary.loadLibrary call isn't recognizing that it's already loaded.

Thanks again!

-- scooter

--
You received this message because you are subscribed to a topic in the Google Groups "Java Native Access" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jna-users/3m6veUJPdz4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/CANQs7dA8Vwre4uWrd%2BipoY6%3DeRKcCwWGG-q2RszSx9hzbjWCRA%40mail.gmail.com.

[Scooter Morris]

Hi Matthias,

Thanks for getting back to me.  Here is what I'm seeing:

    Suppressed: java.lang.UnsatisfiedLinkError: libjnainchi.so: cannot open shared object file: No such file or directory

The required library (libjnainchi.so) is part of my bundle, in (what I hope) is the right location:

jar tf target/chemViz2-1.2.jar

META-INF/
META-INF/MANIFEST.MF
META-INF/maven/
META-INF/maven/edu.ucsf.rbvi/
META-INF/maven/edu.ucsf.rbvi/chemViz2/
META-INF/maven/edu.ucsf.rbvi/chemViz2/pom.properties
META-INF/maven/edu.ucsf.rbvi/chemViz2/pom.xml

beam-core-1.3.5.jar
beam-func-1.3.5.jar
...snip....

jna-5.10.0.jar
jna-inchi-api-1.2.1.jar
jna-inchi-core-1.2.jar
jna-inchi-darwin-aarch64-1.2.jar
jna-inchi-darwin-x86-64-1.2.jar
jna-inchi-linux-aarch64-1.2.jar
jna-inchi-linux-arm-1.2.jar
jna-inchi-linux-x86-1.2.jar
jna-inchi-linux-x86-64-1.2.1.jar
jna-inchi-smiles-1.2.1.jar
jna-inchi-win32-x86-1.2.jar
jna-inchi-win32-x86-64-1.2.jar
junit-4.10.jar
linux-x86-64/
linux-x86-64/libjnainchi.so
signatures-1.1.jar
vecmath-1.5.2.jar

And my manifest has the appropriate Bundle-NativeCode header:

Bundle-ManifestVersion: 2
Bundle-Name: chemViz2
Bundle-NativeCode: linux-x86-64/libjnainchi.so; osname = Linux ; process
 or = x86-64
Bundle-SymbolicName: chemViz2
Bundle-Version: 1.2

Any guidance would be really appreciated.

Thanks!

-- scooter

--
You received this message because you are subscribed to a topic in the Google Groups "Java Native Access" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jna-users/3m6veUJPdz4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/b2eeb5407df12d6a7ecbc507fdf35c3a41e4e4eb.camel%40doppel-helix.eu.

[Tres Finocchiaro]

Can you try to set jna.library.path?

Per:

https://java-native-access.github.io/jna/5.6.0/javadoc/com/sun/jna/NativeLibrary.html#library_search_paths 

In my experience, this can be set at runtime if needed.

If the native resources are bundled, I believe this statement will be relevant... quoting:

Context class loader classpath. Deployed native libraries may be installed on the classpath under ${os-prefix}/LIBRARY_FILENAME, where ${os-prefix} is the OS/Arch prefix returned by Platform.getNativeLibraryResourcePrefix(). If bundled in a jar file, the resource will be extracted to jna.tmpdir for loading, and later removed (but only if jna.nounpack is false or not set).

Assuming the classpath is set properly and jna.nounpack is NOT set to false, the native library should be found, extracted and used.  As a stop-gap you may also try to extract the native library and use the aforementioned technique.

Note, I'm not familiar with OSGi (or how this would impact this behavior), so I apologize if these steps are incorrect for your project.

[Matthias Bläsing]

Hi Scooter,

ok, we are talking about an application library, not the jnidispatch library. I'd like to have a closer look and if I understand it correctly we are talking about an Open Source project here?

So could you give me instructions how to get to the state where you see the problem?

I can build from source, but I need some steps, that I can follow.

Thank you

Matthias

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/10db18ff-1338-4ee5-9477-a5b246d895ed%40gmail.com.

[Scooter Morris]

Hi Matthias,

    Yes, everything is open source.  The app I'm developing is called chemViz2 (https://github.com/RBVI/chemViz2), which is a cheminformatics app for the Cytoscape network analysis system. It uses a jna library: jnainchi (https://github.com/dan2097/jna-inchi).  So, the layers are a bit complex.  The native code I'm trying to load is part of the jnainchi library.  To build this thing, you first need to install Cytoscape, which is the OSGi component.  Run Cytoscape from a command line.  Then you can checkout the "jna" branch of chemViz2 and build it using maven.  Copy the jar file into $HOME/CytoscapeConfiguration/3/apps/installed -- that will install the app and execute the bundle start method.  So far, so good.  The problem comes when I try to use any of the InChI code.  I've put a test file up on my web server: https://www.cgl.ucsf.edu/home/scooter/ChemInforPluginTest.cys  Download that (it's very small) and open it in Cytoscape.  You should see a simple network.  Then do Apps->Cheminformatics tools->Show compound Table->for all nodes.  That should elicit the error shown below.

Thanks so much for your help!!

-- scooter

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/db4552cbd52c1b4ab5b6e1443d27a85c70c6e451.camel%40doppel-helix.eu.

[Matthias Bläsing]

Hi again,

Am Mittwoch, dem 14.08.2024 um 13:51 -0700 schrieb Scooter Morris:

    Yes, everything is open source.  The app I'm developing is called chemViz2 (https://github.com/RBVI/chemViz2), which is a cheminformatics app for the Cytoscape network analysis system. It uses a jna library: jnainchi (https://github.com/dan2097/jna-inchi).  So, the layers are a bit complex.  The native code I'm trying to load is part of the jnainchi library.  To build this thing, you first need to install Cytoscape, which is the OSGi component.  Run Cytoscape from a command line.  Then you can checkout the "jna" branch of chemViz2 and build it using maven.  Copy the jar file into $HOME/CytoscapeConfiguration/3/apps/installed -- that will install the app and execute the bundle start method.  So far, so good.  The problem comes when I try to use any of the InChI code.  I've put a test file up on my web server: https://www.cgl.ucsf.edu/home/scooter/ChemInforPluginTest.cys  Download that (it's very small) and open it in Cytoscape.  You should see a simple network.  Then do Apps->Cheminformatics tools->Show compound Table->for all nodes.  That should elicit the error shown below.

I had a look at the code. The first commit just makes it compile with recent maven versions (using http to load dependencies is discouraged):

https://github.com/matthiasblaesing/chemViz2/commit/ec3955642f7c520334f6f9d20f5d423198d701e1

The second is the relevant one:

https://github.com/matthiasblaesing/chemViz2/commit/7ff74753f2353a465ccecce35bf0c51a076ce531

The idea is, that initialization of JnaInchi must happen with the right classloader. When being loaded JnaInchi  loads its native library, but does not pass the classloader to the NativeLibrary loader. NativeLibrary then uses the thread context classloader to extract the native library. 

The native library is accessible from the bundle classloader as it is part of its classpath. That classloader is made the context classloader for loading JnaInchi. All following calls that use the already loaded library just work.

Side observation: JNA can only be loaded once per JVM it might be worth coordinating with cytoscape so that a single recent version is made available centrally.

HTH

Matthias

[Scooter Morris]

Matthias,

Amazing!!!  I honestly never would have gotten there without your help.  I'll definitely bring up the idea of adding JNA to our standard libraries at our next Cytoscape meeting!  I can't thank you enough -- this is fantastic!

-- scooter

--
You received this message because you are subscribed to a topic in the Google Groups "Java Native Access" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jna-users/3m6veUJPdz4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/5303d685aae324e4780639d2046c64333dd52b33.camel%40doppel-helix.eu.