Loading library in/on main thread

[Sven Zahrend]

I encountered the following issue with Ubuntu Linux/openjdk/Kotlin/Rust development:

I'm loading a uniffied Rust libexample.so in the main thread of my console application.

So far I thought and learned, that native code loaded in the main thread is executed in the main thread.

But, after a lot of examination on the Rust, the Java/Kotlin and the os side, I found out that the NativeLoading procedures produces a second os-thread for the java process, always consecutive to the main thread and with an offset of 4.

On the rust side, this has implications: windowing (winit) does a main thread check and it will fail because processId != threadId, and we all know that the main threadId = processId.

Having the library running on another thread can also have implications for OpenGl context sharing or things like this.

However, is it possible to omit a second jna thread and let the library functions execute in exactly the context of the java main thread (and it's OS counterpart)?

Thank you for some clarification!

[Matthias Bläsing]

Hi,

Am Samstag, dem 01.02.2025 um 16:49 -0800 schrieb Sven Zahrend:
> However, is it possible to omit a second jna thread and let the library functions execute in exactly the context of the java main thread (and it's OS counterpart)?
>

it would have been nice if you would have provided a basis for an
answer.

The JNA codepaths normally don't spin up new threads. The only explicit
threading happens when callbacks are received from native and then, if
I remember correctly, so that the Java thread matches the native
thread.

So I repeat what I already wrote in the github issue:

Create an SSCCE, a short, self contained, compilable example, that
reproduces your problem. Publish it somewhere, where people can
download it (github project for example).

Greetings

Matthias

[Neil C Smith]

On Sun, 2 Feb 2025, 00:49 Sven Zahrend, <sven.z...@gmail.com> wrote:

I encountered the following issue with Ubuntu Linux/openjdk/Kotlin/Rust development:

...

But, after a lot of examination on the Rust, the Java/Kotlin and the os side, I found out that the NativeLoading procedures produces a second os-thread for the java process, always consecutive to the main thread and with an offset of 4.

On the rust side, this has implications: windowing (winit) does a main thread check and it will fail because processId != threadId, and we all know that the main threadId = processId.

Having the library running on another thread can also have implications for OpenGl context sharing or things like this.

However, is it possible to omit a second jna thread and let the library functions execute in exactly the context of the java main thread (and it's OS counterpart)?

What you're seeing is probably the JDK not JNA. The Java main thread isn't the OS main thread.

There's a flag to control this, but as far as I know it's macOS only. See -XstartOnFirstThread at 

https://docs.oracle.com/en/java/javase/23/docs/specs/man/java.html#extra-options-for-macos

Best wishes, 

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : 3rd Floor Suite, 207 Regent Street, London, W1B 3HH, England

[Sven Zahrend]

Hello Matthias,

Thank you so much for your patience and time.

I prepared this one, hoping it is sufficient enough to describe my question:

https://github.com/szsoftware/JvmJnaThreadingExample

Greetings,

Sven

[Sven Zahrend]

Hello Neil,

as far as I understand this option right, then this is about on which OS-Thread runs the first Java Thread of an Application.

My question is about why my native library is not running in the first OS-Thread in which runs my first/Main Java Thread.

I was hoping there are jvmArgs, specific to this jna dependency, where one can determine some behavior.

Greetings,

Sven

[Neil C Smith]

Hi,

On Mon, 3 Feb 2025, 00:36 Sven Zahrend, <sven.z...@gmail.com> wrote:

as far as I understand this option right, then this is about on which OS-Thread runs the first Java Thread of an Application.

My question is about why my native library is not running in the first OS-Thread in which runs my first/Main Java Thread.

These amount to the same thing. The first OS thread isn't the main Java thread. The JDK runs your main() code in another OS thread, which JNA is then using to call into your native library. 

Except on macOS, with the flag I mentioned, I don't know if there's a way to stop the JDK from keeping the first OS thread for itself.

[Matthias Bläsing]

Hi Sven,

thank you for the sample. And indeed you can verify what Neil already suggested. This is the thread dump generated by clhsdb (commandline debugger for hotspot).

As you can see I attach to the process 427997, but the main thread is on thread id 427999 (so while I see a different offset, it matches you description):

matthias@enterprise:~$ sudo /usr/lib/jvm/java-17-openjdk-amd64/bin/jhsdb clhsdb --pid 427997

Attaching to process 427997, please wait...

hsdb> threads

427999 main

State: IN_JAVA

Stack in use by Java: 0x00007212eabfe928 .. 0x00007212eabfe9d8

Base of Stack: 0x00007212eac00000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 427999

...

428007 Reference Handler

State: BLOCKED

Stack in use by Java: 0x00007212bc92b910 .. 0x00007212bc92ba58

Base of Stack: 0x00007212bc92d000

Last_Java_SP: 0x00007212bc92b910

Last_Java_FP: 0x00007212bc92b960

Last_Java_PC: 0x00007212d486c5ab

Thread id: 428007

...

428008 Finalizer

State: BLOCKED

Stack in use by Java: 0x00007212bc82b840 .. 0x00007212bc82ba48

Base of Stack: 0x00007212bc82d000

Last_Java_SP: 0x00007212bc82b840

Last_Java_FP: 0x00007212bc82b8a0

Last_Java_PC: 0x00007212d486c5ab

Thread id: 428008

...

428009 Signal Dispatcher

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc72d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428009

...

428010 Service Thread

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc62d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428010

...

428011 Monitor Deflation Thread

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc52d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428011

...

428012 C2 CompilerThread0

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc42d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428012

...

428013 C1 CompilerThread0

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc32d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428013

...

428014 Sweeper thread

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc22d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428014

...

428015 Notification Thread

State: BLOCKED

No Java frames present

Base of Stack: 0x00007212bc12d000

Last_Java_SP: null

Last_Java_FP: null

Last_Java_PC: null

Thread id: 428015

...

428017 Common-Cleaner

State: BLOCKED

Stack in use by Java: 0x00007212b58fd7d0 .. 0x00007212b58fda58

Base of Stack: 0x00007212b58ff000

Last_Java_SP: 0x00007212b58fd7d0

Last_Java_FP: 0x00007212b58fd838

Last_Java_PC: 0x00007212d486c5ab

Thread id: 428017

...

428019 JNA Cleaner

State: BLOCKED

Stack in use by Java: 0x00007212b56fd870 .. 0x00007212b56fda28

Base of Stack: 0x00007212b56ff000

Last_Java_SP: 0x00007212b56fd870

Last_Java_FP: 0x00007212b56fd8d0

Last_Java_PC: 0x00007212d486c5ab

Thread id: 428019

...

What might be possible: Start from rust. Run a mainloop there. In a secondary thread launch the JVM via its JNI interface. From there create a callback with your initialization code, that is executed from the primary thread (PID==TID) by your mainloop.

That would be an experiment but I would expect the java code then to execute on the mainloop.

Greetings

Matthias

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/jna-users/4236968d-b173-4dea-a538-704248056302n%40googlegroups.com.

[Neil C Smith]

On Mon, 3 Feb 2025, 17:08 'Matthias Bläsing' via Java Native Access, <jna-...@googlegroups.com> wrote:

thank you for the sample. And indeed you can verify what Neil already suggested. This is the thread dump generated by clhsdb (commandline debugger for hotspot).

As you can see I attach to the process 427997, but the main thread is on thread id 427999 (so while I see a different offset, it matches you description):

Incidentally, this may also be interesting and give an indication of what is happening on various platforms.

https://github.com/search?q=repo%3Aopenjdk/jdk%20CallJavaMainInNewThread&type=code

Best wishes, 

Neil

[Sven Zahrend]

Hi Matthias,

thank you again for investigating this and your detailed answer.

I focus to the second part of your answer and I'm happy that you suggest this.
Because it is exactly what I want to test and examine as a next step and be curious about the results.

However, I must admit that this is meanwhile a more or less scientific case what gave me a deeper look behind the scenes
and answered some fundamental questions.
On the Rust side I already found and still I'm finding workarounds.

I hope this conversation will be useful for the community somehow.

Have a good time!

Sven

[Sven Zahrend]

@Neil

That makes absolute sense now.
Thanks for that kind of proof of my or our observations regarding the threads by pointing to the sources.

I followed @Matthias jhsdb examinations and after double reading and code checking it's now clear to me
that "a second thread consecutive to main thread after utilizing jna" was BS.
This "second" thread is always there because it's the main thread.
The only thing what jna calls do is spawning some kind of additional helper/cleaning/whatever threads.

So far it's true that native code runs on the java main thread and therefore JavaThread-ID-1's pid/tid = rust-currentThread-Id.

Therefore I have to deal with that misleading main-thread checking in Rust world.

Anyways,
thank you all for all this enlightenment!

[Neil C Smith]

On Tue, 4 Feb 2025 at 04:59, Sven Zahrend <sven.z...@gmail.com> wrote:
> That makes absolute sense now.
> Thanks for that kind of proof of my or our observations regarding the threads by pointing to the sources.

No problem!

> So far it's true that native code runs on the java main thread and therefore JavaThread-ID-1's pid/tid = rust-currentThread-Id.
>
> Therefore I have to deal with that misleading main-thread checking in Rust world.

I wouldn't necessarily call it misleading. There's a connection here.
While I'm sure it's not the only reason main is run on a secondary
thread, a reason shared with that Rust library is consistency across
OS. On macOS, the first thread has to be reserved by the JVM to run
the event loop, and to allow AWT to work. The same reason that Rust
library needs the first thread on macOS.

I have a project using GLFW, so have dealt with this before - that
runs with the Java options -XstartOnFirstThread
-Djava.awt.headless=true -XX:+IgnoreUnrecognizedVMOptions (the last of
which allows the same options to run on other OS).

I found this comment on your issue on winit (which came up when I
searched for their main thread check) -
https://github.com/rust-windowing/winit/issues/4099#issuecomment-2624346321
The check itself is not misleading, in terms of ensuring consistency,
but they need an escape hatch for circumstances where it is not a hard
error. In the same way that the JVM provides the above escape hatch
for circumstances where it is a hard error.

Anyway, best of luck with your project.