spark crashes when calling .so file using JNA

[xiao qiang]

In one program, we use to build a ByReference space, and start JNA to call the so model in this space. First I call The c model gets a pointer named pointer a, and then I will continue to use jna to pass this pointer to the next c function. When the program first started, everything was normal, but after a while, the spark executor reported an error and returned code 134, and the program terminated.Here is Spark excutor log

[xiao qiang]

If anyone needs sample code, please inform me and I will give a sample structure, but cannot provide .so file, please understand

[Matthias Bläsing]

Hi,,

Am Dienstag, dem 05.09.2023 um 20:40 -0700 schrieb xiao qiang:
> First I call The c model gets a pointer named pointer a, and then I
> will continue to use jna to pass this pointer to the next c function.
> When the program first started, everything was normal, but after a
> while, the spark executor reported an error and returned code 134,
> and the program terminated.

This sounds like a problem with GC. My gut feeling is, that you pass
memory allocated on the java side to native without retaining a strong
reference. If the native code does not make a defensive copy, the
memory object might be GCed and the native memory freed.

You are essentiall creating a use-after-free.

Create a _minimal_ sample and provide a detailed
description/documentation of the native code. Then people might be able
to reason about your approach.

> Here is Spark excutor log image-2023-08-30-15-04-48-265.png

Please _never_ user screenshots to communicate logs. This makes it
unncessary hard to read. Attach the log to your mail or include the
relevant part inline please.

Greetings

Matthias

[xiao qiang]

Thanks!

 Your guess is correct and we have solved the problem after troubleshooting. The problem comes from the fact that there are wrong pointers in the c file, which are used again after being recycled by gc, causing this problem.

Regarding the issue of using screenshots in logs, I will pay attention to it in future consultations. Thanks for your advice

Greetings