jna memory management
Tobias Wolf
Hi community,
I`m one of a team of developers responsible for writing and maintaining a own written server component. Our server component is running highly throughput rates and does of course multi-threading. During huge performance testing I found an big memory leak issue within JNA, but let me explain in detail. JNA allocates native memory everywhere, even at places where a java developer would not think first e.g. IntByReference, PointerbyReference. All pointers of course needs a memory location and therefore must be malloc'ed, depending on the platform default pointer size 4 or 8 bytes, this is being done in the background by calling e.g. the constructor or when the object comes back from a native function mapped by jna. The deallocation is being done also in the background, here is the finalize method overwritten and a dispose method is called which wrapped basically a native free. Here is the first issue, because the finalize method is only for the java garbage collector, even Oracle is pointing this out, they explain this on their web site. The reason is clear because the GC is for removing the java classes and not for freeing custom user code. The solution is to use the Phantom reference class and a reaper thread for freeing the native memory.
We use for this a class named SMART_POINTER, all our user pointers are extending the smart pointer class. The SMART_POINTER_REF class does the magic, it give us now a possibility to clean up native resources after the GC has run. This is what the reaper thread does, it waits on a queue which is being filled by the GC and frees the native resources after GC has removed the java smart pointer classes.
This concept we call implicit freeing of native resources. But this is not enough, because in a multi-treading environment calling malloc and free in different threads can lead to memory leaks, for sure I`ve tested this behavior, but I cannot explain in deep why this is happening. Therefore we need also a concept for explicit freeing native pointers. This we`ve realized in our smart pointers with the "public void free()" method, it is a good developing technique to free native resources immediately after usage to keep your memory footprint small. But this is currently not possible, because the JNA memory class and all inherited classes like IntByReference (see below my draft) does not allow you to call free by yourself! I`m struggling currently to free my callback objects explicitly (don`t know how to get the pointer), because my application consumes to much memory and therefore I would ask you about a JNA design change.
```
public static abstract class SMART_POINTER extends PointerType {
protected AtomicBoolean alreadyFreed = new AtomicBoolean(false);
protected AtomicBoolean autoFree = new AtomicBoolean(true);
protected StackTraceElement[] stacktrace = null;
protected AtomicBoolean hasRef = new AtomicBoolean(false);
public SMART_POINTER() {
log.entry();
if (properties.properties().isDebugMemEnabled()) {
this.stacktrace = Thread.currentThread().getStackTrace();
}
log.exit();
}
public SMART_POINTER(Pointer p) {
super();
log.entry(p);
this.setPointer(p);
this.createRef();
log.exit(p);
}
public SMART_POINTER(Pointer p, boolean autoFree) {
log.entry(p);
this.setPointer(p);
this.setAutoFree(autoFree);
this.createRef();
log.exit(p);
}
public void createRef() {
if (hasRef.compareAndSet(false, true)) {
OpenSSL.refTable.add(new SMART_POINTER_REF(this));
}
}
@Override
public Object fromNative(Object nativeValue, FromNativeContext context) {
SMART_POINTER smartPointer = (SMART_POINTER) super.fromNative(nativeValue, context);
if (smartPointer != null) {
smartPointer.createRef();
}
return smartPointer;
}
public long getNativePointer() {
return Pointer.nativeValue(this.getPointer());
}
public StackTraceElement[] getStacktrace() {
return stacktrace;
}
public void free() {
if (this.alreadyFreed.compareAndSet(false, true)) {
log.trace("freeing pointer: " + this.getClass() + " - " + this.getPointer());
this.nativeFree();
this.setAutoFree(false);
}
}
public void setAutoFree(boolean autoFree) {
this.autoFree.set(autoFree);
}
public boolean isAutoFree() {
return this.autoFree.get();
}
public boolean isNotAlreadyFreed() {
return !this.alreadyFreed.get();
}
public void setAlreadyFreed(boolean freed) {
this.alreadyFreed.set(freed);
}
protected abstract void nativeFree();
public boolean compareAndSetAutoFree(boolean expected, boolean update) {
return this.autoFree.compareAndSet(expected, update);
}
}
public static class SMART_POINTER_REF extends PhantomReference<SMART_POINTER> {
public SMART_POINTER_REF(SMART_POINTER referent) {
super(referent, OpenSSL.refQueue);
}
public SMART_POINTER getSmartPointer() {
Field declaredField;
try {
declaredField = this.getClass().getSuperclass().getSuperclass().getDeclaredField("referent");
declaredField.setAccessible(true);
return (SMART_POINTER) declaredField.get(this);
} catch (Exception e) {
log.catching(e);
}
return null;
}
@Override
public int hashCode() {
SMART_POINTER pointer = getSmartPointer();
final int prime = 31;
int result = 1;
result = prime * result + ((pointer == null) ? 0 : pointer.hashCode());
return result;
}
@Override
public boolean equals(Object obj) {
SMART_POINTER pointer = getSmartPointer();
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
SMART_POINTER_REF other = (SMART_POINTER_REF) obj;
if (pointer == null) {
if (other.getSmartPointer() != null)
return false;
} else if (!pointer.equals(other.getSmartPointer()))
return false;
return true;
}
}
private class ReaperThread extends Thread {
private AtomicBoolean stopping = new AtomicBoolean(false);
private AtomicBoolean stopped = new AtomicBoolean(false);
ReaperThread() {
super("Smart pointer Reaper");
this.setDaemon(true);
}
/**
* Run the reaper thread that will free native objects as their
* associated marker objects are reclaimed by the garbage collector.
*/
public void run() {
while (!stopping.get()) {
SMART_POINTER_REF smartPointerRef = null;
try {
// wait until a SMART_POINTER_REF is ready for deletion
smartPointerRef = (SMART_POINTER_REF) refQueue.remove();
free(smartPointerRef);
refTable.remove(smartPointerRef);
smartPointerRef.clear();
} catch (InterruptedException e) {
if (stopping.get()) {
break;
}
} catch (Exception e) {
// silently ignore...
continue;
}
}
log.trace("Freeing remaining {} Phantom References", refTable.size());
synchronized (refTable) {
Iterator<SMART_POINTER_REF> iterator = refTable.iterator();
while (iterator.hasNext()) {
free(iterator.next());
iterator.remove();
}
}
this.stopped.set(true);
}
private void free(SMART_POINTER_REF smartPointerRef) {
SMART_POINTER smartPtr = smartPointerRef.getSmartPointer();
if (smartPtr.isNotAlreadyFreed() && smartPtr.isAutoFree()) {
if (properties.isDebugMemEnabled()) {
log.trace("a native pointer has not been freed: {}", smartPtr.toString());
log.trace(stacktraceToString(smartPtr.getStacktrace()));
}
smartPtr.free();
}
}
public boolean isStopped() {
return this.stopped.get();
}
/**
* Stops the reaper thread.
*/
public void stopWorking() {
stopping.set(true);
interrupt();
}
}
public class IntByReference extends ByReference {
public IntByReference() {
this(0);
}
public IntByReference(int value) {
super(4);
setValue(value);
}
public void setValue(int value) {
getPointer().setInt(0, value);
}
public int getValue() {
return getPointer().getInt(0);
}
public void free() {
((Memory)this.getPointer()).dispose();
}
}
```
Kustaa Nyholm
Reply-To: "jna-...@googlegroups.com" <jna-...@googlegroups.com>
Date: Wednesday, 22 February 2017 12:42
To: "jna-...@googlegroups.com" <jna-...@googlegroups.com>
Subject: jna memory management
Hi community,
I`m one of a team of developers responsible for writing and maintaining a own written server component. Our server component is running highly throughput rates and does of course multi-threading. During huge performance testing I found an big memory leak issue within JNA, but let me explain in detail. JNA allocates native memory everywhere, even at places where a java developer would not think first e.g. IntByReference, PointerbyReference. All pointers of course needs a memory location and therefore must be malloc'ed, depending on the platform default pointer size 4 or 8 bytes, this is being done in the background by calling e.g. the constructor or when the object comes back from a native function mapped by jna. The deallocation is being done also in the background, here is the finalize method overwritten and a dispose method is called which wrapped basically a native free. Here is the first issue, because the finalize method is only for the java garbage collector, even Oracle is pointing this out, they explain this on their web site. The reason is clear because the GC is for removing the java classes and not for freeing custom user code. The solution is to use the Phantom reference class and a reaper thread for freeing the native memory.
We use for this a class named SMART_POINTER, all our user pointers are extending the smart pointer class. The SMART_POINTER_REF class does the magic, it give us now a possibility to clean up native resources after the GC has run. This is what the reaper thread does, it waits on a queue which is being filled by the GC and frees the native resources after GC has removed the java smart pointer classes.
This concept we call implicit freeing of native resources. But this is not enough, because in a multi-treading environment calling malloc and free in different threads can lead to memory leaks, for sure I`ve tested this behavior, but I cannot explain in deep why this is happening. Therefore we need also a concept for explicit freeing native pointers. This we`ve realized in our smart pointers with the "public void free()" method, it is a good developing technique to free native resources immediately after usage to keep your memory footprint small. But this is currently not possible, because the JNA memory class and all inherited classes like IntByReference (see below my draft) does not allow you to call free by yourself! I`m struggling currently to free my callback objects explicitly (don`t know how to get the pointer), because my application consumes to much memory and therefore I would ask you about a JNA design change.
```
publicstaticabstractclassSMART_POINTER extends PointerType {
protected AtomicBoolean alreadyFreed = new AtomicBoolean(false);
protected AtomicBoolean autoFree = new AtomicBoolean(true);
protected StackTraceElement[] stacktrace = null;
protected AtomicBoolean hasRef = new AtomicBoolean(false);
public SMART_POINTER() {
log.entry();
if (properties.properties().isDebugMemEnabled()) {
this.stacktrace = Thread.currentThread().getStackTrace();
}
log.exit();
}
public SMART_POINTER(Pointer p) {
super();
log.entry(p);
this.setPointer(p);
this.createRef();
log.exit(p);
}
public SMART_POINTER(Pointer p, booleanautoFree) {
log.entry(p);
this.setPointer(p);
this.setAutoFree(autoFree);
this.createRef();
log.exit(p);
}
publicvoid createRef() {
if (hasRef.compareAndSet(false, true)) {
OpenSSL.refTable.add(new SMART_POINTER_REF(this));
}
}
@Override
public Object fromNative(Object nativeValue, FromNativeContext context) {
SMART_POINTER smartPointer = (SMART_POINTER) super.fromNative(nativeValue, context);
if (smartPointer != null) {
smartPointer.createRef();
}
returnsmartPointer;
}
publiclong getNativePointer() {
return Pointer.nativeValue(this.getPointer());
}
public StackTraceElement[] getStacktrace() {
returnstacktrace;
}
publicvoid free() {
if (this.alreadyFreed.compareAndSet(false, true)) {
log.trace("freeing pointer: " + this.getClass() + " - " + this.getPointer());
this.nativeFree();
this.setAutoFree(false);
}
}
publicvoid setAutoFree(booleanautoFree) {
this.autoFree.set(autoFree);
}
publicboolean isAutoFree() {
returnthis.autoFree.get();
}
publicboolean isNotAlreadyFreed() {
return !this.alreadyFreed.get();
}
publicvoid setAlreadyFreed(booleanfreed) {
this.alreadyFreed.set(freed);
}
protectedabstractvoid nativeFree();
publicboolean compareAndSetAutoFree(booleanexpected, booleanupdate) {
returnthis.autoFree.compareAndSet(expected, update);
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Timothy Wall
--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+unsubscribe@googlegroups.com.
Tobias Wolf
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
Neil C Smith
On 23 February 2017 at 07:51, Tobias Wolf <wolf....@gmx.net> wrote:
> I`m
> using in my server component per request/response cycle at least 2 callbacks
> and a few IntByReferences/PointerByReference objects. After some hundred
> thousand cycles I`m experiencing that the native non JVM memory is massive
> growing up and the JVM memory was pinned at startup with the parameters
> "-Xms256m -Xmx512m",
http://stackoverflow.com/questions/1744533/jna-bytebuffer-not-getting-freed-and-causing-c-heap-to-run-out-of-memory
One of the obvious answers to that (other than deallocation hacks with
Cleaner) is to use pooling. Can you not similarly cache your callback
and *ByReference objects, at least on a thread-local level?
Best wishes,
Neil
--
Neil C Smith
Artist & Technologist
www.neilcsmith.net
Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
Timothy Wall
--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+unsubscribe@googlegroups.com.
Tobias Wolf
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
Timothy Wall
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+unsubscribe@googlegroups.com.
Tobias Wolf
Neil C Smith
> Neil has a good idea, pooling makes sense when you use large buffers ... the
with the fact that I think using the GC to manage off-heap memory is a
daft idea. Note that ByteBuffers use phantom references for clean up
already, and that can still be a PITA, and you can find countless
discussions online in many different projects around use of
sun.misc.Cleaner, etc. to support that view!
So, I do kind of agree with you that manual memory management methods
(with caveats attached) makes sense, but I also think that anything
relying on finalization or references for native memory management is
a fundamentally flawed design - manage your own pool of native
resources explicitly.
My opinionated 2c!
Timothy Wall
--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+unsubscribe@googlegroups.com.
Matthias Bläsing
Am Mittwoch, 22. Februar 2017 11:42:32 UTC+1 schrieb Tobias Wolf:
[Examples of a SMART_POINTER construct that should free memory faster than the JNA built-in one]
Could you please give some more context what you want to archive or better how it should work? And maybe provide a complete sample where this is visible?
I have some strong doubts that using PhantomReferences will lead to better memory management. Currently native memory, backing a Memory object, is freed when the finalizer for that object runs. Following the java documentation:
- An object is phantom reachable if it is neither strongly, softly, nor weakly reachable, it has been finalized, and some phantom reference refers to it.
An is phantom reachable _after_ its finalizer has run. So freeing memory based on this would lead to later deallocation, not earlier.
If you need fast Memory clearing I'd subclass Memory and go down the try-with-resource route, allocate Memory in the resource clause of a try-with-resource block and implement a "close" method, that frees that memory. This way you'll get a reliable way for allocation+deallocation.
Greetings
Matthias