Sharing natively allocated memory between Java program and JNI calls.

[John Hening]

Hello,

I would like to consider sharing native memory between JNI and Java. I split it to three cases:

I. Sharing memory between thread calling JNI and thread executing Java program, for example:

[Note that I present a simplified native function- treat it as pseudocode]

native void setMemory(long address, int size, byte value) {
    memset(address, size, value);  
}

long p = Unsafe.allocateMemory(1024);
Thread #1                        |   Thread #2
setMemory(p, 1024, 1);           |   long x = Unsafe.getLong(p);

For my eye it is not possbile to write a Java program that shares a memory between threads executing "normal" Java code and native code because of impossibility to reasoning about memory model. The only solution is to use interprocess-synchronization mechanisms.

II. Sharing memory between Java program and native call within thread.

It should work because of intra-thread consistency. But, how Java compiler can preserve it? Java compiler has no idea what is under the hood of a native call. I am not sure how it is possible to guarantee intra-thread consistency. For example:

long p = Unsafe.allocateMemory(8);                    

setMemory(p, 8, 2);
long x = 0;
x = Unsafe.getLong(p);
assert x != 0

Perhaps I should be sure that assert is evaluated to a true, but I am not? Why?:

JMM ensures that execution will be intra-thread consistency. Ok. But, I don't know how JVM can ensure that here. The only solution I see is to put a memory barrier after every native call. What do you think?

III.

long p1 = Unsafe.allocateMemory(8);
long p2 = Unsafe.allocateMemory(8);

Thread#1                      | Thread#2
Unsafe.putLong(p1, 1);        | if(Unsafe.getLongVolatile(p2) == 1)
Unsafe.putLongVolatile(p2, 1);|    assert Unsafe.getLong(p1) == 1

For my eye assertion is always true. Yes?

[Gil Tene]

Putting Unsafe aside (since nothing is actually defined for Unsafe), you can use DirectByteBuffer to properly abstract off heap memory (including off heap memory shared with other processes) to your Java code. You can then use VarHandles (e.g. via avh = MethodHandles.byteBufferViewVarHandle(long[].class, ByteOrder.LITTLE_ENDIAN)  ) to gain well defined order-enforcing access to the contents of those buffers. Presumably, if the buffer is off-heap (and possibly shared with other processes), the JVM will not be able to reason about the  accesses done by others on the same memory, will not be able to perform ordering-eliminating optimizations that depend on such reasoning for e.g. volatile access, and will thereby conservatively enforce it's end of those ordered operations (like volatile gets and sets, in e.g. prevVal = avh.getAndSet(buffer, index, newVal); ). In addition, varhandles include ordered operations (e.g. prevVal = avh.getAndSetAcquire(buffer, index, newVal); or val = avh. getAcquire(buffer, index); ) that explicitly impose specific ordering requirements against other loads and stores.

[John Hening]

Gil,

thanks for letting me know about VarHandles. I will check it.

I see that you wrote:

since nothing is actually defined for Unsafe

What does it mean exactly that nothing is defined? From what I know Unsafe is being applying in specific solutions.

What is a purpose of a function, for example Unsafe.putLongVolatile, since nothing is defined? Perhpaps, just remember that it is unsafe ;).

Anyway, could someone try to response to my second issue, annotated by II in the first post? It is a black hole in my head.

Especially, I have a sense that my considerations are wild and potentially incorrect.

[Gil Tene]

On Wednesday, June 20, 2018 at 2:26:12 PM UTC-7, John Hening wrote:

Gil,

thanks for letting me know about VarHandles. I will check it.

I see that you wrote:

since nothing is actually defined for Unsafe

What does it mean exactly that nothing is defined? From what I know Unsafe is being applying in specific solutions.

What is a purpose of a function, for example Unsafe.putLongVolatile, since nothing is defined? Perhpaps, just remember that it is unsafe ;).

Since many people use Unsafe in "practical" ways, it is tempting to talk about definitions of behavior for it. But Usafe is a JDK-internal implementation detail, and not part of Java SE or any specification. Since the use of Unsafe is commonplace inside the JDK classes themselves, and JDK class implementations tend to be shated across JDK implementations, most JDKs tend to present "similar" internal behavior for Unsafe, but this similarity is mostly achieved by mimicking implementation, rather than by following some specification or definition. 

Being JDK-internal, literally nothing is actually defined about the behavior of Unsafe calls made from outside of the JDK's own trusted code, and about the behavior (or even the Class signature) that can be expected to exist in the next version or update. JDK classes themselves do not need to worry about such things because they ship with a specific implementation of the Unsafe. You can also think of it as "Unsafe in FooJDK version w.x.y.z is only 'defined' to do what the specific source code in that w.x.y.z version of FooJDK says it does."

If Unsafe.putLongVolatile changed its behavior in some implementation of Java 12 such that when it was called from non-JDK-internal code on odd days of even months it did non-volatile puts at an address that is 224 bytes away from where it seems to be told to store things, and on Feb 29 on leap years it also overwrote ~/.ssh/id_rsa with "0xcafebabee was here", it would be acting well within its defined behavior. It is likely to upset people if that happened, but it would be because of not following "convention" or "tradition". Not because of some non-existent definition was not adhered to.

The is certainly some common wisdom about what Unsafe methods seem to do, and much of that wisdom is anchored in actual reading of the internal implementation code in actual current JDK implementation versions, but that all that wisdom *can* evaporate at any moment with the next JDK version or update.

Anyway, could someone try to response to my second issue, annotated by II in the first post? It is a black hole in my head.

Especially, I have a sense that my considerations are wild and potentially incorrect.

For your case II: Intra-thread consistency is trivially guaranteed by the JVM's compilers when calling native code: they simply avoid reordering loads and stores across calls to opaque methods (methods for which it was unable to reason about limitations on possible memory accesses). Avoiding such reordering guarantees the thread will experience program-order.

Note that you should not reverse this and start relying on assumptions that the compiler will not reorder across native calls. If a future compiler develops the ability to reason about memory operations in the native call (e.g. if the native call is a leaf method and is described as LLVM IR), it could start reordering things across such calls. For example, if code analysis proved that the function being called does not make other calls, has no memory side effects, and does not include any semantic memory ordering operations, the compiler could e.g. safely reorder non-volatile java loads that [in the program] occur after the call such that they execute before the call, and could e.g. safely hoist such loads out of loops that make such calls.

[Ben Evans]

On Wed, Jun 20, 2018 at 4:55 PM, Gil Tene <g...@azul.com> wrote:
> Putting Unsafe aside (since nothing is actually defined for Unsafe), you can
> use DirectByteBuffer to properly abstract off heap memory (including off
> heap memory shared with other processes) to your Java code. You can then use
> VarHandles (e.g. via avh =
> MethodHandles.byteBufferViewVarHandle(long[].class, ByteOrder.LITTLE_ENDIAN)
> ) to gain well defined order-enforcing access to the contents of those
> buffers.

It might also be worth taking a look at Peter Lawrey's Chronicle-Bytes
library (https://github.com/OpenHFT/Chronicle-Bytes) as a possible
alternative to DBB.

Thanks,

Ben

[John Hening]

Gil, thanks! :)

Since many people use Unsafe in "practical" ways, it is tempting to talk about definitions of behavior for it. But Usafe is a JDK-internal implementation detail, and not part of Java SE or any specification. Since the use of Unsafe is commonplace inside the JDK classes themselves, and JDK class implementations tend to be shated across JDK implementations, most JDKs tend to present "similar" internal behavior for Unsafe, but this similarity is mostly achieved by mimicking implementation, rather than by following some specification or definition. 

Being JDK-internal, literally nothing is actually defined about the behavior of Unsafe calls made from outside of the JDK's own trusted code, and about the behavior (or even the Class signature) that can be expected to exist in the next version or update. JDK classes themselves do not need to worry about such things because they ship with a specific implementation of the Unsafe. You can also think of it as "Unsafe in FooJDK version w.x.y.z is only 'defined' to do what the specific source code in that w.x.y.z version of FooJDK says it does."

If Unsafe.putLongVolatile changed its behavior in some implementation of Java 12 such that when it was called from non-JDK-internal code on odd days of even months it did non-volatile puts at an address that is 224 bytes away from where it seems to be told to store things, and on Feb 29 on leap years it also overwrote ~/.ssh/id_rsa with "0xcafebabee was here", it would be acting well within its defined behavior. It is likely to upset people if that happened, but it would be because of not following "convention" or "tradition". Not because of some non-existent definition was not adhered to.

The is certainly some common wisdom about what Unsafe methods seem to do, and much of that wisdom is anchored in actual reading of the internal implementation code in actual current JDK implementation versions, but that all that wisdom *can* evaporate at any moment with the next JDK version or update.

 

So, to use a pair Unsafe.putLongVolatile and loadLongVolatile I have to read a documentation of specific implementation of Java (Unsafe), especially to read implementation of that, and then, possibly, use it. But, I need to remember that after an update it is possible that implementation was changed. Yes?

 

For your case II: Intra-thread consistency is trivially guaranteed by the JVM's compilers when calling native code: they simply avoid reordering loads and stores across calls to opaque methods (methods for which it was unable to reason about limitations on possible memory accesses). Avoiding such reordering guarantees the thread will experience program-order.

Ok, so for an architecture with a weak model of memory every native call (we assume that JVM is unable to reason about memory operation in the native code) should be surronded with memory barriers to prevent hardware before reordering because native code is executed directly by CPU and JVM cannot reason on it. This is why I touch low-level implementation issues like memory barriers: JVM has to deal with that.

[Gil Tene]

No. Nothing needs to be done for intra-thread consistency on any sane CPU I know of. Even weak memory model CPUs maintain in-thread program order. Their weak memory order only applies to what other to interactions with other threads.

 

[John Hening]

By weak memory model I meant a very weak, I didn't say that. So, I meant a model that doesn't ensures about data dependency, for example. But, it is not sane CPU ;). I just try to prove myself that JVM must be careful when a native funcion was called because of the fact it was executed directly on CPU.

[Gil Tene]

On Thursday, June 21, 2018 at 11:37:45 AM UTC-7, John Hening wrote:

By weak memory model I meant a very weak, I didn't say that. So, I meant a model that doesn't ensures about data dependency, for example. But, it is not sane CPU ;). I just try to prove myself that JVM must be careful when a native funcion was called because of the fact it was executed directly on CPU.

There is nothing special about the native function for that. A hypothetical CPU that does not maintain logical order between instructions executing in the thread, as seen by that thread, would present just as much of a problem for the java code itself.

[John Hening]

Gil, thanks :)

you obviously right.

[Remi Forax]

De: "Gil Tene" <g...@azul.com>
À: "mechanical-sympathy" <mechanica...@googlegroups.com>
Envoyé: Jeudi 21 Juin 2018 07:12:40
Objet: Re: Sharing natively allocated memory between Java program and JNI calls.

On Wednesday, June 20, 2018 at 2:26:12 PM UTC-7, John Hening wrote:

Gil,

thanks for letting me know about VarHandles. I will check it.

I see that you wrote:

since nothing is actually defined for Unsafe

What does it mean exactly that nothing is defined? From what I know Unsafe is being applying in specific solutions.

What is a purpose of a function, for example Unsafe.putLongVolatile, since nothing is defined? Perhpaps, just remember that it is unsafe ;).

Since many people use Unsafe in "practical" ways, it is tempting to talk about definitions of behavior for it. But Usafe is a JDK-internal implementation detail, and not part of Java SE or any specification. Since the use of Unsafe is commonplace inside the JDK classes themselves, and JDK class implementations tend to be shated across JDK implementations, most JDKs tend to present "similar" internal behavior for Unsafe, but this similarity is mostly achieved by mimicking implementation, rather than by following some specification or definition. 

Being JDK-internal, literally nothing is actually defined about the behavior of Unsafe calls made from outside of the JDK's own trusted code, and about the behavior (or even the Class signature) that can be expected to exist in the next version or update. JDK classes themselves do not need to worry about such things because they ship with a specific implementation of the Unsafe. You can also think of it as "Unsafe in FooJDK version w.x.y.z is only 'defined' to do what the specific source code in that w.x.y.z version of FooJDK says it does."

If Unsafe.putLongVolatile changed its behavior in some implementation of Java 12 such that when it was called from non-JDK-internal code on odd days of even months it did non-volatile puts at an address that is 224 bytes away from where it seems to be told to store things, and on Feb 29 on leap years it also overwrote ~/.ssh/id_rsa with "0xcafebabee was here", it would be acting well within its defined behavior. It is likely to upset people if that happened, but it would be because of not following "convention" or "tradition". Not because of some non-existent definition was not adhered to.

The is certainly some common wisdom about what Unsafe methods seem to do, and much of that wisdom is anchored in actual reading of the internal implementation code in actual current JDK implementation versions, but that all that wisdom *can* evaporate at any moment with the next JDK version or update.

I agree with everything Gil said and i would add three things:

- first, test again if you win something by using unsafe, unsafe doesn't work well with the auto-vectorization of loops by example, so unsafe was maybe faster in the past, but this is maybe no longer true.

- second, since jdk9 there are now two Unsafes, sun.misc.Unsafe and jdk.internal.misc.Unsafe, the first one is the one which is still accessible and that will die at some point, the second one is the internal one and you can not access it if you are not the JDK (in fact it's more restrictive than that). So it's a good idea to see if you can replace your usage of sun.misc.Unsafe with the VarHandle API which is a public API.

- third, be prepared to do more works in the future if you are using Unsafe.getObject/Unsafe.putObject because those methods doesn't work well with value types (as we (the valhalla team) recently discovered).

[...]

cheers,

Rémi