Cassandra moving to Unsafe.
Kevin Burton
Though this thread would be interesting.
They noted the fact that allocateDirect is no fun because you can't expliclity free()
"이희승 (Trustin Lee)"
Sent from a mobile device.
ngocdaothanh
1. It only have method to put one byte into memory. A loop must be
used to put an array of bytes. Is this slow?
2. The doc about java.nio.ByteBuffer says:
The buffers returned by this method typically have somewhat higher
allocation and deallocation costs than non-direct buffers… It is
therefore recommended that direct buffers be allocated primarily for
large, long-lived buffers that are subject to the underlying system's
native I/O operations. In general it is best to allocate direct
buffers only when they yield a measureable gain in program
performance.
Does Unsafe have this limitation? I read somewhere that ByteBuffer is
implemented with Unsafe.
"이희승 (Trustin Lee)"
I have some questions about Unsafe:
1. It only have method to put one byte into memory. A loop must be
used to put an array of bytes. Is this slow?
2. The doc about java.nio.ByteBuffer says:
The buffers returned by this method typically have somewhat higher
allocation and deallocation costs than non-direct buffers… It is
therefore recommended that direct buffers be allocated primarily for
large, long-lived buffers that are subject to the underlying system's
native I/O operations. In general it is best to allocate direct
buffers only when they yield a measureable gain in program
performance.
Does Unsafe have this limitation? I read somewhere that ByteBuffer is
implemented with Unsafe.
Kevin Burton
ngocdaothanh wrote:I don't think so in the modern VM especially considering VM will aggressively optimize the calls to Unsafe, but without a benchmark, I'm not completely sure.
I have some questions about Unsafe:
1. It only have method to put one byte into memory. A loop must be
used to put an array of bytes. Is this slow?
In my experience I would say it wouldn't be as fast ... and I'd bet money on it :-( ... but I don't have benchmarks so I can't be 100% certain.
Yes. But we can pool it more flexibly with it because we can deallocate the buffer immediately to adjust the pool size. Without Unsafe, we cannot do that because VM usually deallocates too late, making things look like a leak.
Actually with my hack (which is in the same category as the Unsafe hack) you can free() a directly allocated buffer without having to wait for GC.
I'm using it in Peregrine as the GC direct buffers are insane.
I should also note that the JVM gives itself the right to do this but those of us in the worker class must wait for the JVM to do it for us.
I think this is really good evidence that lack of a free() method == broken.
Kevin
ngocdaothanh
> on it :-( ... but I don't have benchmarks so I can't be 100% certain.
I think copyFromByteArray and copyToByteArray of java.nio.Bits can be
used:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/nio/Bits.java#Bits.copyToByteArray%28long%2Cjava.lang.Object%2Clong%2Clong%29
Reading the source code of DirectByteBuffer gives a lot of
information:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/nio/DirectByteBuffer.java
ngocdaothanh
> you can free() a directly allocated buffer without having to wait for GC.
Seeing the source code of java.nio.DirectByteBuffer:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/nio/DirectByteBuffer.java
I think using reflection we can get the Cleaner instance and call
clean() whenever we want to free the memory. This saves a lot of code
because we can still use ByteBuffer without implementing ourselves
something new, based on Unsafe directly.