Netty 4.x supports pooled/unpooled and Unsafe -enabled byte buffers.

[Kevin Burton]

Hey guys.

Netty 4.x is about to go out of RC and cut an official 4.x release.

http://netty.io

Last year there was a large discussion on the list about unsafe and pooled buffers.

I just went and looked through the code and it looks interesting.

Basically they've cleaned up the 'cleaner' issues with buffers so that they can be release and it looks like, if unsafe is enabled, that all the primitive operations will use getInt, etc.

I don't have any benchmark numbers yet but It looks like good work.

My plan is to port Peregrine to it and see what the before/after looks like.

I'm performance testing our merge sort implementation, which uses direct buffers, and it looks like Java is 2.5x faster on the heap vs using Netty 3.x direct buffers.

[Gil Tene]

I see at least one potential (and very serious) bug:

The constructor UnpooledUnsafeDirectByteBuf(ByteBufAllocator alloc, ByteBuffer initialBuffer, int maxCapacity) takes a potentially non-direct-buffer as input, doesn't verify that it's actually a direct buffer, extracts it's address using PlatformDependent.directBufferAddress(buffer), which eventually gets the address via a getLong(buffer, ADDRESS_FIELD_OFFSET), and then later uses that address for direct addressing via unsafe.GetX() methods. The ones intended to never be used on heap objects.

The Unpooled class has a public static ByteBuf wrappedBuffer(ByteBuffer buffer) method that can accept (from end-user code) a non-direct byte buffer, and (if unsafe is supported) create an UnpooledUnsafeDirectByteBuf around it without ever making sure it's really a direct buffer. So now we have unsafe operating on something that is either a random address or a constant heap address (which is just as random over time).

I *think* that this combination makes things go Boom. Unfortunately it will probably be a very latent boom. Not one of those nice SEGV things. Also may not hit you unless timing is just right. More likely to result in psot-QA heap corruption and fun debugging trips with heap dumps coming in from the field.

Is Netty making my point for me in our separate Unsafe related threads?

-- Gil.

[Kirk Pepperdine]

>
> Is Netty making my point for me in our separate Unsafe related threads?

You are evil!!

-- Kirk

[Kevin Burton]

You might be making the point that in Open Source all bugs (even Unsafe ones) are shallow and quickly fixed ;)

It's an RC after all :)

[Jeff Hodges]

Made an issue on the netty repo about this: https://github.com/netty/netty/issues/1485

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Gil Tene]

Thanks. Would have posted it myself but running around jetlagged in Paris right now.

On Tuesday, June 25, 2013 6:30:13 AM UTC+2, Jeff Hodges wrote:

Made an issue on the netty repo about this: https://github.com/netty/netty/issues/1485

On Mon, Jun 24, 2013 at 9:18 PM, Kevin Burton <burto...@gmail.com> wrote:

You might be making the point that in Open Source all bugs (even Unsafe ones) are shallow and quickly fixed ;)

It's an RC after all :)

On Monday, June 24, 2013 9:14:35 PM UTC-7, Kirk Pepperdine wrote:

>
> Is Netty making my point for me in our separate Unsafe related threads?

You are evil!!

-- Kirk

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

["이희승 (Trustin Lee)"]

We do check:

https://github.com/netty/netty/blob/master/buffer/src/main/java/io/netty/buffer/UnpooledUnsafeDirectByteBuf.java#L88

--

You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

-- 
https://twitter.com/trustin
https://twitter.com/trustin_ko
https://twitter.com/netty_project