Re: [vertx:22898] Re: clearing a Buffer

[Tim Fox]

What are you trying to achieve?

On 12/01/15 10:29, Matteo Spennati wrote:

I am allocating a netty ByteBuf which offers the clear() method.

Such buffer needs to be wrapped with new Buffer() in order to be used.

Do you think this is the right approach? I didn't find anything in the documentation.

On Thursday, January 8, 2015 at 4:23:44 PM UTC+1, Matteo Spennati wrote:

Hello,

Can I clear a Buffer instance to reuse it? if yes, how?

Matteo.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alexander Lehmann]

You can create a new Buffer instance to start with an empty buffer, usually you will not use the same buffer for more than one operation.



On Monday, January 12, 2015 at 1:34:05 PM UTC+1, Matteo Spennati wrote:

I want to recycle the Buffer after each time I send a packet otherwise it would keep expanding forever isn't it?

[Tim Fox]

On 13/01/15 07:40, Matteo Spennati wrote:

I would like to clear the indexes and reuse it like I would do in Netty

If you give a Netty buffer to socket.write and clear it immediately afterwards, then there is a good chance the data won't actually get written to the wire, this is because the actual write to the wire is asynchronous and might happen after you have cleared it, so you can't do what you propose in Netty anyway.

So.. just create new buffers as Alex suggests :)

or XNIO, without producing too garbage for the collector.

This is a draft of what I do at the moment, send buffer is a Netty direct buffer.

while(true) {

    sendBuffer.clear();

    sendBuffer.writeBytes(...);

    sock.write(new Buffer(sendBuffer));           

[Alexander Lehmann]

This is a bit similar to thread safety, it will probably work for some cases but not always, e.g. when the rate of connections increases or something.



On Tuesday, January 13, 2015 at 3:20:56 PM UTC+1, Matteo Spennati wrote:

Ok but then my code is working by chance..., I clear such before I reuse it. Then I can't assume that after the write my data is "copied out"... :-\

[Sumit Srivastava]

I encountered the same issue. Tried out PooledByteBufAllocator which in turn ended up with outOfDirectMemoryError when using direct memory and GC overhead limit when using heap memory. How should I proceed.

On Wednesday, 14 January 2015 12:50:52 UTC+5:30, Matteo Spennati wrote:

Ok i changed my code and rather use the netty buffer pool, then wrap such buffer with vertx Buffer.

[Nat]

PooledByteBufAllocator is not safe to be used to wrap with Buffer.buffer(ByteBuf). Buffer.buffer() call circumvents the reference counting on pooled buffer. This creates a big issue with PooledByteBufAllocator as it will never be a good way to make sure the buffer is returned to the pool safely. 

[Julien Viet]

yes Vert.x default Buffer assumes they are unpooled.

you can still come with your own Buffer implementation though.

at least we should make sure that when a Buffer is used in a Vert.x method (for example EventBus.send()), the underlying netty Buffer get released, so one could provide its own Buffer impl that delegates to pooled buffers.

[Sumit Srivastava]

Here's the complete use case,

1] Incoming requests of the order of 100k/sec. Cores: 5

2] Processing involves resolution of 20 parameters and retrieving values for each of them from a hashmap. 

3] Responding with these values amounting to the order of ~4000 bytes

If I keep creating ByteBuf for each request, the GC increases dramatically.

Moved on to UnpooledByteBufAllocator. I just need to make sure the buffer allocated is release as soon as possible. Will it be fine if I release it in the bodyEndHandler() or shall I go for an entirely different approach?