io.netty.buffer.PoolChunk consuming a lot of memory and is a suspected leak in MAT

[Rishabh Handa]

Hello,

I need some help and inputs here, 

We are using JMS which behind the scenes uses AMQPProvider which in turns initializes 

PooledByteBufAllocater (see attached screenshot)

I've gone through the issue: Increased memory footprint in 4.1.43.Final · Issue #9768 · netty/netty (github.com)

According to my understanding the number of chunks allocated depends upon the DEFAULT_NUM_HEAP_ARENA

For an app with 1GB of heap memory - DEFAULT_NUM_HEAP_ARENA = 10 (16MB chunk - by default)

So, maximin no of chunks that it can allocate is 10

So, if only create connection to 2 Queues - the total no of chunks allocated will be 2

But if I create connections to 14 Queues, the no of chunks allocated will be 10

So, from my understanding, whatever is less from the above 2 conditions - decides the number of chunks

So, my question is :

If I set -Dio.netty.allocator.numHeapArenas=0, then the number of chunks allocated becomes 0
So, are there any consequences of doing that??

One thing to note here is, our processes and usage of AMQP are asynchronous (send and receiving messages from queue)

What will happen if I reduce the size of chunk -Dio.netty.allocator.maxOrder=9
and the message received from queue is more than it??
What I mean to ask is, do we use these chunks while reading message from queue 

Will the performance of my application will be affected by setting -Dio.netty.allocator.numHeapArenas=0 ??

Request you kindly help me out here.

Regards,

Rishabh

[Trustin Lee]

Hi Rishabh,

If I set -Dio.netty.allocator.numHeapArenas=0, then the number of chunks allocated becomes 0
So, are there any consequences of doing that??

One thing to note here is, our processes and usage of AMQP are asynchronous (send and receiving messages from queue)

It disables heap buffer pooling completely, which means buffer allocation and deallocation will be managed by JVM's garbage collector entirely. This may be fine depending on your use case. I'd recommend measuring performance to make a final decision.

What will happen if I reduce the size of chunk -Dio.netty.allocator.maxOrder=9
and the message received from queue is more than it??
What I mean to ask is, do we use these chunks while reading message from queue 

It will decrease the maximum capacity of pooled buffers. This means, if you allocate a buffer with the capacity greater than the limit, it will be allocated as an unpooled buffer. Again, this can have performance implication, so please measure performance to decide to choose a smaller maxOrder value.

Will the performance of my application will be affected by setting -Dio.netty.allocator.numHeapArenas=0 ??

Possibly, but the impact might be negligible depending on your performance requirements, so you need to measure to come to a conclusion.

HTH,

T 

 

--
You received this message because you are subscribed to the Google Groups "Netty discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/e6112133-4a56-447b-9b07-3795f030c107n%40googlegroups.com.