Object pooling on the JVM?

[Rajiv Kurian]

What are the valid cases for object pooling on the JVM? What are the invalid use cases for object pooling?

It is generally considered a win in native code since frequent malloc/free pairs can make quite a dent in performance. Object pooling's interaction with GC seems complicated. On the one hand it generates fewer objects that need to be tracked. Conversely it also creates a bunch of long lived objects.

The Netty project is especially interesting to me since I am interested in the properties of pooled buffers for the purpose of holding network data. Netty 4.0 implemented a Jemalloc like memory allocator for it's buffers. It would be great if it's maintainers (I know you guys are on the list :-) ) could talk about the impact on some of their projects and any gotchas/lessons learnt. It would be especially enlightening to learn not only about the impact on time taken to allocate a buffer but also about impact on the rest of the application performance (with less garbage being generated). Would love to hear from other people too.

Thanks!

[Peter Lawrey]

There is a number of types of object pools. In each case the behaviour needs to be optimised for their use case so the cost of using them is faviourable compared with straightforward reallocating and the extra complexity involved.

In the case of buffers for IO you have problem that if you are going to optimise allocation, you can avoid zeroing out the buffer and perform determistic releasing which is otherwise a high proportion of the cost. Ie by using Unsafe allocate and free memory directly. You can use this with file descriptors (which has numerous portability issues) but you might find doing this (when it is an option) means you no longer need an object pool.

On 23 Dec 2013 08:46, "Rajiv Kurian" <geet...@gmail.com> wrote:

What are the valid cases for object pooling on the JVM? What are the invalid use cases for object pooling?

It is generally considered a win in native code since frequent malloc/free pairs can make quite a dent in performance. Object pooling's interaction with GC seems complicated. On the one hand it generates fewer objects that need to be tracked. Conversely it also creates a bunch of long lived objects.

The Netty project is especially interesting to me since I am interested in the properties of pooled buffers for the purpose of holding network data. Netty 4.0 implemented a Jemalloc like memory allocator for it's buffers. It would be great if it's maintainers (I know you guys are on the list :-) ) could talk about the impact on some of their projects and any gotchas/lessons learnt. It would be especially enlightening to learn not only about the impact on time taken to allocate a buffer but also about impact on the rest of the application performance (with less garbage being generated). Would love to hear from other developers too.

Thanks!

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Evan Meagher]

While probably not as in depth as what you're looking for, a post on GC improvements in Netty 4 was featured on Twitter's engineering blog: https://blog.twitter.com/2013/netty-4-at-twitter-reduced-gc-overhead

--
Evan Meagher

[Michael Barker]

Hi,

Here's a some non-comprehensive detail of my personal experiences with object pools.

I quite often use the thread-local of a single object to prevent allocation on a hot path (a particular flavour of object pool).  This works well in my opinion, caveated with the fact that we have only a small number of threads (<20) in our high-performance systems.

We have a third party library that aims to be zero-garbage and does that with a lot of object pooling.  My experience with this approach is that if you have one library that is written in that manner and the rest of the code is more idiomatic Java with lots of short lived objects, then you can end up with some nasty GC behaviour that is difficult to tune for.  If the pools are dynamically grown then you can end up with big pauses due to the promotion of the pooled objects.  Extensive object pooling will result in more data in old and the length of a new GC pause can be affected by the amount of data in old, e.g. card table scanning in CMS to find references from old to new.

So if you want to get to a point where you have no GCs during some operational window, you really need to go the whole hog as a middle ground with some objects pooled and some not will probably be worse than sticking with a solution where all objects are short lived.

Mike.

[Aysylu Greenberg]

As far as potential pitfalls: I worked on a project where the team previously used object pooling and my goal was to parallelize the computation. It resulted in rewriting the application significantly to avoid object pooling for objects that need to be handled by different workers. The problem with those object pools was that they were not thread-safe for performance.

[Peter Lawrey]

Depending on how you write them, an object pool of immutable objects doesn't need to be thread safe in the traditional sense. As long as each thread behaves correctly, that is what matters.

[Rajiv Kurian]

Thanks everyone.