Object Pooling overview

[Alex P]

Hi everyone,

I'm new to this mailing list, but I've been told that there's an interest in optimising software for hardware. We've had to do some work on optimising our software with, so to say, non-traditional methods. 

Although the post has already been written quite some time ago, it's been rather difficult for me to find an audience for such topics, so the planned series of posts (summaries, headlines, notes and a manuscript) has actually ended up sitting unfinished on my machine[1], since I didn't want to spend lots of time writing up and summarising experiences that will never be read by anyone. 

The general topic is "going off-heap", although the title might be a bit misleading, since there are many things that are to be optimised without even going off the heap. You can get a general idea from slides here [2]

I've tried to summarise our experiences with Object Pooling here: http://coffeenco.de/articles/jvm_performance_part_1_object_pooling.html

I'm mostly searching for feedback and people who might be potentially interested in these subjects. Since not everyone is facing similar problems, it's also hard to find a wider audience. I've recently learned about this Mailing List, and it looks like people interested in similar subjects might also be subscribed to it. 

Also, there's another article on lockless concurrency (Atoms & Persistent Data Structures), too: it's going through proof-reading and some final touches at the moment. 

If you're interested in the subject, or would like to collaborate on making such information more accessible and (maybe) popular, please do let know.

Thank you,

Best regards, 

Alex

[1] https://github.com/ifesdjeen/lowlatencyjava/tree/master/manuscript

[2] https://speakerdeck.com/ifesdjeen/going-off-heap

[Martin Thompson]

Hi Alex,

As general feedback on your post/presentation. I'm not clear on the points you are trying to make. You first mention object pools but then talk about a mix of off-heap, lock-free techniques, and then bitwise operations. I'm not sure what you hope the audience will learn. A cynical view is it looks like you are enumerating some stuff you know without any context.

Considerations I always have in mind which I'm not sure if I just missed them or they are not in your material for these topics.

Object pooling on a modern JVM needs to be used with a lot of caution if on heap. If you want to be *totally* allocation free then pooling is often necessary. If you are mixing object pooling with allocating elsewhere in your design then you can greatly increase the cost of the GC when it happens due to retention and card scanning for example. I often say as a rule of thumb don't do object pooling if you are allocating anywhere else in a design. This has some exceptions but is a good rule of thumb.

If going off heap as an alternative to being within the Java managed heap then many more things must be considered rather than just pool. For example, alignment can, if not done correctly, at best cause performance issues and at worst generate a processor exception or give corrupt data in a concurrent environment depending on the CPU architecture.

We are all learning and we don't always get things right. I know for sure I don't! :-) My observation is many in this group love to examine ideas that are put forward with a scientific mindset, i.e. characterise the problem, present a solution as an experiment, and then analyse the data after measuring the experiment. Have you considered taking the techniques you are discussing and measuring examples of their usage compared to the alternatives?

Martin...

[Alex P]

Hi Martin,

I've removed this enumeration on Monday, while preparing for cross-post on another blog. At first I thought there'll be enough people interested in the subject to continue writing same thing as a series of blog-posts, although I've gotten very little feedback and, after talking to some people and checking general interest, decided to invest writing energy into something different for time being. I should have removed this list right away. Thanks for the hint.

As regards to heap/pooling allocations, I can say several things from our experience. We've worked on an in-house stream-processing lib, and had a great need of a little Event (wrapper / routing object). Although, since we were pushing quite a few messages through the system, we kept re-creating those Event objects. After we started pooling those Event objects, we've ended up having a fixed amount of them, which pretty much corresponded to the max throughput we had in the system. It, in fact, reduced GC pauses and allowed us to be allocation-free when routing events, since we never needed to create this additional wrapper object.

Another example of pooling that greatly helped us was using pooled byte buffers (both heap and off-heap) in Netty. Netty has a very good, simple to use and tune pool implementation. I've seen major benefits because of using it. 

It might be that use-cases for pooling are quite limited and there are many better ways to model it, but I've seen benefits myself, and can confirm that it worked very well for us. 

Also, if I were to be completely allocation-free, I wouldn't stay on JVM.

> Have you considered taking the techniques you are discussing and measuring examples of their usage compared to the alternatives?

Sure, but I've decided to leave benchmarks out of the article (also, it's been already almost 8 month since it was first published). With benchmarks it's always hard, especially when they're touching a "theoretical" subject. It'd be hard to simulate the environment that's representing an objective reality. And I couldn't publish our internal benchmarks, either: posting pictures would be fun, but wouldn't bring a lot, and posting code was out of question.

I can't say whether I'm right or wrong, but I did test and verify things out before presenting them to such a smart crowd. 

[Vitaly Davidovich]

It might be better to clarify that you can pool and allocate elsewhere as long as that elsewhere is not on fast/common path.  In other words, you can allocate elsewhere as long as those allocations don't trigger GCs.

Also, if the pooled objects are not reference dense and aren't themselves stored in fields of other objects (e.g. borrowed on-stack only), then they either don't incur any card marking penalty or very little.  Likewise, if the number of pooled instances isn't significant, GC impact can be insignificant  (assuming a GC is triggered afterall).

Generally speaking, it's impossible to be completely allocation free in anything other than trivial examples (and it's a silly goal to aim for unless this is some microcontroller/embedded space).  However, should be allocation free in common/hot paths.

sent from my phone

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vitaly Davidovich]

Alex,

I would continue writing your posts as I suspect some will find them useful, whether they voice that recognition or not.  But, you've mentioned several times that you want (and now lacking) feedback.  What type of feedback are you interested in exactly? I may be wrong, but I suspect majority of active readers on this list (at least ones that also participate in discussions) are already familiar with the material from the two articles you've posted thus far, and may not comment one way or another.  If you're gradually moving to less traveled grounds, enagement may pick up.

Anyway, just my $.02.

sent from my phone

[Martin Thompson]

Good points Vitaly. By allocation free I mean no regular allocation that will eventually cause a GC event within the window which the application must be responsive. I've seen many applications that have a sufficient large young generation that they only GC once per day.

Maybe I should switch from using allocation-free to GC event free :-)

[Alex P]

>  What type of feedback are you interested in exactly?

Your feed back is awesome, I'm very happy. I've also managed to collect a bit feedback from another post. 

I'll try to move to less traveled grounds, let's see how it works out.

Thanks a ton, very much appreciated.