On-Heap vs Off-Heap

[Albert Kam]

Hi guys !

I want to store objects in the memory, so i'm wondering what are the pros and cons for on-heap and off-heap ?

My current conclusions after doing some research are:

- On Heap is faster, but will experience GC pauses

- Off Heap is slower, but no GC pauses

Also another concern would be the memory used. 

From reading https://github.com/omry/banana/wiki/Memory-Management, it seems that off-heap doesnt have the heap extra footprints like 8-12 bytes for each object, which means off-heap is generally more space efficient ?

If this holds true, 8 bytes x 100 million objects will eats up 800MB of heap.

[Niall Gallagher]

Hi Albert,

Off-heap persistence is a bit like storing objects in a database outside the Java heap. The advantage indeed is that you can store a huge number of objects in that database, and the garbage collector won’t have to manage it. 

However, when you run queries to retrieve objects from that database, those objects are copied onto the Java heap.

If your application will frequently retrieve objects for (say) each incoming request, and then discard them when the request has been processed, then the objects which were retrieved for each request will have to be garbage collected.

On the other hand, if you used on-heap persistence, then the required objects would remain on-heap after each request was processed and can be reused as-is by subsequent requests. 

So sometimes, there can be less garbage collection with on-heap persistence than with off-heap persistence.

The increase or decrease in GC that you’ll get for off-heap persistence compared with on-heap, depends on how big the fraction of the collection is, that your queries will retrieve each time.

That said, the modern garbage collectors are pretty much all optimised for quickly garbage collecting short-lived objects. Which actually works quite well with off-heap storage. 

GC of the young generation usually does not cause any pauses.

The retrieval performance for on-heap persistence is also significantly better than for off-heap persistence. This is because object references are the killer performance feature with on-heap persistence. CQEngine’s on-heap indexes store Java object references directly in indexes. Which means results are returned with zero copying of data.

But of course, if you will store millions of objects in the collection, then ultimately you may find that the performance of on-heap persistence will decline due to GC maintenance of the collection. 

So it’s a tradeoff: your best option is to benchmark these options in your application.

When using off-heap persistence, I’d recommend to enable the index merge strategy as well.

It typically gives faster query performance AND lower GC overhead when the collection is off-heap. It hasn’t been done yet, but in a future versions of CQEngine I will probably enable this by default when the primary persistence is off-heap or disk.

Finally - regarding the size of objects when stored off-heap: yes they should be more compact so you can fit more in memory.

CQEngine uses Kryo to serialize objects for off-heap storage off-heap. You can read more about Kryo for further details.

There is actually an OffHeapPersistence.getBytesUses() method which will tell you exactly how much off-heap memory is used.

HTH!

Niall

--
-- You received this message because you are subscribed to the "cqengine-discuss" group.
http://groups.google.com/group/cqengine-discuss
---
You received this message because you are subscribed to the Google Groups "cqengine-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cqengine-discu...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Albert Kam]

Thank you for the great explanation, appreciate it !