Why does EHCache use so much memory on a simple small cache

[su...@rdfined.dk]

1. What version of Ehcache you are currently using;
2. Paste the configuration for the Cache/CacheManager you have an issue with;
3. Add any name and version of other library or framework you use Ehcache with (e.g. Hibernate);
4. Providing JDK and OS versions maybe useful as well.

Hi there

1) 2.8.4

2) <ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd" maxBytesLocalHeap="100M">

<diskStore path="ehcache"/>

<cache name="idf" eternal="false">

<persistence strategy="localTempSwap"/>

</cache>

<cache name="baseforms" eternal="false">

<persistence strategy="localTempSwap"/>

</cache>

<cache name="document-frequency-k90" eternal="false">

<persistence strategy="localTempSwap"/>

</cache>

</ehcache>

3) none

4) JDK 8, Mac OS

I'm using EHCache for caching small list objects containing only a few entries. I'm very puzzled by the memory usage of the EHCache java process. After adding af few hundred items, the process is using more than 1 GB of memory and just keeps growing. I have ruled out memory leaks from my own code, and removing the calls to the put-operation eliminates the problem.

Is this extreme memory usage to be expected or am I missing something?

Regards,

Rune

[Fabien Sanglier]

As per platform support, seems like Java 8 is not supported yet: https://confluence.terracotta.org/display/release/BigMemory+4.1,+Ehcache+2.8+Platform+Support

Can you check if you get same behavior with 1.7 JDK? 

Also, I notice you don't specify any eviction timings (timeToLive, timeToIdle)...is that what you want?

Fabien

[Rune Stilling]

Hi Fabian

I’ll test this with my current example, but I have observed a similar behavior in other setups.

Would timings affect seriously memory usage in my case?

/Rune

--
You received this message because you are subscribed to a topic in the Google Groups "ehcache-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ehcache-users/SelySlrLp18/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ehcache-user...@googlegroups.com.
To post to this group, send email to ehcach...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/CAOEjUMig3R1qrWGSbN8K_MS34bLCxfF34YESGxt%3Dm33QyO_0PA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Fabien Sanglier]

Really, since you specify maxBytesLocalHeap="100M" at the cachemanager level, your used heap usage for all caches should not really be much more than 100MB...

But how deep (object graph) are your objects? (+ size of typical objects?) 

1 possibility I'm thinking of is the "object sizing" library...since you size by Bytes -- maxBytesLocalHeap param -- there is a need to size each objects when they're added... Maybe that sizing is not accurately calculating the object size with your current use-case etc...(possibly related to JDK8)

2 quick tests I would do (in addition to trying with JDK 7 for sure)

 - remove the maxBytesLocalHeap at the top, and put it in each caches instead with the right value (eg. 10MB for one, 5MB for the other, etc...)

 - remove the maxBytesLocalHeap at the top, and use a sizing "by count" mechanism on each cache (using "maxEntriesLocalHeap" param). 

In both case, see if that stabilize your memory usage applied to ehcache.

Thanks,

Fabien

[Rune Stilling]

Hi Fabian

Thanks for your suggestions.

First of all a clarification - When I was talking about memory usage I talked about the memory usage shown to me by the operating system (Mac OS or Windows). I now realize that this memory usage of course includes off-heap storage. I’ll look for a tool that can show me both heap and off-heap memory usage, too get a better understanding of the allocations made by EHCache.

Also it seems that there’s no way to control the off-heap memory usage in the open-source version of EHCache, so it just uses what it sees fit I guess. Correct me if I’m wrong.

Secondly I changed the objects I stored in the cache from List-objects to just plain objects. Now I cannot reproduce the run-away memory usage.

Thirdly - Changing between using Java 7 and 8 did indeed make a difference on general memory usage. Using Java 7 made EHCache use considerably less memory overall.

Thanks,

Rune

--
You received this message because you are subscribed to a topic in the Google Groups "ehcache-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ehcache-users/SelySlrLp18/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ehcache-user...@googlegroups.com.
To post to this group, send email to ehcach...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/CAOEjUMh%3DrB3o40A%2BYDagdjPHAD%2BVv_odixOoYkf874Lu4rnZng%40mail.gmail.com.

[Rune Stilling]

I have now run VisualVM to see what’s going on. I’m still a bit puzzled. I’m using the following setting at the cache manager level: maxBytesLocalHeap=“10M”, and this is the result running the application (adding elements to the cache):

At no time EHCache uses only 10M of memory, most of the time it actually uses a lot more. Is this normal and should those limits be taken with a grain of salt?

/Rune

To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/94378FF6-805D-4500-BD65-DE733D86CE93%40rdfined.dk.

[Rune Stilling]

After doing some more testing it seems clear to me that viewing the heap memory usage of EHCache in VisualVM is not directly comparable to the limits set in the config file. This is due to garbage collection I guess. Even though EHCache frees objects so that it never uses more than the specified amount of heap memory, the freed objects will still be shown as allocated heap memory until garbage collection comes around. This is more clearly seen in this screen shot:

Here I have Xmx64M and maxBytesLocalHeap=“10M”.

So assuming that this is correct I really have only one remaining question:

Also it seems that there’s no way to control the off-heap memory usage in the open-source version of EHCache, so it just uses what it sees fit I guess.

Is that so?

/Rune

Den 14/10/2014 kl. 11.26 skrev Rune Stilling <su...@rdfined.dk>:

I have now run VisualVM to see what’s going on. I’m still a bit puzzled. I’m using the following setting at the cache manager level: maxBytesLocalHeap=“10M”, and this is the result running the application (adding elements to the cache):

<Skærmbillede 2014-10-14 kl. 11.24.14.png>

To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/9BC2316D-DEDB-4F73-A2B3-F227EF2844E1%40rdfined.dk.

[Louis Jacomet]

Hi Rune,

Ehcache open source does not use off-heap memory at all.

For a standalone installation that has an off-heap capability, you need the commercial version of Ehcache, BigMemory Go (http://terracotta.org/products/bigmemorygo)

Regards,

Louis

--
You received this message because you are subscribed to the Google Groups "ehcache-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ehcache-user...@googlegroups.com.

To post to this group, send email to ehcach...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/02C687D0-B62C-41D7-832A-EEB9FE951CE0%40rdfined.dk.

[Alex Snaps]

What's being said is true for the 2.x line, but as of Ehcache3, we've open sourced our offheap implementation:

See http://ehcache.github.io/ for the actual announcement & how to get started with the new API

To unsubscribe from this group and all its topics, send an email to ehcache-users+unsubscribe@googlegroups.com.

To post to this group, send email to ehcach...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/CAOEjUMh%3DrB3o40A%2BYDagdjPHAD%2BVv_odixOoYkf874Lu4rnZng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "ehcache-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ehcache-users/SelySlrLp18/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ehcache-users+unsubscribe@googlegroups.com.

To post to this group, send email to ehcach...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/94378FF6-805D-4500-BD65-DE733D86CE93%40rdfined.dk.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "ehcache-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ehcache-users/SelySlrLp18/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ehcache-users+unsubscribe@googlegroups.com.

To post to this group, send email to ehcach...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/9BC2316D-DEDB-4F73-A2B3-F227EF2844E1%40rdfined.dk.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ehcache-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ehcache-users+unsubscribe@googlegroups.com.

To post to this group, send email to ehcach...@googlegroups.com.