Hazelcast Imap leaking memory v3.10

[parker...@gmail.com]

We have are using Imap on over servers. We are only storing a unique string key and a boolean value

Our data is not more that 1G but over time (in couple of days) we our seeing the heap grow almost to 5G on our 8G RAM server

Upon looking at the heap dump, you can clearly see hazelcast taking up all that space (see attachement). It doesn't cleanup even after GC 

We also have a ttl of 4 hours on those keys 

[Alparslan Avcı]

Hello,

From the screenshot you shared, the heap looks occupied by the data stored on maps.  

Can you please share the following?

1. All your maps' entry counts 

2. All your maps' key and value classes if possible

3. Your cluster configuration

Regards,

Alparslan

[Peter Veentjer]

How many items do you expect to be there in the map? It will be very helpful to see the actual number of map entries.

If you just keep adding items to a map, the memory will keep increase unless the items are removed (e.g. eviction, manual removal etc).

Why do you expect that Hazelcast will remove items? Also the entries in a HashMap will not be gc's because the HashMap has a strong reference to the content. The same goes for HZ (unless configured otherwise).

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+unsubscribe@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/ea392410-303b-41b4-a1ec-f961efbd3a86%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[parker...@gmail.com]

Hi Alparslan,

1) Map size is not more than 300 (However, we are expiring keys and limiting heap usage as you can see in hazelcast.xml)

2) Attached Push.java and HazelcastCacheManager.java .   NOTE: We are only using PUSH_IDENTIFIER_CACHE from the hazelcast.xml

3) Cluster configuration can be found in the hazelcast.xml attached.

We update the map with new keys every time. However, old keys should always be expired and cleaned up because of ttl. When we get imap.size() -> it's always fluctuates around 300-400 at any given time.

Thanks for looking into it

[parker...@gmail.com]

We expect to have not more than 300-400 keys at a given time. 

Here's how we are using it. We have this api server that handles request and for each request, we put a string key and a value in IMap. 

We have ttl along with eviction policy set on the servers (see hazelcast.xml)

Seems like the keys are expiring (verified using size() )  but the data in the partitions never gets cleaned up (was verified using heapdump)

On Sunday, July 29, 2018 at 11:06:23 AM UTC-4, peter veentjer wrote:

How many items do you expect to be there in the map? It will be very helpful to see the actual number of map entries.

If you just keep adding items to a map, the memory will keep increase unless the items are removed (e.g. eviction, manual removal etc).

Why do you expect that Hazelcast will remove items? Also the entries in a HashMap will not be gc's because the HashMap has a strong reference to the content. The same goes for HZ (unless configured otherwise).

On Thu, Jul 26, 2018 at 8:20 PM, Alparslan Avcı <alpa...@hazelcast.com> wrote:

Hello,

From the screenshot you shared, the heap looks occupied by the data stored on maps.  

Can you please share the following?

1. All your maps' entry counts 

2. All your maps' key and value classes if possible

3. Your cluster configuration

Regards,

Alparslan

On Thursday, July 26, 2018 at 5:22:09 PM UTC+1, parker...@gmail.com wrote:

We have are using Imap on over servers. We are only storing a unique string key and a boolean value

Our data is not more that 1G but over time (in couple of days) we our seeing the heap grow almost to 5G on our 8G RAM server

Upon looking at the heap dump, you can clearly see hazelcast taking up all that space (see attachement). It doesn't cleanup even after GC 

We also have a ttl of 4 hours on those keys 

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.

[Parth S]

1. All your maps' entry counts  -> the count is never more than 300

2. All your maps' key and value classes if possible --> see attached

3. Your cluster configuration -> see hazelcast.xml file

Here's how we are using it. We have this api server that handles request and for each request, we put a string key and a value in IMap. 

We have ttl along with eviction policy set on the servers 

Seems like the keys are expiring (verified using size() )  but the data in the partitions never gets cleaned up (was verified using heapdump)

We have Push.java that handles the api request, we use HazelcastCacheManager.java class to get our map config from hazelcast.xml

Note: we are using PUSH_IDENTIFIER_CACHE map only 

Regards,

Parth

--
You received this message because you are subscribed to a topic in the Google Groups "Hazelcast" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hazelcast/VxgVA6LtQlI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hazelcast+unsubscribe@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.