Why heap memory keeps on increasing ? even though

[Amit Aggarwal]

Hello All,

NOTE : I am NOT working on trillionths of node. I am working with few hundreds of node.

 

Its getting more troubling. Scenario is like this.

1. I have developed my plugin in java for neo4j . It uses cypher and other java API to perform operation on neo4j.

2. My webserver continously ( 10 sec delay between each call ) call plugin API which in turn perform some logic ( on nodes and relationship ) to decide whether there is something new for user in database  or not ( it is kind of live updates for user ) .

3. What I observe that neo4j heap size keeps on increasing EVEN THOUGH IT IS RUNNING SAME PIECE OF CODE (and there is no creation of any new node or relationship ) again and again. It should be stable after some time but it keeps on increasing.

I tried all things and have tired now , please help me out ..

Here is my neo4j-wrapper.conf file 

-------------------------------------------------------------------------

# the shipped configuration contains only a read only role called 'monitor' with password 'Neo4j'.

# For more details, see: http://download.oracle.com/javase/6/docs/technotes/guides/management/agent.html

# On Unix based systems the jmx.password file needs to be owned by the user that will run the server,

# and have permissions set to 0600.

# For details on setting these file permissions on Windows see:

#     http://download.oracle.com/javase/1.5.0/docs/guide/management/security-windows.html

#wrapper.java.additional=-Dcom.sun.management.jmxremote.port=3637

#wrapper.java.additional=-Dcom.sun.management.jmxremote.authenticate=true

#wrapper.java.additional=-Dcom.sun.management.jmxremote.ssl=false

#wrapper.java.additional=-Dcom.sun.management.jmxremote.password.file=conf/jmx.password

#wrapper.java.additional=-Dcom.sun.management.jmxremote.access.file=conf/jmx.access

# Some systems cannot discover host name automatically, and need this line configured:

#wrapper.java.additional=-Djava.rmi.server.hostname=$THE_NEO4J_SERVER_HOSTNAME

# Uncomment the following lines to enable garbage collection logging

wrapper.java.additional=-Xloggc:data/log/neo4j-gc.log

wrapper.java.additional=-XX:+PrintGCDetails

wrapper.java.additional=-XX:+PrintGCDateStamps

wrapper.java.additional=-XX:+PrintGCApplicationStoppedTime

wrapper.java.additional=-XX:+PrintPromotionFailure

#wrapper.java.additional=-XX:+PrintTenuringDistribution

# Uncomment the following lines to enable JVM startup diagnostics

#wrapper.java.additional=-XX:+PrintFlagsFinal

#wrapper.java.additional=-XX:+PrintFlagsInitial

# Java Heap Size: by default the Java heap size is dynamically

# calculated based on available system resources.

# Uncomment these lines to set specific initial and maximum

# heap size in MB.

wrapper.java.initmemory=64

wrapper.java.maxmemory=400                 -------------------------------------------------------> I think this is enough for few hundred nodes 

#********************************************************************

# Wrapper settings

#********************************************************************

# path is relative to the bin dir

wrapper.pidfile=../data/neo4j-server.pid

#********************************************************************

# Wrapper Windows NT/2000/XP Service Properties

#********************************************************************

# WARNING - Do not modify any of these properties when an application

#  using this configuration file has been installed as a service.

#  Please uninstall the service before modifying this section.  The

#  service can then be reinstalled.

# Name of the service

wrapper.name=neo4j

[Amit Aggarwal]

I just read whenever you do any traverse or execution on graph it must be exhausted ( and I am not doing this thing in my plugin code , I left iterator of node as it is ) 

So is it possible that I am getting increase in heap size because many objects keeps hanging there in JVM ?

Thanks 

Amit Aggarwal 

[Michael Hunger]

Perhaps you can share your full code? Then it wold be easier to help.

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Amit Aggarwal]

Hello Michael I can share my code but can I send you at you email id ? Just for privacy purpose ..

You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/GSMfnL0Tq2U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

[Amit Aggarwal]

Hello Michael,

I have shared my code with you on you emailId.

Second thing, I am trying myself and analyze heap dump using mat ( eclipse memory analyzer) then I found most of memory (96%) is consumed by java.utils.HasMap$Entry class  and there are more than 64000 such objects .. 

I am not able to attach heapdump becuase it is of 500MB around ..

Thanks 

[Michael Hunger]

Currently traveling will try to have a look when I find a moment

Sent from mobile device

[Amit Aggarwal]

Hello Michael,

No problem I am trying here .. and happy journey .

Please reply to my problem whenever you get time .

You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/GSMfnL0Tq2U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

[Amit Aggarwal]

Hello Michael,

I think I have resolved my problem of continous heap increment ..

It is because I was creating ExecutionEngine everytime for new query

Second thing , I was not exhausting ResouceIterator .

Point first of executionengine was main point in resolving my problem .

@Michael , can you please abandon my source code that I sent to you on your neo id ?
Thanks a lot Michael again for your support ...atleast moral support :p

On 23-Apr-2014 11:41 AM, "Michael Hunger" <michael...@neotechnology.com> wrote:

You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/GSMfnL0Tq2U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

[Michael Hunger]

Yep definitely ;)

These were two things I saw when glancing at your code.

M

Sent from mobile device

[Amit Aggarwal]

Thanks Michael