[Neo4j 2.1.8] Maximum Heap limited to 60GB

[Alex]

Greetings,

I am running Neo4j2.1.8 Embedded with Gremlin2.6.0. Mostly default configurations except for the heap (-Xms -Xmx parameters modified in the gremlin.sh file).
I have about 120GB of RAM on my machine and I wanted to give it an -Xmx of about 110GB since I've read that Neo4j is heavily reliant on memory.

As soon as I give it more than half (from -Xmx61G onwards) and try to instantiate I get the following error:

gremlin> g = new Neo4j2Graph("/tmp/neo4j/");
Requested array size exceeds VM limit
Display stack trace? [yN] y
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
    at java.util.concurrent.atomic.

AtomicReferenceArray.<init>(AtomicReferenceArray.java:94)
    at org.neo4j.kernel.impl.cache.HighPerformanceCache.<init>(HighPerformanceCache.java:100)
    at org.neo4j.kernel.impl.cache.HighPerformanceCacheProvider.newNodeCache(HighPerformanceCacheProvider.java:49)
    at org.neo4j.kernel.impl.core.DefaultCaches.node(DefaultCaches.java:51)
    at org.neo4j.kernel.InternalAbstractGraphDatabase.create(InternalAbstractGraphDatabase.java:573)
    at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:339)
    at org.neo4j.kernel.EmbeddedGraphDatabase.<init>(EmbeddedGraphDatabase.java:59)
    at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:91)
    at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:181)
    at com.tinkerpop.blueprints.impls.neo4j2.Neo4j2Graph.<init>(Neo4j2Graph.java:154)
    at com.tinkerpop.blueprints.impls.neo4j2.Neo4j2Graph.<init>(Neo4j2Graph.java:136)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
    at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
    at groovysh_evaluate.run(groovysh_evaluate:51)
    at groovysh_evaluate$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)
    at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)
    at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:114)

I'm curious to know why this happens. Are there any workarounds for this?

Thank you all for your time,
Alex

[Clark Richey]

The JVM has a hard limit on the size of the array. I’m assuming you are running on a 32 bit os / JVM?

Clark Richey

clark....@gmail.com

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Hunger]

I presume you have neo4j-enteprise dependencies in your classpath.

It tries to set up the high-performance-cache which is an array based cache.

And the array size is based as a percentage of the heap, with the maximum being 2^31 = 2.147.483.648

So you have to adjust those settings, as described here:

http://neo4j.com/docs/stable/configuration-caches.html#high-performance-cache

you can also choose another cache type, e.g. cache_type=none|soft|weak|hpc

Michael