simple cypher query causes OufOfMemoryError of small 700 nodes graph

[Andrea Salvadore]

Given a graph of about 700 nodes and 3.5k relations I am running the following query to find all the nodes connected to mine

start a = node(461)
match a-[*]-b
return b.name

and the server crashes with various OutOfMemoryErrors like

java.lang.OutOfMemoryError: GC overhead limit exceeded
        at org.neo4j.kernel.impl.traversal.TraversalBranchImpl.newNextBranch(TraversalBranchImpl.java:180)
        at org.neo4j.kernel.impl.traversal.TraversalBranchImpl.next(TraversalBranchImpl.java:161)
        at org.neo4j.kernel.PreorderDepthFirstSelector.next(PreorderDepthFirstSelector.java:52)
        at org.neo4j.kernel.impl.traversal.TraverserIterator.fetchNextOrNull(TraverserIterator.java:65)
        at org.neo4j.kernel.impl.traversal.TraverserIterator.fetchNextOrNull(TraverserIterator.java:34)
        at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:55)
        at scala.collection.JavaConversions$JIteratorWrapper.hasNext(JavaConversions.scala:574)
        at scala.collection.Iterator$class.toStream(Iterator.scala:1024)
        at scala.collection.JavaConversions$JIteratorWrapper.toStream(JavaConversions.scala:573)
        at scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1024)
        at scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1024)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:634)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:626)
        at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:175)
        at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:175)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:634)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:626)
        at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:699)
        at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:699)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:634)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:626)
        at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:699)
        at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:699)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:634)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:626)
        at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:175)
        at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:175)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:634)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:626)
        at scala.collection.immutable.Stream.foldLeft(Stream.scala:302)
        at org.neo4j.cypher.internal.pipes.matching.PatternMatcher.traverseRelationship(PatternMatcher.scala:149)
        at org.neo4j.cypher.internal.pipes.matching.PatternMatcher.traverseNextSpecificNode(PatternMatcher.scala:60)



================= OR ==========================

java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.LinkedHashMap.createEntry(LinkedHashMap.java:442)
        at java.util.HashMap.addEntry(HashMap.java:856)
        at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:427)
        at java.util.HashMap.put(HashMap.java:484)
        at sun.util.resources.OpenListResourceBundle.loadLookup(OpenListResourceBundle.java:134)
        at sun.util.resources.OpenListResourceBundle.loadLookupTablesIfNecessary(OpenListResourceBundle.java:113)
        at sun.util.resources.OpenListResourceBundle.handleGetObject(OpenListResourceBundle.java:74)
        at sun.util.resources.TimeZoneNamesBundle.handleGetObject(TimeZoneNamesBundle.java:75)
        at java.util.ResourceBundle.getObject(ResourceBundle.java:387)
        at java.util.ResourceBundle.getObject(ResourceBundle.java:390)
        at java.util.ResourceBundle.getStringArray(ResourceBundle.java:370)
        at sun.util.TimeZoneNameUtility.retrieveDisplayNames(TimeZoneNameUtility.java:118)
        at sun.util.TimeZoneNameUtility.retrieveDisplayNames(TimeZoneNameUtility.java:99)
        at java.util.TimeZone.getDisplayNames(TimeZone.java:463)
        at java.util.TimeZone.getDisplayName(TimeZone.java:414)
        at java.util.Date.toString(Date.java:1042)
        at org.neo4j.server.webadmin.rest.representations.JmxAttributeRepresentationDispatcher.dispatchOtherProperty(JmxAttributeRepresentationDispatcher.java:90)
        at org.neo4j.server.webadmin.rest.representations.JmxAttributeRepresentationDispatcher.dispatchOtherProperty(JmxAttributeRepresentationDispatcher.java:36)
        at org.neo4j.server.helpers.PropertyTypeDispatcher.dispatch(PropertyTypeDispatcher.java:126)
        at org.neo4j.server.webadmin.rest.representations.JmxAttributeRepresentation.getValue(JmxAttributeRepresentation.java:94)
        at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.neo4j.server.rest.repr.ObjectRepresentation$1.get(ObjectRepresentation.java:93)
        at org.neo4j.server.rest.repr.ObjectRepresentation$PropertyGetter.putTo(ObjectRepresentation.java:132)
        at org.neo4j.server.rest.repr.ObjectRepresentation.serialize(ObjectRepresentation.java:144)
        at org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:40)
        at org.neo4j.server.rest.repr.ListSerializer.addMapping(ListSerializer.java:56)
        at org.neo4j.server.rest.repr.MappingRepresentation.addTo(MappingRepresentation.java:52)
        at org.neo4j.server.rest.repr.ListRepresentation.serialize(ListRepresentation.java:60)
        at org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:75)
        at org.neo4j.server.rest.repr.MappingSerializer.putList(MappingSerializer.java:61)

 ========================= OR =================================


Exception in thread "RMI RenewClean-[127.0.1.1:47818]" java.lang.OutOfMemoryError: Java heap space
    at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:76)
    at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:59)
    at sun.rmi.transport.tcp.TCPConnection.getOutputStream(TCPConnection.java:84)
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:220)
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
    at sun.rmi.transport.DGCImpl_Stub.dirty(Unknown Source)
    at sun.rmi.transport.DGCClient$EndpointEntry.makeDirtyCall(DGCClient.java:360)
    at sun.rmi.transport.DGCClient$EndpointEntry.access$1600(DGCClient.java:171)
    at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:574)
    at java.lang.Thread.run(Thread.java:722)
11:40:40.754 [770659873@qtp-197061332-1137] ERROR org.mortbay.log - Error for /db/manage/server/jmx/kernelquery
java.lang.OutOfMemoryError: Java heap space
    at java.io.BufferedWriter.<init>(BufferedWriter.java:105) ~[na:1.7.0_09]
    at java.io.BufferedWriter.<init>(BufferedWriter.java:88) ~[na:1.7.0_09]
    at com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:188) ~[jersey-core-1.9.jar:1.9]
    at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128) ~[jersey-core-1.9.jar:1.9]
    at com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88) ~[jersey-core-1.9.jar:1.9]
    at com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58) ~[jersey-core-1.9.jar:1.9]
    at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1437) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) ~[jersey-server-1.9.jar:1.9]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) ~[servlet-api-2.5-20081211.jar:na]
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) ~[jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) ~[jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.Server.handle(Server.java:326) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.25.jar:6.1.25]
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.25.jar:6.1.25]
11:43:30.812 [1376426182@qtp-197061332-1138] ERROR org.mortbay.log - /db/manage/server/monitor/fetch/1354275521
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2367) ~[na:1.7.0_09]
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) ~[na:1.7.0_09]
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) ~[na:1.7.0_09]
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:535) ~[na:1.7.0_09]
    at java.lang.StringBuffer.append(StringBuffer.java:322) ~[na:1.7.0_09]
    at java.io.StringWriter.write(StringWriter.java:94) ~[na:1.7.0_09]
    at org.codehaus.jackson.impl.WriterBasedGenerator._flushBuffer(WriterBasedGenerator.java:1812) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.impl.WriterBasedGenerator.writeRawLong(WriterBasedGenerator.java:603) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.impl.WriterBasedGenerator.writeRaw(WriterBasedGenerator.java:543) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.impl.WriterBasedGenerator.writeNumber(WriterBasedGenerator.java:732) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializers$DoubleSerializer.serialize(StdSerializers.java:221) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializers$DoubleSerializer.serialize(StdSerializers.java:209) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.StdContainerSerializers$IndexedListSerializer.serializeContents(StdContainerSerializers.java:122) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.StdContainerSerializers$IndexedListSerializer.serializeContents(StdContainerSerializers.java:71) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.AsArraySerializerBase.serialize(AsArraySerializerBase.java:86) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serializeFields(MapSerializer.java:262) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:186) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:23) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serializeFields(MapSerializer.java:262) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:186) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:23) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:610) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1613) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.neo4j.server.rest.domain.JsonHelper.createJsonFrom(JsonHelper.java:102) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.formats.JsonFormat.complete(JsonFormat.java:68) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:44) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.format(OutputFormat.java:182) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.formatRepresentation(OutputFormat.java:132) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.response(OutputFormat.java:119) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.ok(OutputFormat.java:55) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.webadmin.rest.MonitorService.getData(MonitorService.java:118) ~[neo4j-server-1.9.M01.jar:1.9.M01]


I tried changing the heap size in neo4j-wrapper.conf with no luck.


How is it possible that with such a small graph I am getting OutOfMemoryErrors? Am I doing something wrong?

I am running neo4j community 1.9 M01

Pleas find attached the log file.

Thanks for your help

[Andres Taylor]

The memory consumption of Cypher has been worked on, and the code for it will be released in the next milestone (1.9.M02). Basically, "never-ending" patterns like the one you have, will not die a horrible OOM death. You will get a lazy iterator which buffers nothing, so heap usage is much smaller.

You'll still run into the same problem if you have an eager query, like aggregating on the values of such a pattern, or doing an updating query.

The next milestone of 1.9 is due out any time now.

Andrés

--
 
 

--

The best way to ask for Cypher help: http://console.neo4j.org/usage.html 

[asalvadore]

HI Andres, thanks for your reply.
Is there any other smart way to run the same query without running in this problem?

I am just trying to find disconnected sub-graphs...

[Lasse Westh-Nielsen]

Andrea,

http://docs.neo4j.org/chunked/stable/query-match.html#match-all-shortest-paths
- that algorithm is guaranteed to terminate whereas your infinite
paths one will cycle forever if your graph has cycles.

- Lasse

> --
>
>

[Lasse Westh-Nielsen]

Andrea,

In fact: http://console.neo4j.org/r/gbdgrj - I guess you then count
and see if the number matches the total number of nodes.

- Lasse

[Andres Taylor]

Something like this?

http://console-test.neo4j.org/?id=v9vv98

start d=node(1), e=node(*) 

match p = shortestPath( d-[?*]->e ) 

return e, e = last(p) as reachable

It uses the fact that optional paths (the question mark in the match clause) will be null if no path is found. 

Linked to console-test, because console.neo4j.org an older version of Neo4j that seems to have a problem with that query.

HTH,

Andrés

[Andrea Salvadore]

Hi Lasse, thanks for your input. Running your query I still get the same error....

Exception in thread "GC-Monitor" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2367)
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:638)
    at java.lang.StringBuilder.append(StringBuilder.java:224)
    at org.neo4j.kernel.impl.cache.MeasureDoNothing.run(MeasureDoNothing.java:85)
14:58:29.689 [1560764416@qtp-690349086-5] ERROR org.mortbay.log - /db/data/cypher
java.lang.OutOfMemoryError: Java heap space
    at org.neo4j.server.plugins.PluginManager.getExensionsFor(PluginManager.java:102) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.Serializer.injectExtensions(Serializer.java:49) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:39) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.ListSerializer.addMapping(ListSerializer.java:56) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.MappingRepresentation.addTo(MappingRepresentation.java:52) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.ListRepresentation.serialize(ListRepresentation.java:60) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:75) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.ListSerializer.addList(ListSerializer.java:61) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.ListRepresentation.addTo(ListRepresentation.java:67) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.ListRepresentation.serialize(ListRepresentation.java:60) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:75) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.MappingSerializer.putList(MappingSerializer.java:61) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.CypherResultRepresentation.serialize(CypherResultRepresentation.java:50) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:42) ~[server-api-1.9.M01.jar:1.9.M01]

    at org.neo4j.server.rest.repr.OutputFormat.format(OutputFormat.java:182) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.formatRepresentation(OutputFormat.java:132) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.response(OutputFormat.java:119) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.ok(OutputFormat.java:55) ~[neo4j-server-1.9.M01.jar:1.9.M01]

    at org.neo4j.server.rest.web.CypherService.cypher(CypherService.java:68) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_09]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_09]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_09]
    at java.lang.reflect.Method.invoke(Method.java:601) ~[na:1.7.0_09]
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) ~[jersey-server-1.9.jar:1.9]
14:58:29.689 [1363027059@qtp-690349086-6] ERROR org.mortbay.log - /db/manage/server/monitor/fetch/1354287472
java.lang.OutOfMemoryError: Java heap space
    at org.codehaus.jackson.util.DefaultPrettyPrinter.<init>(DefaultPrettyPrinter.java:24) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.impl.JsonGeneratorBase.useDefaultPrettyPrinter(JsonGeneratorBase.java:120) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.neo4j.server.rest.domain.JsonHelper.createJsonFrom(JsonHelper.java:99) ~[neo4j-server-1.9.M01.jar:1.9.M01]

    at org.neo4j.server.rest.repr.formats.JsonFormat.complete(JsonFormat.java:68) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:44) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.format(OutputFormat.java:182) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.formatRepresentation(OutputFormat.java:132) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.response(OutputFormat.java:119) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.ok(OutputFormat.java:55) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.webadmin.rest.MonitorService.getData(MonitorService.java:118) ~[neo4j-server-1.9.M01.jar:1.9.M01]

    at org.neo4j.server.webadmin.rest.MonitorService.getData(MonitorService.java:96) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_09]
    at java.lang.reflect.Method.invoke(Method.java:601) ~[na:1.7.0_09]
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) ~[jersey-server-1.9.jar:1.9]

    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) ~[jersey-server-1.9.jar:1.9]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) ~[jersey-server-1.9.jar:1.9]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) ~[servlet-api-2.5-20081211.jar:na]
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) ~[jetty-6.1.25.jar:6.1.25]
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) ~[jetty-6.1.25.jar:6.1.25]

--

--

		Andrea Salvadore
		Senior Software Engineer
		T: +44 208 742 1600 E: andrea.s...@openbet.com W: www.openbet.com	OpenBet Chiswick Park Building 9 566 Chiswick High Rd London W4 5XT

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postm...@openbet.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by OpenBet for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. OpenBet Technologies Limited. Registered Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company registered in England and Wales. Registered no. 6712030. VAT no. GB927523623

[Lasse Westh-Nielsen]

Andrea,

How big is your heap?

 - Lasse

--
 
 

[Andrea Salvadore]

1GB

[Andrea Salvadore]

Actually this query seems to work - Thanks Andres, I slightly changed yours

start d=node(1), e=node(*)
match p = shortestPath( e -[?*]-d )
where (p Is Not Null)
return e

[Andres Taylor]

This query gets you the same result, but is much more readable, I think.

start a=node(461), b=node(*) 

where a-[*]-b

return b.name

Andrés

[Andrea Salvadore]

Andres, that is the query I started  with :)

That is causing the OOE, unless using the shortestPath function

Thanks

[Andres Taylor]

On Fri, Nov 30, 2012 at 6:41 PM, Andrea Salvadore <andrea.s...@openbet.com> wrote:

Andres, that is the query I started  with :)

No it isn't. :P

You started with:

start a = node(461)
match a-[*]-b
return b.name

I suggest you instead use:

start a=node(461), b=node(*) 

where a-[*]-b

return b.name

I'll point out the differences and why one kills the database and the other doesn't.

The first query has a single start point - a. From that point, you ask Cypher to find all possible paths outgoing from your start point. From each of these paths, you take the end point and get the property name from it. It can return a very large result set - you wrote that you have 3.5K relationships - Cypher will try to find ALL possible paths that can be constructed using these relationships only once per path. 

In my query, I use two start points - a and b. a will only ever point to a single node, but b will contain all nodes in your graph.  It then filters out any b-node that is not somehow connected with the a node. I do this with the same pattern that you used, but I used it in the WHERE clause and not the MATCH clause. This query will at most return as many nodes as there are in your graph, and never more.

HTH,

Andrés

[Andrea Salvadore]

HI Andrés, I didn't see the where / match difference.
BTW I tried the second query and it kills the database as well.

--
 
 

[Andrea Salvadore]

Andrés thanks for your help. It actually make perfect sense what you are saying. I missed that query difference :)

Looking at the stack trace, it seems there is some problem in the query serialization?

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2367) ~[na:1.7.0_09]
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) ~[na:1.7.0_09]
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) ~[na:1.7.0_09]
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:535) ~[na:1.7.0_09]
    at java.lang.StringBuffer.append(StringBuffer.java:322) ~[na:1.7.0_09]
    at java.io.StringWriter.write(StringWriter.java:94) ~[na:1.7.0_09]
    at org.codehaus.jackson.impl.WriterBasedGenerator._flushBuffer(WriterBasedGenerator.java:1812) ~[jackson-core-asl-1.9.7.jar:1.9.7]

    at org.codehaus.jackson.impl.WriterBasedGenerator.writeString(WriterBasedGenerator.java:451) ~[jackson-core-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.impl.WriterBasedGenerator.writeNumber(WriterBasedGenerator.java:727) ~[jackson-core-asl-1.9.7.jar:1.9.7]

    at org.codehaus.jackson.map.ser.StdSerializers$DoubleSerializer.serialize(StdSerializers.java:221) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializers$DoubleSerializer.serialize(StdSerializers.java:209) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.StdContainerSerializers$IndexedListSerializer.serializeContents(StdContainerSerializers.java:122) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.StdContainerSerializers$IndexedListSerializer.serializeContents(StdContainerSerializers.java:71) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.AsArraySerializerBase.serialize(AsArraySerializerBase.java:86) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serializeFields(MapSerializer.java:262) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:186) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:23) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serializeFields(MapSerializer.java:262) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:186) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.std.MapSerializer.serialize(MapSerializer.java:23) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:610) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1613) ~[jackson-mapper-asl-1.9.7.jar:1.9.7]
    at org.neo4j.server.rest.domain.JsonHelper.createJsonFrom(JsonHelper.java:102) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.formats.JsonFormat.complete(JsonFormat.java:68) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:44) ~[server-api-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.format(OutputFormat.java:182) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.formatRepresentation(OutputFormat.java:132) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.response(OutputFormat.java:119) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.rest.repr.OutputFormat.ok(OutputFormat.java:55) ~[neo4j-server-1.9.M01.jar:1.9.M01]
    at org.neo4j.server.webadmin.rest.MonitorService.getData(MonitorService.java:118) ~[neo4j-server-1.9.M01.jar:1.9.M01]

    at org.neo4j.server.webadmin.rest.MonitorService.getData(MonitorService.java:96) ~[neo4j-server-1.9.M01.jar:1.9.M01]

[Michael Hunger]

Use the header: X-Stream:true

Then the results are not built up in memory but directly streamed to the client.

But this is also an indication that you're returning _a lot_ of data.

Michael

--