Unable to memory map warning in a read only embedded db

[zahariadn]

Hello, 

I am constantly getting the unable to memory map warning while using a read only embedded db , without any further configurations. 

Is there any way to solve this issue and more importantly does it affect the performance of my database?

The concept of my work is that i use the read only db in a web services application thus i have multiple concurrent reads over the db.

This is the exception i get. I am also attaching the messages.log file. 

Any help will be appreciated. 

Thank you

Nikolas

WARNING: [/Users/nikolaszachariades/Desktop/UoM/dissertation/DBMS4BN/GraphDB/gdbCreator/neo4jDB/neostore.relationshipstore.db] Unable to memory map

org.neo4j.kernel.impl.nioneo.store.MappedMemException: Unable to map pos=18715 recordSize=33 totalSize=617595

at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.<init>(MappedPersistenceWindow.java:61)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:590)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.expandBricks(PersistenceWindowPool.java:565)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:141)

at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:519)

at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:333)

at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:122)

at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:105)

at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:108)

at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:608)

at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:400)

at org.neo4j.kernel.impl.core.NodeImpl.loadInitialRelationships(NodeImpl.java:352)

at org.neo4j.kernel.impl.core.NodeImpl.ensureRelationshipMapNotNull(NodeImpl.java:338)

at org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:184)

at org.neo4j.kernel.impl.core.NodeImpl.getRelationships(NodeImpl.java:240)

at org.neo4j.kernel.impl.core.NodeProxy.getRelationships(NodeProxy.java:98)

at org.neo4j.kernel.StandardExpander$RegularExpander.doExpand(StandardExpander.java:579)

at org.neo4j.kernel.StandardExpander$RelationshipExpansion.iterator(StandardExpander.java:194)

at org.neo4j.graphalgo.impl.path.ShortestPath$DirectionData$1.createNestedIterator(ShortestPath.java:314)

at org.neo4j.graphalgo.impl.path.ShortestPath$DirectionData$1.createNestedIterator(ShortestPath.java:309)

at org.neo4j.helpers.collection.NestingIterator.fetchNextOrNull(NestingIterator.java:67)

at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:55)

at org.neo4j.graphalgo.impl.path.ShortestPath$DirectionData.fetchNextRelOrNull(ShortestPath.java:384)

at org.neo4j.graphalgo.impl.path.ShortestPath$DirectionData.fetchNextOrNull(ShortestPath.java:326)

at org.neo4j.graphalgo.impl.path.ShortestPath$DirectionData.fetchNextOrNull(ShortestPath.java:263)

at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:55)

at org.neo4j.graphalgo.impl.path.ShortestPath.internalPaths(ShortestPath.java:163)

at org.neo4j.graphalgo.impl.path.ShortestPath.findSinglePath(ShortestPath.java:139)

at neo4jGraphDB.neo4jDBConnector.calculateShortestPath(neo4jDBConnector.java:1183)

at SBMLWebServices.RequestHandler.getshortestPath(RequestHandler.java:205)

at SBMLWebServices.ShortestPathResource.getXml(ShortestPathResource.java:48)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:167)

at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:70)

at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:279)

at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:86)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136)

at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:74)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1347)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1279)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1229)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1219)

at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:419)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1534)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)

at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)

at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:98)

at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162)

at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:326)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:227)

at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:170)

at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:822)

at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:719)

at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1013)

at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)

at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)

at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)

at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)

at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)

at com.sun.grizzly.ContextTask.run(ContextTask.java:71)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)

at java.lang.Thread.run(Thread.java:680)

Caused by: java.io.IOException: Invalid argument

at sun.nio.ch.FileChannelImpl.truncate0(Native Method)

at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:718)

at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.<init>(MappedPersistenceWindow.java:53)

... 73 more

WARNING: [/Users/nikolaszachariades/Desktop/UoM/dissertation/DBMS4BN/GraphDB/gdbCreator/neo4jDB/neostore.propertystore.db] Unable to memory map

org.neo4j.kernel.impl.nioneo.store.MappedMemException: Unable to map pos=24602 recordSize=41 totalSize=504341

at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.<init>(MappedPersistenceWindow.java:61)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:590)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.expandBricks(PersistenceWindowPool.java:565)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:141)

at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:519)

at org.neo4j.kernel.impl.nioneo.store.PropertyStore.getLightRecord(PropertyStore.java:297)

at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getPropertyRecordChain(ReadTransaction.java:185)

at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.loadProperties(ReadTransaction.java:215)

at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.nodeLoadProperties(ReadTransaction.java:239)

at org.neo4j.kernel.impl.persistence.PersistenceManager.loadNodeProperties(PersistenceManager.java:113)

at org.neo4j.kernel.impl.core.NodeManager.loadProperties(NodeManager.java:682)

at org.neo4j.kernel.impl.core.NodeImpl.loadProperties(NodeImpl.java:131)

at org.neo4j.kernel.impl.core.Primitive.ensureFullProperties(Primitive.java:569)

at org.neo4j.kernel.impl.core.Primitive.getProperty(Primitive.java:152)

at org.neo4j.kernel.impl.core.NodeImpl.getProperty(NodeImpl.java:50)

at org.neo4j.kernel.impl.core.NodeProxy.getProperty(NodeProxy.java:155)

at SBMLDocumentBuilder.SBMLDocBuilder.createShortestPathDoc(SBMLDocBuilder.java:903)

at SBMLWebServices.RequestHandler.getshortestPath(RequestHandler.java:206)

at SBMLWebServices.ShortestPathResource.getXml(ShortestPathResource.java:48)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:167)

at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:70)

at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:279)

at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:86)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136)

at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:74)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1347)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1279)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1229)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1219)

at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:419)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1534)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)

at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)

at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:98)

at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162)

at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:326)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:227)

at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:170)

at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:822)

at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:719)

at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1013)

at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)

at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)

at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)

at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)

at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)

at com.sun.grizzly.ContextTask.run(ContextTask.java:71)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)

at java.lang.Thread.run(Thread.java:680)

Caused by: java.io.IOException: Invalid argument

at sun.nio.ch.FileChannelImpl.truncate0(Native Method)

at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:718)

at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.<init>(MappedPersistenceWindow.java:53)

... 61 more

[Peter Neubauer]

Hi there,
I take it that the application is working right? Not bring able to memory map because of more than one process accessing the db files is going to slow things down a bit, but I am not sure how much. For a full speed setup, try setting up an ha cluster where every cluster node operates at full speed under one of your readers?
/peter

Send from mobile.

--
 
 

[zahariadn]

Yes my application is functioning fine. Although i have some memory problem when i am making some neighborhood queries using cypher, for queries with hop=6 and above. As far as i can tell am running out of memory but i dont get a java heap out of memory is more like a GC thrashing. The process continues the execution but it never finishes.   

Currently the stored graph in the database contains 13047 nodes and 28 285 relationships.

About the HA solution, what kind of configuration i must provide when instantiating a the service? From what i understand from the manual this setup is when using multiple machines. 

Thanks 

Nikolas

[Michael Hunger]

Can you share your cypher query?

Thx

Sent from mobile device

--
 
 

[zahariadn]

this my cypher query:

        ExecutionResult result = engine.execute(

                  "start r=node({id}) "

                + "match path=(r)-[*"+hopnum*2+".."+hopnum*2+"]-(r2) "

                + "where r2.type=\"reaction\" AND r.type=\"reaction\" "

                + "AND ALL(x in relationships(path) "

                + "where type(x)=\"HAS_PRODUCT\" OR type(x)=\"IS_REACTANT\" "

                + "OR type(x)=\"IS_MODIFIER\" ) "

                + "return r.id,r2.id,path",params);

On Sunday, 19 August 2012 00:45:27 UTC+1, zahariadn wrote:

[Peter Neubauer]

Hi there,
you can set up HA also in one machine, see
https://github.com/neo4j/neo4j-enterprise-local-qa and execute it to
get a 3-node cluster running on your local machine.

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.

> --
>
>