UnderlyingStorageException: Unable to load position

[Eugeny Kozhanov]

Hi to all!

I use multiprocessing in Java for work with neo4j server.

I use timeout: 

taskFutures = service.invokeAll(tasks, timeout, TimeUnit.SECONDS);

Some tasks is cancelled by timeout. Before it I connect to the server and I see follow exception: "UnderlyingStorageException: Unable to load position[..."

Can you help me?

TIA.

[Michael Hunger]

Can you share more code and the full exception ?
Also possible errors from the server logs ? data/log and data/graph.db/messages.log

Thanks

Michael

> --
>
>

[..Simba]

我如何退订该订阅呢各位

------------------ Original ------------------

From:  "Eugeny Kozhanov"<vin...@gmail.com>;

Date:  Tue, Sep 11, 2012 07:19 PM

To:  "neo4j"<ne...@googlegroups.com>;

Subject:  [Neo4j] UnderlyingStorageException: Unable to load position

[段贺新]

我也在纳闷 如何退订呢

--
 
 

[Eugeny Kozhanov]

Full exception text:

{

  "message" : "Unable to load position[4238] @[38142]",

  "exception" : "org.neo4j.kernel.impl.nioneo.store.UnderlyingStorageException: Unable to load position[4238] @[38142]",

  "stacktrace" : [ "org.neo4j.kernel.impl.nioneo.store.PersistenceRow.readPosition(PersistenceRow.java:101)", "org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:177)", "org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:559)", "org.neo4j.kernel.impl.nioneo.store.NodeStore.loadLightNode(NodeStore.java:183)", "org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.nodeLoadLight(ReadTransaction.java:85)", "org.neo4j.kernel.impl.persistence.PersistenceManager.loadLightNode(PersistenceManager.java:82)", "org.neo4j.kernel.impl.core.NodeManager.getNodeByIdOrNull(NodeManager.java:420)", "org.neo4j.kernel.impl.core.NodeManager.getNodeById(NodeManager.java:435)", "org.neo4j.kernel.EmbeddedGraphDbImpl.getNodeById(EmbeddedGraphDbImpl.java:327)", "org.neo4j.kernel.EmbeddedGraphDatabase.getNodeById(EmbeddedGraphDatabase.java:109)", "org.neo4j.server.rest.repr.RepresentationFormat.getNode(RepresentationFormat.java:259)", "org.neo4j.server.rest.repr.RepresentationFormat.getNode(RepresentationFormat.java:246)", "org.neo4j.server.rest.repr.RepresentationFormat.convertNode(RepresentationFormat.java:231)", "org.neo4j.server.rest.repr.RepresentationFormat$1.convertNode(RepresentationFormat.java:147)", "org.neo4j.server.plugins.ParameterList.getNode(ParameterList.java:349)", "org.neo4j.server.plugins.NodeTypeCaster.get(NodeTypeCaster.java:30)", "org.neo4j.server.plugins.ParameterExtractor.extract(ParameterExtractor.java:45)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:53)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)", "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(Method.java:616)", "com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)", "com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)", "com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)", "com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)", "com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)", "com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)", "com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)", "com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)", "com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)", "com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)", "com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)", "com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)", "com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)", "com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)", "com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)", "javax.servlet.http.HttpServlet.service(HttpServlet.java:820)", "org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)", "org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)", "org.neo4j.server.statistic.StatisticFilter.doFilter(StatisticFilter.java:62)", "org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)", "org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)", "org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)", "org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)", "org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)", "org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)", "org.mortbay.jetty.Server.handle(Server.java:326)", "org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)", "org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)", "org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)", "org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)", "org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)", "org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)", "org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)" ]

[段贺新]

你做it 开发 ？主攻前端

--
 
 

[Eugeny Kozhanov]

from server log:

SEVERE: /db/data/node/0

org.neo4j.kernel.impl.nioneo.store.UnderlyingStorageException: Unable to load position[0] @[0]

at org.neo4j.kernel.impl.nioneo.store.PersistenceRow.readPosition(PersistenceRow.java:101)

at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:177)

at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:559)

at org.neo4j.kernel.impl.nioneo.store.NodeStore.loadLightNode(NodeStore.java:183)

at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.nodeLoadLight(ReadTransaction.java:85)

at org.neo4j.kernel.impl.persistence.PersistenceManager.loadLightNode(PersistenceManager.java:82)

at org.neo4j.kernel.impl.core.NodeManager.getNodeByIdOrNull(NodeManager.java:420)

at org.neo4j.kernel.impl.core.NodeManager.getNodeById(NodeManager.java:435)

at org.neo4j.kernel.EmbeddedGraphDbImpl.getNodeById(EmbeddedGraphDbImpl.java:327)

at org.neo4j.kernel.EmbeddedGraphDatabase.getNodeById(EmbeddedGraphDatabase.java:109)

at org.neo4j.server.rest.web.DatabaseActions.node(DatabaseActions.java:117)

at org.neo4j.server.rest.web.DatabaseActions.getNode(DatabaseActions.java:232)

at org.neo4j.server.rest.web.RestfulGraphDatabase.getNode(RestfulGraphDatabase.java:216)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:616)

at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)

at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)

at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)

at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)

at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)

at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)

at org.neo4j.server.statistic.StatisticFilter.doFilter(StatisticFilter.java:62)

at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)

at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)

at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)

at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)

at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)

at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)

at org.mortbay.jetty.Server.handle(Server.java:326)

at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)

at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926)

at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)

at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)

at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)

at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)

at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Caused by: java.nio.channels.ClosedChannelException

at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:109)

at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:654)

at org.neo4j.kernel.impl.nioneo.store.PersistenceRow.readPosition(PersistenceRow.java:91)

... 49 more

[Michael Hunger]

And what exactly do you do?

read operations / update-ops?

Any other messages in the logfiles?

Michael

> --
>
>

[Eugeny Kozhanov]

task's class:

class RouterTaskDirect implements Callable<List<List<NodeData>>> {  

        ...

   public List<List<NodeData>> call() throws Exception {

    Transaction tx = this.gdb.beginTx();

    List<List<NodeData>> p;

    try {

    p = findDirectPaths( this.fromId, this.toId, this.transport );

    tx.success();

    }

    finally {

    tx.finish();

    }

       return p;

   }  

        ...

        }

create task:

          tasks.add( new RouterTaskDirect ( nodeFrom, nodeTo, transport, graphDb ) );

and solve all tasks:

          taskFutures = service.invokeAll(tasks, timeout, TimeUnit.SECONDS);

My question: how I must use timeout with neo4j server?

[Eugeny Kozhanov]

in function findDirectPaths I read DB.

[Michael Hunger]

For server you don't need tx.
What does findDirectPaths do?

It could be that server connections stay open and are not properly closed.

Michael

> --
>
>

[Michael Hunger]

How ?

Michael

--
 
 

[Eugeny Kozhanov]

nodeFrom.getRelationships()

rel.start

rel.end

etc.

[Eugeny Kozhanov]

How connection may properly closed if I use .invokeAll with timeout?

[Michael Hunger]

Ouch, via REST you should use a traverser or cypher. Otherwise you get hit by many roundtrips.

What is the version of your neo4j server and the java-rest-binding?

Michael

--
 
 

[Eugeny Kozhanov]

server:  1.6.2

java-rest:  where I can see this?

[Michael Hunger]

How did you include it? e.g. in your maven pom or library directory.

Michael

--
 
 

[Eugeny Kozhanov]

1.6.2

[Eugeny Kozhanov]

My plugin call functions from class Dijkstra. All work normally. But if one of call was terminated by timeout, next call of my plugin return this error. Any idea? TIA. 

[Michael Hunger]

Any means to share your project / dataset for us to have a look?

Michael

Am 12.09.2012 um 09:19 schrieb Eugeny Kozhanov:

> My plugin call functions from class Dijkstra. All work normally. But if one of call was terminated by timeout, next call of my plugin return this error. Any idea? TIA.
>

> --
>
>

[Eugeny Kozhanov]

I send it on your e-mail.

[Michael Hunger]

Thanks, don't know when I find the time but I'll try.

Michael

--
 
 

[Michael Hunger]

Could you try to run it against a more recent version of neo4j to see if the issue still persists ? e.g. 1.8.RC1 ?

Thanks

Michael

Am 12.09.2012 um 11:55 schrieb Eugeny Kozhanov:

--
 
 

[Eugeny Kozhanov]

I trying in 1.8RC2 - the same error.

[Eugeny Kozhanov]

It's work on the small example! I will refactor my plugin.

Thank you very much!

[Peter Neubauer]

Did you ever find what the error was Eugeny?

Cheers,

/peter neubauer

Neo4j 1.8.RC1 "Vindeln Vy" -
http://blog.neo4j.org/2012/09/neo4j-18rc1-really-careful-ftw.html

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.

> --
>
>

[Eugeny Kozhanov]

Unfortunately, it is work only for one task in the pool (((

[Eugeny Kozhanov]

How I request the server in read-only mode? May be, it's help?

[Eugeny Kozhanov]

Why it's work with one thread, but don't work with >1 threads?

I can create N plugins - by one for each task, but I think that it was wrong way...

[Eugeny Kozhanov]

It's work if I use one thread:

ExecutorService service = Executors.newFixedThreadPool( 1 )

And don't work in other case:

ExecutorService service = Executors.newFixedThreadPool( 2 )

[Eugeny Kozhanov]

Oh, I'm sorry. There aren't work. The question is open...

[Eugeny Kozhanov]

Instead:

List<Future< List<List<Node>> >> taskFutures = service.invokeAll( tasks, timeout*1000, TimeUnit.SECONDS );

I write:

List<Future< List<List<Node>> >> taskFutures = service.invokeAll( tasks );

service.awaitTermination(timeout, TimeUnit.SECONDS);

It's work! 

[Michael Hunger]

Did you remove the state from the plugin?

You have a different timeout in the second call?

Michael

> --
>
>

[Naveen K. Kadiyala]

Please remove my email from the group

Regards,
Naveenkumar Kadiyala

--


CONFIDENTIALITY NOTICE: This e-mail and any files attached contain confidential information of Next Sphere technologies. If you are not the intended recipient, or a person responsible for delivering it, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error, please destroy the original transmission and its attachments without reading or saving in any manner.

[Eugeny Kozhanov]

Yes, I remove the state from the plugin.

There is very strange timeout in the second call: if I set timeout=5seconds, than very simple task is resolved in 5 seconds! Why don't respond earlier?

[Eugeny Kozhanov]

Basic steps of create of my plugin you can find here: http://vinger4.blogspot.com/2012/09/create-neo4j-plugin.html