How to solve DeadlockDetectedExceptions ?

[Nicolas Delsaux]

Hi

question title is quite clear, but requires some (may ?) contextual informations.

I'm running a JavaEE application that uses neo4j 1.5.3 as storage back-end. This graph is used through the following application stack

 * The neo4j DB is started through neo4j-jca-connector (https://github.com/alexsmirnov/neo4j-connector) in embedded mode

 * That jca connector is accessed as a totally standard JavaEE resource in my application code (ie injected through @Resource(mappedName = "jca/Neo4j/resource", type=Neo4JConnectionFactory.class))

 * To not write our objects to neo4j ourselves, we also use an adaptation layer called gaedo-blueprints (http://riduidel.github.com/gaedo/site/0.4.1/gaedo-blueprints/index.html) which access neo4j through blueprints 1.1. (well, it appears that, beside developping my enterprise application, i'm also gaedo lead and only developper).

So, after a recent modification in our code base, we started having recuring DeadlockDetectedException.

Those exceptions always shows up deep in a stack of re-throwed exceptions, with messages likes

Caused by: org.neo4j.kernel.DeadlockDetectedException: JavaEETransactionImpl: txId=18 nonXAResource=null jtsTx=com.sun.jts.jta.TransactionImpl@70faf570 localTxStatus=0 syncs=[com.sun.ejb.containers.ContainerSynchronization@6d65042c] can't wait on resource RWLock[Node[6]] since => JavaEETransactionImpl: txId=18 nonXAResource=null jtsTx=com.sun.jts.jta.TransactionImpl@70faf570 localTxStatus=0 syncs=[com.sun.ejb.containers.ContainerSynchronization@6d65042c] <- RWLock[Node[132736]] <- JavaEETransactionImpl: txId=18 nonXAResource=null jtsTx=com.sun.jts.jta.TransactionImpl@70faf570 localTxStatus=0 syncs=[com.sun.ejb.containers.ContainerSynchronization@6d65042c] <- RWLock[Node[6]]
at org.neo4j.kernel.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:219)
at org.neo4j.kernel.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:247)
at org.neo4j.kernel.impl.transaction.RagManager.checkWaitOn(RagManager.java:186)
at org.neo4j.kernel.impl.transaction.RWLock.acquireWriteLock(RWLock.java:321)
at org.neo4j.kernel.impl.transaction.LockManager.getWriteLock(LockManager.java:130)
at org.neo4j.kernel.impl.core.NodeManager.acquireLock(NodeManager.java:737)
at org.neo4j.kernel.impl.core.RelationshipImpl.delete(RelationshipImpl.java:141)
at org.neo4j.kernel.impl.core.RelationshipProxy.delete(RelationshipProxy.java:50)
at com.tinkerpop.blueprints.pgm.impls.neo4j.Neo4jGraph.removeEdge(Neo4jGraph.java:314)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService.deleteCollection(BluePrintsBackedFinderService.java:355)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService.doDelete(BluePrintsBackedFinderService.java:279)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService.access$300(BluePrintsBackedFinderService.java:56)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService$2.doPerform(BluePrintsBackedFinderService.java:256)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService$2.doPerform(BluePrintsBackedFinderService.java:252)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService$TransactionalOperation.perform(BluePrintsBackedFinderService.java:72)
at com.dooapp.gaedo.blueprints.BluePrintsBackedFinderService.delete(BluePrintsBackedFinderService.java:252)

As our application do not directly deal with neo4j nodes, I can't even have the slightest idea of what are the affected nodes.

So, here finally come my question : to your mind, what is the best way to deal with DeadlockDetectedException ? (both in termes of code modification to do to directly recover from them and ways to investigate how they happen and how to avoid them).

Thanks

[Mattias Persson]

An common way to recover from it is to retry transaction that encounters a deadlock, even a couple of times. Deadlocks can happen when multiple threads (transactions) modifies the same entities (nodes or relationships) at the same time, especially if they modify (i.e. lock or try to lock) them in different order. Modifying the entities in the same order in transactions will also prevent this from happening.

2012/11/16 Nicolas Delsaux <nicolas...@gmail.com>

--
 
 

--
Mattias Persson, [mat...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com