Lost in "unable to commit transaction" errors, during move from 1.9.3 -> 2.0.0 (now all reads require transactions)

[M. David Allen]

As I'm updating code for 2.0.0, I'm wrapping a lot of old code that only serves to inspect a graph (not update it) in transactions, using the new idiom:

try ( Transaction tx = myDb.beginTx() ) {
   accessSomeData();
   tx.finish();
}

After the try block finishes, I'm getting exceptions of this form:

org.neo4j.graphdb.TransactionFailureException: Unable to commit transaction
    at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:134)
    at blah.blah.mycode

Caused by: javax.transaction.RollbackException: Failed to commit, transaction rolled back
    at org.neo4j.kernel.impl.transaction.TxManager.rollbackCommit(TxManager.java:623)
    at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:402)
    at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:122)
    at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:124)
    ... 70 more

What are the various causes of this, and how can I troubleshoot them?

This is all code that ran without any problem on 1.9.3 - so I'm thinking I should look into areas of difference there.

Sometimes this happens when iterating over the results of executing a cypher query from java.  Sometimes it happens when I'm using a TraversalDescription I built.

Strangely enough, since these are read-only operations, I can *ignore* the failure exception, and everything seems peachy (the data came back from the graph database just fine).   I'm just wondering why they're happening.

Any suggestions or pointers?

[Johannes Mockenhaupt]

You need to call tx.success() at the end of the block rather than
tx.finish(). The latter is done through the try-with-resources statement.
For further details see the JavaDoc of the Transaction class, which
explains this nicely with an example.

> --
> You received this message because you are subscribed to the Google
> Groups "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to neo4j+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[M. David Allen]

Sorry, this was my mistake on the original post -- I'm not calling tx.finish() at all, but rather tx.success() at the end of the try block.

The original question stands though - what are the various reasons why a transaction would fail when nothing inside of the transaction is modifying the graph in any way?   E.g. when it says "failed to commit" -- exactly what is being committed in a set of operations that theoretically should all be read-only?

[Johannes Mockenhaupt]

Now that I think about it again, why should a call to tx.success() be
needed at all on a r/o tx? Default is to not commit and rollback. Only
close() needs to be called to clean up resources. Maybe Neo4j throws
because you want it to commit something when there's nothing to commit
;-) Seriously though, with or without a call to success(), a r/o
transaction shouldn't throw I think, so this might be a bug ...
hopefully someone from Neo can enlighten us.

> > an email to neo4j+un...@googlegroups.com <javascript:>.

> > For more options, visit https://groups.google.com/groups/opt_out

> <https://groups.google.com/groups/opt_out>.

[Mattias Persson]

Hi David,

unfortunately I think the root cause of the failure to commit is lost in the thrown exception. You can have a look in messages.log for the cause, or just attach you message.log here with a rough timestamp when this happened. When we dig up that root cause we can argue if it's strange and unexpected, or not :)

Btw, whether anything actually gets committed or not is dictated by the presence of write operations in there, but the code path in the transaction manager is the same.

2013/12/31 M. David Allen <allen....@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mattias Persson, [mat...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com

[M. David Allen]

So this was kind of my thinking as well.   I'll play with  messages and see if I can get more information to post on the source of the bug.

The other thing that's a big pain in the butt about calling success() is that it breaks a certain coding pattern.  I used to have code that looked like this, for example:

try (Transaction tx = db.beginTx()) {
   return new MyObject(myNode.getProperty("foo"), myNode.getProperty("bar"));
}

See the getProperty() call has to be wrapped in a transaction, so I can't just return something that calls getProperty(), instead I have to do this:

try (Transaction tx = db.beginTx()) {
  String foo = ""+myNode.getProperty("foo");
  String bar = ""+myNode.getProperty("bar");
  tx.success();
  return new MyObject(foo, bar)
}

For code like that, the old finally { tx.finish(); } block was a lot easier.

David

[Mattias Persson]

2014/1/2 M. David Allen <allen....@gmail.com>

So this was kind of my thinking as well.   I'll play with  messages and see if I can get more information to post on the source of the bug.

Cool

The other thing that's a big pain in the butt about calling success() is that it breaks a certain coding pattern.  I used to have code that looked like this, for example:

try (Transaction tx = db.beginTx()) {
   return new MyObject(myNode.getProperty("foo"), myNode.getProperty("bar"));
}

See the getProperty() call has to be wrapped in a transaction, so I can't just return something that calls getProperty(), instead I have to do this:

try (Transaction tx = db.beginTx()) {
  String foo = ""+myNode.getProperty("foo");
  String bar = ""+myNode.getProperty("bar");
  tx.success();
  return new MyObject(foo, bar)
}

For code like that, the old finally { tx.finish(); } block was a lot easier.

True, although that pattern existed before as well, but not for pure read-transactions. You can actually leave out tx.success() for pure read-transactions since commit/rollback will not have any persistence effects either way, but it would introduce an assumption that might change later on.

[M. David Allen]

OK, so here's an exception stack for a failed transaction on a read-only set of operations, and attached is a full messages.log output.

I've reviewed the messages.log myself, and I honestly can't find anything useful in there.   Is there any additional setting I can use for an embedded database that will produce more information?

Finally, to keep it simple I'm using a test stub below.  I blew away the entire database (deleted directory from the disk) then ran a simple test stub which populates the database with a bunch of stuff (all of that went fine) and then does a few simple queries.   The transaction that fails does so AFTER successfully executing this cypher query, and doing various kinds of property reading.

start r=relationship:relationship_auto_index(workflow={wf}) return r limit 50                

{wf} is bound to a String value passed in as part of a HashMap<String,Object>.   After the cypher query executes, I just create a bunch of domain objects, which requires reading properties from the nodes that come back. 

Jan 03, 2014 9:51:42 AM org.mitre.provenance.db.neo4j.Neo4JStorage getMembers
SEVERE: Failed transaction: Unable to commit transaction

org.neo4j.graphdb.TransactionFailureException: Unable to commit transaction
    at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:134)

    at org.mitre.prov.db.neo4j.Neo4JStorage.getMembers(Neo4JStorage.java:503)
    at org.mitre.prov.test.Stub.main(Stub.java:25)

Caused by: javax.transaction.RollbackException: Failed to commit, transaction rolled back
    at org.neo4j.kernel.impl.transaction.TxManager.rollbackCommit(TxManager.java:623)
    at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:402)
    at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:122)
    at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:124)

    ... 2 more

Thanks,
David

[Craig Taverner]

I had similar problems and found it was due to nested read-only transactions where the inner tx did not call success(), but the outer one did. I blogged the details here: http://blog.amanzi.org/2014/08/neo4j-read-only-transactions-still-need.html. Perhaps this helps.