WARNING: Found record in transaction with the same RID but different instance

[mike.o...@fpx.com]

Here is an interesting item. We have a test program that basically builds an AVL tree in the database and does insert and delete operations on that tree. When running this test we are constantly seeing the following warning message in the output:

com.orientechnologies.common.log.OLogManager log
WARNING: Found record in transaction with the same RID #12:18 but different instance. Probably the record has been loaded from another transaction and reused on the current one: reload it from current transaction before to update or delete it 

However, we are currently only running one thread with a single transaction active at a time. A single transaction will group inserts and deletes together, and the app is designed to have multiple threads (and we see it when multiple threads are active as well), but it still shows up with only a single thread.

The following snippet is the logic we use for querying. We have a Long in each node that is indexed. We get the vertex RID from the index based on the id we want, then get the full record for that.

private static ODocument getNode(final OGraphDatabase db, final Long nodeId) throws InvalidTreeException {

    OIndex index = db.getMetadata().getSchema().getClass("Node").getClassIndex("Node.id");

    return (ODocument) db.getRecord((OIdentifiable) index.get(nodeId.toString()));

} 

Does db.getRecord() not act within a transaction?

[mike.o...@fpx.com]

Forgot to add that is currently running against the 1.5.0 release.

[Artem Orobets]

Hi Mike,

Could you try 1.6.0-SNAPSHOT. This bug has been fixed there.

Best regards,

Artem Orobets

Orient Technologies

the Company behind OrientDB

2013/8/15 <mike.o...@fpx.com>

Forgot to add that is currently running against the 1.5.0 release.

--
 
---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[mike.o...@fpx.com]

I'm now running this against the latest 1.6.0-SNAPSHOT (using Gradle, just updated the dependency from Sonatype today) and I still see this issue.

[mike.o...@fpx.com]

We are still seeing this issue even with an updated 1.6.0-SNAPSHOT (updated at 4:15am today). It doesn't seem to impact the system at all, but the console is constantly output this message.

I figure we're doing something to cause it, as we have other code that interacts with Orient that does not produce this message. 

[Andrey Lomakin]

Hi,

Do you use server ?

Do you have this error logs in server console ?

--

Best regards,
Andrey Lomakin.

[Andrey Lomakin]

One more question,

Do you have intermediate TX commit messages in console ?

Or you create you scheme before tests are executed ?

[Sylvain Spinelli]

>Does db.getRecord() not act within a transaction?

No, the record is only keep in the transaction (until commit) when you save (create or update) or delete it.

If Cache Level 1 is enabled, you will often get the same instance when you load record twice, but not always : if memory is low, cache level 1 is cleared and 2 instances with the same RID can be referenced. This cache behavior is an important source of randomly bug.

Note 1: When you test your code, you should also test with Cache level 1 disabled (memory cleared simulation).

Note 2: In my Orient version I add a method IDatabase.keepRecordInTransaction(pRec) to ensure the instance is definitively referenced by the transaction as soon as I load it. It's really necessary when you update complex graphs.

Note 3: I know only Orient 1.3 and lower versions. Perhaps something has changed in newer versions.

2013/9/15 Andrey Lomakin <lomakin...@gmail.com>

[mike.o...@fpx.com]

Right now we are just running it local, using plocal as the storage type (although we've seen the warning with both local and plocal). We're still developing the code, and it's easier to test in local mode. We are currently updating to the latest snapshot and will test with a server this morning.

We see no other messages in the log/console.

We have a bootstrap process that drops and creates the database on it's own connection when the process starts, and then creates the schema on another separate connection. Each thread will also use it's own, separate connection. We setup the bootstrap process that way as we branch for local/remote connections. Right now were are not using a pool for the threads.

[mike.o...@fpx.com]

All of our tests deal with multiple threads, so as a rule we always turn off the level 1 cache; too many problems otherwise.

What does your keepRecordInTransaction method do?

What's confusing me is we have other code that does not produce this warning, and while they are separate codebases we aren't seeing any functional difference between them.

[Sylvain Spinelli]

>What does your keepRecordInTransaction method do?

Here is an example (all is done in the same thread and one transaction) :

db.begin();

ODocument doc1 = getNode(id1);
doc1.set(...);

[...dispatch events... and imagine you have in an other method : ]

     ODocument doc2 = getNode(id1); // doc1 and doc2 are different instances
     doc2.set(...);
     doc2.save()
[end dispatch events]

doc1.save(); //WARNING... and updates in doc2 are LOST.

db.commit();

WITH db.keepRecordInTransaction() :

db.begin();

ODocument doc1 = getNode(id1);
db.keepRecordInTransaction (doc1);
doc1.set(...);

[...dispatch events... and imagine you have in an other method : ]

     ODocument doc2 = getNode(id1); // doc1 and doc2 reference the SAME instance
     doc2.set(...);
     doc2.save()
[end dispatch events]

doc1.save(); // NO WARNING : the same instance is save twice in the transaction.

db.commit(); // updates in the same instance (via doc1 and doc2 variables) are correctly commited.

2013/9/16 <mike.o...@fpx.com>

[Andrey Lomakin]

HI

All of our tests deal with multiple threads, so as a rule we always turn off the level 1 cache; too many problems otherwise.

1-st level cache was redeveloped.

Could you check the latest version with 1-st level cache switched off.

1-st level cache prevents to have several instances of the same record and as result no concurrent conflicts if two instances were loaded..

What does your keepRecordInTransaction method do?

This behavior is implemented in 1.6-SNAPSHOT.

[Andrey Lomakin]

On Mon, Sep 16, 2013 at 5:43 PM, Andrey Lomakin <lomakin...@gmail.com> wrote:

ould you check the latest version with 1-st level cache switched off.

Could you check the latest version with 1-st level cache switched on.

Sorry for typo.

[Sylvain Spinelli]

> This behavior is implemented in 1.6-SNAPSHOT.

Cool !

[mike.o...@fpx.com]

The warning was still displayed with level 1 cache on, and we ran into a lot of varied Exceptions as well. That might point to some logic issues in our code, and we have identified a potential cause based on Sylvain's posts as our logic could load the same record into two different variables.

We ran into a separate issue with indexes trying to run our code against a server, so I'm going to post a separate issue about that.

[Andrey Lomakin]

On Mon, Sep 16, 2013 at 6:18 PM, <mike.o...@fpx.com> wrote:

The warning was still displayed with level 1 cache on, and we ran into a lot of varied Exceptions as well. That might point to some logic issues in our code, and we have identified a potential cause based on Sylvain's posts as our logic could load the same record into two different variables.

I am looking forward for your updates after code verification.

It would be good to fix this issue  till 1.6 will be released.

[mike.o...@fpx.com]

I think we've got this narrowed down. We're going to try to replicate the issue in a testcase, but the flow looks like this:

We have a tree (doesn't matter, but it's an AVL tree implementation) made of vertices and edges. As part of a transaction we are deleting an edge between two nodes and creating an edge between a different set of nodes. We are using plocal so that RIDs are not reused, but the newly created edge is getting the same RID has the just deleted edge. This produces that warning as we now have two edges in the transaction with the same RID, but are different instances. 

I thought plocal did not reuse RIDs. 

[mike.o...@fpx.com]

We've created code that reproduces this issue. I've created a Gist for it at https://gist.github.com/mikeosterlie/f2c557aadb54370623da. When the transaction is committed the warning is shown.

[Andrey Lomakin]

Hi Mike,

If you disable 1-st level cache you will have these exception because it is main responsibility of given cache to prevent such cases.

You wrote that you got exceptions when you enbable 1-st level cache could you send them ?

--
 
---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[mike.o...@fpx.com]

We adjusted the test code to explicitly turn on the level 1 cache and the warning is still shown.

We tried again in the main code and now we're not seeing any difference (no additional exceptions). We did do a bit of refactoring and also updated again to the latest SNAPSHOT (dated this morning), so that may have changed the behavior. So now, however, we are seeing no difference in output with level 1 cache enabled or disabled. The warning is shown either way.

[Shishya]

Hi,

I see such warnings in 1.7rc1 as well.

Does it impact any data integrity?

Regards

[Artem Orobets]

Hi Shlshya,

Do you see the warnings with enabled 1st level cache?

Could you provide a test case to let me reproduce the case?

Best regards,

Artem Orobets

Orient Technologies

the Company behind OrientDB

For more options, visit https://groups.google.com/d/optout.

[prabhat]

Hi,

My Level-1 cache was off. But I tested with v1.7rc2 and there are no warnings now

Thanks

Prabhat Kumar Singh

--

---
You received this message because you are subscribed to a topic in the Google Groups "OrientDB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/orient-database/TA0voVLymgk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to orient-databa...@googlegroups.com.