Questions about transactions

[Dmitri Shubin]

Hi,

Some questions about transactions in WiredTiger.

1. Can read in explicit or in implicit transaction return WT_DEADLOCK?

2. Can write in implicit transaction return WT_DEADLOCK?

'Concurency Control' section (http://source.wiredtiger.com/2.3.1/transactions.html#transactions_concurrency) says "... concurrent transactions updating the same value will fail with WT_DEADLOCK. Some applications may benefit from application-level synchronization to avoid repeated attempts to rollback and update the same value."

Does it mean that even updating transactions cannot block on some synchronization primitive, but will return WT_DEADLOCK in case of contention?

(E.g. pthread_mutex_trylock() is used instead of pthread_mutex_lock()).

Thanks!

[Keith Bostic]

On Friday, September 5, 2014 8:06:05 AM UTC-4, Dmitri Shubin wrote:

 

1. Can read in explicit or in implicit transaction return WT_DEADLOCK?

2. Can write in implicit transaction return WT_DEADLOCK?

'Concurency Control' section (http://source.wiredtiger.com/2.3.1/transactions.html#transactions_concurrency) says "... concurrent transactions updating the same value will fail with WT_DEADLOCK. Some applications may benefit from application-level synchronization to avoid repeated attempts to rollback and update the same value."

There are two answers to this question, the theoretical answer and the practical one: theoretically, applications should anticipate that WT_DEADLOCK can be returned from any operation, it insulates the application from future code changes in the engine, and, of course, there's always the possibility of an unexpected code path in the engine.  Practically speaking, the only place where a WiredTiger application will see deadlock returns is if there's an obvious collision between two transactions updating a value.

 

Does it mean that even updating transactions cannot block on some synchronization primitive, but will return WT_DEADLOCK in case of contention?

(E.g. pthread_mutex_trylock() is used instead of pthread_mutex_lock()).

Updating transactions in WiredTiger shouldn't block on synchronization primitives -- WiredTiger isn't a locking engine underneath, there is serialization going on at some points, but there's no blocking in read/write paths.

[Dmitri Shubin]

On Friday, September 5, 2014 4:25:42 PM UTC+4, Keith Bostic wrote:

There are two answers to this question, the theoretical answer and the practical one: theoretically, applications should anticipate that WT_DEADLOCK can be returned from any operation, it insulates the application from future code changes in the engine, and, of course, there's always the possibility of an unexpected code path in the engine.  Practically speaking, the only place where a WiredTiger application will see deadlock returns is if there's an obvious collision between two transactions updating a value.

Ok, I see.

So this sentence

"WiredTiger uses optimistic concurrency control algorithms. This avoids the bottleneck of a centralized lock manager and ensures transactional operations do not block: reads do not block writes, and vice versa."

means that instead of blocking WT_DEADLOCK is returned in case of collisions.

At first I thought it means that reads are totally lock-free (like in LMDB).

Thanks!

[Keith Bostic]

On Friday, September 5, 2014 9:26:44 AM UTC-4, Dmitri Shubin wrote:

 

means that instead of blocking WT_DEADLOCK is returned in case of collisions.

At first I thought it means that reads are totally lock-free (like in LMDB).'"

Reads are lock-free -- WiredTiger is an MVCC engine inside, so there's no reason for a read to block.

[Martti Laiho]

Hi all, as a member of DBTechNet.org of database teachers I'm writing tutorial on MongoDB Transactions at https://drive.google.com/file/d/1cp0v04Vc24HFiBp6hrF4DLlLKCyup-vs/view?usp=drive_link.

I hope that somebody has time to read if the WiredTiger part  of the text is OK in "big picture" level.

All comments and corrections are most welcome

[Will Korteland]

Hi Martti,

Thanks for your interest in WiredTiger. There are a few issues with the document as-is, this is what we spotted after a quick review:

> SI transaction cannot see new records inserted by concurrent transactions after its start timestamp, the phantoms are possible

I don’t think this is correct - phantom reads should not be possible under snapshot isolation, only read-uncommitted and read-committed.

> Latest committed records of interest in the document collections in databases are always loaded first for reading or manipulating in the WiredTiger shared cache of Mongod server.

I don’t quite understand this - are you talking about pre-loading records into the cache?

> For record update a new version of the record is created stamping it with the transactionID … and current timestamp, and the old version is moved into History Store set of records supporting snapshots of concurrent transactions.

The record is only moved to the history store once reconciliation is performed (either as part of checkpoint, or eviction).

> delta collection

This term isn’t familiar to me - it makes sense, but might it be worth clarifying that it’s not a term WiredTiger uses internally.

For figure 3 and 4: step 4 is part of reconciliation, as above.

For figure 4: I think having the delete and insert flows on the same diagram is a little confusing, so it was hard to review. But I think step 2 is wrong - if we're deleting we don’t do a copy, we mark it as deleted.

> On WiredTiger transaction commit phase the manipulated records in the snapshot delta will be flushed into the collection in the MongoDB database and their latest versions in the cache will be visible to concurrent transactions.

There’s a few misunderstandings here:

• Data is only made durable in the commit phase if using commit durability, but inserting the new record into the collection is part of reconciliation.

• It’s not just the “latest version in the cache”, but the latest version outright - if it gets evicted, we’ll read it back into the cache.

• "visible to concurrent transactions": this is only partly true, it’s still subject to visibility rules.

> WiredTiger does checkpoint operation by default every 60 seconds by internal transaction

This isn’t wrong per se, but the word "transaction" implies something else, I’d probably use the word “thread” here.

> On WiredTiger transaction rollback the whole snapshot delta set of records is simply removed from the Cache.

This isn’t correct - there's an aborted flag that gets applied to those records but they remain in the cache until the next reconciliation.

Hopefully that helps!

- Jie Chen and Will Korteland

--
You received this message because you are subscribed to the Google Groups "wiredtiger-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wiredtiger-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/65bf9040-65cf-4265-9102-3c2c5c342b2an%40googlegroups.com.