Status of 2pc effort

[Yuri Finkelstein]

Hello,
I’m curious about the status of 2pc branch in git. It looks like some significant effort was made to expose transaction control api via iproto and even implement global transactions across shards based on tests that I see there.
Is there some writeup on the design and the approach and on how this will all fit ? Presumably the router becomes global transaction coordinator. But there should be more to it:
2 phase locking?
Transaction side effects need to become visible only after commit decision which means transaction should stay in prepared and locked state until then yet remain durable (written to WAL) and prepared in memory?
What kind of isolation level is targeted ? Is serializable level achievable?
Would appreciate any additional info.
Thanks
Yuri

[Konstantin Osipov]

Hi,

To proceed forward with this branch we need:

- agree on extension of the write ahead log format. The log should
be able to contain prepare records. We're synchronizing this
effort with our work on raft support

- work on a common mvcc transaction manager for vinyl and memtx. right
now memtx uses a very naive transaction manager when each
transaction runs in isolation on each node until it commits. If
it's a 2pc transaction, it has to block the scheduler on this
node until a distributed commit is achieved.
This in turn implies:
* rewriting memtx memory allocator to support multiple
consistent read views, right now it supports only 1, which
is used by the snapshot thread periodically
* making begin/commit messages available in the binary
protocol, which in turn implies

- implementing cluster-wide monotonic but cheap transaction
identifiers for distributed transactions.

The list goes on and on actually. Many of issues from this list
are in progress, namely, raft and extension of the write ahead log
is our top priority and @georgyk has written a draft patch for it.

To sum up, this was an experimental exploration branch, which we
used to come up with a more step-by-step plan of implementing this
feature in midterm future.

* Yuri Finkelstein <yuri...@gmail.com> [18/12/21 20:21]:

--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov

[Yuri Finkelstein]

Ok, thanks for quick and detailed response.

I was thinking that the work in vinyl to support 2pc would be less than that in memtx because you already have most of the plumbing. Rocksdb has a good design page on 2pc and it’s clear that 2pc is a relatively small increment if one uses pessimistic locking.

Just one more question. Where is RAFT work being performed? I can only see the word RAFT mentioned in several github issues but not in code.

Get Outlook for iOS

 

---

From: tara...@googlegroups.com on behalf of Konstantin Osipov <kostja...@gmail.com>
Sent: Friday, December 21, 2018 10:16 AM
To: tara...@googlegroups.com
Subject: Re: Status of 2pc effort

 

--
You received this message because you are subscribed to the Google Groups "Tarantool discussion group (English)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tarantool+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Konstantin Osipov]

* Yuri Finkelstein <yuri...@gmail.com> [18/12/21 21:38]:

> Ok, thanks for quick and detailed response.
> I was thinking that the work in vinyl to support 2pc would be less than that in memtx because you already have most of the plumbing. Rocksdb has a good design page on 2pc and it’s clear that 2pc is a relatively small increment if one uses pessimistic locking.
>
> Just one more question. Where is RAFT work being performed? I can only see the word RAFT mentioned in several github issues but not in code.

remotes/origin/g.kirichenko/synchro-draft
remotes/origin/g.kirichenko/synchro-draft-2
remotes/origin/g.kirichenko/wal.draft