What makes WT need reconciliation compared to Innodb ?

57 views
Skip to first unread message

bai...@gmail.com

unread,
Feb 3, 2021, 6:42:41 AM2/3/21
to wiredtiger-users
Hi.
I try to find the original source about `reconciliation` in database fields, but no result (Obviously [1] is not what WT means). So,

1. Who was the first one that coined the term ?
2. Compared to Innodb, why does WT need reconciliation ? (I guess Innodb page is fixed and just synced to disk pages, whereas WT's page is not)

Keith Bostic

unread,
Feb 3, 2021, 11:45:42 AM2/3/21
to wiredtiger-users
On Wednesday, February 3, 2021 at 3:42:41 AM UTC-8 bai...@gmail.com wrote:
Hi.
I try to find the original source about `reconciliation` in database fields, but no result (Obviously [1] is not what WT means). So,

1. Who was the first one that coined the term ?

Not really sure, and I'm not sure if it's unique to WiredTiger or not.
 
2. Compared to Innodb, why does WT need reconciliation ? (I guess Innodb page is fixed and just synced to disk pages, whereas WT's page is not)

Database engines have more-or-less differences in the on-disk representation vs. the in-memory representation of the data. Some need make only small changes when writing cached data to disk, others (like WiredTiger), have in-memory representations that very different from their on-disk representations. What WiredTiger calls "reconciliation" is simply the process of converting the in-memory representation to the on-disk representation.
 

bai...@gmail.com

unread,
Feb 3, 2021, 10:12:07 PM2/3/21
to wiredtiger-users
Thanks for replying. 

Then what's the benefit of making in-memory representation differ a lot from on-disk representation ?  I guess reducing storage overhead may not be its goal, since compression does the job.

I search the git commit history of wiredtiger project,  and find reconciliation could be dated back to 2010. (https://github.com/wiredtiger/wiredtiger/commit/6782f0ba0)
So it seems to be an intrinsic thing in developer's mind since its beginning. What fundamental thing that make reconciliation an essential part of wiredtiger ?

baoshang young

unread,
Feb 4, 2021, 2:17:09 AM2/4/21
to wiredtiger-users
I think it maybe gets better concurrency to make in-memory representation differ from on-disk representation. In a memory page, it has linkedlists and skiplists. Writer will not block reader for the same key.

Keith Bostic

unread,
Feb 4, 2021, 10:57:20 AM2/4/21
to wiredtiger-users
On Wednesday, February 3, 2021 at 11:17:09 PM UTC-8 zaor...@gmail.com wrote:
I think it maybe gets better concurrency to make in-memory representation differ from on-disk representation. In a memory page, it has linkedlists and skiplists. Writer will not block reader for the same key.

Yes, exactly right. 
Reply all
Reply to author
Forward
0 new messages