Tokyo Cabinet B+ Tree Tunning ...over a billion records

93 views
Skip to first unread message

Harsh Agarwal

unread,
May 14, 2014, 9:44:41 PM5/14/14
to tokyocabi...@googlegroups.com

I am using a C implementation of Tokyo Cabinet and implemented a B+ Tree.

I have not tuned the parameters.
I have only set Database size to large (BDBTLARGE)..using  tcbdbtune 
i.e. tcbdbtune( variable, 0, 0, 0, -1, -1, BDBTLARGE);
My performance chokes at about 1,497,600,000 records.

But since I am using default parameters
bucket size(no of pages) is defaulted to 32,749.
Total records in leaf node (r-leaf)= 128
Total Records in non leaf nodes (r-non-leaf) = 256.

Total No of Pages(Nodes)  n = 32,749
Assuming it is a complete binary tree.
Hence if leaf is no of leaf nodes
2* leaf - 1 = 32,749.
...
Hence no of leaf nodes  (leaf) is 16375.
No of non leaf nodes  (non-leaf) is 16374.
 

Hence the B+ tree should be saturated when x records are stored.
where 
x= leaf*(r-leaf) + (non-leaf) * (r-non-leaf)
   = 16375*128 + 16374*256 
   = 6,287,744


Hence as per my calculations the database should have choked at around 6 million records and not at a billion records.

Can someone please explain what happened so that i can tune accordingly...
Thanks a lot...



Reply all
Reply to author
Forward
0 new messages