Tokyo Cabinet Performance Issue

[Hi Guy]

Hi,

I have more than 100 million key-value pairs (one key can have multiple values). I am using Tokyo Cabinet's BDB (B+Tree DB) whose key, values are Integers (32 bit byte-array). I have used following tuning parameters:

bdb.tune(256 , 512 , 10000000 , 12 ,16 , BDB.TLARGE)

bdb.setcache(2048 , 1024 )

bdb.setxmsiz(1024*1024*2)

bdb.setcmpfunc (BDB.CMPINT32)        [all of them gives true]

while running using: -Xms 2.5 GB -Xmx 2,5GB to JVM.

Also, run with varying bnum and also with default setting. However, it seems no performance gain. Even if, to store just 1 million key-value pair it takes more than 5 minutes. Another thing,  varying bdb.setxmsiz does not help also. It is completely against of their benchmark. Can anybody help me what is going wrong.

[nmmm]

bit off-topic , but I am interested how you do backup of your
database?

[Sander]

My feeling is that their benchmarks are valid only if a good part of the database gets cached by the operating system in RAM.

However, there is one thing that you might try.

Your cache setting is rather small. And I had an experiment, when giving a GB of cache for a tree database (on kyotocabinet) improved the performance a lot