[pmemkv] cmap performance

[Pradeep Fernando]

Hi All,

I ran pmemkv-tool's db-bench with pmemkv/cmap engine and obeserved random-fills with

12 us update cost. 

I use Intel pmem device with a DAX file-system. I have given the full output at the end of this text. 

Is the numbers are faithful/consistent with the numbers you have seen?

In a different test, I directly use the concurrent hash-map code found in libpmemobj-cpp repo to create a hashmap with <unsigned long, pstring<32>>. For this particular hashmap,

I'm seeing  < ~1 us update costs ( which is drastically cheaper than pmemkv reported numbers). Can there be such drastic difference in update performance because of

hashmap key data type ? ( pstring<8> to unsigned long ). 

Hence I wan't to cross check the validity of my performance numbers.

thanks in advance,

--Pradeep

[pfernando@aep01 pmemkv-tools]$ PMEM_IS_PMEM_FORCE=1 ./pmemkv_bench --db=/mnt/pmem1/pmemkv --db_size_in_gb=1 --key_size=8 --value_size=32 --readwritepercent=50  --engine=cmap
Date:       Thu Aug  8 08:21:13 2019
CPU:        96 * Genuine Intel(R) CPU 0000%@
CPUCache:   33792 KB
Path:       /mnt/pmem1/pmemkv
Engine:     cmap
Keys:       8 bytes each
Values:     32 bytes each
Entries:    1000000
RawSize:    38.1 MB (estimated)
------------------------------------------------
open         :     180.096 millis/op;
readrandomwriterandom :       6.120 micros/op 163406 ops/sec;    4.3 MB/s ( reads:500000 writes:500000 total:1000000 found:106865)
removed      :       2.424 millis/op;
open         :      52.931 millis/op;
fillrandom   :      12.350 micros/op 80974 ops/sec;    3.1 MB/s
overwrite    :       6.496 micros/op 153938 ops/sec;    5.9 MB/s
removed      :       1.838 millis/op;
open         :      53.409 millis/op;
fillseq      :      15.219 micros/op 65707 ops/sec;    2.5 MB/s
readrandom   :       2.491 micros/op 401430 ops/sec;   15.3 MB/s (1000000 of 1000000 found)
readseq      :       1.703 micros/op 587297 ops/sec;   22.4 MB/s (1000000 of 1000000 found)
readrandom   :       2.052 micros/op 487307 ops/sec;   18.6 MB/s (1000000 of 1000000 found)
readmissing  :       2.059 micros/op 485771 ops/sec;   18.5 MB/s (1000000 of 1000000 found)
readrandom   :       2.053 micros/op 487136 ops/sec;   18.6 MB/s (1000000 of 1000000 found)
deleteseq    :      10.331 micros/op 96800 ops/sec;

[Sergei Vinogradov]

Hi Fernando,

The numbers you get with pmemkv_bench utility looks correct. But you did apple to orange comparison when you run test for concurrent_hash_map<unsigned long, pstring<32> >.

In the Cmap engine we use concurrent_hash_map<pmem::obj::string, pmem::obj::string>. And pmem::obj::string is a variable-length string (that internally allocates char buffer) while pstring<32> is a fixed-length string (that internally hash just char[32] array, e.g. there is no buffer allocation). Cmap engine targeted for more generic use case rather than fixed size value. 

So, for fair comparison you should compare against concurrent_hash_map<pmem::obj::string, pmem::obj::string>. If your use case assume fixed size values and unsigned long keys than you can get performance benefit by using concurrent_hash_map<unsigned long, pstring<32> > directly.

Let us know if you need more clarifications.

четверг, 8 августа 2019 г., 20:15:57 UTC+3 пользователь Pradeep Fernando написал:

[Pradeep Fernando]

Thanks for the clarification Sergei!.

--Pradeep

--
You received this message because you are subscribed to the Google Groups "pmem" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmem+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pmem/d2d9f9ea-bb00-42d9-bca2-005921d1b42a%40googlegroups.com.

--

Pradeep Fernando.