memcachedb performance at 50 million for random set operations terrible
tom gogii
gets to about 50 million records. I'm using a hash database backing in
the BDB.
my startup:
memcacheDB -m 10000 -b 4096 -N -A 4096 -B hash
my keys are pretty simple digits 1 to 50 million.
Random/Sequential Read performance is fast 100 milliseconds on a miss,
then 1 millisecond.
Sequence write are fast, I get about 1000 inserts/s.
However, if I try doing any sort of random set operations for even a
small range from 1 to 20,000, my random set operations speed is only
30 sets/second. I've tried multithread and single threaded, and my
memcache set operations timeout frequently.
Looking on the server, iotop tells me that disk IO is pretty terrible,
and the server load is roughly 6.
Anyone have any suggestions? Perhaps 50 million records is too many
for memcacheDB with the berkely backend for random updates.
Steve Chu
Have you tried to do some tuning? Such as changing -L to 8192 and -T to 15.
50millions is not a small number for single daemon, have you think
about partition? hash your data into 4-8 daemons.
> --
> You received this message because you are subscribed to the Google Groups "memcachedb" group.
> To post to this group, send email to memca...@googlegroups.com.
> To unsubscribe from this group, send email to memcachedb+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/memcachedb?hl=en.
>
>
--
Best Regards,
Steve Chu
http://stvchu.org