Hi
I have used your test setup to test performance of a very simple memcached replacement written in java. Thanks for finding time to write such a useful tool.
I have few suggestions:
1) In client server performance tests following metrics are important:
- cpu usage of the client
- cpu usage of the server
- network bandwidth consumed (IN and OUT)
- latency
If the test infrastructure could capture these details, it would much much easier to find out the bottleneck. That is client, server or network.
2) When I ran the tests, I couldn't saturate (CPU) of a single memcached instance. The server was a single 1.6Gz Quad core intel processor. I am not sure if you need three servers for this test. I was getting close to 80K TPS after few modification to the benchmark code.
3) All the non-blocking client libraries are using a single thread and single connection for IO. I could get about 50% higher TPS numbers by changing the client code to use 4 threads (connections) and assigning different connections to different client threads (non NIO threads).
4) Multiple threads in the test scenario seem to test only two things. One the ability of the client OS to do context switches fast enough and the ability of the client library to use mgets. I think just testing mget performance with less number of threads would give us a better comparison of client libraries.
5) Another thing orthogonal to the tests which I noticed was that memcached hit rate continues to deteriorate if flush_all is not called between different runs. Do I need to switch on slab reallocation?
thanks!
Rohit Karlupia