MemcacheDB benckmark

[Akinobu Mita]

Hello,

I tried to do the same benchmark with http://memcachedb.org/benchmark.html

Steve's benchmark on thread edition shows:
- key: 16 value: 100B, 8 concurrents, every process does 2,000,000
set

23564 w/s

- key: 16 value: 100B, 8 concurrents, every process does 2,000,000
get

64257 r/s

But I could not get enough performance.

- 13343 w/s
- 12733 r/s

Is there anything wrong that I did? Here is the detail of my
benchmark.

Software versions:
- memcachedb: svn revision 98 (--enable-threads)
- db-4.8.24
- libevent-1.4.13-stable

Machine spec:
- 4GB memory
- Intel Core 2 Quad CPU Q9550 2.83GHz

Log:
$ memcachedb -r -u mita -H /home/mita/data1 -N -t 4 -v

$ ./mcben.py --threads=8 --requests=2000000 --length=100 --command=SET

Benchmarking (be patient).....
Thread name: Thread-6; Time cost: 1198.150774 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-2; Time cost: 1198.624658 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-4; Time cost: 1198.881415 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-7; Time cost: 1198.852324 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-3; Time cost: 1199.308162 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-8; Time cost: 1199.165258 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-5; Time cost: 1199.527734 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-1; Time cost: 1200.012967 seconds; Requests:
2000000; Errors: 0
done.

Server name: 127.0.0.1:21201
Command: SET
Thread number: 8
Requests per thread: 2000000
Value Length: 100 bytes
Avg. time cost per thread: 1199.065412 seconds
Throughout: 1511 kbytes/sec
Requests per second: 13343 req/sec
Time cost per request: 0.074942 ms
Total requests: 16000000
Total errors: 0

$ ./mcben.py --threads=8 --requests=2000000 --length=100 --command=GET

Benchmarking (be patient).....
Thread name: Thread-8; Time cost: 1255.482045 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-2; Time cost: 1256.266260 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-1; Time cost: 1256.829289 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-6; Time cost: 1256.459424 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-4; Time cost: 1256.922189 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-5; Time cost: 1256.753656 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-7; Time cost: 1256.720970 seconds; Requests:
2000000; Errors: 0
Thread name: Thread-3; Time cost: 1257.156268 seconds; Requests:
2000000; Errors: 0
done.

Server name: 127.0.0.1:21201
Command: GET
Thread number: 8
Requests per thread: 2000000
Value Length: 100 bytes
Avg. time cost per thread: 1256.573763 seconds
Throughout: 1442 kbytes/sec
Requests per second: 12733 req/sec
Time cost per request: 0.078536 ms
Total requests:

[Steve Chu]

hi,

mcben.py is just a toy that written with python, while the case in the
benchmark page is using libmemcached.

> --
>
> You received this message because you are subscribed to the Google Groups "memcachedb" group.
> To post to this group, send email to memca...@googlegroups.com.
> To unsubscribe from this group, send email to memcachedb+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/memcachedb?hl=en.
>
>
>



--
Best Regards,

Steve Chu
http://stvchu.org

[Gregory Burd]

Which version of Berkeley DB were you using? Berkeley DB 4.8 performs
much better on multi-core/multi-cpu tests with many threads.

-greg

[Akinobu Mita]

On Nov 28, 4:31 am, Gregory Burd <greg.b...@oracle.com> wrote:
> Which version of Berkeley DB were you using?  Berkeley DB 4.8 performs
> much better on multi-core/multi-cpu tests with many threads.

The version is db-4.8.24 but I used mcben.py as Steve said it is just
toy.
So I tried again with libmemcached and I got good performance.

38717 w/s
136938 r/s

Here's the benchmark program

#!/bin/sh

spawn()
{
time $@ &
}

SERVER=127.0.0.1:21201

THREADS=8
REQUESTS=2000000
VALUE_LENGTH=100

OPT="-s $SERVER -n $REQUESTS -l $VALUE_LENGTH"

export TIMEFORMAT='%R'

for i in `seq $THREADS`;do
spawn /home/mita/memcachedb-benchmark $OPT -i 0 -w
done

wait
sync; sync; sync

for i in `seq $THREADS`;do
spawn /home/mita/memcachedb-benchmark $OPT -i 0 -r
done

wait

/*
* memcachedb-benchmark
*/

#include <err.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <getopt.h>
#include <libmemcached/memcached.h>

static void do_memcached_set(memcached_st *memc, const char *key,
size_t key_length, const char *value, size_t value_length)
{
memcached_return ret;

ret = memcached_set(memc, key, key_length, value, value_length, 0,
0);
if (ret != MEMCACHED_SUCCESS) {
warn("memcached_set: %s", memcached_strerror(memc, ret));
}
}

static void do_memcached_get(memcached_st *memc, const char *key,
size_t key_length)
{
memcached_return ret;
char *value;
size_t value_length;
uint32_t flags;

value = memcached_get(memc, key, key_length, &value_length,
&flags, &ret);
if (ret != MEMCACHED_SUCCESS)
warn("memcached_get: %s", memcached_strerror(memc, ret));

free(value);
}

static int id;

static void test(const char *server, const int value_length,
const int requests, const int command)
{
char key[20];
char *value;
struct memcached_st memc;
struct memcached_server_st *servers;
memcached_return ret;
int i;

value = malloc(value_length);
if (!value)
err(EXIT_FAILURE, "out of memory");
memset(value, '*', value_length);

memcached_create(&memc);
servers = memcached_servers_parse(server);
ret = memcached_server_push(&memc, servers);

if (ret != MEMCACHED_SUCCESS)
err(EXIT_FAILURE, "memcached_server_push failed");

for (i = 0; i < requests; i++) {
sprintf(key, "%04d-%011d", id, i);

if (command == 'w')
do_memcached_set(&memc, key, 16, value, value_length);
else
do_memcached_get(&memc, key, 16);
}

memcached_server_list_free(servers);
memcached_free(&memc);
}

static int value_length = 100;
static int requests = 100000;
static char *server = "127.0.0.1:21201";
/* 'r' for Read, 'w' for Write request */
static int command = 'w';

static void parse_options(int argc, char **argv)
{
int c;

while ((c = getopt(argc, argv, "n:k:l:s:i:rw")) != -1) {
switch(c) {
case 'n':
requests = atoi(optarg);
break;
case 'l':
value_length = atoi(optarg);
break;
case 's':
server = optarg;
break;
case 'i':
id = atoi(optarg);
break;
case 'r':
command = 'r';
break;
case 'w':
command = 'w';
break;
default:
err(EXIT_FAILURE, "invalid option: %c", c);
}
}
}

int main(int argc, char **argv)
{
parse_options(argc, argv);

test(server, value_length, requests, command);

return 0;
}

[Greg Burd]

Akinobu,

Excellent, thanks for testing again. Can you tell me more specifics about your platform/hardware/os/etc.? How does this compare with other solutions? With Memcached? Are you going to do a system test?

Thanks again,

-greg

[Akinobu Mita]

On Nov 30, 11:02 pm, Greg Burd <GREG.B...@oracle.com> wrote:
> Akinobu,
>
> Excellent, thanks for testing again.  Can you tell me more specifics about your platform/hardware/os/etc.?  How does this compare with other solutions?  With Memcached?  Are you going to do a system test?

The system is:

- 4GB memory
- Intel Core 2 Quad CPU Q9550 2.83GHz

- CentOS 5.4 (2.6.18-128.el5)
- x86_64
- Ext3 filesystem on LVM
- SATA

I have not compared with other solutions yet.

We're now developing a distributed computing system that requires
persistent key-value storage. So we tried MemcacheDB.