brutis: php memcache benchmarking tool

[zyou...@gear6.com]

I wrote a memcached benchmarking tool in PHP to help out with our QA
process. I thought it might be useful to other memcached users,
developers
and the community for sizing and regression. (Currently supports set
and get
operations)

Please check it out on:

Google Code:
http://code.google.com/p/brutis/

Github:
http://github.com/zyounker/brutis/tree/master

Feedback appreciated!

Thanks,
-Zach

[Henrik Schröder]

Hi Zach,

Can you turn on/off compression of data? Can you turn on/off connection pooling? If not, then your tool is ok for sizing apps that use the PHP client, but pretty useless for sizing apps that use any other client. Those two things probably do a lot more for the performance of memcached in an application than any other setting you could tweak.

Also, you're doing MD5 checksums of your stored data, which means that the CPUs of your test clients will spend ~90% of their time doing this, instead of stressing the memcached cluster, since MD5 is a expensive hash function. This probably skews the results, or cause you to need a lot more test clients than you would otherwise.


/Henrik

[Zach Younker]

The memcache servers added via the -x command line option are added to a
connection pool. Forks do not share connection pools, neither do
clients. This means depending on how you configure your test, you can
utilize connection pooling or not.. I think if this is important it
could be restructured into a command line switch to make it easier to
switch on and off.

Compression would be a simple matter and will look into adding it a
command line switch..

As for the MD5 checksum, there is a command line switch to disable it.
I don't think i want to remove this function, because it doesn't hurt
anything having it. Maybe this should be disabled by default?

Thanks for the suggestions!

-Zach

Henrik Schröder wrote:
> Hi Zach,
>
> Can you turn on/off compression of data? Can you turn on/off
> connection pooling? If not, then your tool is ok for sizing apps that
> use the PHP client, but pretty useless for sizing apps that use any
> other client. Those two things probably do a lot more for the
> performance of memcached in an application than any other setting you
> could tweak.
>
> Also, you're doing MD5 checksums of your stored data, which means that
> the CPUs of your test clients will spend ~90% of their time doing
> this, instead of stressing the memcached cluster, since MD5 is a
> expensive hash function. This probably skews the results, or cause you
> to need a lot more test clients than you would otherwise.
>
>
> /Henrik
>
> On Fri, Apr 10, 2009 at 22:20, zyou...@gear6.com

> <mailto:zyou...@gear6.com> <zyou...@gear6.com

[Henrik Schröder]

You could also switch from using MD5 to a cheaper hash function, FNV is a popular choice. If you only want to verify that the data you get back is correct, the cheapest possible hash function is good enough for you.


/Henrik

[Zach Younker]

Not sure FNV is available in PHP, at least a quick search on google
didn't result in much.

More commonly available hash functions:
http://us3.php.net/manual/en/function.hash-algos.php

Seems the cheapest out of these would be md2
http://us3.php.net/manual/en/function.hash.php#89574

Might be worth trying to switch it to md2 as a quick solution.

> <mailto:zyou...@gear6.com> <mailto:zyou...@gear6.com
> <mailto:zyou...@gear6.com>> <zyou...@gear6.com
> <mailto:zyou...@gear6.com> <mailto:zyou...@gear6.com

[Joseph Engo]

We ported it over quite a ways back and to be honest I don't remember
what we did to make this all work. The code is public domain (as per
fnv.h) I _think_ this version has the correct int size for 64 bit.
We ran into a problem with the end result being calculated different
on 32 bit vs 64 bit. But honestly, I am not a C guy so I am not much
help. Hopefully, this is the correct version :D

[Joseph Engo]

http://joped.com/Downloads/php_fnv.tar.gz

---
http://linkped.com - Get that link I sent ya ?

[Zach Younker]

Thanks for the link. Where did this package come from? (can't seem to
find it's home)

Doesn't compile in its current state:

In file included from /home/zyounker/Desktop/fnv/fnv.c:1:
/home/zyounker/Desktop/fnv/php_fnv.h:1:3: error: invalid preprocessing
directive #Ported
make: *** [fnv.lo] Error 1

Had to comment out the 1st line of php_fnv.h to get it to compile. After
that it compiled and was able to use it.

Thanks,
-Zach

[Joseph Engo]

Well, the code to handle FNV is from http://isthe.com/chongo/tech/comp/fnv/
Packaging it into PHP was done by 2 of our engineers. We never had
a chance to release it, I am going to see if I can post it as a google
project.

That first line was my fault. I wanted to make sure they got credit
for doing the port, so I added those comments in there. But like I
said, I don't know C so that was my fail.

---
http://jokeped.com - The greatest uncensored jokes!

[Zach Younker]

It would be great if you could get this released, specifically a pecl or
pear package would be nice.

I'd love to include it in brutis as it is obviously a much faster hash
then the ones available.

If you need any help getting it packaged up, feel free to contact me
directly and I can help out.