Have you seen autobench? <http://www.xenoclast.org/autobench/>
I don't want to dissuade you (autobench is getting old, and has a number of shortcomings); just wanted to make sure you're aware.
Also, while I'm writing to the list -- we *really* need to get the event-driven httperf production-ready and released; it's becoming a big limiting factor in testing newer servers like node.js.
Cheers,
--
Mark Nottingham mn...@yahoo-inc.com
Autobench is actually what lead me to create the new package. Based on
the documentation and my experience it only allows you to run
benchmarks within a given range of rates.
In contrast, autohttperf allows you to specify an error 'threshold'
and other start parameters and will continue to increase the rate
until the benchmark detects 'threshold' amount of errors in the
benchmark between the distributed clients. At this point it will then
run for a specified number of rounds to finish off the benchmark. If
the benchmark is in an error state and the server recovers, i.e. does
not produce further errors, then the cooldown steps are reset and the
benchmark is allowed to continue, exiting the error state.
In addition, the number of connections is based on how long you want
each 'round' of benchmarking to take, rather than being a number
picked from thin air. If you want your benchmark to take 2 minutes per
round, then you want a total of rate * 120 connections for an ideal
server. This limits the amount of time you spend waiting for
benchmarks to complete.
Really, the problem I had with autobench is there was no easy way to
determine precisely WHAT ranges and parameters you should be testing
with. My package is for poking and prodding to get that information. I
very well may use autobench on the final set of benchmarks, once I
know the ranges I need to test against.
> I don't want to dissuade you (autobench is getting old, and has a number of shortcomings); just wanted to make sure you're aware.
>
> Also, while I'm writing to the list -- we *really* need to get the event-driven httperf production-ready and released; it's becoming a big limiting factor in testing newer servers like node.js.
That would be welcome, and I would be happy to test anything for you
if you have it available!
- Jim