[ANN] Ruby Web Bench 0.0.0 (RWB)
pat eyler
Last week I had an itch that ab (the web performance tester shipped with
apache) just didn't scratch, so I decided to sprinkle some Ruby dust over
the problem and see what that bought me. RWB is a light weight (and alpha
quality) performance/load testing tool for websites. It features a couple of
nice features:
- you can build up a weighted list of URLs, then test them at a given
total number of requests and level of concurrency. Each request will
pull a random request from your weighted list.
- given a group or family of URLs, you can specify them using a
weighted base URL and an array of extensions. Each time a request is
pulled from the group, it will get a random extension from the array.
- Reports are given for the run overall, and for each URL or URL group.
Once I'm a bit happier with the engine and the reporting, RWB will be turned
into a DSL (rather like Rake) to make building and running load or performance
tests even easier.
For now though, I'd love to hear what works well (or doesn't), what additional
features people would like to see, etc.
Go grab your copy from: http://www.red-bean.com/~pate/
BTW, here's a sample report:
Concurrency Level: 50
Total Requests: 1000
Total time for testing: 4.148434 secs
Requests per second: 241.054817311786
Mean request time: 4 msecs
Overall results:
Shortest time: 18 msecs
50.0%ile time: 41 msecs
90.0%ile time: 55 msecs
99.9%ile time: 81 msecs
Longest time: 81 msecs
Results for http://www.example.com :
Shortest time: 41 msecs
50.0%ile time: 53 msecs
90.0%ile time: 62 msecs
99.9%ile time: 81 msecs
Longest time: 81 msecs
Results for http://www.example.com/nonesuch :
Shortest time: 29 msecs
50.0%ile time: 41 msecs
90.0%ile time: 51 msecs
99.9%ile time: 65 msecs
Longest time: 65 msecs
Results for http://www.example.com/entries :
Shortest time: 18 msecs
50.0%ile time: 41 msecs
90.0%ile time: 50 msecs
99.9%ile time: 69 msecs
Longest time: 69 msecs
Results for http://www.example.com/search?:
Shortest time: 28 msecs
50.0%ile time: 40 msecs
90.0%ile time: 55 msecs
99.9%ile time: 69 msecs
Longest time: 69 msecs
--
thanks,
-pate
-------------------------
zed...@zedshaw.com
since the overall request time's 50%tile is 41msecs witha shortest of 18.
Also, where's the standard deviation?
Zed A. Shaw
http://www.zedshaw.com/
<snip>
>
> Concurrency Level: 50
> Total Requests: 1000
> Total time for testing: 4.148434 secs
> Requests per second: 241.054817311786
> Mean request time: 4 msecs
> Overall results:
> Shortest time: 18 msecs
> 50.0%ile time: 41 msecs
> 90.0%ile time: 55 msecs
> 99.9%ile time: 81 msecs
> Longest time: 81 msecs
<snip>
pat eyler
> Nice.
thanks
> I think your calculations on the mean request time might be wrong
> since the overall request time's 50%tile is 41msecs witha shortest of 18.
Yeah, I noticed that while I was playing with another report on the bus ride
home :(
Look for a fixed version tomorrow (after a Tuesday morning
commute).
> Also, where's the standard deviation?
On the list of things to do. :)
Any other comments/requests?
>
> Zed A. Shaw
> http://www.zedshaw.com/
>
> <snip>
> >
> > Concurrency Level: 50
> > Total Requests: 1000
> > Total time for testing: 4.148434 secs
> > Requests per second: 241.054817311786
> > Mean request time: 4 msecs
> > Overall results:
> > Shortest time: 18 msecs
> > 50.0%ile time: 41 msecs
> > 90.0%ile time: 55 msecs
> > 99.9%ile time: 81 msecs
> > Longest time: 81 msecs
> <snip>
>
>
>
--
thanks,
-pate
-------------------------
Bob Hutchison
On Nov 7, 2005, at 7:52 PM, pat eyler wrote:
>
> On the list of things to do. :)
>
> Any other comments/requests?
>
Hi,
This looks really quite handy -- I think I've had the same issues as
you have had with ab. In a few days I'll try it on a couple of
systems I'm working on.
Since you are asking for suggestions, one thing that would be nice,
maybe it is already there and I just didn't notice it, is a warmup
phase. In the applications I'm working on the first couple hits on a
URL causes work to be done that I don't really want timed.
Cheers,
Bob
----
Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/>
Recursive Design Inc. -- <http://www.recursive.ca/>
Raconteur -- <http://www.raconteur.info/>
pat eyler
>
> On Nov 7, 2005, at 7:52 PM, pat eyler wrote:
>
> >
> > On the list of things to do. :)
> >
> > Any other comments/requests?
> >
>
> Hi,
>
> This looks really quite handy -- I think I've had the same issues as
> you have had with ab. In a few days I'll try it on a couple of
> systems I'm working on.
>
> Since you are asking for suggestions, one thing that would be nice,
> maybe it is already there and I just didn't notice it, is a warmup
> phase. In the applications I'm working on the first couple hits on a
> URL causes work to be done that I don't really want timed.
Not there yet, but I love the idea, so I'll add it shortly. How does
something like this look:
(in the RWB::Runner class)
warmup(num_times) # this would just walk through the
# array of urls and
url_groups num_times
# times, making num_times
* array.length
# total requests
rand_warmup(num_requests) # this would randomly select urls to request,
# making num_requests
total requests.
>
> Cheers,
> Bob
>
> ----
> Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/>
> Recursive Design Inc. -- <http://www.recursive.ca/>
> Raconteur -- <http://www.raconteur.info/>
>
>
>
>
--
thanks,
-pate
-------------------------
Bob Hutchison
On Nov 8, 2005, at 10:32 AM, pat eyler wrote:
> On 11/8/05, Bob Hutchison <hu...@recursive.ca> wrote:
>>
>> On Nov 7, 2005, at 7:52 PM, pat eyler wrote:
>> Since you are asking for suggestions, one thing that would be nice,
>> maybe it is already there and I just didn't notice it, is a warmup
>> phase. In the applications I'm working on the first couple hits on a
>> URL causes work to be done that I don't really want timed.
>
> Not there yet, but I love the idea, so I'll add it shortly. How does
> something like this look:
>
> (in the RWB::Runner class)
> warmup(num_times) # this would just walk through the
> # array of urls and
> url_groups num_times
> # times, making num_times
> * array.length
> # total requests
>
> rand_warmup(num_requests) # this would randomly select urls to
> request,
> # making num_requests
> total requests.
Looks good. Maybe with the ability to sepecify a set of URLs to
warmup on and an order within them? Maybe too much work for small gain?
Cheers,
Bob