benchmarking perl clients: speed and durability

[Vick Khera]

I'm checking out the various clients for Perl to use against the C
server, becuase I need the persistent queue feature. My goal is to
have a system to which I can submit jobs and just know they'll get
done. I also need the speed.

I'm comparing Gearman::Client, Gearman::XS::Client, and
AnyEvent::Gearman::Client.

For the server, I'm running gearmand 0.9 as installed by FreeBSD ports
(I suppose I should upgrade the port, too...), with sqlite3 as the
persistent queue. I'm not running any workers -- just queueing up the
background jobs as that is my main need at this time.

I run the server like this:

gearmand -t3 -q libsqlite3 --libsqlite3-db=/tmp/gearmanqueue

Then I run a simple loop that submits 10k background jobs with a
payload of about 520 bytes. I delete the sqlite file and restart the
server on each test. Everything is running on a single server, which
is a Pentium D 64-bit server with 4Gb of RAM doing not so much else.

Here are some typical numbers:

For AnyEvent::Gearman::Client:
loop to create tasks: 8.51s
waiting for all tasks to be submitted in event loop: 72.07 seconds
total: 80.58 seconds

However, durability is not so great. Only 9990 jobs in this test made
it to the persistent queue. The client ran the "task created"
callback for each of the 10k tasks, so the 10 tasks were just missing
in the ether somewhere. The asynchronous nature of the event loop
style
program is just so tempting, but losing work is unacceptable.

For Gearman::Client:
loop to create tasks: 72.89s

Again, durability is a problem here. On this sample run, only 9999
jobs were written to the sqlite queue. This is actually the most
reliable run I've had with this client -- usually I lose upwards of
12-15 jobs. Again, the client reported back to the program a
successful submission for every single submission.

For Gearman::XS::Client:
loop to create tasks: 71.85s

Durability here is 100%. In the many times I've run this script it
has *never* lost a single task submission to the server. However, as
it is a 1:1 mapping of the C to perl, it is very unnatural to program
this style. On the other hand, not losing work is much more important
to me. Plus, I can wrap this up into a more perl-ish style interface,
and get the best of both worlds.


Conclusion:

For my project, I will be settling on the XS client, because it is
fast and it has never lost a job in all of my testing. Also of note
is that the original perl client, even though it supports "high
priority" flag for submitting the jobs, does not submit jobs that show
up as such in the sqlite queue table.