asynchronous client design

[Keith Rarick]

I came across this recently:

http://svn.effbot.org/public/stuff/sandbox/memcached/async_memcache.py

It strikes me as an excellent architecture, and particularly
appropriate for beanstalkd. For example, properly handling
DEADLINE_SOON messages would be trivial in a worker built on this
strategy. You simply enqueue your delete commands while waiting for a
reservation. When DEADLINE_SOON is received, those delete commands
will be processed immediately and automatically.

Also, command pipelining is built in (note the placement of the
command.send() call). Even more advanced stuff like using TCP_CORK for
batching commands into fewer packets would be pretty straightforward.

I will try to figure out a way to use this design in the ruby client.
Unfortunately, ruby doesn't have anything as good as python's
asyncore. There are a few event-based socket libraries, but none are
that great, and I'm loath to add a dependency.

Python guys, you might want to steal some ideas from here too. :)

kr

[Dustin]

Keith Rarick wrote:

> It strikes me as an excellent architecture, and particularly
> appropriate for beanstalkd. For example, properly handling
> DEADLINE_SOON messages would be trivial in a worker built on this
> strategy. You simply enqueue your delete commands while waiting for a
> reservation. When DEADLINE_SOON is received, those delete commands
> will be processed immediately and automatically.
>
> Also, command pipelining is built in (note the placement of the
> command.send() call). Even more advanced stuff like using TCP_CORK for
> batching commands into fewer packets would be pretty straightforward.

I've been thinking about building a beanstalk client based on my
java memcached client. It has a dedicated thread for IO, but the
concept is the same. It's all asynchronous and uses java Futures in a
way to allow you to synchronize whenever or however you want.

This means that most write operations, in particular, are
approximately instant. multi-ops (sparse gets, for example) provide a
good example of fire-and-maybe-forget kinds of things where you can
issue commands where you only conditionally care about the results
based on success (in the case of a read) or failure (in the case of a
write) while still letting you block on either.

The neat thing about running all the stuff through a queue like this
(as you said) is that the code that packetizes the commands from
objects in the queue fills a buffer before transmission so commands
naturally get combined into larger packets. I do that by hand when
filling buffers, but that's just because I had to packetize them
anyway. While I'm in there, I can do other nice ops like combining
multiple sequential gets into a single deduplicated multi-get request
and appropriately dispatch results as they come back over the wire.

Since the whole thing sits on whatever multiplexer java supports
best on your platform, it can easily deal with a large number of
servers all sending responses at the same time which makes the multi-
server-reserve problem a lot easier to deal with.

Of course, my code's a lot more to read. :)

[Erich]

Nice, Ill need to look into this arch a bit more, for the python
client. I already got the start of that going in the libevent
connection, but it needs some reworking for
this style of doing stuff.