tmm1-ampq vs. Lizzy vs. Carrot vs. Qusion vs. Bunny

[ph7]

Hi,

I am in the process of selecting a ruby API to interface with
RabbitMQ. My use-case is a classic one: Some Rails processes running
under passenger are going to offload tasks to a farm of workers via
RabbitMQ.

While navigating blog posts and wikis I have a hard time figuring out
the tradeoffs between the different options though. Could somebody
give me the big picture on synchronous vs. event-based APIs in this
context (for the passenger processes and the workers which are likely
to have a different lifecycle)

Also I would love to hear about tradeoffs/advantages/pitfalls of:
- tmm1-ampq
- Lizzy
- Carrot
- Bunny
- Qusion
- ??? : any other option on the table

Thanks in advance,
- Philippe

[Richard Heycock]

Excerpts from ph7's message of Wed Oct 21 16:17:02 +1100 2009:

Of the list above only tmm1-ampq, Carrot & Bunny are amqp bindings. The
others sit above the bindings.

Since you are going to use passenger you'd probably also want to look at
Qusion which provides a mapping between the synchronous passenger (or
mongrel, ...) and the asynchronous tmm1-ampq.

I suspect Lizzy would not be of interest to you though you could use it
implement your workers which would make for quite a nice solution
(I do something similar).

I've used tmm1-ampq for about a year or so in a fairly high volume
production environment and it has worked extremely well. It is a little
hard to get your head around sometimes but, for me, that was as much
getting to grips with event driven programming. I also use Bunny, I've
only used if for a few months (again in a production environment) but
I've had no problems with it.

I use both tmm1-ampq and Bunny because, while I mainly use an event
driven approach, I do sometimes need to make synchronous calls from
rails. You could implement synchronous using fibers and tmm1-ampq but
you'd need ruby 1.9.

oldmo's booklet is something you should probably read (and to be honest
you should probably read anything he's written).

http://oldmoe.blogspot.com/2009/10/ruby-19x-web-servers-booklet.html

oldmo has written about using rails and fibers but I couldn't find it.
Anyone?

rgh

[Jake Mallory]

I highly recommend using rosetta_queue which is a wrapper around
broker adapters. It comes with 2 adapters for AMQP (bunny and tmm1-
amqp) It has a nice API and simplifies the common messaging patterns,
ie publish/subscribe.

http://wiki.github.com/bmabey/rosetta_queue

As to the tradeoffs IMO, right now in ruby 1.8 I'd say the biggest
trade off is complexity. The bunny client is synchronous and requires
nothing special to be done in the passenger environment. The tmm1-amqp
client's evented model becomes far more appealing in 1.9 or jruby
where the threads are better (or real) where I believe it will perform
better and be worth the added complexity. We've also seen increased
complexity when trying to test tmm1-amqp in cucumber features.

Others may disagree with me about this or have different views about
the tradeoffs; they are likely smarter them me so my recommendation is
to make a quick risk/knowledge assessment and take the plunge.

[Daniel DeLeo]

Hi,

I wrote Qusion. To be clear, it's just a wrapper around the tmm1 AMQP client, with some special magic that figures out what server you're on and sets up whatever threading and server callbacks are required. There's also support for a amqp.yml config file if you want to use it.

While I am of the opinion that the code is solid, I would urge you to consider using Bunny (my personal choice for synchronous AMQP) in your server and tmm1-amqp in the worker daemon. I've seen quite a few recommendations for this setup. The reason, as Jake said previously, is that Ruby 1.8's threading is not great. Aman (tmm1) gave a presentation about this at Ruby Hoedown, and though I didn't attend, I found the slides to be very informative. You can see them here: http://timetobleed.com/ruby-hoedown-slides/ 

Though it's the most popular thing I've put on github to date, I think the primary use case for Qusion is when another library or plugin is forcing you to use the evented AMQP library in the server (workling, for example). I've been meaning to update the README to this effect for some time, but have been to lazy to do so. 

If you are going to roll your own solution, you may be interested in Daemon-kit (http://github.com/kennethkalmer/daemon-kit) which has support and a code generator for an AMQP worker daemon. You might also want to have a look at minion, which I haven't had the opportunity to use yet, but looks like the background job solution I would want to write if I were to write one: http://github.com/orionz/minion

As for Rosetta Queue, I haven't had the opportunity to use it either, though it does have some support for a test environment, which is a plus. If you stick with tmm1-amqp for producer or consumer, you might like the moqueue library I wrote which helps you avoid the pitfalls of testing EventMachine-using code: http://github.com/danielsdeleo/moqueue

Hope that helps,

Daniel DeLeo

[Philippe Hanrigou]

Richard, Jake and Daniel thanks a lot for your feedback. This is very helpful and well appreciated. My current belief is that indeed there is little to gain from using an asynchronous API in my use case. I wanted to make sure that I was not missing something and get a feel for how "battle-tested" all these clients were.

At this point and for my particular use case:

* I am inclined to go with Bunny (simple API, code and paradigm + apparently a bigger mindshare than Carrot)

* I am not especially thrilled at the idea of  introducing rosetta_queue: I am not really interested in switching between messaging broker/patterns : RabbitMQ sounds like a stable choice for the foreseeable future. I also believe I can handle the testing part with good design and ad-hoc stubbing/mocking at the right level in my unit-testing (I have done so in the past with other message brokers).

* I see little value in embracing daemon-kit or minion as they seem to be centered in an evented/asynchronous model. It seems that even in the workers I am probably just as well using a synchronous API too, right? In particular I'd rather have less throughput but be sure that I have custom ack logic I can rely on (http://pivotallabs.com/users/jpalermo/blog/articles/952-rabbitmq-amqp-gem-and-eventmachine#ixzz0UapniSPV).

Does that makes sense, or is there anything that I am missing or fundamentally misunderstand?

Richard, thanks also for the reference to Muhammed Ali's writing: This is one of the most balanced and insightful piece I have read on the matter.

Cheers,
- Philippe

[Daniel DeLeo]

Phillippe,

Just a quick note on the passage you highlighted: The tmm1-amqp library supports a prefetch option (don't remember if this is RabbitMQ only or supported by other Brokers--but not an issue for you in any case). Using prefetch will limit the amount of messages that RabbitMQ will try to force down your worker's throat. This is covered at the bottom of the post you reference, where the author describes that the issue was with acks being "stuck" at a level below the AMQP library by the combined behavior of EM and RabbitMQ, and that setting the prefetch limit allowed EM ample time to send the acks. So acks will work fine if you tune this setting to a reasonable value.

Of course, you should do whatever you think gives you the best chance to deliver the best software.

Daniel DeLeo

[Philippe Hanrigou]

Thanks for the clarification Daniel!

[Jake Mallory]

Philippe,

We started with tmm1-amqp. Because we use cucumber heavily in our
development we found that testing the full stack became more difficult
with tmm1-amqp. I work with Ben and Chris who maintain rosetta_queue
and we wrote an adapter for bunny. We now use it everywhere and it
works great. Using rosetta_queue makes it very easy to try the
different amqp clients.

Another reason to consider rosetta_queue is for it's great logging.

Ultimately I'm glad to hear you are going to take the plunge.

On Oct 21, 12:39 pm, Philippe Hanrigou <philippe.hanri...@gmail.com>
wrote:

> Richard, Jake and Daniel thanks a lot for your feedback. This is very
> helpful and well appreciated. My current belief is that indeed there is
> little to gain from using an asynchronous API in my use case. I wanted to
> make sure that I was not missing something and get a feel for how
> "battle-tested" all these clients were.
>
> At this point and for my particular use case:
>
> * I am inclined to go with Bunny (simple API, code and paradigm + apparently
> a bigger mindshare than Carrot)
>
> * I am not especially thrilled at the idea of  introducing rosetta_queue: I
> am not really interested in switching between messaging broker/patterns :
> RabbitMQ sounds like a stable choice for the foreseeable future. I also
> believe I can handle the testing part with good design and ad-hoc
> stubbing/mocking at the right level in my unit-testing (I have done so in
> the past with other message brokers).
>
> * I see little value in embracing daemon-kit or minion as they seem to be
> centered in an evented/asynchronous model. It seems that even in the workers
> I am probably just as well using a synchronous API too, right? In particular
> I'd rather have less throughput but be sure that I have custom ack logic I

> can rely on (http://pivotallabs.com/users/jpalermo/blog/articles/952-rabbitmq-amqp...
> ).

>
> Does that makes sense, or is there anything that I am missing or
> fundamentally misunderstand?
>
> Richard, thanks also for the reference to Muhammed Ali's writing: This is
> one of the most balanced and insightful piece I have read on the matter.
>
> Cheers,
> - Philippe
>