Scaling web service with amqp and thin

[Kevin Nuckolls]

I've been implementing a lightweight asynchronous web service backend
that will push a payload from jQuery or any other ajax app to a
rabbitmq exchange for logging of events that occur in the browser. My
current implementation of the code can handle roughly 3500 requests /
second on an 8 core ec2 instance with 8 thins behind a nginx load
balancer. I think rabbitmq is the bottleneck right now. I'm going to
set up a cluster of rabbitmq nodes and benchmark the code against that
next.

I'm concerned about one section of the code pertaining to the
channels. Currently publish is being called on one channel over one
connection for all the current requests on a given thin instance. Is
this wise? I get the best performance this way, but I don't want lost
messages. Would there be any reason to use a pool of channels
instead?

An object of this class gets instantiated when the thin server is
fired up. From that point on it pushes all incoming payloads through
the publish method.

class AMQPPublisher
# Set up the connection and one channel
def initialize
connection = AMQP.connect(:host => 'localhost')
channel = MQ.new(connection)
@xchange = channel.direct('blah', :durable => true)
end

# Push the payload to the queue for later processing
def publish
@xchange.publish("data to publish", :key => 'mykey')
end
end


Thanks,
-Kevin

[Aman Gupta]

I would ask in the rabbitmq mailing list, but I don't think there is
any advantage to multiple channels if you're just publishing messages.

Aman

[Kevin Nuckolls]

That's what I was thinking. Just wanted to pass it by those who've
been playing with amqp for awhile. Thanks Aman.

-Kevin

On Oct 14, 9:14 pm, Aman Gupta <themastermi...@gmail.com> wrote:
> I would ask in the rabbitmq mailing list, but I don't think there is
> any advantage to multiple channels if you're just publishing messages.
>
>  Aman
>