Using amqp to loadbalance http requests

[Aleks Clark]

This might sound kinda silly, but when I was pondering on how to
loadbalance our 'super enterprisey' application, I thought it would be
nice if HTTP requests could be put into a queue, and then each
application server would consume a request as it finished with the
previous request, putting http replies into the queue for the proxy
server that generated the initial request. I'm not sure if I would do
this in ruby, but it seems like a super-sexy way to handle LB.
Thoughts?

--
Aleks Clark

[Michael Klishin]

Aleks Clark:

With existing Ruby libraries and tools that likely will be too complex. AMQP is great at separating
concerns between applications, moving operations from the user-facing Web app to other apps/runtimes
or broadcasting events different parts of the system need to react to.

In your suggested solution, the hard part will be responding back to the browser. It won't be a problem to identify
messages and responses but if your Web app is in Ruby and built using common Web frameworks, it will be
really hard (and slow, and too expensive to maintain) to respond to clients because Ruby Web apps are usually
completely stateless and making them stateful opens a huge can of worms: you will fight most of libraries
you may choose Ruby for in the first place.

This approach would work well with something JVM based and Erlang, where stateful servers are quite common (arguably,
in Erlang they are dominant). With Node.js it may work well but I don't
think juggling callbacks of events flying in both directions is going to be a piece of cake.

I hope that answers your question.

MK

[AlexKane]

Braintree does something like this using redis

[Aleks Clark]

No, I don't want to make it stateful. What I'd like is basically for
the application to see the AMQP client as an HTTP client. All the AMQP
front end does is take an incoming HTTP request, serialize it, and
pass it to the message queue. The AMQP client on the backend receives
it and translates it into a normal http request that gets sent to the
application. The application does whatever it wants, and returns an
HTTP response, just like it would if it were being accessed normally.
This request then gets passed back to the front end via AMQP and the
front end translates it into the HTTP response that the end user is
waiting for. This way I can spawn lots of front end threads, say if
I've got a mixture of slow and fast clients, and my application
servers can complete requests as fast as they receive them, because
the AMQP backend client receives the HTTP request as fast as it is
generated. AlexKane's suggestion is interesting as well, I shall have
to look into it. Redis is as high on my 'I'd like to use this to do
awesome things' as AMQP is :)

--
Aleks Clark

[Michael Klishin]

Aleks Clark:

> All the AMQP
> front end does is take an incoming HTTP request, serialize it, and
> pass it to the message queue. The AMQP client on the backend receives
> it and translates it into a normal http request that gets sent to the
> application. The application does whatever it wants, and returns an
> HTTP response

Like I said, the hard part will be responding back to the browser. Most of Ruby Web app ecosystem was not designed
with this architecture in mind.
…

> Redis is as high on my 'I'd like to use this to do
> awesome things' as AMQP is :)

With pretty much any messaging solution you will face very similar challenges. Two other good options are ZeroMQ and Apache Kafka.

Check out SockJS, a Web messaging solution from the RabbitMQ team. They have released a RabbitMQ plugin recently
(which introduces no new features for clients to support) and there is a Ruby implementation.

MK

[Aleks Clark]

I don't think it'd be that difficult...I can just have the load
balancer thread accept the request as a normal web server would, wait
for the amqp response, and return the response. I don't need
websockets or streaming (at the moment). The braintree 'broxy' is
exactly what I need, although for fault-tolerance I would prefer not
to rely on a single machine running redis. then I've got the amqp
brokers to worry about though. And the load balancers for those. Guh.
guess I'll be doing a lot of googling, I want to magically not have
single points of failure (even HA setups). is there a button for that
somewhere? >.> maybe I'll just use HA proxy.

--
Aleks Clark

[Michael Klishin]

Aleks Clark:

> I want to magically not have
> single points of failure (even HA setups). is there a button for that
> somewhere?

Take a look at Kafka's design goals:
http://incubator.apache.org/kafka/design.html

MK

[Aleks Clark]

wow that's complex :) lots of food for thought there. Thanks guys <3

--
Aleks Clark

[Aleks Clark]

if anyone else is interested, mongrel2 is using zeromq to basically do
what I was proposing. great minds think in the same gutters and all
:P

--
Aleks Clark