LMAX: How do you implement HTTP services with Disruptor?

[Toby DiPasquale]

Hi all (specifically the LMAX people),

I'm writing an HTTP service using Disruptor and I've come across a
quandary. Using Netty for HTTP handling in front of a Disruptor ring
buffer causes entries to enter a queue. Netty accepts connections on
one set of threads ("boss threads") and then enqueues them for post-
accept processing by another set of threads ("worker threads"). This
introduces a queue into the mix (and an unbounded one, at that).

The point of Disruptor, as I understand it, is that eliminating queues
makes the work less contentious and faster. How does LMAX implement
their HTTP services on top of Disruptor? Do you have a single socket-
acceptor thread? Are you using raw NIO and inserting HTTP requests
directly into the ring buffer? Or do you have multiple socket-
accepting threads that are massaging the HTTP request into a more
suitable Entry object before inserting into the ring buffer?

Thanks for all the help you can give me on this.

--
Toby DiPasquale

[Martin Thompson]

Toby,

Good question that does not have a simple answer.

We have evolved with this over time. HTTP is fundamentally a
synchronous model designed to retrieve documents. We love
asynchronous systems :-) A lot of our system uses Comet style
programming for the web communication. We typically do not respond
directly to HTTP requests. We generate an event based on that input,
and via a number of layers, place it in a Disruptor. When our system
responds to that incoming event we push it down a comet connection
using either long poll or a HTTP chunk-encoded stream. This allows us
to separate the request from the response. As WebSockets become more
common I'd like to move to this pattern to have full duplex
communications. For our web server we use Caucho Resin that has good
support for Comet style programming and WebSockets.

Martin...

[Olivier Deheurles]

Same here: HTTP requests with a correlation ID to send commands from
client to server and asynchronous response(s) via a "Comet" connection
(this mechanism can be used for a RPC style request response or
opening a stream of events: 1 request, N responses).

Lots of trading systems use HTTP push in their system to feed realtime
data in their RIA (market data), but they often don't generalise that
for request response style interactions between the client and the
server. I personaly think it's a mistake, the asynchronous mechanism
described by Martin offer way better scallability compared to more
traditional aproaches.

Martin, did you evaluated other Comet/Web push vendors (Caplin,
Kaazing, Lightstreamer, My-Channels, etc) or did you choosed directly
to build that yourself (actually I don't know Caucho Resin, so maybe
everything is build-in)?

No problem so far with some institutional/corporate client's proxies
and/or firewals?

Olivier

[David Yu]

It can be done with mongrel2 + zeromq.

Your single zeromq handler (producer) will then dispatch messages to consumer(s) via disruptor pipeline/diamond.

The last node in the pipeline/diamond graph is responsible for generating the response back to mongrel2 via zeromq.

Cheers

--
When the cat is away, the mouse is alone.
- David Yu

[Martin Thompson]

Yes we evaluated lots of providers including Jetty, Lightstreamer and
Virgil. At the time I knew the CTO of Caucho quite well and convinced
him to do a async implementation for us. He has since come to love
this approach and is now on the Servlet async and WebSockets
specification helping guide them. I'd highly recommend the Caucho
guys for building one of the most reliable and scalable web containers
I know off.

[Rajiv Kurian]

I found this thread when searching for networking solutions based on the disruptor.

Assuming you have multiple connections. Do you guys have a single thread reading from each connection or a thread per connection or a threadpool. In all but the first case this would map to multiple producers right?

Thanks,

Rajiv

[Michael Barker]

I this case it happens to be multiple threads as that is what the web
container that we are using provides.

Mike.

> --
> You received this message because you are subscribed to the Google Groups
> "Disruptor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lmax-disrupto...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Rajiv Kurian]

Got it. Thanks Mike!