Advice on the implementation of a real-time trading system

[Mauro Asprea]

Hey guys, I just posted a Question in Stackoverflow and I would like to know if any of you could help me out ;) http://bit.ly/VKfMLF

I know this doesn't applies to celluloid itself, but the nature of this gem makes it a likely candidate with experience people things like real time systems :D

Advice on the implementation of a real-time trading system
I would love to have some advice on the implementation of a real-time trading system. Mainly, my challenges lie in the real-time part - ensuring efficiency and reaching scale.

Full question: http://bit.ly/VKfMLF 

Thanks!

[Robert Pankowecki]

Perhaps this might be interesting:

http://martinfowler.com/articles/lmax.html
http://www.infoq.com/presentations/LMAX-Disruptor-100K-TPS-at-Less-than-1ms-Latency

Robert Pankowecki

[Charles Remes]

I have written a trading system in Ruby. It is primarily focused on futures markets where the market data is measured in megabytes per second rather than gigabytes per second (like the equity markets). 

I based it off of zeromq which inspired me to write ffi-rzmq, zmqmachine (an event machine clone that uses zeromq sockets), and rzmq_brokers (a ruby version of the zeromq "majordomo" pattern). I had great success with this effort but ultimately I have decided to abandon it in favor of a refactoring session to use celluloid (and eventually dcell).

Anyway, here are the things to consider in no particular order.

** Pipelines and Subscribers

There are always lots and lots of subscribers in any trading system. You have your "signal generator" which watches market data (and perhaps other types of events) in order to generate a buy or sell signal. Next, you'll have an "execution strategy" which watches for those signals and acts upon them. Next you will have a "reporting system" that watches for orders, trades and related events. 

So, I would recommend familiarizing yourself with programming patterns for achieving a pipeline pattern and a publisher/subscriber (observer) pattern. Doing this efficiently is very easy with zeromq.

** Finite state machines

If you don't understand (or know about) FSMs, learn. They are *key* to properly handling all of the events that can be generated by a trading system. They are the only sane way to break down a complex lifecycle into manageable parts.

For example, think about the lifecycle of an order. You might be tempted to say that when you send an order, it always goes through and is accepted by the exchange. Nothing could be further from the truth. Sometimes orders are rejected and there could be a dozen (or more) reasons for why and each different rejection type requires different handling. 

You will want to solve this with if/else chains, but they will rapidly spiral out of control. FSM to the rescue.

** Async

Everything in a trading system should be async. Blocking operations will kill your performance. 

Database operations are one of the main items to watch out for. If feasible, I recommend reading tables from your DB into memory and accessing them that way. Any time you need to write to the DB, hand it off to a separate thread/process in a "fire and forget" type manner. Let the subordinate thread handle error recovery if the write fails.

FSMs also help immensely when dealing with async code. The use of celluloid mitigates this somewhat but doesn't eliminate the overall need.

** Technologies

I recommend looking at:

zeromq

mongodb

zookeeper

Good luck.

cr

[Tony Arcieri]

Depending on your definition of "realtime" Ruby isn't really a suitable language for these sorts of applications in general (or rather, no existing Ruby implementation is suitable)

You can try putting something together with JRuby and Disruptor, and if you manage to avoid allocating any memory it might work. That's about all I can think of.

--
Tony Arcieri

[Mauro Asprea]

Thanks Robert, I'll take a look at the material and get back to you.

[Mauro Asprea]

Thanks cr,

  I currently use State Machines to model some of the trade's logic and Mongodb for placing all the incoming data (which is fast!)

Regarding the architecture you mention, I had something similar in my head. So thanks, this somehow re affirms my thoughts :D. In particular the "PubSub" and Service Bus patterns.

I guess I will continue reading all the stuff others mentioned and get back to you guys ;) 

Thanks!

[Mauro Asprea]

Thanks Tony. 

One the very few things I've tested till now, I managed to have good performance using JRuby and Apache Camel to solve the issue of getting the data from our providers do some stuff and save that in the DB, while "simultaneously" pushing the data to the webclients.

I'll take a look into Disruptor and let you know my thoughts ;)

Thanks!

[Grant Rodgers]

Might want to look into Storm, especially if you're already using the jvm.

http://storm-project.net/