gaming machine learning or how to get rich

[josef...@gmail.com]

Suppose financial traders hire machine learning guys and are using twitter news for sentiment analysis to direct their trades,

suppose we have a large enough group that can create a significant amount of twitter traffic,

suppose we have enough capital to invest a large amount in short selling,

then we can get rich.

suppose we can do this often enough, then these financial traders either have to hire more or smarter machine learners or give up and "release" them.

----

Just thinking, after hearing some stories about high frequency trading where milliseconds are important.

Josef

- You're cheating: your evaluation sample doesn't look at all like the training and test sample

- so what? life changes, sometimes

[Kyle Kastner]

I looked into this some for a thesis project - not the gaming the system part, but the twitter stream analysis. Though I have since changed to another topic, there was a pretty simple algorithm and powerful algorithm for trend prediction written by a student @ MIT - see http://dspace.mit.edu/bitstream/handle/1721.1/85399/870304955.pdf 

I believe they aggregate from other sources as well (Google news, the market itself, etc.) though the feeds from there are much slower, they *should* also be more reliable. If they are wise, it is probably some kind of Kalman filter to do "sensor fusion" to fuse together the estimates from all these different sources with different variance ... 

Similar to the description here http://www.slideshare.net/antoniomorancardenas/data-fusion-with-kalman-filtering-21838422 , only your "sensors" are variable information sources rather than gyros, magnetometers, etc. Building the state-space model might be difficult though! However, we will also have a really nice KF (and I think EKF?) to play with in statsmodels soon enough... 

If they are reliant enough on twitter that one *could* poison their stream, that is pretty poor programming IMO. It would be interesting to see if you could exploit a lack of Twitter volume in some other language to affect HFT people who play in the regional markets... since there will be less sources to aggregate you might be able to do something interesting.

Kyle