JArbitrager release 1.02

[nonlinear5]

Latest release download: http://jarbitrager.googlecode.com/files/JArbitrager-1.02.zip
Market data: http://jarbitrager.googlecode.com/files/marketData-1.02.zip
Release Notes: http://code.google.com/p/jarbitrager/wiki/ReleaseNotes
Issues: http://code.google.com/p/jarbitrager/issues/list

Changes in this release:
-- Replaced timer-driven mechanism with the data-driven mechanism
-- Reworked charting and dispatchers
-- Added two new sample strategies

Up to release 1.01, JArbitrager used 1-second snapshots to record the
data. Starting from this release, it's all data driven. Specifically,
a new line is added to the dataset if and only if either bid or ask
price changes for either of the instruments defined in a pair. This
means that sometimes there would be multiple lines with the same
timestamp, and sometimes the lines will be separated by several
seconds or even minutes. The previously recorded data can still be
used, and backtest and optimization results should be exactly the same
as before.

This new methodology should make multi-user tests much more
consistent. I will be running all sample strategies in the forward
test mode tomorrow, Friday, July 8, 9:15am to 3:55pm. I need a couple
of testers to do the same, so that we can compare the results.

[new_trader]

great work!
I am in your test tomorrow.

Will our planned downloader make sense with this new data format? Has
the new format any implications on this?

[Eugene Kononov]

Will our planned downloader make sense with this new data format? Has
the new format any implications on this?

We'll use the downloader one way or another, but we'll have to experiment with different formats to see which one makes most sense.

[nonlinear5]

Switching to data-driven mechanism seems to have increased the
frequency of trading. Here are my intermediate results as of 10:12am:
http://groups.google.com/group/jarbitrager/web/intermResultsJuly9.PNG

[new_trader]

confirmed!
my intermediate results as of 10:25
ES_NQ: 1 trades, +$3
ES_SPY: 12 trades, +$72
ES_YM: 6 trades, $107
EUR_EUR: 31 trades, +$114
YM_DIA: 9 trades, +$52
GBP_GBP (new): 17 trades, +$12
NQ_QQQQ (new) : 18 trades, -$142

[new_trader]

sorry, I have posted wrong data.
here is the right one with timestamp 10:38
http://groups.google.com/group/jbooktrader/web/JAB1038.png

[nonlinear5]

Looking pretty spectacular so far.

On Jul 9, 10:40 am, new_trader <loc...@gmx.de> wrote:
> sorry, I have posted wrong data.

> here is the right one with timestamp 10:38http://groups.google.com/group/jbooktrader/web/JAB1038.png

[new_trader]

unbelievable! I think you've got it now :-) !!!

[dafa]

Yes, very promising, great work nonlinear
as 10:40am, result on my box:
http://jbooktrader.googlegroups.com/web/JAB102_10-40am.PNG?gsc=tymGtQsAAAC7diUm7dLVnwHQNfwvO-9l

[Gab]

A few questions for the testers :

How many seconds you are in the trade in average?

The longest trade was? 

The shortest trade was?

[new_trader]

look at my YM_DIA trading report:
http://groups.google.com/group/jbooktrader/web/JAB_YM_DIA.png

[Gab]

On Fri, Jul 9, 2010 at 11:21 AM, new_trader <loc...@gmx.de> wrote:

look at my YM_DIA trading report:
http://groups.google.com/group/jbooktrader/web/JAB_YM_DIA.png

Thanks. It looks good. Above 1 minute most of the times. No trade got sent 1 sec after another one (which would probably result in a loss). 

I see that you're always in the market for that strategy. Can you explain how it works?

[nonlinear5]

Can you guys post an interim update at 2pm EST? Thanks.

[new_trader]

intermediate results at 14:00:
http://groups.google.com/group/jbooktrader/web/JAB_1400.png
still looks more than excellent!

[nonlinear5]

2pm results: http://groups.google.com/group/jarbitrager/web/2pmResults.PNG

[nonlinear5]

Yes, looking good in absolute terms, although I expected that we would
be matching closer.

[dafa]

my 2pm update:

http://jbooktrader.googlegroups.com/web/JAB102_2PM_dafa.PNG?gsc=0GcmWwsAAAAnl9Go7p2VCE10KYQSBPEr

On Jul 9, 2:02 pm, new_trader <loc...@gmx.de> wrote:

[new_trader]

> Yes, looking good in absolute terms, although I expected that we would be matching closer.

yes, significant deviation. have we done this test ever with JBT?
perhaps here we have also this deviations?

has anyone insight on how IB is distributing their data, what is the
architecture behind it.
I think distributing tick data with quaranteed equal values for all
subscribers will be hard for them.
The question also is what is the role of TWS/IBAPI? which kind of
processing is TWS/IBAPI applying to the raw stream which comes from
the IB servers?
there are a lot of green frogs between the origin of the data and JAB.
and each quake of a frog can potentially modify the data...

[nonlinear5]

>
>  yes, significant deviation. have we done this test ever with JBT?
> perhaps here we have also this deviations?
>

Yes, we've run many multi-user test with JBT. Using NTP clock improved
the consistency quite a lot. I've also traded live the same strategies
with the other guy, and compared the results, they were matching
closely. Jarbitrage is using a different data stream (market bid/ask
instead of market depth), so I actually expected to have an even
closer match, but it looks like we are missing something.

> has anyone insight on how IB is distributing their data, what is the
> architecture behind it.
> I think distributing tick data with quaranteed equal values for all
> subscribers will be hard for them.
> The question also is what is the role of TWS/IBAPI? which kind of
> processing is TWS/IBAPI applying to the raw stream which comes from
> the IB servers?
> there are a lot of green frogs between the origin of the data and JAB.
> and each quake of a frog can potentially modify the data...

Florent and other guys in JBT did some detailed analysis and
comparison of the recorded "raw" data stream coming from IB to API
clients. From what I remember, all data messages were consistent
(meaning that user A had the same set of messages as user B). However,
in some cases, the *order" of messages was not the same. For example
user A first got a message about level 3 changing, and then about
level 2 changing, while user B got the same messages in the reverse
order. This can be explained by the TCP/IP retransmissions, which
occur from time to time.

[new_trader]

have just browsed through com.jarbitrager.platform.trader.Trader.java
and compared it to com.jbooktrader.platform.trader.Trader.java:
it seems that in JBT the actions in updateMktDepth() are much leaner
than in tickPrice().
in updateMktDepth only marketDepth.update() is called. in this
function only small things happen.

the code path in tickPrice seems more complicated:
strategy.setTime(System.currentTimeMillis());
strategy.process();

Dispatcher.getInstance().fireModelChanged(ModelListener.Event.StrategyUpdate,
strategy);
strategy.saveSnapshot();

can it be we have kind of a timing problem here?
especially saveSnapshot() is doing disk IO which always has the risk
of high latency.

you have a power machine, I for example only have a laptop with a slow
hard drive.

[dafa]

In the past hour ,I ran "trade" mode against paper IB account(in
separate computer and
account ) vs the one ran forward test. looks like lost money in
all pairs.
For example in SPY-ES pair in forward mode, almost all are
profitable(ie,
small profitable 5.2,9.8)
07/09/10 15:12:22.182 EDT 57 1 1073 -500 107.83 9.8 5.2 388.8
07/09/10 15:19:12.675 EDT 58 -1 1072.75 500 107.77 9.8 7.7 396.5
but in paper account all trades are lost money(ie below two trades in
SPY-ES):
07/09/10 15:15:41.039 EDT 4 1 1073.5 -500 107.856 9.8 -21.8 -66.3
07/09/10 15:16:51.863 EDT 5 -1 1073.25 500 107.85 9.8 -19.3 -85.6

[new_trader]

I have done this too.
had the same bad results in the paper account. I think it once again
shows that data from the paper account is useless :-(

[new_trader]

final results Jul 9:
http://groups.google.com/group/jbooktrader/web/JAB_finalresults_20100709.png

[nonlinear5]

Final results for July 9: http://groups.google.com/group/jarbitrager/web/finalJuly9.PNG

[dafa]

July 9 result:
http://jarbitrager.googlegroups.com/web/JAB102_4PM0709_dafa.PNG?gsc=ru5bFRYAAABXRys7guwSXvnhwGheaA4buy7lU7T7D1l6smgL4x7o7Q

[nonlinear5]

Yes, this may explain it. In JBT, data is still being written to disk,
but in a separate thread. In JArbitrager, it's the same thread that
processes API messages, runs strategies, and writes to file. I'll make
some adjustments to see how the consistency is affected.

Thanks again for your participation in the test, folks!

[nonlinear5]

Consolidated results for July 9, for future reference to compare with
the next multi-user test:

new_trader
--------------------------
ES_NQ: 2 trades, -$124
ES_SPY: 60 trades, +$372
ES_YM: 13 trades, +$182
EUR_EUR: 98 trades, +$381
GBP_GBP: 29 trades, +$71
NQ_QQQQ 40 trades, +$51
YM_DIA: 54 trades, +$426

nonlinear
--------------------------
ES_NQ: 2 trades, -$94
ES_SPY: 65 trades, +$468
ES_YM: 9 trades, +$128
EUR_EUR: 111 trades, +$522
GBP_GBP: 27 trades, +$101
NQ_QQQQ 44 trades, +$39
YM_DIA: 55 trades, +$369

dafa
--------------------------
ES_NQ: 2 trades, -$124
ES_SPY: 64 trades, +$418
ES_YM: 10 trades, +$264
EUR_EUR: 92 trades, +$377
GBP_GBP: 21 trades, +$61
NQ_QQQQ 36 trades, +$19
YM_DIA: 51 trades, +$425

[new_trader]

> Yes, this may explain it. In JBT, data is still being written to disk,
> but in a separate thread. In JArbitrager, it's the same thread that
> processes API messages, runs strategies, and writes to file. I'll make
> some adjustments to see how the consistency is affected.
>
> Thanks again for your participation in the test, folks!

great that you will modify JAB in this way!
i am looking forward to our upcoming sessions next week :-)

[nonlinear5]

I looked at my results for ES_SPY on July 9. With the exception of the
first and the last trade, all trades were profitable. The profit per
trade is extremely small, between $5 and $10. This means that even the
smallest slippage, such as a buy executing at 107.10 instead of
assumed fill price of 107.09 would turn the strategy into a consistent
loser. Here are potential sources for this slippage:

1. The strategy calls the evaluate() method after one of the
instruments bid has changed, but the ask update didn't have a chance
to fire. For example, current bid/ask is 100.00/100.01. The new best
bid comes in at 99.99. The evaluate() method is invoked, but the new
offer of 100.00 didn't have a chance to fire yet. Similar thing can
happen with the ask. Basically the problem is with the evaluate()
method triggering in the middle of the bid/ask update for the same
instrument.

2. Similar thing as above, except that the evaluate() method triggers
in between the updates of the two instruments. For example,
instrument1 is 100.00/100.01 and instrument2 is 100.00/100.01. Next,
the bid/ask for instrument1 becomes 100.10/100.11. The evaluate()
triggers at this point, sees the widened spread, and decides to sell
short instrument1 and buy instrument2. Before the order is executed,
the new bid/ask for instrument2 arrives at 100.10/100.11. As a result,
the order will be executed with a 10 cent slippage.

If this is indeed what's happening, the backtest and forward test
results would be different for the live results. I am thinking about
how to best model the backtest to more faithfully capture what would
happen in real trading. I suppose one way is to allow a time delay, so
that the orders will be assumed to be filled at the prices which are 1
second after the time the order is placed. If anyone can think of
another solution, please let us know.

[new_trader]

this is indeed a big issue.
an additional point would be if IB is artificially slowing down the
execution of the orders to prevent their customers from arbitrage
advantages.
As JAB is placing MARKET orders we are beyond the point of control.
I do not know if placing LIMIT orders would be a possible solution -
besides the fact that limit orders would be much harder to program
than market orders.

Another question is if the effect you described is "averaging out"
over time by having this situation in the positive and negative
direction.
It would be definitely a good idea to build the backtesting algorithm
more conservative.

At the end we will only know if we do some live trading with JAB! this
is the real source of truth.
I will be willing to do so even on the risk of loosing some bugs. When
we have some more significant backtest data I will jump into this.

[nonlinear5]

What I will do tomorrow is run JAB with the paper trading account in
the *trading* mode. We know that the data feed for the paper trading
account is different from the real account. However, for the purposes
of this experiment, all we need is to determine is the difference
between the trading results and the backtest results using the data
recorded during the session. If there is slippage, we would see the
live trading results much worse than the backtesting results.

[new_trader]

On Jul 12, 8:23 pm, nonlinear5 <eugene.kono...@gmail.com> wrote:
> What I will do tomorrow is run JAB with the paper trading account in
> the *trading* mode. We know that the data feed for the paper trading
> account is different from the real account. However, for the purposes
> of this experiment, all we need is to determine is the difference
> between the trading results and the backtest results using the data
> recorded during the session.  If there is slippage, we would see the
> live trading results much worse than the backtesting results.

surely a very good approach. the problem is that as a result we know
how the paper account is behaving.
I could imagine that when we are trading with the real account we will
have another behaviour.
For example the behaviour of how orders are filled would be totally
different between the two account types.

Is your intention to get an impression of the general "mechanics"
behind the order mechnanism IB uses?

I am just thinking about to let JAB run for 30-60 minutes in live
trading mode with the real account using a strategy with instruments
which are not to heavy for the margin.

[nonlinear5]

> I am just thinking about to let JAB run for 30-60 minutes in live
> trading mode with the real account using a strategy with instruments
> which are not to heavy for the margin.

Yes, this would be the ultimate test. I think the EUR_EUR pair has the
smallest required margin (at around $9K)

[new_trader]

> Yes, this would be the ultimate test. I think the EUR_EUR pair has the
> smallest required margin (at around $9K)

from my side I would be in. you mentioned you were thinking about some
improvements concerning the comparability of our forward test results,
which were differing significantly.
would it be recommendable to wait for these modifications before going
into a live test?

[nonlinear5]

> from my side I would be in. you mentioned you were thinking about some
> improvements concerning the comparability of our forward test results,
> which were differing significantly.
> would it be recommendable to wait for these modifications before going
> into a live test?

I originally wanted to move the code which runs the strategy into a
separate thread, but I am now thinking that it would make it less
consistent, not more consistent, when comparing to other people's
results. But regardless of that, let me put the next release out,
because it contains other improvements.

[new_trader]

> I originally wanted to move the code which runs the strategy into a
> separate thread, but I am now thinking that it would make it less
> consistent, not more consistent, when comparing to other people's
> results.

why should it make things worse? because of the problems you metioned
in above post in point 1. and 2. where in the evaluate() method a new
bid is not arriving?
perhaps in tickPrice() the strategy should only be updated if a
corresponding bid/ask pair is available? so you do not update
strategies in every tickPrice(), but only when consistent bid/ask
pairs are available.

> But regardless of that, let me put the next release out,
> because it contains other improvements.

which improvements?