Special multi-user test for Friday, May 9

[nonlinear5]

This is a special multi-user test. It has two purposes:

a. Test drive a new mechanism of capturing market depths. This new
mechanism is actually much more straighforward than what has been used
in all the previous releases. In this "special build" release, market
depths are captured as they occur, without any kind of timing.

b. Create a full history of all market depths messages so that we can
figure out how to best capture market depths in their "completed"
states.

If you would like to participate in this test, please follow these
steps:

1. Download the special JBT build from
http://jbooktrader.googlecode.com/files/JBookTrader-specialBuild.zip

2. Setup, build, and run this special build the same ways as you did
it with regular JBT releases.

3. Start TWS and log in to your real TWS account. No actual trades
will be performed in the "forward test" mode.

4. At any time before 9:20 EDT, start JBT and run the following
strategies in the "forward test" mode: Test1, Test2, Test3.

5. As the trades occur, please report them in this thread (number of
trades and P&L for each strategy).

6. At any time after 16:10 EDT, please report your final results.


During the test, JBT will write raw market depth messages to files:
Test1rawMarketDepth.txt, Test2rawMarketDepth.txt, and
Test3rawMarketDepth.txt. Each one will look like this:

050808, 20:54:20.715, bid, 3, update, 134, 1388.25
050808, 20:54:38.465, ask, 4, update, 38, 1389.75
050808, 20:54:38.465, ask, 5, update, 64, 1390.0
050808, 20:54:39.465, ask, 4, update, 39, 1389.75
050808, 20:54:44.715, ask, 5, update, 65, 1390.0
050808, 20:54:55.465, bid, 1, update, 22, 1388.75
050808, 20:54:55.465, bid, 1, update, 21, 1388.75
050808, 20:55:11.465, bid, 1, update, 22, 1388.75
050808, 20:55:13.965, ask, 4, update, 44, 1389.75
050808, 20:55:13.965, ask, 5, update, 60, 1390.0
050808, 20:55:14.965, ask, 3, update, 36, 1389.5
050808, 20:55:15.309, ask, 3, update, 37, 1389.5

The format is:
column1: date
column2: time
column3: bid or ask side
column4: book level (1 to 5)
column5: operation (insert, update, delete)
column6: size for the level
column7: price for the level

We need testers! Thanks for your participation.

[nonlinear5]

Small correction: step #4 must be done at any time before 9:15 EDT.

[Kelvin]

Can you give us a brief explaination of what is the difference between
test1, test2 and test3?

[nonlinear5]

> Can you give us a brief explaination of what is the difference between
> test1, test2 and test3?
>

All three tests are essentially equivalent to Scalper in JBT 2.12. The
difference is that none of the three use any kinds of timers to
capture market depths. Instead, market depths are captured on every
single update, so nothing is ever skipped or missed. Test1 uses raw
market depths to calculate the indicator, and Test2 and Test3 do some
averaging. At this point, I am not interested in absolute performance,
but in the *differences* in performance among us as we run these
tests. Hopefully, we'll have much more similar results today compared
to what we observed in all previous tests.

[rickty]

Test1: long, 3.85
Test2: long, 3.85
Test3: long, 3.85

[nonlinear5]

3 trades. All three tests went long on 9:20.

[Eugene Kononov]

Folks, please report your trades as they occur. This will make it easier for us to determine what happened if we go out of sync.

[Kelvin]

same here

[Shane]

same here

[rickty]

Test1: 1 trades, long, 141
Test2: 1 trades, long, 141
Test3: 2 trades, short, 136

[Kelvin]

Test1: 1 trades, long, 141
Test2: 1 trades, long, 141

Test3: 2 trades, short, 124

[nonlinear5]

Test1 and Test3 went short. Test2 is still long. 5 trades.

[Kelvin]

test 1 still long, still 4 trades

[rickty]

Test1: 1 trades, long, 128
Test2: 1 trades, long, 128
Test3: 3 trades, long, 181

[Kelvin]

Now:
Test1: 2 trades, short,
Test2: 2 trades, short,
Test3: 4 trades, short,

On May 9, 9:47 am, Kelvin <baoxing...@gmail.com> wrote:
> test 1 still long, still 4 trades
>
> On May 9, 9:46 am, nonlinear5 <eugene.kono...@gmail.com> wrote:
>
>
>

> > Test1 and Test3 went short. Test2 is still long. 5 trades.- Hide quoted text -
>
> - Show quoted text -

[rickty]

Test1: 1 trades, long, 186
Test2: 1 trades, long, 186
Test3: 4 trades, short, 239

[rickty]

Test1: 2 trades, short, 199
Test2: 2 trades, short, 199

[Shane]

all short 2 trades each

[ecthx]

I'm not sure if I should post my results because I'm running three JBT
instances
and this may affect the results but I will post anyway

Linux:
Special build:
Test1 5 trades 1
Test2 3 trades 1
Test2 5 trades 1
Windows:
Special build
Test1 2 trades -1
Test2 2 trade -1
Test2 2 trades -1


2.04
Scalper 2 trades -1

2.12
Scalper 2 trades -1

[rickty]

Test1: 3 trades, long, 206
Test2: 3 trades, long, 206
Test3: 5 trades, long, 234

[Kelvin]

Now:
Test1: 3 trades, short,
Test2: 3 trades, short,
Test3: 5 trades, short

JBT 212
Scalper: 2 trades, short

[Shane]

all long 3 trades each

[Kelvin]

sorry, last post has errors, forget it, now it should be:

Now:
Test1: 3 trades, long,
Test2: 3 trades, long,
Test3: 5 trades, long

JBT 212
Scalper: 2 trades, short

> > all short 2 trades each- Hide quoted text -

[nonlinear5]

Test1: 5 trades, long, +266
Test2: 3 trades, long, +200
Test3: 5 trades, long, +241

[nonlinear5]

Looks like we all are in sync, discounting a few trades. Everyone
should be long on all three tests as of 10:00 EDT.

[Kelvin]

nonlinear, we are already out of sync. Geeze, it is strange
I will go out for breakfast, and return after half an hour.

[rickty]

Test1: 3 trades, long, 144
Test2: 3 trades, long, 144
Test3: 6 trades, short, 192

[ecthx]

Linux:
Special build:
Test1 5 trades 1
Test2 3 trades 1

Test2 6 trades -1

Windows:
Special build
Test1 2 trades -1

Test2 3 trade 1

[rickty]

Test1: 3 trades, long, 194
Test2: 4 trades, short, 151
Test3: 6 trades, short, 142

[rickty]

Test1: 4 trades, short, 164
Test2: 4 trades, short, 114
Test3: 6 trades, short, 117

[nonlinear5]

> Test1: 4 trades, short, 164
> Test2: 4 trades, short, 114
> Test3: 6 trades, short, 117

Same here, although my combined P&L is around +$600

[ecthx]

Test1 6 trades -1
Test2 4 trades -1
Test2 6 trades -1

P&L aroud $500

[Kelvin]

Linux:
Test1: 4 trades, long,
Test2: 4 trades, long,
Test3: 6 trades, long

Windows:

JBT 212
Scalper: 2 trades, short

[Kelvin]

the lastp post should be all short

> > P&L aroud $500- Hide quoted text -

[nonlinear5]

> Linux:
> Test1: 4 trades, long,
> Test2: 4 trades, long,
> Test3: 6 trades, long
>

Hmm. Can you check that again to be sure?

[nonlinear5]

>the lastp post should be all short

Oh, ok, good.

[ecthx]

Test1 7 trades 1
Test2 5 trades 1
Test2 7 trades 1

P&L around $700

[rickty]

Test1: 5 trades, long, 197
Test2: 5 trades, long, 172
Test3: 7 trades, long, 162

[Shane]

all long 5 trades each

[ecthx]

Test1 8 trades -1
Test2 6 trades -1
Test2 8 trades -1

P&L aroud $800

[nonlinear5]

Same here. Looks like all of us are in sync. Very encouraging.

[rickty]

Test1: 6 trades, short, 242
Test2: 6 trades, short, 217
Test3: 8 trades, short, 207

[nonlinear5]

> Test1: 6 trades, short, 242
> Test2: 6 trades, short, 217
> Test3: 8 trades, short, 207

Same: all three are short.

[rickty]

Test1: 6 trades, short, 217
Test2: 6 trades, short, 204
Test3: 10 trades, short, 235

[ecthx]

Test1 8 trades -1
Test2 6 trades -1
Test2 10 trades -1

P&L around $800

[rickty]

Test1: 7 trades, long, 300
Test2: 7 trades, long, 275
Test3: 11 trades, long, 318

[ecthx]

Test1 10 trades 1
Test2 8 trades 1
Test2 12 trades 1

P&L around $1050

[rickty]

Test1: 8 trades, short, 332
Test2: 8 trades, short, 320
Test3: 12 trades, short, 351

[Shane]

all short 8 trades  +345

[nonlinear5]

All three are long. P&L is around +$1060.

[rickty]

Test1: 9 trades, long, 365
Test2: 9 trades, long, 340
Test3: 13 trades, long, 358

[rickty]

Test1: 10 trades, short, 373
Test2: 10 trades, short, 373
Test3: 14 trades, short, 378

[rickty]

Test1: 11 trades, long, 401
Test2: 11 trades, long, 419
Test3: 15 trades, long, 406

[rickty]

It's flipping around alot, I'm taking a break.

[ecthx]

Test1 19 trades 1
Test2 17 trades 1
Test2 21 trades 1

P&L around $1500

it trades so often that it's hard to keep up
with posting updates

[nonlinear5]

All three are long. Combined P&L is around +1530. Printing money like
I've never seen before. All three behaving like true scalping
strategies. 57 trades.

[ecthx]

Test1 22 trades -1
Test2 20 trades -1
Test2 24 trades -1

P&L around $1600

[nonlinear5]

> it trades so often that it's hard to keep up
> with posting updates

Indeed. Let's wait for a quet time of at least 5 minutes (i.e. 5
minutes of no trades) before posting updates.

[Shane]

all short

[rickty]

Test1: 18 trades, short, 522
Test2: 20 trades, short, 587
Test3: 22 trades, short, 515

[nonlinear5]

Test1: 20 trades, short, 606
Test2: 18 trades, short, 516
Test3: 24 trades, short, 475

Looking good.

[Shane]

Test1: 18 trades, short, 460
Test2: 16 trades, short, 420
Test3: 18 trades, short, 460

[Eugene Kononov]

It's flipping around alot, I'm taking a break.

Yes, it looks like changing the way of capture also changed the nature of the strategies. It also made our results consistent.

[Kelvin]

Test1: 20 trades, -1 375
Test2: 20 trades, -1 400
Test3: 24 trades, -1 356


JBT 212
Scalper: 10 trades, -1 236

[rickty]

> Yes, it looks like changing the way of capture also changed the nature of
> the strategies. It also made our results consistent.

What is the difference in the code?

[Gab]

Test1: 14 trades, -1   475
Test2: 12 trades, -1   435
Test3: 12 trades, -1   288

[rickty]

Test1: 19 trades, long, 404
Test2: 21 trades, long, 457
Test3: 23 trades, long, 373

[Gab]

Test1: 15 trades,  1   500
Test2: 13 trades,  1   458
Test3: 13 trades,  1   308

[ecthx]

Test1 23 trades 1
Test2 21 trades 1
Test2 25 trades 1

[nonlinear5]

All three are long. Gave up some of the gains in the last 30 minutes.
Combined P&L is around +$1200.

[Gab]

Same here, My combine P&L is 1200$, but I have less trades than you

[ecthx]

Don't worry Gab I have similar results on my Windows Box

Linux:

Test1 23 trades 1
Test2 21 trades 1
Test2 25 trades 1

Windows:
Test1 15 trades 1
Test2 17 trades 1
Test2 13 trades 1

[ecthx]

Test1 24 trades -1
Test2 22 trades -1
Test2 26 trades -1

P&L around $ 1200

[Shane]

Test1    19   trades     1
Test2    17   trades     1
Test2    21   trades     1

P&L around $ 1250

[nonlinear5]

Here are some observations, based on today's test so far:

1. All of us seem to be in sync. That tells me that memory settings
and network connections have very little impact on performance. Our
differences in the previous tests can be attributed to the differences
in timing the market depths capture.

2. I expected Test1 to make a lot more trades than Test2, and Test2 to
make a lot more trades than Test3. I had this expectation because
Test2 and Test3 average the depths, while Test1 does not. However,
that did not materialize. All three seem to be flipping with about the
same frequency. My guess is that the reason it's happening is because
when IB sends a market depth update, it's has been already averaged.
We've seen from the CME data set that in real trading, market depths
happen every 10 milliseconds or so, while TWS sends them at the rate
of about every 300 milliseconds. That means that the market depths
coming from TWS are already sampled and/or averaged, and further
averaging by JBT doesn't change the MD too much.

3. It appears that all three tests trade with much higher frequency. I
am not sure if it's good or bad yet. The P&L today is impressive, but
one day of testing doesn't tell us much.

4. If we decide to promote this test build to next JBT release, I am
not sure what to do with the historical data file. To make it
consistent with "no timing", the file would have to become about 5 to
10 times larger for every recorded day.

[nonlinear5]

Same here.

[rickty]

Currently,
Test1: 19 trades, long, 342
Test2: 23 trades, long, 498
Test3: 23 trades, long, 323

[Gab]

1. All of us seem to be in sync. That tells me that memory settings
and network connections have very little impact on performance. Our
differences in the previous tests can be attributed to the differences
in timing the market depths capture.

If network connection and memory settings have little impact on our results, would you attribute the difference to  the OS and the hardware of everyone?

3. It appears that all three tests trade with much higher frequency. I
am not sure if it's good or bad yet. The P&L today is impressive, but
one day of testing doesn't tell us much.

I tend to think that's a good thing.  The more a strategy trade, the more it's likely that a *missed* trade would not affect the entire day.  I agree that we have to test a lot more, but the results today seem in the right way.

4. If we decide to promote this test build to next JBT release, I am
not sure what to do with the historical data file. To make it
consistent with "no timing", the file would have to become about 5 to
10 times larger for every recorded day.

Maybe we would have to start sharing work to optimize the strategy.

[Eugene Kononov]

Test1: 23 trades, long, 467
Test2: 21 trades, long, 377
Test3: 27 trades, long, 360

I am off to lunch. Will update in an hour.

[ecthx]

Test1 25 trades 1
Test2 23 trades 1
Test3 27 trades 1

P&L around $ 1400

[Shane]

>>4. If we decide to promote this test build to next JBT release, I am
not sure what to do with the historical data file. To make it
consistent with "no timing", the file would have to become about 5 to
10 times larger for every recorded day.

I say, the more "raw" we get w/ this the better.  Don't worry about the file size.

[Shane]

Test1: 19 trades, long, 380
Test2: 17 trades, long, 340
Test3: 21 trades, long, 445

[Gab]

I say, the more "raw" we get w/ this the better.  Don't worry about the file size.

Yes, I agrre with you Shane, if the data is already sampled by IB, we should take the data as raw as possible from IB.  If it makes the file bigger, then we'll need to upgrade our computers!

[Kelvin]

Test1: 23 trades, long, 335
Test2: 23 trades, long, 360
Test3: 27 trades, long, 291


JBT 212
Scalper: 11 trades, long 143


It seems there still some difference between us, say Shane's trades is
much less than us.

[ecthx]

Kelvin. Are you using Windows or Linux?

[Gab]

It seems there still some difference between us, say Shane's trades is
much less than us.

Yes, and I trade less than all of you:
 

Test1: 15 trades,  1   325
Test2: 13 trades,  1   271
Test3: 13 trades,  1   121

[Kelvin]

I used two computers,

the special test version is on my Ubuntu OS,
and the JBT2.12 is on my Windows XP.

[Shane]

I want to get this working with the IB feed, but ultimately, long term, I think there will be more opportunities by going with the direct feed from CME.

[ecthx]

Linux
Test1: 26 trades, -1
Test2: 24 trades, -1
Test3: 28 trades, -1
P&L around $1000

Windows
Test1: 15 trades, 1
Test2: 17 trades, 1
Test3: 16 trades, -1
P&L around $1000

[ecthx]

Linux
Test1: 27 trades, 1
Test2: 25 trades, 1
Test3: 29 trades, 1
P&L around $1000

Windows
Test1: 17 trades, 1
Test2: 17 trades, 1
Test3: 17 trades, 1
P&L around $1000

[Kelvin]

I am confused that if the test is independent to timming, the
performance of Linux and windows should be the same. But, we see the
difference from ecthx's results.

There might be two reasons:
(1) The current special version still depends on the timing somehow.
(2) The data is sent by UDP, and there might be some lost data.

[ecthx]

Yes it looks weird but I had the same results with 2.12 two days ago.
Yesterday however, both computers were running Linux and results were
the same.
I don't think there is any missing data because both computers are
standing next to each other and
they are using same TWS.

[ecthx]

Hey guys, did you check the size of today's market data files.
38MB!!!!

[nonlinear5]

Test1: 25 trades, long, +307
Test2: 25 trades, long, +345
Test3: 31 trades, long, +216

[ecthx]

Test1: 27 trades, 1
Test2: 25 trades, 1

Test3: 31 trades, 1
P&L around $1000

[Gab]

Hey guys, did you check the size of today's market data files.
38MB!!!!

Oh boy, I'm up to 43MB now.  That's a big differene!

The other day, the test ecthx made showed that two LINUX computers produced the same results when a windows and a LINUX showed differences.  That tend to indicate that the OS has a big influence in the results.

[Kelvin]

Special test, Ubuntu 8.04
Test1: 25 trades, long, 226
Test2: 27 trades, long, 292
Test3: 29 trades, long, 207

JBT 212
Scalper: 11 trades, long 118