Market data issues today

[nonlinear5]

I noticed that JBT stopped updating the ES price today, and this is an
indication that something is wrong with the market depth. So, I looked
it up in TWS, and sure enough, there are problems. See the attached
pic which shows TWS Market Depth and TWS BookTrader side by side.

There are multiple issues there:
1. All the bids in TWS BookTrader match those in TWS Market Depth, but
the totals do not match (although the difference is not that large)

2. On the ask side, TWS Market Depth shows multiple levels at the same
price, while TWS BookTrader seems to accumulate them into one level.

3. The total ask size in TWS Market Depth is very different from the
total ask in TWS Market Depth.

There is some market depth validation logic in JBT MarketDepth.java.
When it detects that something is gross in market depth, it stops
updating the price. This is what is happening today.

My question to the group is, what IS market depth based on these
attached snapshots and how should JBT deal with these discrepancies?

[Eugene Kononov]

Snapshot attached.

[Eugene Kononov]

Another update. This time, I am showing three different views of the same data: TWS BookTrader, TWS Market Depth, and JBT main window. The market data seems to have been fixed now, and JBT updates the price. However, the total sizes still don't match between TWS BookTrader and TWS Market Depth.

[Martin Koistinen]

I've seen numerous times when IB's Book Trader window has differences from IB's Market Depth window.  The important thing, of course, is that JBT matches Market Depth.

[Eugene Kononov]

I've seen numerous times when IB's Book Trader window has differences from IB's Market Depth window.  The important thing, of course, is that JBT matches Market Depth.

From what I've seen, JBT's market depth always matches what's in the IB Market Depth window, even when market depth itself is suspicious, as in examples above. If we establish that this is, in fact, what matters (that is, that JBT matches IB Market Depth), then we could simply remove the validations from the JBT code. That would mean that JBT would assume that market depth coming from IB is always valid. With this approach, all the JBT variables (market depth, price, and indicators) would update without interruptions.

I still would like to know why IB BookTrader is often different, though. I guess we'll have to ask IB people.

[nonlinear5]

I looked at IB BookTrader window a lot before, but I just noticed that
the cumulative bid and cumulative ask size numbers are updated very
infrequently there, something in the order of once per minute or so.
That means it's meaningless to compare them to those in the IB market
depth window where the cumulative numbers are changing several time
per second. So, perhaps, the issues that I reported above are not as
severe as they appeared to me first.

[ecthx]

My opinion is that TWS BookTrader is simply smarter tool when it comes
to handling
market depth. Removing JBT validation logic would solve our problems
but then can we trust signals generated from data that might be
distorted?
I remember that last year we did some testing with raw data. Back then
our main concern
was timing and discrepancies among users, but maybe we should do some
more testing?
We could record raw data with every operation listed (insert update
delete) and then analyze it
line by line.

More extreme example :)
http://jbooktrader.googlegroups.com/web/depth16.png?gsc=TqIuRAsAAADBZL-Hty4GHhz5EyyI8Kvm

[MKoistinen]

Very weird. Does anyone know who to contact at IB to get to the bottom
of this? I suppose posting something to the forums might yield some
answers, but speaking directly to someone at IB might tell us which of
these two windows to trust.

In the meantime, what should we be doing with our code? When we
detect a discrepancy, should we:

1) Do nothing (current code stops updating prices, continue with this
practice)
2) Trust IB BookTrader, continue on as normal.
3) Trust IB Market Depth, continue on as normal.
4) Trust nothing, setPosition(0) in all strategies on this underlying
until things match.

My feeling is that option 4 is the safest thing to do, BUT, I can
easily see that this may happen many times per day intermittently and
I'd hate for entire days to go to waste (or worse) by getting flat on
a little hiccup that may occur.

No. 1 is possibly the worst thing to do currently, as it leaves our
positions open without any knowledge of what is happening to the UL
price. The only thing that will close our positions here, is the end
of any defined TradingSchedule.

So, considering this, I suggest we go to Nos. 2 or 3. Which one, I
can't be sure. :)

There may be other options too.

On 17 July, 21:19, ecthx <marcin.pikul...@gmail.com> wrote:
> My opinion is that TWS BookTrader is simply smarter tool when it comes
> to handling
> market depth. Removing JBT validation logic would solve our problems
> but then can we trust signals generated from data that might be
> distorted?
> I remember that last year we did some testing with raw data. Back then
> our main concern
> was timing and discrepancies among users, but maybe we should do some
> more testing?
> We could record raw data with every operation listed (insert update
> delete) and then analyze it
> line by line.
>

> More extreme example :)http://jbooktrader.googlegroups.com/web/depth16.png?gsc=TqIuRAsAAADBZ...

[Eugene Kononov]

1) Do nothing (current code stops updating prices, continue with this
practice)
2) Trust IB BookTrader, continue on as normal.
3) Trust IB Market Depth, continue on as normal.
4) Trust nothing, setPosition(0) in all strategies on this underlying
until things match.

JBT always matches the IB Market Depth Window, even when the data becomes suspicious. I don't think there is a way to match IB BookTrader, so there is really no option 2.

I've added some diagnostic logic to MarketDepth.java and I'll run in on Monday to see the extend of this problem. I've attached the source code if anyone wants to do the same. It logs all the discrepancies in market depth to event report.

[Eugene Kononov]

Diagnostics code attached.

[MKoistinen]

Thanks, Eugene,

I already feel better knowing that you're "on it" :) I'll also do the
same, perhaps we can compare notes after close tomorrow. Perhaps this
is affecting some more than others?

[Eugene Kononov]

I already feel better knowing that you're "on it" :)  I'll also do the
same, perhaps we can compare notes after close tomorrow.  Perhaps this
is affecting some more than others?

On Friday, I restarted TWS several times to see if the problem would be solved, but it persisted. This makes me think that it's affecting everyone in the same manner. But I will be interesting to compare.

Incidentally, you can also spot the market depth problems by looking at the performance chart after a backtest. Since JBT stops updating the price when market depth becomes invalid, it leaves the characteristic horizontal line patterns on the chart when this happens. I attached an example (not from Friday, but from another day).

[Eugene Kononov]

Very weird. Does anyone know who to contact at IB to get to the bottom
of this?  I suppose posting something to the forums might yield some
answers, but speaking directly to someone at IB might tell us which of
these two windows to trust.

There is an "API Chat" with the IB API Specialists (I've never used it):
http://chatsrv1.interactivebrokers.com/fchat/chat/flashchat.php

There is also API support by email. I used it a few times, it was marginally helpful:
a...@interactivebrokers.com

[Eugene Kononov]

I made some last minute changes to MarketDepth.java. Please use this version (attached) if you would like to run JBT with the diagnostic reporting on Monday.

[nonlinear5]

Market depth is looking clean so far today. I had three discrepancies
at 09:01:49, 09:04:31, and 10:42:07 (all times are in EDT), but each
one lasted only 1 second. What do you see, Martin?

>  MarketDepth.java
> 9KViewDownload

[nonlinear5]

OK, I had a total of 6 instances when market depth was invalid in the
period from 6:30 to 16:15 EDT today. In all cases, it lasted for 1
second and market depth returned to normal after that. My inclination
is to add some logic to JBT to close open positions and stop trading
if the market depth stays invalid for longer than, say, 5 minutes, and
leave everything else as it is right now.

[Martin Koistinen]

That sounds pretty reasonable to me.  Can I suggest that you make the time-threshold a preference panel item?  And perhaps an on/off switch for this?

[Eugene Kononov]

On Mon, Jul 20, 2009 at 6:25 PM, Martin Koistinen <mkois...@gmail.com> wrote:

That sounds pretty reasonable to me.  Can I suggest that you make the time-threshold a preference panel item?  And perhaps an on/off switch for this?

Right, OK.

[ecthx]

Eugene can you tell me what part of JBT 7.01 validation logic rejects
this book state:
http://groups.google.com/group/jbooktrader/attach/6d73aad569adff53/marketDataIssues.png?part=4&view=1



On Jul 20, 6:10 pm, Eugene Kononov <eugene.kono...@gmail.com> wrote:

[Eugene Kononov]

Eugene can you tell me what part of JBT 7.01 validation logic rejects

this book state:
http://groups.google.com/group/jbooktrader/attach/6d73aad569adff53/marketDataIssues.png?part=4&view=1

JBT 7.01 would not reject it, because the validation logic in 7.01 is somewhat relaxed, as it allows the same price on multiple levels. However, one can tell that something is seriously wrong with the market depth by looking at this snapshot, specifically on the "ask" side. Even if we assume that IB BookTrader somehow "intelligently" corrects the IB Market Depth by aggregating the sizes, we still see only 6 ask levels. This is *very*  suspicious. No one in the world wants to sell above 939.5?

For the next release, I am making the following changes.

1. Market depth validation will be stricter. Specifically, market depth would be considered valid if all four of the following conditions are true:
a. The number of bid levels equals the number of ask levels and is non-zero
b. The bid price of level N is smaller than the bid price of level N-1 for all levels
c. The ask price of level N is greater than the ask price of level N-1 for all levels
d. The best bid price (at level 0) is smaller than the best ask price (at level 0)

2. If market depth is detected as invalid, the last known valid balance would be used to update the indicators.

3. If market depth has been invalid for significantly long period of time P, the corresponding strategy will close its open position (if any) and stop trading. The parameter P can be set in the preferences. I am thinking the default should be somewhere between 5 and 15 minutes.

I am open to discussion about these changes.

[Eugene Kononov]

Here is another instance of what I think is corrupted market depth. There is a hole on the ask side. If you folks are running TWS at the moment, can you confirm that you see the same thing?

[ecthx]

yep, I still see a hole at 948

>  marketDataIssues4.png
> 32KViewDownload

[nonlinear5]

> yep, I still see a hole at 948

Well, at least we know that it's corrupted for all of us. Same here,
no offers at 948.

[ecthx]

IB BookTrader and MarketDepth work beautifully with stocks, its really
hard to spot
anything suspicious just by looking at both windows. When IB
BookTrader encounters multiple levels at the same price
it aggregates them because it was designed to do that. It basically
assumes that these orders are from different market
makers. Most of the time it does the same thing with the futures even
though it shouldn't.
So I guess when we see multiple levels at the same price (futures
only!) in Market Depth window IB Book Trader doesn't show
correct sizes either.
My theory is that multiple levels might be caused by an uptick or
downtick before complete update of every level.
I still don't understand why those errors in the book remain for
minutes at a time. Once price settles for a while
IB server should update every row and everything should go back to
normal.
Maybe some sort of book preprocessing would solve it.

[nonlinear5]

Market depth issues resurfaced in the last two days.

[Eugene Kononov]

Here is the snapshot from yesterday.

[Eugene Kononov]

OK, here is what I am thinking of doing.

The currently implemented market depth validation rules appear to be too strict. In particular, the ES bid levels frequently have the same price (as shown in the pic above), and they persist for many minutes (if not hours). Under these conditions, JBT flags market depth as "invalid", discards it, and uses the last known "valid" depth to calculate the indicators. Clearly, if this continues for longer than several seconds, the indicators become "stale" and they no longer reflect the current book balances.

Disabling the market depth validation does not seem like a good option. Let's look at the above example again. The cumulative bid size is 7502, the cumulative ask size is 10967. So, it appears that the balance is negative:
balance = 100 * (7502-10967) / (7502+10967) = -18.76

However, this snapshot has only 5 *actual* bid levels (1000, 1000.25, 1000.5, 1000.75, and 1001), while the *actual* number of ask levels is 10. To compensate for this, what I propose is dividing the cumulative bid size by the number of *actual* bid levels, and dividing the cumulative ask size by the number of *actual* ask levels, before calculating balance. With this adjustment, we would get:
adjusted cumulative bid = 7502 / 5 = 1500
adjusted cumulative ask = 10967/ 10 = 1097
balance = 100 * (1500-1097) / (1500+1097) = +15.52

What do you folks think?

[JBTrader]

Makes a lot of sense...
the only thing is how "comaptible" new figure will be with the others
(already recorded)

thanks

[Eugene Kononov]

Makes a lot of sense...
the only thing is how "comaptible" new figure will be with the others
(already recorded)

When all levels are present (which is the case most of the time), the new balance would be exactly the same as the old balance:

(cumulativeBid - cumulativeAsk) / (cumulativeBid + cumulativeAsk) = (cumulativeBid/10 - cumulativeAsk/10) / (cumulativeBid/10 + cumulativeAsk/10)

[ecthx]

It doesn't look right to me.
Since there is only 5 correct bid levels we can assume that
approximate cumulative bid size is 3500 vs 10967. No matter how you
calculate the balance it should be negative (even if we take 7502 as a
correct
cumulative bid size).

[nonlinear5]

> It doesn't look right to me.
> Since there is only 5 correct bid levels we can assume that
> approximate cumulative bid size is 3500 vs 10967.

Where does 3500 come from?

[ecthx]

I assumed that that when we see multiple levels at the same price only
one
of them can be taken into account.
When we see:
1000.50 size 560
1000.50 size 414
1000.50 size 1811
only one of these orders is correct, and you can easily spot which one
by
looking at the live market depth window.
3500 is not exactly right, I simply picked one order out of every
multiple price
segment to estimate the bid size and that's where 3500 comes from.

[Eugene Kononov]

I assumed that that when we see multiple levels at the same price only
one
of them can be taken into account.
When we see:
1000.50  size 560
1000.50  size 414
1000.50  size 1811
only one of these orders is correct, and you can easily spot which one
by
looking at the live market depth window.

I don't think that's the right assumption, though. In this example, I think the actual bid size at 100.5 is 560+414+1811=2785. This is what IB BookTrader window would show.

[ecthx]

I have to admit that i didn't do any in-depth analysis and my
conclusions
come simply from looking at market depth window so I might be
wrong :)
I think that when we see multiple levels at the same price both Market
Depth
window and Book Trader are showing incorrect sizes.
TWS Book Trader aggregates those multiple orders because it assumes
they are from different
market makers and as I mentioned in my earlier post it works just fine
for stocks.
When IB server updates order at 1000.5 with new size all other sizes
at that price should be
discarded (if they exist on different levels).

[ecthx]

Tomorrow I will test this code change to the MarketDepth.java
case Update:
if (position < levels) {
MarketDepthItem item = items.get(position);
item.setSize(size);
item.setPrice(price);

for (int i = 0; i < levels; i++) {
if (item.getPrice() == items.get(i).getPrice() &&
i != position) {
items.get(i).setSize(0);
}
}

}
I also disabled part of the validator that checks for multiple levels.

[Eugene Kononov]

Tomorrow I will test this code change to the MarketDepth.java
case Update:

Well, the thing is, we don't know if our changes are legitimate, unless we have some explanation for the duplicate price levels. I sent an email to API support asking them to clarify this.

[ecthx]

I stumbled across this thread on elitetrader.com:
http://elitetrader.com/vb/showthread.php?s=&threadid=164903&perpage=6&pagenumber=9
It got IB's attention and they are working on a fix.

[Eugene Kononov]

Hmm, interesting. Hopefully we will not see that "multiple levels at the same price" nonsense after the fix.