Summary of 2012 and outlook for 2013

[nonlinear]

Hello JBTers, and happy new year!

It's been relatively quiet in the JBT discussion forums, but for me personally, a lot of things have happened:

1. Another JBookTrader version was released, which added to the stability of the platform.

2. I traded with JBookTrader live, every day, methodically, and without interference. It paid off, as my account returned 50%. I was trading 8 strategies, 4 short-only, and 4 long-only. Each strategy is independent, and trades 1 ES contract. The performance chart (from the IB report) is below.

3. In 2013, I am scaling it up to 14 ES strategies, and starting to accumulate the data for another instrument (the 30-yr bond US future).

4. Three months ago, I started tracking the performance of my IB account with RapaCap, which is the service to match best traders with the capital. Based on the short term tracking performance, I made it to the top 10 traders:

http://rapacapintro.com/account/leaderboard/landingpage

My account is shown as nonlinear/JBookTrader in the leaderboard.

5. I am continuing to work on JBookTrader to improve its robustness, features, and performance. Sometime in January, I'll release the next version of JBookTrader, which would include:

a) improved performance of the "brute force" and the "divide-and-conquer" optimizer

b) automated holiday detection

c) portfolio manager (for better handling position sizing when running multiple strategies)

d) new performance metric

e) enhancement to trading schedule

f) re-subscription feature for long-running JBT sessions

g) other minor changes and improvements

Feel free to share your success stories (and failure stories) with JBookTrader in this thread.

Good luck to you all in 2013,

Eugene Kononov.

[JeanMorinGmail]

Congratulations Eugene,

I continued to follow the group even if I'm no longer active in the group and in the Trading

Bravo

Jean

Jean

Envoyé de mon iPhone

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.
To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/mhl8rn0h7q0J.
To post to this group, send email to jbook...@googlegroups.com.
To unsubscribe from this group, send email to jbooktrader...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.

[Klaus]

Dear Eugene, 

my congratulations to the great work and of course to the very successful trading!

It seems your dream of successful fully automated trading comes true - and pays off.

Have a great 2013!

 Klaus

[Dyno Brium]

Congratulations. I am happy to hear your successful story. I hope to hear more successful stories, either from you or other persons.

The PnL chart looks good except the bump in June and July. Probably due to your short strategies. Overall very good performance.

[Eugene Kononov]

Congratulations. I am happy to hear your successful story. I hope to hear more successful stories, either from you or other persons.

From various random bits. I get a sense that not a lot of people are trading live, but some are collecting data. 

 

[Eugene Kononov]

 

- Another idea not related to the above, and not even implying any change to JBT, would be to compile a sort of 'feared events' data set, or black swans if you like, including only days in which the market behaved weirdly, to backtest a strategy before launching. This is done often in safety systems engineering (which is more related to my background than futures trading!).

This can be done without any code modifications. Simply backtest your strategy using the data file which contains the "weird" events. For example, the infamous "flash crash" May 6, 2010 biased the optimization of my strategies so much (albeit in a good way), that I decided to exclude this day from my optimization data set. However, I backtest against this day when I consider new candidate strategies for trading.

[Judson Wilson]

On Tue, Jan 8, 2013 at 1:43 PM, Iñaki Andersen <inakia...@gmail.com> wrote:

- It seems that the strategies work fine over a period of weeks following a certain market trend or market conditions. However, as JBT strategies are intraday, they do not take into account the overall trend of the last weeks. However, my experience is that they can be quite sensitive to it. 

For example, perhaps the platform could easily generate a file (from the logged data or from IB) with some daily values e.g. the open, close, max, min, volume, etc. so that strategies use this info when launched everyday e.g. to not enter (or control position sizing in the new version?). Maybe it goes a bit agains the JBT phylosophy, and maybe there are other ways to avoid entering in adverse periods, but I thought it was worth commenting... 

This was one of my next areas of work, but I am on indefinite hold for the moment (grad school). 

The main issue I found is that every few months the contract switches. This creates a disconnect in your data. This is a long known-about issue, and blending the edges goes back probably to the beginning of quantitative futures or options trading.

My proposed solution is to use the underlying ES index. This would be price only, not volume. Volume is a strange beast to think about when it comes to roll-over. I was thinking a simple open/high/low/close bar for each day would be very useful for creating basic notions of long term trend and volatility.

[Iñaki Andersen]

Thanks Eugene and Judson for your answers. I agree with Judson that using the ES index seems a priori the best solution, to avoid contract switch distortions in the JBT logged data, and volume seems not so important. I'll keep you posted if I have any progress in any of those areas.

[Eugene Kononov]

Alexander,

I responded to your private mail, but it bounced back. Sounds like a corporate account. Please provide another email.

On Wed, Jan 9, 2013 at 2:36 PM, <editio...@swissonline.ch> wrote:

Hi Eugene,

this sounds great. You have done a tremendous job! By the way I have learned a lot about java programming while inspecting your code. Thank you! It's a pity that I had no time last year to follow up your idea with book trading but I will have more time this year. I have started to collect data for CL, GC and ZC this year. Unfortunately, IB does not allow more than 3 markets at the time. Some time ago you have offered your strategies for sale. I would be interested in it and sent you an e-mail in this regard.

All the best to you,

Alexander

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.

To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/pDroCu55r6QJ.

[editio...@swissonline.ch]

It sounds corporate but it is not. I'll send another e-mail.

[Marcus Williford]

Looking at the code, and this feature request.  I think we can use IB API to get the historical data, since we all pay for it! 

My suggestion is to create a new type of object, which lazy loads a 1yr security historical data for any security. Let's call it SecurityHistory (think of this like a macro marketbook, with no book data). 
Next, we need to have a tier for HistoricalIndicators, which can get access to SecurityHistory.  These HIstoricalIndicators can have state, for EMA calculations, etc.  Then, any Strategy can create as many HistoricalIndicators it wants.  I don't think this would break the existing object oriented model too much ;). 

If you want, I can write this as an example and share it with the team for review.  Any other thoughts about how to organize it, or features it needs before I start?

Marcus

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.

To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/Uditd7bns3YJ.

[Eugene Kononov]

On Wed, Jan 9, 2013 at 5:02 PM, Marcus Williford <mwill...@gmail.com> wrote:

Looking at the code, and this feature request.  I think we can use IB API to get the historical data, since we all pay for it! 

My suggestion is to create a new type of object, which lazy loads a 1yr security historical data for any security. Let's call it SecurityHistory (think of this like a macro marketbook, with no book data). 
Next, we need to have a tier for HistoricalIndicators, which can get access to SecurityHistory.  These HIstoricalIndicators can have state, for EMA calculations, etc.  Then, any Strategy can create as many HistoricalIndicators it wants.  I don't think this would break the existing object oriented model too much ;). 

This is certainly doable. My only objection is that it would complicate things quite a bit. Specifically, for the purposes of backtesting and optimization (as well as trading and forward-testing), you would now need 2 historical data files (one containing 1-sec market depth samples, and the other one containing historical prices with some lower resolution). On top of that, from what I hear from Judson, the market depth historical data file would contain the future prices (as it's now), but the second one would contain the underlying index (such as SPY, if I am trading the ES).  

I think there may be a simpler solution, and I am willing to look into it, but I'd like to see the evidence that the multi-week and multi-months trends affect the intraday strategies. The average holding period of my strategies is about 30-40 minutes. I really don't anticipate that these types of strategies would be affected by taking long-term trends into consideration. More importantly, all my strategies are actually anti-trend (or mean-reversion, using a different terminology). However, there are of course so many ways to trade (and to auto-trade), that it would be silly for me to say that long term trends don't matter. If someone has such a strategy, it's still possible to use it with JBT as it is, by simply specifying filters (such as EMAs and SMAs) as indicators of long length. As Judson noted, the discontinuity associated with contracts rollovers would be a problem, but that can be hacked for the "proof of concept" purposes. If I see such as proof of good performance, I'd be motivated to enhance JBT so that it can support the time frames larger than a day.

 

[Marcus Williford]

I see, there is a flaw with my plan in backtesting and optimization.  My design proposal was to use IB to get historical data (so no data file needed, and this would have made it almost automatic for the user), but this has a limitation of only going back 1-year in history.  So, to backtest, It would ALSO need to support this historical index file you pointed out.  This does add some complexity, as you again need to know if you have valid date, gaps, then what to do if you do.  I guess I don't see another way, unless we use some other online data source dependency that can be automatic, and goes back longer in history for backtesting and optimization automatically.  Is there such a free db of open/close for popular indexes, I'm kinda weak in the where do get data part of this.

Summary:  If the user can deal with only backtesting and optmization over the last year (minus and period time required for indicators to settle), then it would be pretty easy to support.

Is 1-year enough valid data to satisfy most backtesters?  Is it worth a demo branch to try this setup?

I think I agree with Judson on using index as the historical trend line, so it would be ES index (not the real securities).

From a trading perspective, I think if a macro indicator was known, it might help influence entry/exit points with respect to short/long bias.  How, I don't know, but I'm sure someone would attempt to make this part of a valid strategy modification (even for short-lived ones).

Marcus

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.

[Judson Wilson]

> I think there may be a simpler solution, and I am willing to look into it,
> but I'd like to see the evidence that the multi-week and multi-months trends
> affect the intraday strategies.

Yes, the classic chicken and egg problem ;) I favor attempting
anything sane and logical, but obviously the burden is on the person
who actually wants the feature in the first place. I am fine keeping
these features as patches, as having a bloated code base in a trading
app is asking for bugs that cost real money.

I have a hunch that some basic parameters would work well as a simple
function of historical volatility. These parameters could be levels
at which a trade could be opened or closed, for instance, or perhaps
indicators that trading at a given time could be dangerous.

> This is certainly doable. My only objection is that it would complicate
> things quite a bit. Specifically, for the purposes of backtesting and
> optimization (as well as trading and forward-testing), you would now need 2
> historical data files (one containing 1-sec market depth samples, and the
> other one containing historical prices with some lower resolution). On top
> of that, from what I hear from Judson, the market depth historical data file
> would contain the future prices (as it's now), but the second one would
> contain the underlying index (such as SPY, if I am trading the ES).

My thought on the OHLC daily bars is that they give a lot of
information for very little actual data.

I do not think adding the feature would be "hard", but it wouldn't be
trivial. Much much easier than trying to do full book pairs trading.
And repairing the data by hand would be a cinch if a day is lost
(assuming the data is available, but you could probably extrapolate
something "good enough" from a plethora of readily available sources.

A years worth of data is less than 1000 datapoints. So doing a "setup
long term indicators" routine at strategy start would be simple.

Except, yes, i just remembered the one thing that was complicated: we
would need to implement a trading schedule calendar, to know when the
exchanges are open and closed.

On Wed, Jan 9, 2013 at 3:55 PM, Eugene Kononov <eugene....@gmail.com> wrote:
>

> --
> You received this message because you are subscribed to the Google Groups
> "JBookTrader" group.

[Judson Wilson]

I wouldn't use the IB historical data route. I've had a lot of trouble
with it (on stocks) in the past. Better to keep our own data and keep
it as a known good reference.

The trade-off you want to make is more-difficult data reading (as
opposed to just reading from a file) at the gain of not having to
record anything.
Recording indexes is pretty simple, but adding in all the glue around
it does require more work (I found the GUI to be one of the trickier
parts). You would still need a lot of this glue, regardless. I don't
think recording our own files is too much additional code compared to
the IB historical data method.

[Marcus Williford]

Ok, so you prefer to have us calculate high/low/close for the day from the market book data?  Or, should we still use IB to get today's high/low/close (in the event we are recording)?

What if our jbook is briefly turned off while a high occurs.  Shouldn't we trust IB more than ourselves due to this possibility? 

Just flushing out the details.

Regarding the GUI, I think I can do this with no GUI impact.  This would simply be another type of indicator that a Strategy can get a reference too.  This would keep things really simple for the initial impl, and reduce the risk of any impact to the existing trading code.

Marcus

[Eugene Kononov]

Regarding the GUI, I think I can do this with no GUI impact.  This would simply be another type of indicator that a Strategy can get a reference too.  This would keep things really simple for the initial impl, and reduce the risk of any impact to the existing trading code.

Not sure how you'd do this without affecting the GUI. Wouldn't you need to add the corresponding controls to the backtesting and optimization dialogs to specify the *second* historical data file? The GUI changes aside, there is a lot more to do in the back end to somehow synchronize these data files.

There is a much simpler way out, though. We can add another column to the existing recording format, to record whatever we define as a "long term trend". For example:

# This historical data file was created by JBookTrader

# 1. date in the MMddyy format

# 2. time in the HHmmss format

# 3. book balance

# 4. price

# 5. volume

# 6. long-term trend (in relative strength units) 

timeZone=America/New_York

010713,080904,4.0,1455.875,10,95

010713,080905,4.61,1455.875,5,95 

010713,080906,4.63,1455.875,1,95

010713,080907,4.64,1455.875,0,95

010713,080908,4.59,1455.875,1,95

010713,080909,4.63,1455.875,0,95

That way, very few things would need to change, and all components of the system would be intacts. The downside is, of course, you've got to collect this for a year or so, before it becomes usable.

[nonlinear]

Alexander, here is the bounced header:

Delivery to the following recipient failed permanently:
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 552 552 5.2.0 iFZf1k00B3rVV9d08FZgTe automated process detected unsolicited content (state 17).

Looks like your spam filter needs to be relaxed.

[Judson Wilson]

> Ok, so you prefer to have us calculate high/low/close for the day from the
> market book data? Or, should we still use IB to get today's high/low/close
> (in the event we are recording)?
>
> What if our jbook is briefly turned off while a high occurs. Shouldn't we
> trust IB more than ourselves due to this possibility?

There are several different ways to get this data from IB in
real-time. I would have to look at my raw-recorder data again to see
what comes up. I feel like there is some specific open/high/low/close
data for the entire day that gets broadcast, but I am not sure. If
you mess up a day then we should record something that says the data
is bad. Then we can hand edit the file later (should be easy to find
the values).

> Regarding the GUI, I think I can do this with no GUI impact. This would
> simply be another type of indicator that a Strategy can get a reference too.
> This would keep things really simple for the initial impl, and reduce the
> risk of any impact to the existing trading code.

In the long term we would need a visual display that the index's
stream is alive. At least, I would like that.
As Eugene says, we would probably need some modifications to the
backtesting window.

> There is a much simpler way out, though. We can add another column to the
> existing recording format, to record whatever we define as a "long term
> trend". For example:

I admire the simplicity of the idea, but I want more data. More
columns could work. It would probably be a significant amount of
additional time for the optimizer. Also, I'm not sure how you would
read the data between strategy restarts - except for the obvious way
(read the end of the file) which itself is a little tricky for really
long files.

[Marcus Williford]

I think maybe a phased approach might work.  And, for sure I'd recommend a separate branch for this experiment.

Here is my current idea, base on our thread.  I think it addresses most of the requirements:

1.  Create a directory where index long term data files will live, no UI needed if we start with a static file name.  Yes, this is ugly long term, but this is just for phase1.  It gets us off the ground, and means we can postpone the ability to name the file yourself until other parts are proven.

2.  Create a SecurityHistory object, or every security found in this directory, which is loaded from this file location.  It will parse this file, and make available the data to other classes.

3.  A HistoricalIndicator object is then created, this is created by the user in code, EMA, RSI, etc....  I'll supply some exmples.  It also needs some isValid(date) indication, so downstream strategies will know if it is valid for a given date.  It will know if it is valid by making sure that the SecurityHistory object that it needs exists, and has enough data, with no gaps, etc.

4.  Setup a demo Sample strategy, and adds this HistoricalIndicator as an indicator, and it can use params too!.  So, it looks to the strategy just like another Indicator object.  Optimization should work fine.  The only error case is when isValid() returns false, in which case the Strategy should not trade, and should probably log this. 

5.  Come up with a way to populate and maintain our data file(s), my first suggestion was through IB data feed (auto-fill the entire year, then each day), maybe user will supply the file, maybe IB supplies it "if and only if" the user didn't. 

6.  The file format could have enough information for the IB system to query quotes.

Phase 2:

1.  Allow the UI to add a security (which creates the underlying file with the right info), also configure how the file is to be maintained, though IB or some other.

2.  Allow the UI to show the status of the data, if it is valid, date ranges it covers, etc.

3.  During backtesting and Optimization, interrogate the HistoricalIndicator dependency for valid date ranges, warning the user if Optimization or Backtesting is going to be invalid due to lack of this historical indicator data.

I "think" that covers most of the issues.

Advantages:

-  We could grow this to have the right checks it needs, and to help the user maintain the index data.

-  Users can write their own Indicator logic for historical stuff, so they can decide what is important, we just give them access to high/low/open/close

-  Strategies can have generic use of this, just like today's indicators.  The system just needs to be aware of what Historical Indicators it depends on, and for what dates it is valid.

This is probably how I'd code a beta version of this.  If there are other feature priorities, they should be considered too.  Is this one of the highest requested features?

Marcus

[Eugene Kononov]

Is this one of the highest requested features?

Not to my knowledge. Judson is obviously the one who is driving this requirement. I'd rather do other things. The two of you guys are welcome to work on this in a branch, if you'd like. 

[Marcus Williford]

Eugene,

I have the fix for bug# 28, strategy rollover.

Let me know if you want this in default branch, or if I should create a feature branch for review.

Release notes:

-  I didn't assume that closePosition() will be a success, so I check the currentPositions instead.  This makes sure that we really did close that old contract before rolling it over.

-  Because of how I did the check, I added a timestamp to make sure I don't check too often when outside of trading hours.

-  We need to know how to recreate a contract, so for now I am using an optional override, this way we don't break existing Strategy code.  I updated all checked in base classes to override this getNewContract() method.  

So, I think this was a reasonable balance of maintaining backward compatibility, yet encouraging this override for all future contracts.

Let me know where you want it for review.

Marcus

On Wed, Jan 9, 2013 at 7:17 PM, Eugene Kononov <eugene....@gmail.com> wrote:

Is this one of the highest requested features?

Not to my knowledge. Judson is obviously the one who is driving this requirement. I'd rather do other things. The two of you guys are welcome to work on this in a branch, if you'd like. 

--

[Eugene Kononov]

Very impressive, Marcus. Good to have you in the project. Please check this (and future) code changes to default branch. I'll review from there.

[Borg Alexander]

Eugene, I have set up another address. Please try Alexand...@swissonline.ch.

[Eugene Kononov]

Same thing with this address. I give up.

alexand...@swissonline.ch

Technical details of permanent failure:

Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 552 552 5.2.0 mJCe1k03R3mJnoZ07JCeko automated process detected unsolicited content (state 17).

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.

To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/PFIuDNjnAKsJ.

[Borg Alexander]

I can't understand that your e-mails not even show up in my spam folder. I've never had that. Surely the problem is on my provider's side. However I have located you in Skype. Please send your reply as a file via Skype. Thank you!

[Eugene Kononov]

I can't understand that your e-mails not even show up in my spam folder. I've never had that.

I made too may attempts to reach you, so I don't want to try anymore. Feel free to ask your questions here, and I'll be happy to respond.

[Iñaki Andersen]

Hi,

For those wanting to trade or log data from more than 3 instruments, I
think what you can do is to register a 2nd person to trade in your
account, and this may allow you to get market data from up to 6
instruments. For those interested I'd recommend to confirm before with
IB (for example through the chat). I have have registered a 2nd person
but I am not using more than 3 instruments.

Regards,
I�aki

editio...@swissonline.ch escribi�:

> Hi Eugene,
>
> this sounds great. You have done a tremendous job! By the way I have
> learned a lot about java programming while inspecting your code. Thank
> you! It's a pity that I had no time last year to follow up your idea
> with book trading but I will have more time this year. I have started
> to collect data for CL, GC and ZC this year. Unfortunately, IB does
> not allow more than 3 markets at the time. Some time ago you have
> offered your strategies for sale. I would be interested in it and sent
> you an e-mail in this regard.
>
> All the best to you,
> Alexander
>

> --
> You received this message because you are subscribed to the Google
> Groups "JBookTrader" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/jbooktrader/-/pDroCu55r6QJ.

[Iñaki Andersen]

Hi all,

First, thanks very much to the responses to the idea of looking at long
term trends. I am not as familiar as you with Java and JBT (but
improving...), so my 'simplest' idea is:

- To get the S&P index daily data from another source, e.g.
http://finance.yahoo.com/q/hp?s=%5EGSPC&a=00&b=10&c=2000&d=00&e=10&f=2013&g=d
. It allows to even download a csv file with the data we want. It'd be
relatively easy to make a script that updates the file every day
independently from JBT, or even make JBT get if a more definitive
solution is implemented.

- Everytime the strategy starts to run or there is a day change (from IB
in trade mode or from the file in backtest), the strategy recalculates a
trend indicator that looks at the historical file (daily
open/close/min/max) based on the day info and the indicator input
parameters (e.g. period of days to look at). According to this
indicator, it determines the number of contracts (0,1,...) to use for
that day and it doesn't run it anymore until the next day/start.

I guess it may require some changes in the indicators management as it
has been suggested but it doesn't seem too difficult (maybe I'm wrong
though). I'll keep you posted if/when I get to something. Of course any
kind of support would be welcome.

Cheers,
I�aki



Eugene Kononov escribi�:

[Judson Wilson]

I have raw ES index data from IB for (guessing here) 6 months?

If someone wants to write a program to synthesize a simpler dataset, i
can possibly dig up a subset of it for you to play with. But I am
pretty busy for the next week.

On Thu, Jan 10, 2013 at 2:22 PM, Iñaki Andersen <inakia...@gmail.com> wrote:
> Hi all,
>
> First, thanks very much to the responses to the idea of looking at long term
> trends. I am not as familiar as you with Java and JBT (but improving...), so
> my 'simplest' idea is:
>
> - To get the S&P index daily data from another source, e.g.
> http://finance.yahoo.com/q/hp?s=%5EGSPC&a=00&b=10&c=2000&d=00&e=10&f=2013&g=d
> . It allows to even download a csv file with the data we want. It'd be
> relatively easy to make a script that updates the file every day
> independently from JBT, or even make JBT get if a more definitive solution
> is implemented.
>
> - Everytime the strategy starts to run or there is a day change (from IB in
> trade mode or from the file in backtest), the strategy recalculates a trend
> indicator that looks at the historical file (daily open/close/min/max) based
> on the day info and the indicator input parameters (e.g. period of days to
> look at). According to this indicator, it determines the number of contracts
> (0,1,...) to use for that day and it doesn't run it anymore until the next
> day/start.
>
> I guess it may require some changes in the indicators management as it has
> been suggested but it doesn't seem too difficult (maybe I'm wrong though).
> I'll keep you posted if/when I get to something. Of course any kind of
> support would be welcome.
>
> Cheers,

> Iñaki
>
>
>
> Eugene Kononov escribió:

[Eugene Kononov]

I have the fix for bug# 28, strategy rollover.

Let me know if you want this in default branch, or if I should create a feature branch for review.

Release notes:

-  I didn't assume that closePosition() will be a success, so I check the currentPositions instead.  This makes sure that we really did close that old contract before rolling it over.

-  Because of how I did the check, I added a timestamp to make sure I don't check too often when outside of trading hours.

-  We need to know how to recreate a contract, so for now I am using an optional override, this way we don't break existing Strategy code.  I updated all checked in base classes to override this getNewContract() method.  

So, I think this was a reasonable balance of maintaining backward compatibility, yet encouraging this override for all future contracts.

Let me know where you want it for review.

Marcus, what's your preferred review method? Maybe we could have a Google+ session or something.

I have not tested it yet, but I think there are two things missing. First, when a new contract is created, it's associated with the corresponding market book. Second, the new data file is created with the corresponding file name. I don't think these two things are happening when you "rollover" the contract.

[Marcus Williford]

I think you are correct that I missed the marketbook dependency created when the code runs TraderAssistant.createMarketBook(), then creates and instrument and sets it.  I'll focus on this, and try to resolve it.  So, I'll need to study how to either create a new one, or modify the existing one (and switch files). I'll continue to test this, as my CL security is hitting the volume crossover soon.  

I think if we have "issues" for each bug/feature, you can make comments in the issues system.  This might work best, to keep us from spaming the list with specific details of code review.  So, I'll change the issues status fixed when I "think" i have fixed them, but you can use the "verified" state to indicate that you have performed a code review.  If you find a bug, like in this case, please change the state from my fixed state to something like "started", so I"ll know that I need to do more work.

Does that work?

Marcus

[Eugene Kononov]

Yes, that works for me, too. Thanks.

[Judson Wilson]

For code review, you guys might want to check out BitBucket
(mercurial's equivelent to github).

Marcus could host his experimental code there, for example.

Just throwing that out there.

On Thu, Jan 10, 2013 at 4:51 PM, Eugene Kononov

[Marcus Williford]

Judson,

If you have a standard format you want to use for daily date/open/high/close/low/volume(optional), send over a sample.  Otherwise, I'll just make something up for testing.

Marcus

[Judson Wilson]

I would use csv format like the lines in the JBT standard backtest
file. Just leave out the hours/minutes/seconds/etc and change the
columns.

[Borg Alexander]

My Questions:

Are the offered strategies based on the same idea (thresholds of tension/force = balanceVelocity – factor * priceVelocity) or is there an entirely new approach?

Are the 5 Long strategies (or the 6 Short strategies) programmed as the same code with just different parameters?

The optimization period (often called the “In Sample” period) was Oct 1, 2010 to Jan 9, 2012, i.e. 15 months. What period is covered by the results shown on the paper "Best-Performing JBookTrader Strategies)? If that is the same period, could you provide a backtest covering just the year 2012 (the “Out of Sample” period) using the optimized parameters from the “In Sample” period?

PS: UPS Cablecom of Switzerland is currently investigating the reason why your e-mail did not get through.

[Eugene Kononov]

Are the offered strategies based on the same idea (thresholds of tension/force = balanceVelocity – factor * priceVelocity) or is there an entirely new approach?

The offered strategies are based on the same idea, which is to detect the extreme imbalances in the exchange limit order book and to enter the position when this happens. When the balance returns to normal range, the position is closed. The difference with the "paid" strategies is that they use improved methodology of such detection, and are carefully optimized, backtested, and forward tested. I trade all 14 of them in real time, every day. My live results from late October 2010 can be seen here: http://rapacapintro.com/account/leaderboard/landingpage (I am listed as nonlinear/JBookTrader)

 

Are the 5 Long strategies (or the 6 Short strategies) programmed as the same code with just different parameters?

There are now 14 instead of 11, I updated the page:

https://docs.google.com/document/d/1cYjooxpxqUftqkvUWkb3CBlTFyQSx2jmi16KDVXeykk/view

The intent is diversification across different optimization parameters, short and long bias, and time frames.

 

The optimization period (often called the “In Sample” period) was Oct 1, 2010 to Jan 9, 2012, i.e. 15 months. What period is covered by the results shown on the paper "Best-Performing JBookTrader Strategies)? If that is the same period, could you provide a backtest covering just the year 2012 (the “Out of Sample” period) using the optimized parameters from the “In Sample” period? 

I periodically re-optimize my strategies to let them evolve with the market. The results are shown for the backtest period from October 2010 to December 2012.  

 

 

[Borg Alexander]

Thanks Eugene for your reply.

I realize that the price for the strategies you trade has doubled since I first came back here two weeks ago. While this is probably still worth the price, it's too much for me right now. I was willing to take the offer of the previous 11 strategies, and if you agree this would be still fine with me without having your latest additions.

Periodically re-optimizing strategies is important, as market conditions change. How this is done however depends on the person who does the optimizing. It could well be that this worked so great for you because you did the periodical optimizing. For a person who buys a system it would be important to see how a system optimized in your manner for 2009-2011 turned out during 2012. That's why I asked for it.

Do you have an idea of the maximum draw down of your system while trading all strategies simultaneously? From a risk management perspective, this would be the figure to look at. Of course I could sum up all individual draw downs but the actual maximum draw down is probably much less. 

Alexander

[Eugene Kononov]

Do you have an idea of the maximum draw down of your system while trading all strategies simultaneously? From a risk management perspective, this would be the figure to look at. Of course I could sum up all individual draw downs but the actual maximum draw down is probably much less. 

I don't pay much attention to the Max DD metric. I think the entire measure is misleading. For example, consider two strategies, A and B, and here are the respective trades:

A: {+200, -100, +200, -100, +200, -100, +200}

B: {+200, -100, -100, -100, +200, +200, +200}

Notice that the distribution of trades is exactly the same, and so are the profit factors, expectancy, net profit, standard deviation, and Kelly. The only thing that is different is Max DD, which is $100 for strategy A, and $300 for strategy B. Does that make strategy A better than strategy B? No, they are the same strategy where the sequence of trades is shuffled.

In my optimization runs, I rely almost exclusively on the PI, and the CPI (which is a new performance measure, to be introduced in the next JBT release).

[Judson Wilson]

You are making a dangerous assumption that the distribution of
wins/losses in time is random. If losses tend to happen sequentially,
that's a recipe for an account blowup - one that obviously can be
mitigated by trading smaller sizes.

Just trying to make a point.

I think the larger issue is the fact that JBT has no notion of
instantaneous draw-down.

> --
> You received this message because you are subscribed to the Google Groups
> "JBookTrader" group.

[Eugene Kononov]

I think the larger issue is the fact that JBT has no notion of
instantaneous draw-down.

What's instantaneous draw-down? Do you mean the max DD while the trade is open? 

[Judson Wilson]

Yes. Could be critical for heavy leverage or flash-crash like events -
although its hard to back test black-swans.

[Eugene Kononov]

Yes. Could be critical for heavy leverage or flash-crash like events -
although its hard to back test black-swans.

Ok. Yes, in JBT, the Max DD is only updated when the trade is closed. The reason for this is that according to my performance profiler, the calculation of max DD was one of the most expensive operations during optimization. However, that was a while ago, so it may be worthwhile to revisit this.

[Borg Alexander]

Your example with the two strategies A and B is not exactly what I meant. I talk about the cumulated drawdown of A and B at any given time. Suppose you run 5 strategies concurrently and suddenly all 5 go short at more or less the same time, and all lose money. Even when all strategies have a drawdown of less that 500 over a certain period of time, the cumulated drawdown could theoretically rise over 2000.

Im my opinion, in order to calculate the maximum number of contracts traded to meet specific risk requirements, this possibility has to be taken into account.

[Eugene Kononov]

Your example with the two strategies A and B is not exactly what I meant. I talk about the cumulated drawdown of A and B at any given time. Suppose you run 5 strategies concurrently and suddenly all 5 go short at more or less the same time, and all lose money. Even when all strategies have a drawdown of less that 500 over a certain period of time, the cumulated drawdown could theoretically rise over 2000.

Right, I was really addressing a different point.  With regards to the Max DD while running all the strategies, I have not analyzed it.

[Javier]

Right, I was really addressing a different point.  With regards to the Max DD while running all the strategies, I have not analyzed it.

First of all, congratulations to Eugene for his JBT project dedication and 2012 live trading results!  This is a real proof that a new way of algorithmic trading based on market book analysis is possible.

Secondly, in my case I optimized some sample strategies but they were really quiet during all the past year, weeks and months passed with 0 trades... On the other hand, I'm also quite worried about the risk of real time Max DD, I mean when the trade(s) are still open! I think it will be good to spend some time looking at this.

During 2013 I'd like to spend more time on two main points:
- optimizing methodologies
- new strategies development

I'd like to start discussions about this two previous points using the current JBT release.

What do you guys think? Shall we create a new JBT section just for strategies?


Have a safe trading!
Javier

[Eugene Kononov]

Secondly, in my case I optimized some sample strategies but they were really quiet during all the past year, weeks and months passed with 0 trades..

Same thing for me. All the sample strategies (and the real ones that I trade) have the same premise: enter the trade when the order book is extremely out of balance, and exit the trade when the balance returns to normal range. In 2012 (and 2013 so far), there were not that many occurrences to enter the trade, compared to 2011. I am not sure what that means. I could be simply because the volatility is low, or it could be that the irregularities in the ES order book have dissipated due to more efficient trading. It's tempting to adjust the sensitivity threshold lower to trigger the trades when less "stretching" occurs in the balance, so that more trades are generated, but if the market returns to the 2011 mode, this would mean losses. So, I'm just patiently waiting, and yes, it may mean weeks and sometimes months without taking a trade.

[Ali Farahani]

Greetings,

I should first mention that I don't believe historical data can "predict" the future. However, I do believe that price and volume data (within short time frames) provide a fast moving window of opportunity for profitable trading. In 2009 and 2010 I used JSystemTrader for index trading using a version of the Money Flow Index strategy. I have been skeptical about Order Book based trading for the following reasons:

1. Book data does not necessarily reflect the true intentions of market participants (i.e. dark liquidity, managed order placement, etc.);

2. Although I believe it's new information that drives the behavior of the market, it is Price (and volume) that embodies the sum of all the information available at the time of each trade (of course, volume is the other factor).

It would be productive to explore new frameworks for trading strategies that use Price and Volume. Of particular interest to me are also variables such as the dynamics of the Ask-Bid versus filled price.

I do understand that all such functionality are also available in JBookTrader (thanks to Eugene's masterful work); my suggestion is more on the strategy development side than it is on the technical (no pun intended) side.

Kind regards,

Ali

[Eugene Kononov]

Hello JBTers, 

I'll make the quarterly updates in this thread.

1. JBookTrader is about to be released (I know, I promised it in January)

2. I continued to trade my ES strategies with JBT, and added a couple of CL strategies. The performance chart (from the IB report) is attached below. I was slightly down in January, but made gigantic gains in February and March.

3. I am still tracking my live trading performance with RapaCap, which is the service to match best traders with the capital. Despite my good absolute performance, my relative standing with RAPA actually came down, as my gains came with a lot of volatility, and as a lot of new traders signed up with RAPA:

http://rapacapintro.com/account/leaderboard/landingpage

My account is shown as nonlinear/JBookTrader in the leaderboard.

4. In February 20123, I also started publishing the trading signals from JBookTrader to Collective2. My Collective2 system is listed here:

http://www.collective2.com/cgi-perl/system78983390

Among the 20,700+ strategies published on Collective2, JBookTrader system is currently rated as #3 based on annual return, and #8 based on Sharpe's ratio.

5. JBookTrader is my full time job now, as I found a partnership with a company which is interested in making a commercial spin-off from the open source model.

Best to you all,

Eugene Kononov.

[Klaus]

Dear Eugene, 

this is really cool results all over!

And my congratulations that you can turn your invention into your full time profession!

Klaus

[Mick O'Donnell]

Yes, congratulations on this great news. I'm only new to the JBT circle, but it's a fantastic bit of software for strategy traders. I hope the open source model continues to thrive into the future and I get a chance to commit some code to the project.

Michael.