Pre-testing etc

[Allan Deacon]

I don't know if anyone else is doing this (or something similar) but this is what I basically do now:

 

I look for 'possible' strategies using lots of timeframes but with, say only a 200 population over 10 generations, 100% in sample.

 

When I find something interesting, I do a run of 2000 generations (with 20% OOS) - unchecking 'reset on build'. At the end of each generation run, while the build continues (you usually have time) I quickly sort the OOS table on Coeff Corr and then Net profit. If there are any equity curves that look promising, I stop the build and save it using a sequential number e.g. Build 143a / b/ c etc. I log everything on a speadsheet. Then I resume the build and repeat. What I usually find, is that the 'promising' candidates (as Mark points out) get shunted down the table and disappear as the fittest in sample take the lead.

 

I can then re-visit the saved builds and explore further.

 

Not perfect and certainly not scientific but it seems to work for me and the strategies it finds are certainly better that some of my home brewed ones.

 

Allan.

[Mark Knecht]

Allan,
See Mike's Breakout Bulletin for his cut at the same sort of thing.

What I like about Mike's method, and it potentially fits well what
you are doing also, is that it's a measure of both the data being used
as well as the metrics chosen along with their weighting. In Mike's
paper he suggests running Builder a number of times (say 25 runs) and
looking at the average of all of the OOS results. If most of the 25
runs produced profitable OOS results then this data with these metrics
are worth further exploration. (Or that's my take on compressing 3000
of Mike's words into 25 of my own!) ;-)

- Mark

[Allan Deacon]

Mark,

Many thanks for that. I did have a quick scan of Mike's paper but other
matters cropped up. I will re-visit this - sounds good.

Thanks for the comments on the metrics also. I did look at the new
procedures but I think, like you, I was so keen to start using the new
version of Builder....

Allan.